Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Log cf-ray headers from CloudFlare, if available #3576

Open
wants to merge 8 commits into
base: main
Choose a base branch
from

Conversation

heckj
Copy link
Collaborator

@heckj heckj commented Jan 1, 2025

This adds a replacement for the default route logging middleware from Vapor that adds in the cf-ray id, if any, from cloudflare.

This expands the logging data a bit, but gets cf-ray data exposed in the logs from the server. I'm not 100% confident how the cf-ray header is spelled and passed through, so I set this up as a low-cost experiment. My idea would be to(if this is acceptable) running this in prod for a week or two, do a log dump, and use analytics on the request logs to identify a bit more of normal-use pattern to be able to tune a rate-limiter-like cache to observe, and hopefully ameliorate, excessive bot-load, like what happened on Dec 30th.

Right now it's dumping out --nil-- in the logger metadata. More than happy to tune this to be in the log message instead of the metadata, or just not log it if the header doesn't come through. I do expect to see this in a server fronted by CloudFlare, but I don't have an easy experimental case to know for sure.

@heckj heckj requested a review from finestructure January 1, 2025 22:32
@heckj heckj self-assigned this Jan 1, 2025
@cla-bot cla-bot bot added the cla-signed label Jan 1, 2025
@heckj
Copy link
Collaborator Author

heckj commented Jan 2, 2025

yeah, the --nil-- was only because I wanted to get a explicit reinforcement if I got the spelling of the header correct, and highlight where the output would be while doing the dev work - since none of that is behind CF

@finestructure
Copy link
Member

Let's get this running on dev and we'll see how it goes.

@heckj heckj force-pushed the heckj/cfrayRequestLogging branch from 85d6fc0 to 4278425 Compare January 17, 2025 20:40
@heckj
Copy link
Collaborator Author

heckj commented Jan 17, 2025

@daveverwer @finestructure - I think it might be useful to have this in place, although now I'm wondering if I shouldn't augment it to expose more detail. My thought it to have the details available from the SPI logs to do post-attack analysis that encompass where the attacks came from, and perhaps their user-agent reporting as well.

I'm not sure what is available through the CloudFlare analysis tooling, so if it's easier to view/analyze from these elements there, then I'll close this out - no use having duplicated content. I'm afraid the analysis side of mitigating the pressure of random crawlers on the site is a bit hidden to me, so this is kind of stab in the dark.

(just rebased this on the latest updates to main)

If this is obviously redundant to what's available, or potentially useful but insufficient to give a view to do useful analysis, saw the word either way. I can close this out as not needed, or bolster it up.

@heckj heckj requested a review from daveverwer January 17, 2025 20:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants