Tell HN: Cloudflare verification is breaking the internet
1. "Verify you are a human"
2. Check the box or perform some other type of rain dance
3. "Please stand by, while we are checking your browser..."
4. Repeat step 1
I'm on Fedora Linux 37 using Firefox 110.
The workaround is to use Chrome.
After experiencing this dozens of times and getting annoyed of needing to use Chrome, I finally went and deleted all my cookies and cache which I had been dreading to do.
It did not help.
I don't have a CloudFlare account so I wrote up a detailed post on their community forums. I offered a HAR file and was willing to do diagnostics. It received no responses and it was auto-closed.
It's unacceptable that CloudFlare is breaking the internet while offering no community support.
Edit: I'm in Texas. I'm not using a VPN or Tor, just AT&T Fiber. I don't have ad-blockers. No weird extensions. Nothing special (besides being on Linux).
Edit2: Since this got traction, I opened a new community post: https://community.cloudflare.com/t/infinite-verify-you-are-a-human-loop/503065
To be clear, I'm not against CloudFlare doing DDoS protection, etc., but it can't be breaking the internet while ignoring community posts on it.
Edit3: The CloudFlare team has engaged. Thank you HN!
339 comments
[ 34.7 ms ] story [ 6865 ms ] threadObviously most small sites are not actively targeted by bots and using reCAPTCHA is a waste of money and people's time. But if you are, reCAPTCHA is a godsend.
It's not so much that "people … are opposed to reCAPTCHA", but that for some they can't make it work.
Speaking of bullshit restrictions designed to encourage compliance with surveillance, have imgur links just straight up stopped working for anyone else recently? I'm coming from a datacenter IP. I assume it's just some heavy handed part of the cost cutting push they announced.
CloudFlare is merely the symptom of a greater set of problems, which it attempts to mitigate.
If you want to be angry about something, be angry that bruteforce attacks are common, guzzle resources and usually yield zero legal repercussions in most cases.
The centralized solution is going to be a government-owned/controlled MITM service like CloudFlare. No doubt with actual ID for verification.
I don't see the decentralized solution happening any time soon.
Massive attacks existed long before CloudFlare ever did. If you're implying there's a conspiracy that CloudFlare is attacking others directly or indirectly to sell their solutions, I'd be extremely careful as that's defamatory and almost certainly false.
Furthermore, most CloudFlare users only use the free plan and thus cost CloudFlare money. Isn't that curious?
There are too many bots out there that are very inconsiderate and do not limit or throttle themselves.
We have one right now that crawls every single webpage (and we have 10's of thousands) every couple days, without any throttle or limit. It's likely somebody's toy scraper, and currently it's doing no harm, but not everyone has the server resources we have.
The point is - if you are dealing with inconsiderate bots, a captcha of some type is pretty nearly a bullet proof way to stop them.
With that said, Cloudflare usually is smart enough to detect unusual patterns, and present a challenge to only those who they believe are bots or up to no good. If every person gets a challenge, then the website operator is either experiencing an active attack, or has accidentally set their security configuration too high.
Not any more.
At least in our experience.
Of the three common types, only one has a serious interest in breaking captcha's - but they also have a serious interest in not getting too much attention by abusing your services deliberately. ie. if a bot is misbehaving, it's going to get our attention, and we're going to look into what it's doing, where it came from, who operates it, etc, and possibly take some action if appropriate or available. Those actions may not necessarily be limited to the technical space either...
This is just our experience. Other industries will have their own sets of challenges to deal with.
1. Most sites only have this problem due to inefficient design. You are literally complaining about handling 1 request every 2 seconds! That's like a "C10μ problem."
2. How many IPs are these bots coming from? Rate limiting per source IP wouldn't be nearly as intrusive.
3. There are much less obtrusive ways of imposing resource requirements on a requester, like say a computational challenge.
> You are literally complaining about handling 1 request every 2 seconds
I don't know where this came from. The inconsiderate bots tend to flood your server, likely someone doing some sort of naïve parallel crawl. Not every website has a full-stack in-house team behind it to implement custom server-side throttles and what-not either.
However, like I mentioned already, if every single visitor is getting the challenge, then either the site is experiencing an attack right now, or the operator has the security settings set too high. Some commonly-targeted websites seem to keep security settings high even when not actively experiencing an attack. To those operators, remaining online is more important than slightly annoying some small subset of visitors 1 time.
100,000 / (86400 * 2) = 0.58 req/sec.
I acknowledge that those requests are likely bursty, but you were complaining as if the total amount was the problem. If the instantaneous request rate is the actual problem, you should be able to throttle on that, no?
I can totally believe your site has a bunch of accidental complexity that is harder to fix than just pragmatically hassling users. But it'd be better for everyone if this were acknowledged explicitly rather than talked about as an inescapable facet of the web.
Again, not every website is the same, and not every website has a huge team behind it to deal with this stuff. Spending 30-something developer hours implementing custom rate limiting and throttling, distributed caching, gateways, etc is absurd for probably 99% of websites.
You can pay Cloudflare $0.00 and get good enough protection without spending a second thinking about the problem. That is why you see it commonly...
If your website does not directly generate money for you or your business, then sinking a bunch of resources into it is silly. You will likely never experience this sort of challenge on an ecommerce site, for instance... but a blog or forum? Absolutely.
I'd say you're really discounting the amount of hassle people get from these challenges. Some sites hassle users every visit. Some hassle users every few pages. Some hassle logged in users. Some just go into loops (as in OP). Some don't even pop up a challenge and straight up deny based on IP address!
And since we're talking about abstract design, why can't Cloudflare et al change their implementations to throttle based on individual IPs, rather than blanket discriminating against more secure users? Maybe you personally have taken the best option available to you. But that doesn't imply the larger dynamic is justifiable.
Cloudflare does not do this - I've made that point several times. The website operator either has the security setting cranked to a paranoid level (which is not the default, btw), or they are experiencing an attack. Those are the only two scenarios where Cloudflare is going to inject a challenge as frequently as you assert.
Normally Cloudflare will only challenge after unusual behavior has been detected, such as inhuman numbers of page requests within a short duration, or the URL/forms are being manipulated, etc. The default settings are fairly unobtrusive in my experience.
If you are also complaining about generic captchas on forms and what-not, that's a different thing entirely. Those exists as anti-bot measures, naturally, but also as anti-human measures. We simply do not want a pissed customer to send us 900 contact-us form requests one drunken evening...
This is a bit of intent laundering. By Cloudflare providing ridiculous options, some people are going to take it because more "security" must be better.
> Normally Cloudflare will only challenge after unusual behavior has been detected, such as ...
or people using more secure browsers like Firefox with resistFingerprinting = 1. I suspect this is a significant blind spot for site operators. Have you personally tried your own site with RFP=1, TOR browser bundle, VPN from a datacenter IP, etc?
> generic captchas on forms ... exists as anti-bot measures, naturally, but also as anti-human measures. We simply do not want a pissed customer to send us 900 contact-us form requests one drunken evening
My whole point is it's a bit disingenuous to throw out large quantities of things as the argument, when the hassles are often thrown up on the very first request. I'm not complaining about the sites that throw up CAPTCHAs after the third failed login, but rather the ones that do it on the first attempt!
And sure, I don't have a good map of which types of hassles are specifically Cloudflare versus others of their ilk. And I certainly don't know how often Cloudflare doesn't cause problems, as it doesn't stand out. I just know there is too much indefensible surveillance-based user-hassling in general and OP's anecdote is right in line with my standard browsing experience on many sites these days.
Yes, and it is not an issue for us. Again, this is up to site operators to decide for themselves. The defaults are sane, and Cloudflare makes it very clear what each level of their security configuration does. It is up to the site operator to decide how they want their site to behave. Perhaps, simply avoid sites that bother you? That list will grow by the day, unfortunately.
> TOR browser bundle, VPN from a datacenter IP, etc
Nobody, and I mean nobody, cares about this traffic. We're in the ecommerce space, so perhaps by that I mean nobody in the ecommerce space cares. We do not want TOR traffic. We do not want random-cloud-ip-vpn traffic. These are more often than not where our fraud bots/attempts originate, and we are not alone.
Recognize, if you are using TOR, or browsing regularly via a datacenter IP VPN - you are in an extreme minority and unfortunately lots of folks before you have used these services for bad things.
I personally like TOR, and VPNs. This is no slight against them - but the facts are undeniable here.
> surveillance-based user-hassling
You also referenced canvas-based fingerprinting, and seem to assume that's how these things work. Some might, but many are much more dumb than that. Usage-pattern based challenges are fairly simple when you understand what normal traffic looks like.
Even if they did, I'd still avoid imgur since they censor even worse than reddit.
It wants to do a bit of cryptography, which means that if scripts/WASM/etc are disabled, you can be out of luck.
CAPCHA/RECAPCHA is the internet version of the infamous "regatta" question on SAT [1].
[1] https://www.clearchoiceprep.com/sat-act-prep-blog/the-most-i...
It is funny how our five year old daughter can recognise what American school buses look like, simply through media exposure, despite the fact that buses in our country look completely different (and our school buses don't look different from public transport buses, since they are the exact same buses and drivers, just scheduled on school routes instead of public ones.)
Sometimes I can get rather critical of American cultural imperialism, but this kind of thing is more at the amusing than concerning end of that spectrum. It is a good example though of how many American businesses are happy to offer their products outside the US with minimal or no attempt at localisation–and either don't realise the reality of that lacking localisation, or do yet don't care. It is particularly a problem I think with other English-speaking countries, where people just assume that if the language is the same everything else must be, or else their idea of the differences is limited to a handful of well-known items like date formats
(Comment written from memory, I may have details wrong.)
Did you get that sorted out?
Asking because we (sqlitebrowser.org, dbhub.io) have a bunch of Hetzner dedicated servers that are nowhere near fully utilised. Could probably figure something decent out using those, as Hetzner doesn't charge for bandwidth.
How much space have you got by the way?
For these boxes, once they're set up they tend to not grow all that much in disk space.
I've got a Firefox extension that tells me if a site appears to be using Cloudflare - and I avoid all the ones I can
But I'm stuck with that stupid Cloudflare slowdown screen for the portal to my dr's office
Maybe time for a boycott of sites using cloudflare /s :)
I also wonder how hart this is for people who are blind, I think they would have a very hard time. Seems to me blind people in the US could use cloudflare using the American Disability Act.
Pull the "/s" off, and you've already got one person (me) on your side :)
Even if you were doing any, or all of these things, you are no less a legitimate internet user than anyone else. This whole "rain dance" supplication to show you are worthy of browsing a web site has got to go. Stop visiting sites who treat their users this badly!
We are trying to frame people who are trying to protect their privacy as "suspicious" rather than saying that we want to track them better.
You want to determine who is manually asking vs automated so you can ignore requests that aren't manually generated.
At the server or datacenter level, it's call a firewall :)
When not in a vehicle and there are no cops around, I do the New Yorker thing: I completely ignore signals and focus on traffic. The prima facie and prime directive is safety over conformance. I will not waste my life at the behest of some Christmas lights.
The problem is the individual sites aren’t making these highly technical decisions, people are using what seems to them an innocuous security product.
Not visiting a random website places no pressure on CloudFlare to change, since there’s no way to correlate your choice with the decision to use CloudFlare.
I'm wondering how long it will be before we have memory holes considering how, apart from the internet archive, there is perpetual bitrot and silent updates.
there's no way to solve this problem without having some sort of tracking system to determine who's a legitmate user.
and when some legitimate users really, really look like bot traffic because they circumvent whatever methods we use to determine whether traffic is coming from real people, they might sometimes get blocked along with the bots. they're going to complain about that, and the only thing we can do is listen to their complaints.
It's not a hard concept. Unless you think you're too big to care, you fix issues you cause. (If you are too big to care, I hope the laws regulating anticompetitive behaviours hit you)
Solution 2: Everyone in engineering and management at CF can access internet only while marked as the same level of trust as the lowest one currently assigned to cgnat-s and tor exits. No exceptions, but they can contact support like any external user.
> If it were coming from one IP address nobody would be bothered by it, it would be easily solved with rate-limiting rules on the firewall.
DDOS works by sending more traffic than your upstream bandwidth can carry (e.g. you have 100 Gbit link and they send 40 Tbit of UDP packets to you). Firewall won't help here. The protocol I am talking in a comment above would solve the problem by blocking this traffic close to its source.
Not by any stretch of the imagination.
My preferred solution would be domain validated identities with long lived, global reputation alongside some type of attestation. For example, if I have a GitHub account with 'example.com' as a verified domain, GitHub could attest 'example.com seems to be a real user or organization that behaves well'. It would be similar to the web of trust concept in GPG, but technology is to the point where it could actually be built in a way that makes it usable. Money that you're spending, or the way you interact in well known communities, could have the side effect of bolstering your reputation everywhere.
My most feared solution would be a similar system of attestation, but using Passkey since it would solidify the role of the current big tech companies as the arbiters of everything online. For example:
Those companies, as Passkey providers, would, for all intents and purposes, be your 'anchor identity' online and they'd be in a good position to attest to you behaving like a normal, non nefarious participant.I think Apple would be the company that could sell that kind of change to normal users. It could be done in a way that's anonymous because all you really need is an attestation that says 'Apple certifies this user is in good standing'. Apple is very good at selling those kinds of changes as being privacy focused and I think their user base would go for it if it were framed as 'good people' (aka Apple device owners) getting a superior experience that isn't available to the 'bad people' (aka bots, bad actors, and outliers).
If it worked, Google would follow with Android. Anyone else large enough for their opinion of you to count (Microsoft, Facebook, etc.) could probably compete, but it doesn't work for startups or small, less known providers.
In my opinion, as soon as authentication moves to something like domains or digital signatures where 3rd party attestations become simple, we could see a lot of new ideas that focus on reputation and related solutions / services.
https://support.apple.com/en-us/HT213449
https://developer.apple.com/wwdc22/10077
While its build with privacy in mind, I have some deep concerns on making these current big players gatekeepers to identity on the web
Too bad that basically means you can't surf the internet anymore as a majority of websites use Cloudflare. One of my Firefox installations on Linux are also plagued by this. I can't use Firefox to browse the web.
CloudFlare blocks me from a part of the internet when I use anonymizing tools like Tor. I assumed they just do that to fingerprint and track you. Even the crypto thing to get a dozen or so passes after solving a riddle never worked.
So I have just moved on to websites protected by Akamai, or virtually anything but CloudFlare. It's not just a political decision btw. It's just easier to move on than to try to fight CloudFlare or to become viral on HN to get support.
It shouldn't be up to the user to adapt, but to the website.
This is just whining. I don't necessarily like it either, but you conveniently ignore all the reasons why that rain dance supplication exists in the first place. All ears if you have a better solution for DDoS attacks, malicious bot traffic, etc.
On Firefox it hasn’t worked for a long time.
People use CloudFlare to solve a multitude of problems, some of which include automated attacks by bots, which would make the website unavailable in the first place.
If you're going to use a non-mainstream browser then you're going to compromise in some way. If people are going to defend their website against attackers then there's compromise.
CloudFlare isn't the problem, it's a symptom of other problems left unsolved. Is it a compromise? Yup. What's the alternative? Not using it and thus having constant downtime?
What they seem to be doing is just presenting a CAPTCHA to anything at all unusual. Which is actually kind of strange, given the vast amount of raw data available to them. They should be able to learn real indicators.
I'm actually not even sure Cloudflare is primarily responsible for most of this... exactly. The problem is more likely that Cloudflare gives its users a lot of knobs to twiddle, and most of the users are probably not up to twiddling them correctly. That could be the main source of these problems.
And there are so many possible combinations that it would be hard for Cloudflare to really test them, or even think about how all the knobs might interact.
Taking away knobs would be a good start, but there may be reasons they don't think they can do that. Probably reasons that are more about their customers' perceptions than about their customer's real needs.
Come to think of it, isn't one of those knobs the ability to turn off PrivacyPass? I don't have access to a Cloudflare account at the moment, but I seem to remember that it was.
This is a good point.
Bots are going to try to make their traffic look as legit as possible, which means spoofing the most common browsers with the most common setups.
So if a User-Agent is reporting that it's running Firefox, it's actually more likely that it's legit traffic, as bots wouldn't try to pretend to be an uncommon setup.
The "non-mainstream browser" comment is about zigging when the overwhelming majority of people zag. It's a self-inflicted problem.
Don't like it? Pick a different set of compromises, whether that's using Privacy Pass or a Chromium-derivative browser. God forbid people work on the problem itself rather than just complain that they don't like compromising.
You can't demand "taking away knobs" with taking away control of the end user on CloudFlare whilst simultaneously lamenting that using Firefox (browser choice is a "knob") yields compromises. It's hypocritical.
Why shouldn't people be free to deny Tor users access to their server? Why shouldn't people be free to self-inflict their set of compromises on themselves with their choice of browser? Why shouldn't people be free to mitigate bruteforce attacks? It may not align with your views or beliefs, but that service provider is free to do as they please within the extent of the law. Doesn't make it ethical, but your access to the service depends on x.
Life is compromise.
It definitely is. And there are even bot detection services that can detect puppeted Chrome installs pretty reliably (I ran into that when I tried to scrape some data about the housing market). Blink, WebKit, and Gecko are the only common browsers and the rest is a long tail. If you pick an uncommon browser (Lynx, Ladybird) you're an outlier in most automated scans but still end up with a smaller total browser market share than even the small bots. Another reason to be suspicious of uncommon clients is that puppeted Chromium builds with special flags to prevent bot detection don't run on a hacked security camera/router/TV box/NAS/IoT box.
If you're being extorted by someone who paid $50 to DDoS your business for a month, you're going to turn up the DDoS protection knobs. The annoying tracking, cyberstalking and CAPTCHA services are mere symptoms of the underlying problem.
I wouldn't want to use an internet where Cloudflare doesn't give you knobs to turn. You'll end up with websites either not being protected from DDoS attacks or several layers of CAPTCHAs for everyone. Sometimes you need to turn up the protections when the defaults don't work well enough but the defaults shouldn't be high enough to cover those scenarios.
Also, I'm using Falkon browser every day - ever heard of that? I have to switch user agent to be allowed in some places which is ridiculous.
It may be profitable to use CAPTCHA for AI training, but that's not only annoying, it's also unethical because one (like me) may be unwilling to engage in such activity. Also, CAPTCHAs involving houses, number plates or bikes are absolutely invading someone's privacy.
Owners of websites (e.g. shops) themselves have more options to show captchas only in critical moments: when performing heavy searches, registering, checking out, posting. Again, it doesn't have to be intrusive or disruptive. But I understand this way takes more professional approach and probably requires programming, which is not what _every_ store owner can probably afford.
Probably because it takes fractions of a cent to solve those grainy distorted captchas?
Whereas its not so trivial to get an extra IP address, extra computer, etc.
I can’t think of any alternative that would still be as onerous a bar to spammers and bots but also be less restrictive for genuine users. Other then linking real IDs, which has its own can of worms.
Is an independent indicator, or is statistically correlated?
> And there are even bot detection services that can detect puppeted Chrome installs pretty reliably (I ran into that when I tried to scrape some data about the housing market).
Interesting. The arms race continues...
> Blink, WebKit, and Gecko are the only common browsers and the rest is a long tail. If you pick an uncommon browser (Lynx, Ladybird) you're an outlier in most automated scans but still end up with a smaller total browser market share than even the small bots
The post I was responding to was calling Firefox an "uncommon browser".
> If you're being extorted by someone who paid $50 to DDoS your business for a month, you're going to turn up the DDoS protection knobs. The annoying tracking, cyberstalking and CAPTCHA services are mere symptoms of the underlying problem.
Wouldn't most Cloudflare users prefer that Cloudflare notice that attack, adjust the settings by itself, and send them an email saying "You appear to be under attack; we've enabled X, and lowered the thresholds for Y and Z"? And then notice when the attack seemed to be slowing down, and put things back the way they were?
I'm normally not a fan of machines acting like they know better than I do... but the machines probably do know better than Cloudflare's average customer.
At the very least, they could probably find ways to discourage people from messing with knobs they don't understand, and more ways to make the specific costs obvious, even if those knobs ultimately stayed available.
Don't like it? Use Privacy Pass or pick another set of compromises.
With that said, when I was still willing to subject myself to Mozilla in any form, I never found CloudFlare to be a problem, and I've used it since it launched. If people use Firefox, they're probably a privacy wonk LARPing an imagined threat model, and are using a shitty cheap VPN used by countless attackers, it's unlikely to be the browser itself unless they're doing something weird with extensions.
Nah. I'll just not use the website. They obviously don't want me there anyway.
I'm sure nobody cares about my blocks, but we've got to start somewhere if we want websites to change their behaviour.
I'm confident you use CloudFlare every day, many times per day.
What kind of nonsense is this? Firefox is a mainstream browser used by millions of people every day. Maybe you're confusing it with Tor?
You could had just try it in the porn mode. Another option is to use a different profile or a portable version.
https://support.mozilla.org/en-US/kb/profile-manager-create-...
https://portableapps.com/apps/internet/firefox_portable (Windows only, I guess)
There are some things around local storage which isn't cleard even if you clear cookies.
You can thank abusers and spammers for ruining the internet for you, not website operators trying to deal with spam/bots.
I've had my most inconsequential service taken offline with a $5 booter because the user wanted to brag on Discord. You can bet I default to Cloudflare now.
It's not just for the website operator either. All of my users suffer when $5 botnets take down my server too. And it's cheaper and cheaper to do that every year thanks to the internet of shit.
So I'm not sure who this "Tell HN" PSA is for. Are the baddies going to read about your inconvenience and stop being baddies so we don't need to use captchas anymore?
And yes, it's annoying that we live in that world. In 1999 you could probably assume a request was human with a User-Agent regex.
In 2024, your smart toaster could be saturating your AT&T Fiber uplink without you even knowing while you're rage-posting in Cloudflare's forums about HAR files and how you're not a bot.
As mentioned, it works fine in Chrome on the same computer. CloudFlare has engaged and is investigating, thanks to this HN post.
I propose that we make owners of shitty devices responsible for their actions. if my internet of shit thermostat begins spamming people, that would be my responsibility, if it participates in a ddos, that would become my responsibility.
But similar to somebody's leaky car, good luck finding them and enforcing they actually clean it up.
Yet read-only access to websites, which by definition can't be used for spam, is also locked behind Cloudflare. The same old excuse every time - they're given a legitimate inch for security, but take a mile.
Most telling is that you don't even get heavily rate-limited access to a website without passing Cloudflare's filter. Because then your actual behavior could be used to determine if/how much of a DDoS threat you are. But that would take away Cloudflare's excuse to monitor users, so they prefer to use absolutes.
Not sure how the math works out for ad-supported sites, but it pretty strongly favors "moderately-aggressive automated blocking" for those taking direct payments.
But Cloudflare often enough blocks users from reading content pages. Cloudflare could just serve their cached static content instead of showing Captchas.
Sure, a general solution is better, but since everything today is docker running node.js running without a modicum of caching or appfw in front, not surprised things are so fragile
>> This isn't true, though.
What??
Also, you are probably missing my point: it's not like sites don't need protection, it's the unfriendliness of how CF implements it.
No, definitely not. I'm completely incapable of logging into several different services that have Cloudflare's protection (including their own website) if I use Chrome on my iPad. If I try on mobile Safari on the same device (which has basically an empty history), it goes through just fine.
Something is broken.
Sounds very dystopian and DRM-y for me. You would have to enforce this at the OS or even hardware level (because otherwise bots will just lie). And probably mandatory by law.
Unfortunately I don't think we can force devices to differentiate between "user-originated" and automated traffic without changing internet fundamentals and locking it down.
At some point balance will tip where an internet with only authenticated traffic will be seen as more usable and preferred by the masses, then anonymity is over. New AI systems generating unfathomable amounts of human-like garbage to flood all UGC platforms might actually become the tipping point fairly soon.
If a Cloudflare-like service is a required part of internet infrastructure, then each ISP and hosting provider should offer their own, equivalent service. By law, if necessary, if the economics don't work out otherwise. Because having a single company be the arbiter and monitor of who may visit any website is, well, bad.
The other day I stopped the Cloudflare CAPTCHA for a day just to see what would happen and the next day I saw fake orders with disputes and credit card testing which costed my business thousands.
I don't think this is a major problem for consumers, but for merchants, without CAPTCHA it is even worse for merchants.
I think I'll keep the CAPTCHA turned on, not sure if there is an alternative though.
I am also in Texas. Also using Mozilla Firefox on Fedora, on Spectrum / Road runner / Charter.
https://support.mozilla.org/en-US/kb/profile-manager-create-...
I personally don't mess with profiles. I download firefox developer binaries and put them in ~/bin folder which uses a different profile by default (no extensions for web dev test).
Some related issues:
- https://forum.gitlab.com/t/cant-open-the-signin-page-it-keep...
- https://gitlab.com/librewolf-community/browser/linux/-/issue...
- https://github.com/arkenfox/user.js/issues/1253
When I change the protocol and get the redirect back to https there's another "/" which is added after the domain such that "domain/path" becomes "domain//path". This repeats if I continue to change the protocol and hit the redirect such that "domain//path" will become "domain///path" (I noticed this because there was like 6 of them).
Apologies if this is indeed caused by my browser settings; I've been unable to find the cause if that's the case.
I suppose I better update it now, sorry for the inconvenience.
I see them using some VPNs and using Tor, but that makes sense, because that's super close to the type of traffic that these filters were designed to block.
I suspect people behind CGNAT and other such technologies may be flagged as bots because one of their peers is tainting their IP address' reputation, or maybe something else is going on on a network level (i.e. the ISP doesn't filter traffic properly and botnets are spoofing source IPs from within the ISPs network?).
Try ten Google dorks for finding open Apache directory listings; your IP address gets reCAPTCHA prompts for every single search query for minutes. Share that IP address with thousands of people, and suddenly thousands of people get random Google/Cloudflare prompts.
With the difference being that you get your own /48 or /56 and suffer from only your own behaviour.
If you're behind CG-NAT because your ISP can't get enough IPv4 addresses, then you suffer from the behaviour of other people.
IPv6 is way better than cgnat, but ISPs are still doing their own internal routing for much smaller blocks. Meaning the block itself is functionally the equivalent of a shared IPv4 for abuse prevention purposes.
But also, I could just not know about the ISPs giving out /48s. My window to this is from the abuse prevention side.
You'll be able to to get them from any geo-location easy as pie.
So it's worse. You'll be even less trustworthy unless you register as trustworthy and keep it, which means tracking. The same as having a fingerprint or login now.
As pro argument that sucks, it's the opposite.
This is a thing that is absolutely happening, I got temporarily shadowbanned for spam on Reddit the day I switched to T-Mobile Home Internet which is CGNAT'd, and I didn't post a single thing
I'm actually kind of glad more people are becoming aware of this problem, and hope it finally spurs more interest in mechanisms that divorce network identity from IP addresses -- including the work Cloudflare is doing on Privacy Pass!
Who said that? I don't see anyone saying that.
All this says is "This explains why I get blocked while using tor or vpns", It does not say they agree with it or accept it etc.
It only says they are not suprised that it happens, that they understand the mechanism by which it happens, not that they accept or agree with it.
They might or might not also think it's fine and reasonable. I can't say they don't approve any more than you can say they do.
However, the sad fact is that Tor is abused for a LOT of malicious traffic, much more so than any VPN provider, let alone normal ISPs using CGNAT. The anonymity combined with its free nature make it very attractive for bad people to use Tor for bad things without any reasonable fear of getting caught.
An outright block for Tor traffic is definitely out of the question, but adding CAPTCHAs to sensitive things (like account signups, expensive queries, etc.) is sadly a requirement these days.
Blocking exit nodes does nothing to protect your website's security, but it sure as hell cleans up the logs and false positives in your security logs. It's not just Tor, though, there are also some home ISP networks that don't seem to care about the botnets operating inside their network.
Maybe it is just per use case. Or they think I'm a bot as I keep looking at sites every couple hours... Which might be actually common with these sites.
The Cloudflare verification has become a sick or sadistic joke now. It's often just used to annoy people, and no matter if they pass the tests, denies access anyway. If the test is not going to determine access, then don't provide it, and just wholesale be up front on mindlessly or frivolously blocking people and entire IP ranges.
For security, an actor needs to be tested and marked as secure, or else tested again before every interaction.
For privacy, an actor must not be marked, lest observers could correlate several interactions and make conclusions undesirable for the actor.
It does not make the infinite loop produced by CLoudflare any more reasonable though.
One far-fetched idea is to use ZKP proofs to prove that you were verified, without disclosing anything your identity. But that's likely overkill.
Anyway, I think Cloudflare already works on something better with turnstile, the "privacy preserving captcha" and private access tokens [0].
[0] https://blog.cloudflare.com/turnstile-private-captcha-altern...
https://privacypass.github.io/
Theoretically, to penalize the user you need to identify the user. And for that you need to maintain long term identity.
Trivial counter-examples include proof-of-work (see HashCash) or cryptocurrency micropayments (not necessarily settled on-chain, so transaction fees are not an issue for the user).
The idea was to prove that a token exists without disclosing the token itself, nor any sort of 1:1 substitution.
That sort of thing is definitely possible, that's not the conundrum. What they said is one of the conundrums I have to admit. If the server doesn't know who the user is, then the server doesn't know it's a valid user vs a bot.
But I only agree it's a problem. I don't agree it's a problem without a solution.
CloudFlare claims to support Privacy Pass, which is supposed to use a zero-knowledge scheme to solve for this for Tor users.
Unforunately, the integration has been broken for a very long time and bug reports aren't tended to.
https://blog.cloudflare.com/cloudflare-supports-privacy-pass...
https://privacypass.github.io/
https://github.com/privacypass/challenge-bypass-extension/is...
"Cloudflare is not happy with anything that is not Cloudflare"
ftfy :)
If you get locked out of your hotel room, do you call Assa Abloy to complain?
Complain to the site that their site doesn't work. They are the ones that install and configure their security software.
An increasing numnber of shops on the street have locks that silently open if you look like the right kind of person, but lock if you don't look right.
And most people look right, so they don't even realise the lock is there.
Here's some real-life equivalents to a web application firewall:
https://www.flickr.com/photos/ibran/595450232/
https://www.manythings.org/signs/im/shirt_and_shoes_required...
https://media.istockphoto.com/photos/restaurant-dress-code-p...
And the shop owner just asked for "good security" or is even on default settings.
If a shop is operating under a license to be a business, it is open to everyone who does not do something wrong.
Choosing to use a different browser is not the same as being nude or so dirty as to be a health risk, or commiting violence or property damage or other disruptive acts.
Even private businesses, at least in the US, are not allowed to mysteriously be out of stock of everything only for the black family. They aren't allowed to have a "dress code" that says "be white".
A dress code for a business can say "you must have shoes". It can even say "you must have appropriate shoes that meet this necessary safety standard" like work boots on a job site. It can not say "you must have Louis Vuitton shoes".
The closest valid analogy to block an http client would be for failing to adhere to agreed standard open protocol specs, ie, not being a functional http client.
I think web services currently just get away with being in the wrong simply because the various legal systems simply haven't caught up with how various established legal principles apply to various on-line things.
Not every digital event has a direct analog in pre-internet reality, but most legal principles are priciples, and there is always some way to apply them, merely that way has not been hashed out yet for most things, because most of the people making the decisions just have very little deep understanding of what all goes on or how everything works, and so they have no way to judge what constitutes reasonable or unreasonable, or where to assign obligation and responsibility in most situations.
To date, it's still simply too easy for any lawyer to say almost anything halfway reasonable sounding, and all the relevant judges, jurors, and politicians just go with it, pretty much based on which lawyer or lobbyist told the better sounding story, or their own personal prejudices and interests.
Everyone has some way to make themselves look like the reasonable victim and someone else the reasonable culprit, but that doesn't mean those situations don't have a definitively correct answer, just that it hasn't been unpacked and hashed out yet by anyone in a position that matters, and to date, a lot of court cases are still arriving at essentially coin flip results, and most cases never even get past the point of a user being annoyed with a faceless web site and just living with it, let alone get decided either the right or the wrong way by some judge.
Now that CloudFlare has engaged with this problem, I'll give them some time to try to fix it, and if they don't, I'll start complaining to every website that uses this CloudFlare feature.
Website operators can override Cloudflare the same way.
https://developers.cloudflare.com/waf/tools/ip-access-rules/...
but not a single one has the false positive rate that cloudflare has
cloudflare only accepts the very standart users, and locks a lot of others out. and then they offer no convenient way to prove you're a legitimate user, to access the website.
and they have to fix it, because they sell their protection to admins who don't want to set it up themselves. They have the knowledge and are tasked to do that
If captchas are so important - serious point, perhaps different ones are the way to go?
I apologize in advance if this is more of a setting of difficulty from Cloudflare on Recaptcha, and Hcaptcha potentially being able to be set just as difficult/cost you as much time to get past/etc
That isn't just a reCaptcha thing. HCaptcha will definitely do that as well -- and if anything it's worse, because some of the "identify this AI-generated image" challenges are pretty awful. (At one point, I recall it asking me to "select the ladybugs" with nine images all containing round spotted bugs in slightly different shades of red and orange.)
I also understand Linux is an obscure use case but I do wonder how many other "normal" use cases out there have been ensnared. Given the lack of a "help" link on the verification page, an average user is powerless.
Hows the "chatting" going?
Right now I'm reviewing about:config for non-standard settings. I did find that I did set general.useragent.override at some point and I forgot about it; however, unsetting it didn't help. I went through all other non-default settings and haven't found anything yet.
If it was the latter, I'm sorry to CloudFlare as this was user error.
However, I do think the two meta points still stand:
1. Better diagnostics: perhaps a FAQ page that lists common issues such as an overridden general.useragent.override, etc. (obviously without giving anything away to bad people, but I'm sure certain things such as this can be pointed out)
2. Better responsiveness in the community forum particularly to this category of errors which blocks public internet activity.
Call me skeptical but does feels more of an "okay this feature we wanted is breaking stuff and people are noticing. Lets turn it off for now".
Something new will break in probably a month.
I wouldn't call Linux an obscure use case, it's particularly great for workstations and old laptops that struggle with running Windows.
Sucks if your website is unique.
Small browsers (like mine) are basically unusable now because of this. Theyre significantly squeezing everyone into chrome/safari. Ours is even chromium based, so super annoying.
https://coveryourtracks.eff.org
https://www.amiunique.org/fp
https://chrome.google.com/webstore/detail/canvas-blocker-fin...
https://addons.mozilla.org/en-US/firefox/addon/canvasblocker...
https://blog.cloudflare.com/eliminating-captchas-on-iphones-...