Tell Cloudflare: You're Breaking Tor/VPN/Shared IPs Again
I searched "list of sites behind cloudflare" and got hunter.io which exhibits this behavior.
The catpcha page now contains a mix of embarrassingly wrong statements and nonsense propaganda like:
- "Checking if the site connection is secure"
- "Did you know the first botnet in 2003 took over 500-1000 devices? Today, botnets take over millions of devices at once"
- "Did you know 43% of cyber attacks target small businesses?"
- "Did you know bots historically made up nearly 40% of all Internet traffic?"
You (just you) are making the web a dystopia. There was not rampant blocking before Cloudflare, and even during Cloudflare nobody else does this except a small amount of copycat companies, which is still your fault for spreading the bad idea.
You broke Tor between 2010-2018, but it was far worse: Everyone had to solve a captcha per session per website to visit any Cloudflared website at all. We had to solve two per site per session if they had a separate "cdn" subdomain. And you used Recaptcha, which gave unreadable captchas to Tor. Every single captcha had ambiguous text, which you have a 1/4 chance of guessing right (since they typically had at least 2 ambiguous characters). Recaptcha itself also just blocks some IPs (!!). When it wasn't blocking our IP it would still also often say "connection failed" in a popup for what was obviously no real reason while loading or submitting the captcha. Those are just the main issues with Recaptcha, which was and still is horse shit. This is still your fault though because even with a normal captcha instead of Recaptcha, we'd still have to solve 50 captchas just for say a 1 hour research session. All of the behavior explained here happened regardless of whether we used Tor Browser or any other browser.
And then you "fixed" this in 2018 by just skipping the captcha for anyone that emulated Tor Browser. This is terrible flawed protocol implementation and was documented as harmful and stupid in early RFCs, and I have no idea why HNers put up with it considering only shit companies like Microsoft would do this back in the early 2000s. There was absolutely no correlation of how easy it was to hack a site and whether they had a WAF. Nobody uses Cloudflare for the WAF or anti scraping, so please stop pretending there is any dilemma here, just remove it by default and let the idiots who like that stuff enable it if they want it. Then we'd be back to the 2003-2010 web where only a few unhinged web admins mass blocked IPs.
JUST STOP MASS BLOCKING. You literally have no reason to. The only reason would be if you have an agenda to stop people from anonymously accessing information.
55 comments
[ 3.1 ms ] story [ 111 ms ] threadThis is my take, who cares were people accessing the site is from. The only places that should kind-of care are Banking Sites.
Does everybody get those attacks? Probably not, however, Cloudflare centralizes the attacks into a single IP reputation database so, if at some point, a certain node was abused on x site that uses Cloudflare, anybody who is routed through that node will have a poor experience browsing CF sites.
This approach of centralizing IP reputations has its own flaws and benefits, Tor Nodes aren't inherently given a bad reputation, it just happens that if 90 people are using the tool for all the good things, 2 assholes can abuse the IPs and have them blacklisted on almost any website, whether it's Cloudflare, Imperva, Akamai, PX, you name it. Cloudflare is the most known name but there are tons of other E2E/B2B providers that don't show up as often.
For example if you are running a SaaS website which only caters to customers in the US what is the advantage of letting IPs from China or Russia access your service? Those IPs are not going to utilize your service because you don’t offer services outside of the US; since the IPs cannot be used for legitimate actions they can therefore only be used for illegitimate actions and should be blocked.
But even if you don't mind doing so... Our company didn't have any outside-the-US employees - until we did. We hired one remote in Canada, and we hired some contractors in the Philippines and Portugal. So you're creating a situation where either you have to fix your ability to let people connect from outside the US right now (with all the security issues that may cause, which you also have to fix right now), or else you lose a customer each time something like that happens.
I’m honestly a bit surprised to see so many people attempting to counter the idea that some businesses don’t need to be accessible from every region in the world.
just imagine for a moment people may want to send gifts for their loved ones.
That's literally being hostile to your customers where they may be under more pressure than normal to resolve whatever they're trying to achieve.
As someone who has dealt with massive influx of requests caused by bad actors who use everything in their arsenal to mask their behavior, yes they do. It may not be ideal by any means, but DoS attacks can be very, very expensive and being extra trigger happy on the ban is the difference between an operational site to most to an unavailable site for all.
I resorted to putting my website under a Cloudflare firewall. I benefit from it by getting web traffic stats and keeping away the bad requests. Many bots are set up with Tor and automate their browsing.
It would be impossible for me to pay fees in order to fend off DDOS attacks. My site is awalkaday.art. The amount of bad requests, per Cloudflare classification, goes up to 70-90% of regular web visitors. Imagine if you have to service 9 unidentifiable/shady customers, out of 10 people visiting your online store? It's not financially wise to allow anyone to deplect hardware resources (i.e: servers) of a new internet services. Moreover, these bad requests don't improve the growth or bring in revenue. The world wide web is too wild. Its unregulated in the sense that anyone from anywhere can start and successfully run an army of bots to scour the Internet for specific purposes.
I'm happy that Cloudflare can help mitigate some security issues before they occur.
Or do they just make them up?
If you mean that those requests were actually "good" (real humans) but CF mistakenly mark them as bad... then yes, this probably happens. For example, I have old tablet which regularly gets CF captcha, and sometimes this causes me to abandon some websites rather than solve it. I am sure that this appears in the logs as "bad" request -- after all, how can you tell frustrated human from defeated bot/DDOS?
But for many sites, the question is not "captcha vs same content no captcha" but rather "captcha vs significantly reduced functionality"... so at least for me, occasional captcha is worth not having to login / having full-resolution images etc...
Seemed fishy, but never questioned it.
> The problem is Tor exit nodes often have very bad reputations due to all the malicious requests they send, and you can do a lot of harm just with GETs. Content scraping, ad click fraud, and vulnerability scanning are all threats our customers ask us to protect them from and all only take GET requests.
https://blog.cloudflare.com/the-trouble-with-tor/
CF has become part of the web's security theater. Insecure WordPress install? Secure WP install that may become vulnerable sometime next week? No PreparedStatements in your framework's DB queries? Not to worry, just throw CF at it! Don't fix your bloated server frameworks and autogenerated DB schemas, just throw CF at it!
Who cares if modest hardware 2 decades ago could serve the same volume of requests? You're living in the future with the latest and greatest flavor of the week framework. Just throw CF at it!
This struck me as insane the first time I saw it. Not too surprising the biggest MITM op on the world engages into what is essentially psychological warfare.
aka marketing
As a result tor is de-facto banned, as well. But we don't target tor specifically, we just handle it as any other activity.
You are not entitled to access a website and the website doesn't know if you're a "good" or "bad" person automatically.
Put another way, if 9/10 of all phone calls/text messages sent to you are antagonistic at all hours of the day with NO filtering, would you accept that?
If so please post your unfiltered phone number that you will always answer, or your email address that has no spam filtering on it.
When most of the traffic on the internet is junk, some level of filtering is required.
How does a website determine if you're a "good" person, or a hacker who will destroy them otherwise?
Perhaps you want to take another go, only this time don't push a false narrative?
It was possible to reach sites without cloudflare before and view the content.
Now its primarily being used as a stopgap to block and de-anonymize people. The website doesn't need to determine if you are good or bad.
They simply need to manage their resources, and send the response to requests.
That is if they are actually in the business of providing something to someone like content or something else. Otherwise the business they are really talking about is data brokering and surveilling without really disclosing it, and that's another discussion completely.
Most of the traffic on the internet isn't junk, somewhere about 30% is protocol overhead, some small percentage is discarded during path and routing, and a large percent is the data people ask for.
There are tactics you can use as a website owner that target bad actors without blocking en-masse. Server side checks that characterize specific actors that are not related to IP or ASN. Those can then be easily targeted, and its not hard to set that kind of response up as an automated response.
I'm saying what Cloudflare should do as reasonable engineers. It's not a question of entitlement. IP blocking is never valid aside from temporary DDoS mitigation. Eventually, those IPs will get reassigned or the attacker will get bored, and you have to stop blocking at that point or else you will just block legitimate users, since IP addresses do not represent individual people.
Your analogy is invalid. The captcha we're talking about here is not for filtering messages. To further nip this argument in the bud: Most sites that have a Cloudflare captcha to view anything at all will still require more captchas, email confirmation, and often phone confirmation before being able to sign up and post. Then this last idea that IP blocking stops hackers is just not even plausible (queue flood of posts arguing about the diminishing returns of repurposing firewalls/av/block lists for mitigation of low quality automated exploit attempts).
I've had to deal with websites getting attacked, most of the traffic comes from datacentre IPs which Tor exit nodes are usually hosted on. We block most if not all datacentre IPs from reaching our sites, we've seen a huge decrease in attack attempts since doing this.
You're choosing to access the website using a shared IP that bad actors use, there's only so many ways webmasters can protect their websites. This one sounds like it's on you for using Tor.
Is people using iCloud Private Relay get blocked as well?
I spend real money browsing from these rotating datacenter IPs, to many different online stores, and I've only run into a rare few that won't let me. If a website doesn't work, especially for the casual browsing of shopping, then that will very much influence my purchasing decisions.
If your website relies on surveillance to make money, meaning you don't particularly appreciate my efforts to stop the abuse of my privacy, then I understand how we're at an impasse. But if you're running an honest business and think that heavy handed blocking is only pushing away illegitimate traffic, please do reconsider.
I do completely understand that you need to rate limit things that mutate the site or perform a lot of non-cacheable reads (and thus always hit your own server(s)). But if you have some sort of account system, I'd urge you to make it so that logged in users bypass IP-based limiting. Thus someone coming from a non-naive IP may hit a bunch of CAPTCHAs to login, but once they're logged in they should have an easy experience (with the idea that if the account turns out to be abusive, you ban it). An account is basically a longstanding nym.
Also, it's unkind to put privacy in scare quotes. It's true that there is very little absolute privacy with the sorry state of web technologies, but that doesn't mean throwing the whole concept out. I personally find value even in little things like some ad that slips through saying "List of <whatever> in <city not related to me>", rather than getting my city correct, normalizing the idea that it's right for what I'm reading to be reading back at me, and perhaps enticing me to click.
So I guess I should just let all the bots and wordpress hack attempts through to waste the very limited resources of my server and network connection?
Seems like a great idea.
/sarcasm
What ever happened to read-only websites? Using one's browser to render static content? An underrated concept in 2023, when almost every website wants to run some sort of code on your computer.
Your pain is felt, OP!
I find it kind of crazy that the crowd here is so willing to excuse that kind of environmental pollution, knowing how important a healthy open ecosystem is to all of our work.
That said the op is wrong about vpn blocks, users able to afford a vpn with nodes in a first world country have no problems with access.