Tell HN: Cloudflare is blocking Pale Moon and other non-mainstream browsers
Cloudflare's Browser Intergrity Check/Verification/Challenge feature used by many websites, is denying access to users of non-mainstream browsers like Pale Moon.
Users reports began on January 31:
https://forum.palemoon.org/viewtopic.php?f=3&t=32045
This situation occurs at least once a year, and there is no easy way to contact Cloudflare. Their "Submit feedback" tool yields no results. A Cloudflare Community topic was flagged as "spam" by members of that community and was promptly locked with no real solution, and no official response from Cloudflare:
https://community.cloudflare.com/t/access-denied-to-pale-moo...
Partial list of other browsers that are being denied access:
Falkon, SeaMonkey, IceCat, Basilisk.
Hacker News 2022 post about the same issue, which brought attention and had Cloudflare quickly patching the issue:
https://news.ycombinator.com/item?id=31317886
A Cloudflare product manager declared back then: "...we do not want to be in the business of saying one browser is more legitimate than another."
As of now, there is no official response from Cloudflare. Internet access is still denied by their tool.
553 comments
[ 3.2 ms ] story [ 515 ms ] threadWhat are you protecting cloudflare?
Also they show those captchas when going to robots.txt... unbelievable.
It is either that or keep sending data back to the Meta and Co. overlords despite me not being a Facebook, Instagram, Whatsapp user...
For example even Cloudflare hasn't configure their official blog's RSS feed properly. My feed reader (running in a DigitalOcean datacenter) hasn't been able to access it since 2021 (403 every time even though backed off to checking weekly). This is a cachable endpoint with public data intended for robots. If they can't configure their own product correctly for their official blog how can they expect other sites to?
If you are writing some kind of malicious crawler that doesn't care about rate-limiting, and wants to scan as many sites as possible for the most vulnerable to get a list together to hack, you will scan robots.txt because that is the file that tells robots NOT to index these pages. I never use a robots.txt for some kind of security through obscurity. I've only ever bothered with robots.txt to make SEO easier when you can control a virtual subdirectory of a site, to block things like repeated content with alternative layouts (to avoid duplicate content issues), or to get a section of a website to drop out of SERPs for discontinued sections of a site.
This is not relevant because Cloudflare will cache it so it never hits your origin. Unless they are adding random URL parameters (which you can teach Cloudflare to ignore but I don't think that should be a default configuration).
Again, I think you are correct with more sane defaults, but I don't know if you've ever dealt with a network admin or web administrator that hasn't dealt with server-side caching vs. browser caching, but it most definitely would end up with Cloudflare losing sales because people misunderstood how things work. Maybe I'm jaded, at 45, but I feel like most people don't even know to look at headers by default when they feel they hit a caching issue. I don't think it's based on age, I think it's based on being interested in the technology and wanting to learn all about it. Mostly developers that got into it for the love of technology, versus those that got into it because it was high paying and they understood Excel, or learned to build a simple website early in life, so everyone told them to get into software.
It's also a pretty safe assumption that Cloudflare is not run by morons, and they have access to more data than we do, by virtue of being the strip club bouncer for half the Internet.
Absolutely true. But the programmers of these bots are lazy and often don't. So if Cloudflare has access to other data that can positively identify bots, and there is a high correlation with a particular user agent, well then it's a good first-pass indication despite collateral damage from false positives.
They do not - not definitively [1]. This cat-and-mouse game is stochastic at higher levels, with bots doing their best to blend in with regular traffic, and the defense trying to pick up signals barely above the noise floor. There are diminishing returns to battling bots that are indistinguishable from regular users.
1. A few weeks ago, the HN frontpage had a browser-based project that claimed to be undetectable
For now
Half these imbeciles don't even change the user-agent from the scraper they downloaded off GitHub.
I employ lots of filtering so it's possible the data is skewed towards those that sneak through the sieve - but they've already been caught, so it's meaningless.
"Google is adding code to Chrome that will send tamper-proof information about your operating system and other software, and share it with websites. Google says this will reduce ad fraud. In practice, it reduces your control over your own computer, and is likely to mean that some websites will block access for everyone who's not using an "approved" operating system and browser."
https://www.eff.org/deeplinks/2023/08/your-computer-should-s...
If you really do have a better way to make all legitimate users of sites happy with bot protections then by all means there is a massive market for this. Unfortunately you're probably more like me, stuck between a rock and a hard place of being in a situation where we have no good solution and just annoyance with the way things are.
This approach clearly blocks bots so it's not enough to say "just don't ever do things which have false positives" and it's a bit silly to say "just don't ever do the things which have false positives, but for my specific false positives only - leave the other methods please!"
So it's OK for them to do shitty things without explaining themselves because they "have access to more data than we do"? Big companies can be mysterious and non-transparent because they're big?
What a take!
This hostility to normal browsing behavior makes me extremely reluctant to ever use Cloudflare on any projects.
Cloudflare just blocks you without recourse nowdays.
looks like someone is due for a class action
I'd presumed it was just the VM they're heuristically detecting but sounds like some are experiencing issues on Linux in general.
Looks like there’s a plugin for that https://chromewebstore.google.com/detail/user-agent-switcher...
Turnstile is the in-page captcha option, which you're right, does affect page load. But they force a defer on the loading of that JS as best they can.
Also, turnstile is a Proof of Work check, and is meant to slow down & verify would-be attack vectors. Turnstile should only be used on things like Login, email change, "place order", etc.
Also, Turnstile is definitely not a simple proof of work check, and performs browser fingerprinting and checks for web APIs. You can easily check this by changing your browser's user-agent at the header level and leave it as-is at the header level; this puts Turnstile into an infinite loop.
A cheeky response is "their profit margins", but I don't think that quite right considering that their earnings per share is $-0.28.
I've not looked into Cloudflare much, I've never needed their services, so I'm not totally sure on what all their revenue streams are. I have heard that small websites are not paying much if anything at all [1]. With that preface out of the way–I think that we see challenges on sites that perhaps don't need them as a form of advertising, to ensure that their name is ever-present. Maybe they don't need this form of advertising, or maybe they do.
[1] https://www.cloudflare.com/en-gb/plans/
I scrape hundreds of cloudflare protected sites every 15 minutes, without ever having any issues, using a simple headless browser and mobile connection, meanwhile real users get interstitial pages.
It's almost like Cloudflare is deliberately showing the challenge to real users just to show that they exist and are doing "something".
Somehow, Safari passes it the first time. WTF?
If I am met with the dreaded cloudflare "Verify you are a human" box, which is very rare for me, I dont bother and just close the tab.
Why does it need web workers, when it worked fined without them on Waterfox Classic firefox 56 fork that hasn't been updated in water?
I know it happens, but also I've run plenty of servers hooked directly to the internet (with standard *nix security precautions and hosting provider DDoS protection) and haven't had it actually be an issue.
So why run absolutely everything through Cloudflare?
If you are just a small startup or a blog, you'll probably never see an attack.
Even if you don't host anything offensive you can be targeted by competitors, blackmailed for money, or just randomly selected by a hacker to test the power of their botnet.
Also you can buy a cheaper ipv6 only VPS and run it thru free CF proxy to allow ipv4 traffic to your site
I do. Many people I know do. In my risk model, DDoS is something purely theoretical. Yes it can happen, but you have to seriously upset someone for it to maybe happen.
A while ago, my company was hiring and conducting interviews, and after one candidate was rejected, one of our sites got hit by a DDoS. I wasn't in the room when people were dealing with it, but in the post-incident review, they said "we're 99% sure we know exactly who this came from".
Take that story for instance. Here's how that goes in the physical world, just to show how unbelievably ridiculous it is.
So you didn't get the job? What's your next step?
I'll stop by their office and keep people from entering the front doors by running around in front of them. That'll show those bastards.
I've only been here 1.5 years but sounds like we usually see 1 decent sized DDoS a year plus a handful of other "DoS" usually AI crawler extensions or 3rd parties calling too aggressively
There are some extensions/products that create a "personal AI knowledge base" and they'll use the customers login credentials and scrape every link once an hour. Some links are really really resource intensive data or report requests that are very rare in real usage
Why was that not enough to mitigate the DDoS?
I work on a "pretty large" site (was on the alexa top 10k sites, back when that was a thing), and we see about 1500 requests per second. That's well over 10k concurrent users.
Adding 10k requests per second would almost certainly require a human to respond in some fashion.
Each IP making one request per second is low enough that if we banned IPs which exceeded it, we'd be blocking home users who opened a couple of tabs at once. However, since eg universities / hospitals / big corporations typically use a single egress IP for an entire facility, we actually need the thresholds to be more like 100 requests per second to avoid blocking real users.
10k IP addresses making 100 requests per second (1 million req/s) would overwhelm all but the highest-scale systems.
The attacker was using residential proxies and making about 8 requests before cycling to a new IP.
Challenges work much better since they use cookies or other metadata to establish a client is trusted then let requests pass. This stops bad clients at the first request but you need something more sophisticated than a webserver with basic rate limiting.
So how is Cloudflare supposed to distinguish legitimate new visitors from new attack IPs if you can't?
Because it matches my experience as a cloudflare user perfectly if the answer were "they can't"
They also do IP and request risk scores using massive piles of data they've collected
In the past you could ban IPs but that's not very useful anymore.
The distributed attacks tend to be AI companies that assume every site has infinite bandwidth and their crawlers tend to run out of different regions.
Even if you aren't dealing with attacks or outages, Cloudflare's caching features can save you a ton of money.
If you haven't used Cloudflare, most sites only need their free tier offering.
It's hard to say no to a free service that provides feature you need.
Source: I went over a decade hosting a site without a CDN before it became too difficult to deal with. Basically I spent 3 days straight banning ips at the hosting company level, tuning various rate limiting web server modules and even scaling the hardware to double the capacity. None of it could keep the site online 100% of the time. Within 30 mins of trying Cloudflare it was working perfectly.
Very true! Though you still see people who are surprised to learn that CF DDOS protection acts as a MITM proxy and can read your traffic plaintext. This is of course by design, to inspect the traffic. But admittedly, CF is not very clear about this in the Admin Panel or docs.
Places one might expect to learn this, but won't:
- https://developers.cloudflare.com/dns/manage-dns-records/ref...
- https://developers.cloudflare.com/fundamentals/concepts/how-...
- https://imgur.com/a/zGegZ00
That said, their Magic Transit and Spectrum offerings (paid) provide L3/L4 DDoS protection without payload inspection.
I incorrectly interpreted your comment as one of the multitude of comments claiming nefarious reasons for proxying without any thought for how an alternative would work.
Magic Transit is interesting - hard to imagine how it would scale down to a small site though, they apparently advertise whole prefixes over BGP, and most sites don't even have a dedicated IP, let alone a whole /24 to throw around.
You're right that Cloudflare has written many high-quality blog posts on the workings of the Internet, and the inner workings at Cloudflare. Amusingly, they even at times criticize HTTPS interception (not their use of it) and offer a tool to detect: https://blog.cloudflare.com/monsters-in-the-middleboxes/
I still believe that this information should be displayed to the relevant user configuring the service.
There are many types of proxies, and MITM decryption is not an inherent part of a proxy. The linked page from the Admin Panel is https://developers.cloudflare.com/dns/manage-dns-records/ref... and links to pages like "How Cloudflare works" (https://developers.cloudflare.com/fundamentals/concepts/how-...) which still do not mention HTTPS interception. It sounds like you found a link I didn't. In the past someone argued that I should've looked here: https://developers.cloudflare.com/data-localization/faq/#are...
But if you look closer, those are docs for the Data Localization Suite, an Enterprise-only paid addon.
It is sad that in this day and age, when you buy a car you need to sign a legal exclaimer that you understand it requires gasoline to run.
Again, there are many forms of proxies and DDOS protection that do not rely on TLS interception, just as there are cars that do not rely on gasoline. Cloudflare has many less technical home users who use their service to avoid sharing their IP online, avoid DDOS, or access home resources. I do not think the average Internet user is familiar with these concepts. There are many examples of surprised users on subreddits like /r/homelab.
Also, how would their certificates work if they don’t see content?
That's a weird question to ask to someone that went out of their way to describe a non-caching situation.
> Also, how would their certificates work if they don’t see content?
Can you be more specific? I'm not sure which feature you're asking about or how it uses certificates.
But the answer is likely "that feature isn't necessary to provide DDOS protection".
Just turning off some features gets them just about there. It wouldn't take rearchitecting things. Those features being bundled by default means very little for the difficulty.
Which itself shifted from complaining that you aren’t warned that coffee is hot, to - after implicitly agreeing that it should be obvious it’s hot - complaining that it they didn’t have to make it as hot.
Great! Offer an alternative! Everyone would be more than happy.
That is a much much easier to reach bar.
It's like if a restaurant sells cheeseburgers, and I want a hamburger. "How do they figure out ~~what~to~cache~~ the cheese to ketchup ratio without adding cheese?" They can just skip that part. I'm not asking for sushi and supporting that by saying "sushi is possible".
It would also be trivial for google and facebook to turn off all ads and logging of your activity. They would need to do strictly less than they do now. It would benefit all users too!
In CF case they would have to build a completely different infrastructure to detect bots using different technology to what they have now, including different ways around false positives for legitimate users. While perhaps nothing new in the sense that you claim “this is possible”, i see no one else offering this mythical “possible” product.
I would be the first in line to your offering of free cheeseless hamburgers. Where do i sign up?
My argument has never shifted.
But the reason the argument shifted was because someone specifically asked about how you'd do DDoS protection without those downsides.
And you continued asking how it could be done.
> It would also be trivial for google and facebook to turn off all ads and logging of your activity. They would need to do strictly less than they do now. It would benefit all users too!
Isn't cloudflare supposedly not tracking private information in the websites they proxy...? If you think they make money off it, that's pretty bad...
> In CF case they would have to build a completely different infrastructure to detect bots using different technology to what they have now, including different ways around false positives for legitimate users.
I disagree.
> I would be the first in line to your offering of free cheeseless hamburgers. Where do i sign up?
First you need to put me into a situation where my business can compete with cloudflare while doing exactly the same things they do. Then I will be happy to comply with that request.
The hard part of this situation is not the effect of that tiny change on profitability, it's getting into a position where I can make that change.
They are at the very least tracking the users and using that tracking as part of the heuristics they use in their product.
Whether they sell the data for marketing, i don’t know, hopefully not but conceivably, yes.
To which, > I disagree.
Yes, we’ve established that you disagree and explicitly claim “it’s possible to offer ddos protection without mitm”
and now further that “dropping the extra feature of caching” would not adversely affect their technology or their business”
Great, claims though entirely unsupported and in the latter case obviously false if you know anything about how it works.
In particular, they would need to sponsor the free accounts via much poorer economies of scale due to not being able to cache anything, and would not help at all with a “legitimate ddos” such as being on the front page here
They can do that without seeing the proxied contents. So your analogy to asking facebook or google to stop ads and tracking is completely broken.
> and now further that “dropping the extra feature of caching” would not adversely affect their technology or their business”
Yes. (Well, it was stated much earlier but I guess you didn't notice until now?) You're the one saying it would be a problem, do you have anything to back that up?
> in the latter case obviously false if you know anything about how it works.
Caching costs a bunch of resources and still uses lots of bandwidth, what's so obvious about it? And cloudflare users can already cache-bust at will, so it's not exactly something they're worried about.
https://developers.cloudflare.com/cache/how-to/cache-rules/s...
> would not help at all with a “legitimate ddos” such as being on the front page here
Which is not the scenario people were worrying about.
And an average web server can handle that.
there are many others. just buy a book for industries that value privacy or pay someone.
Then self host from your connection at home, don't pay for the VPS :). That's what I've been doing for over a decade now and still never saw a (D)DoS attack
50 mbps has been enough to host various websites, including one site that allows several gigabytes of file upload unauthenticated for most of the time that I self host. Must say that 100 mbps is nicer though, even if not strictly necessary. Well, more is always nicer but returns really diminish after 100 (in 2025, for my use case). Probably it's different if you host videos, a Tor relay, etc. I'm just talking normal websites
Bandwidth hasn't been a limiting factor for years for me.
But generating dynamic pages can bring just enough load for it to get painful. Just this week I had to blacklist Meta's ridiculously overactive bot sending me more requests per second than all my real users do in an hour. Meta and ClaudeBot have been causing intermittent overloads for weeks now.
They now get 403s because I'm done trying to slow them down.
The only time I had a problem was when gitea started caching git bundles of my Linux kernel mirror, which bots kept downloading (things like a full targz of every commit since 2005). Server promptly went out of disk space. I fixed gitea settings to not cache those. That was it.
Not ever ddos. Or I (and uptimerobot) did not notice it. :)
If it was actually a traffic based DDOS someone still needs to pay for that bandwidth which would be too expansive for most companies anyway - even if it kept your site running.
But you can sell a lot of services to incompetent people.
Cloudflare offers protection for free.
My experience form 15 years working in the hosting industry is that volumetric attacks are extremely rare but customers that turn to Cloudflare as a solution are more often than not DDOS-ing them self because of bad configured systems, but their junior developers lack any networking troubleshooting skills.
Many of the AI scrapers don't identify themselves, they live on AWS, Azure, Alibaba Cloud, and Tencent Cloud, so you can't really block them and rate limiting also have limited effect as they just jump to new IPs. As a site owner, you can't really contact AWS and ask them to terminate their customers service in order for you to recover.
The availability part on the other hand is maybe something that's not so business critical for many but for targeted long-term attacks it probably is.
So I think for some websites, especially smaller ones it's totally feasible to not use Cloudflare but involves planning the hosting really carefully.
Only issues I had to deal with are when someone finds some slow endpoint, and manages to overload the server with it, and my go to approach is to optimize it to max <10-20ms response time, while blocking the source of traffic if it keeps being too annoying after optimization.
And this happened like 2-3 times over 20 years of hosting the eshop.
Much better than exposing users to CF or likes of it.
Every other attempt at DDoS has been ineffective, has been form abuse and credential stuffing, has been generally amateurish enough to not take anything down.
I host (web, email, shells) lots of people including kids (young adults) who're learning about the Internet, about security, et cetera, who do dumb things like talk shit on irc. You'd think I'd've had more DDoS attacks than that rather famous one.
So when people assert with confidence that the Internet would fall over if companies like Cloudflare weren't there to "protect" them, I have to wonder how Cloudflare marketed so well that these people believe this BS with no experience. Sure, it could be something else, like someone running Wordpress with a default admin URL left open who makes a huge deal about how they're getting "hacked", but that wouldn't explain all the Cloudflare apologists.
Cloudflare wants to be a monopoly. They've shown they have no care in the world for marginalized people, whether they're people who don't live in a western country or people who simply prefer to not run mainstream OSes and browsers. They protect scammers because they make money from scammers. So why would people want to use them? That's a very good question.
> Cloudflare wants to be a monopoly. They've shown they have no care in the world for marginalized people, whether they're people who don't live in a western country or people who simply prefer to not run mainstream OSes and browsers.
"Marginalized" has a specific connotation, sure, but people can be marginalized for reasons other than, or in addition to, those that fit the connotation.
I have relatively fast internet, so maybe it's fast enough to absorb a lot of the problems, but I've had good enough luck with some basic Nginx settings and fail2ban.
[1] a small little mini gaming PC running NixOS.
Google itself tried to push crap like Web Environment Integrity (WEI) so websites could verify "authentic" browsers. We got them to stop it (for now) but there was already code in the Chromium sources. What makes CloudFlare MITMing and blocking/punishing genuine users from visiting websites?
Why are we trusting CloudFlare to be a "good citizen" and not block unfairly/annoy certain people for whatever reason? Or even worse, serve modified content instead of what the actual origin is serving? I mean in the cases where CloudFlare re-encrypts the data, instead of only being a DNS provider. How can we trust that not third party has infiltrated their systems and compromised them? Except "just trust me bro", of course
(And if I were doing this on my own, rather than trusting Cloudflare to do it, I would almost surely decide that I don't care enough about Pale Moon users to fix an otherwise good rule that's blocking them as a side effect.)
I witnessed this! Last time I checked, in the default config, the connection between cloudflare and the origin server does not do strict TLS cert validation. Which for an active-MITM attacker is as good as no TLS cert validation at all.
A few years ago an Indian ISP decided that https://overthewire.org should be banned for hosting "hacking" content (iirc). For many Indian users, the page showed a "content blocked" page. But the error page had a padlock icon in the URL bar and a valid TLS cert - said ISP was injecting it between Cloudflare and the origin server using a self-signed cert, and Cloudflare was re-encrypting it with a legit cert. In this case it was very conspicuous, but if the tampering was less obvious there'd be no way for an end-user to detect the MITM.
I don't have any evidence on-hand, but iirc there were people reporting this issue on Twitter - somewhere between 2019 and 2021, maybe.
If there were an alternative that would provide the same benefits at roughly the same cost, I would definitely be willing to take a look, even if it meant I needed to spend some time learning a different way to configure the service from the way I configure Cloudflare.
It's like talking about getting murdered - it happens, and there are statistics, but if you're literally expecting everyone to change their whole lives based on the fact that some people are murdered, with zero consideration for the where, why and how, you're doing it wrong.
For a random site from the internet, sure, because a random blog is probably too small to be noticed.
Forums, even relatively niche ones, unfortunately do suffer DDoS from their disgruntled users. (Or competitors of the same fandom. Or from the disgruntled part of a rivaling fandom.)
> It's like talking about getting murdered - it happens, and there are statistics, but if you're literally expecting everyone to change their whole lives based on the fact that some people are murdered, with zero consideration for the where, why and how, you're doing it wrong.
All analogies fail somewhere, but this is probably one of those which easily falls apart. Injuries are probably better. In a random population, there are a relatively small proportion of injuries, but some jobs (like construction) tend to have a significantly higher number of injuries compared to a mean person, in the same manner that a DDoS on a random website is unlikely but certain types of websites are DDoS magnets.
I would like to know if there are alternatives somewhere close to the same cost, where I don't need to use Cloudflare. I don't enjoy annoying customers, or even dealing with sales and marketing, but I have built lots of software where I get to control the technology, and can get a new website up and running in 3 hours, with a ton of built-in functionality. I've spent about 12 years reducing the amount of memory the Umbraco CMS uses, compared to normal installs, and I love that aspect of my career. If I could get my clients to pay more and not use Cloudflare, I would happily go that route, believe me!
I do get a "your browser is unsupported" message from the forums.
I noticed another platform (wallapop, a kind of ebay/craigslist here in Spain) that does the same. It never works well in a browser, even in chrome. I think they're just trying to bully their users to their app, which has 30+ trackers in it.
I have not tried less mainstream browsers, just FF and Chrome.
CAPTCHAs are barely sufficient against bots these days. I expect the first sites to start implementing Apple/Cloudflare's remote attestation as a CAPTCHA replacement any day now, and after that it's going to get harder and harder to use the web without Official(tm) Software(tm).
Using Linux isn't what's getting you blocked. I use Linux, and I'm not getting blocked. These blocks are the results of a whole range of data points, including things like IP addresses.
What usually works for me is to close the browser, reload, and try again.
I agree that this exposes the risk of relying overmuch on handful of large, opaque, unaccountable companies. And as long as Cloudflare's customers are web operators (rather than users), there isn't a lot of incentive for them to be concerned about the user if their customers aren't.
One idea might be to approach web site operators who use Cloudflare and whose sites trigger these captchas more than you'd like. Explain the situation to the web site operator. If the web site operator cares enough about you, they might complain to Cloudflare. And if not, well, you have your answer.
But if someone has a site that is failing, feel free to post it and I will give it a try.
It's probably dependent on the security settings the site owner has choosen. I'm guessing bot fight mode might cause the issue.
This is all as it was intended.
Kagi exists and has been production quality and better than Google for over two years already. (At this point I think it is even better than old google.)
On the other, Pale Moon is an ancient (pre-quantum) volunteer-supported fork of Firefox, with boatloads of known and unfixed security bugs - some fixes might be getting merged from upstream, but for real, the codebases diverged almost a decade ago. You might as well be using IE 11.
Cloudflare not supporting Pale Moon has no impact on the rest of us. Matter of fact today is the first time I'm hearing of this browser I will never end up using.
Which are..?
"'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/132.0.0.0 Safari/537.36'"
That means my browser is pretending to be Firefox AND Safari on an Intel chip.
I don't know what features Cloudflare uses to determine what browser you're on, or if perhaps it's sophisticated enough to get past the user agent spoofing, but it's all rather funny and reminiscent just the same.
As a part of some browser fingerprinting I have access to at work, there's both commercial and free solutions to determine the actual browser being used.
It's quite easy even if you're just going off of the browser-exposed properties. You just check the values against a prepopulated table. You can see some of such values here: https://amiunique.org/fingerprint
Edit: To follow up, one of the leading fingerprinting libraries just ignores useragent and uses functionality testing as well: https://github.com/fingerprintjs/fingerprintjs/blob/master/s...
I forgot the script open, polling for about 20 minutes, and suddenly it started working.
So even sending all the same headers as Firefox, but with cURL, CF seemed to detect automated access, and then eventually allowed it through anyway after it saw I was only polling once a minute. I found this rather impressive. Are they using subtle timings? Does cURL have an easy-to-spot fingerprint outside of its headers?
Reminded me of this attack, where they can detect when a script is running under "curl | sh" and serve alternate code versus when it is read in the browser: https://news.ycombinator.com/item?id=17636032
If it's a https URL: Yes, the TLS handshake. There are curl builds[1] which try (and succeed) to imitate the TLS handshake (and settings for HTTP/2) of a normal browser, though.
[1] https://github.com/lwthiker/curl-impersonate
They're kinda covered because IE also sent Mozilla/5.0 (or 4.0, 2.0, [..]).
The internet is so much better like this! There is a 2010 lightweight mobile version of Google, and m.youtube with obviously cleaner and better UI and not a single ad (apparently it's not worth to show you ads if you still appear to be using iphone 6)
Why not adwall the user instead, showing only ads until they upgrade the device or buy premium?
Yes, it is, both your TLS and TCP stacks are unique enough that such spoofing can be detected. But there are a lot of other things that can be fingerprinted as well.
That's not the case, that ua is Chrome on MacOS. The rest is backward compatibility garbage
If I understand correctly, this is why I've said on previous Cloudflare threads that they've managed to design a game they can never win. They project a certain omniscience, but then all this sh*t happens. We need to persuade them to stop playing.
Think about it this way: when a framework (many modern websites) or CAPTCHA/Challenge doesn't support an older or less common browser, it's not because someone's sitting there trying to keep people out. It's more likely they are trying to balance the maintenance costs and the hassle involved in allowing or working with whatever other many platforms there are (browsers in this case). At what point is a browser relevant? 1 user? 2 users? 100? Can you blame a company that accommodates for probably >99% of the traffic they usually see? I don't think so, but that's just me.
At the end, site owners can always look at their specific situation and decide how they want to handle it - stick with the default security settings or open things up through firewall rules. It's really up to them to figure out what works best for their users.
"Challenges are not supported by Microsoft Internet Explorer."
Nowhere is it mentioned that internet access will be denied to visitors not using "major" browsers, as defined by Cloudflare presumably. That wouldn't sound too legal, honestly.
Below that: "Visitors must enable JavaScript and cookies on their browser to be able to pass any type of challenge."
These conditions are met.
I'm unsure what part of this isn't clear, major browsers, as long as they are up to date, are supported and should always pass challenges. Palemoon isn't a major browser, neither are the other browsers mentioned on the thread.
> * Nowhere is it mentioned that internet access will be denied to visitors not using "major" browsers *
Challenge pages is what your browser is struggling to pass, you aren't seeing a block page or a straight up denying of the connection, instead, the challenge isn't passing because whatever update CF has done, has clearly broken the compatibility with Palemoon, I seriously doubt this was on purpose. Regarding those annoying challenge pages, these aren't meant to be used 24/7 as they are genuinely annoying, if you are seeing challenge pages more often than you are on chrome, its likely that the site owner is actively is flagging your session to be challenged, they can undo this by adjusting their firewall rules.
If a site owner decides to enable challenge pages for every visitor, you should shift the blame on the site owners lack of interest in properly tunning their firewall.
There are actually hundreds of smaller chromium forks that add small features, such as built-in adblock and have no issues with neither Cloudflare nor other captchas.
Because in the end, the result is connection denial. I don't want to connect to Cloudflare, I want to connect to the website.
I read that part. They still do not indicate what may happen, or what is their responsibility -if any- for visitors with non-major browsers.
Not claiming this is "on purpose" or a conspiracy, but if these legitimate protests keep getting ignored then yes, it becomes discrimination. If they can't be bothered, they should clearly state that their tool is only compatible with X browsers. Who is to blame for "an incorrectly received challenge"? The website? The user who chooses a secure, but "wrong" browser not on their whitelist?
Cloudflare is there for security, not "major browser approval pass". They have the resources to increase response times, provide better support and deal with these incompatibility issues. But do they want to? Until now, they did.
The problem with this setup, is that it sacrifices on both security (because it needs to keep false positives at a minimum, even if that means allowing some known bots) and user experience (because situations like the one you have will occur from time to time). When you enable a challenge page on CF, it will work as-is and you have no ruling over it, the most you can do is skip the page for the browsers having false positives.
If CF gave site owners a clearer view of what they are blocking and let them choose which rules to enforce (within the challenge page), it would be much easier to simply say that the customer running CF doesn't want you visiting their page/doesn't care about few false positives.
I use uptodate Firefox, and was blocked from using company gitlab for months on end simply because I disabled some useless new web API in about:config way before CF started silently requiring it without any feature testing or meningful error message for the user. Just a redirect loop. Gitlab support forum was completely useless for this, just blaming the user.
So we dropped gitlab at the company and went with basic git over https hosting + cgit, rather than pay some company that will happily block us via some user hostile intermediary without any resolution. I figured out what was "wrong" (lack of feature testing for web API features CF uses, and lack of meaningful error message feedback to the user) after the move.
Some things that I had found helpful when working with Gitlab is to add ".patch" on the end of commit URLs, and changing "blob" to "raw" in file URLs. (This works on GitHub as well.) It is also possible to use API, and sometimes the data can be found within the HTML the server sends to you without needing any additional requests (this seems to work on GitHub more reliably than on Gitlab though).
You could also clone the repository into your own computer in order to see the files (and then use the git command line to send any changes you make to the server), but that does not include issue tracker etc, and you might not want all of the files anyways, if the repository has a lot of files.
It sometimes blocks me on fairly major browsers, such as google chrome ( but on an older Ubuntu ).
I love how so many of these apologists talk about stuff like "maintenance costs", as though it's impossible to write code that's clean and works consistently across platforms / browsers. "Oh, no! Who'll think of the profits?!?"
If you had any technical knowledge, you'd know that "maintenance costs" are only a thing when you code shittily or intentionally target specific cases. A well written, cross-browser, cross-platform CAPTCHA shouldn't have so many browser specific edge cases that it needs constant "maintenance".
In other words, imagine you're arguing that a web page with a picture doesn't load on a browser because nobody bothered to test with that browser. Now imagine you're making the case for that browser being so obscure that nobody would expend the time and money. Instead, why aren't you pondering why any web site with a picture wouldn't be general enough to just work? What does that say about your agenda, and about the fact that you want to make excuses for this huge, striving-to-be-a-monopoly, for-profit company?
This is the internet and everybody is a field expert the moment they want to win an argument, best of luck with that.
The Cloudflare tool does not complete its verifications, resulting in an endless "Verifying..." loop and thus none of the websites in question can be accessed. All you get to see is Cloudflare.
Is it worth giving the internet to them? Is something so fundamentally wrong with the architecture of the internet that we need megacorps to patch the holes?