If you're such a high volume site then why not pay for CDN/DDOS Protection? I'm amazed so many people feel this is something they deserve for free, and then complain when there's some kind of limitation on the free thing they're receiving.
If someone gave me a free car and I suddenly discovered that it had stopped working at 8:30am on Monday morning because the free car plan didn't support peak hour, just when I needed it most, I'd be annoyed as hell. This would be the case no matter how much I got paid at my job. Getting a nasty surprise because you didn't realise a product was inferior is upsetting for reasons that should be pretty obvious.
Admittedly the cloudflare free plan does say it's for things "that aren’t business-critical", but what does that actually mean in terms of resource quota and expected uptime?
I don't care as much for DDOS protection as for free easy SSL which gets semi annoying to set up for me personally. I assumed cloudflare was an option from their advertisment and never worried about it. Now I suppose I will.
I know from experience that they might take some actions if you take too much of a DDoS on their free plan but I never heard of that on some usual traffic, especially not when it's as low as the blog suggests (sub 10 rpm).
The attacks my blog received were in the thousand requests per second area when it got suspended.
I've been using Cloudflare for a site that has around 400k requests per day on a Free plan (although I did buy a dedicated certificate and a domain from them) and Cloudflare has been very reliable to me except few hiccups in the past. I've had an instance of Cloudflare failing only on a single region, but after talking to support it turns out to be some route caching issue on their side, which subsequently fixed. I would not assume anything until there's some clarification on Cloudflare side. It might just be a bug.
I am talking internally with the customer support team about what happened to this customer's traffic. It is definitively not the case that doing "four requests per minute" gets you rate limited.
A great first step would be to take responsibility for what the customer has proven to happen. "It is definitively not the case that doing" is directly at odds with what the author provides, you might want to reframe your message to "it shouldn't be like this but unfortunately it is like this right now, sorry" rather than dismissing the author with that they are not correct.
you can (and should) put monitors on the outside of your infrastructure as well as inside. I've multiples hitting the login page and robots file where I work, and never got an unaccounted 503
Like I wrote, for some integrations it is not possible to gather all the logs. Also how will you know that a client accessing your website in a browser gets 503 instead of your web page?
You don't, but your monitors will show the 503 happening, how often, on which endpoint operation and in which regions; that will give you a pretty good picture of whether is actually your CDN layer or something else triggering the 503
While we are on the subject of rate limiting and such, does Cloudflare allow using its service for serving video files for a media heavy social media app where we store the videos on Backblaze B2?
> The Services are offered primarily as a platform to cache and serve web pages and websites. Unless explicitly included as part of a Paid Service purchased by you, you agree to use the Services solely for the purpose of (i) serving web pages as viewed through a web browser or other functionally equivalent applications, including rendering Hypertext Markup Language (HTML) or other functional equivalents, and (ii) serving web APIs subject to the restrictions set forth in this Section 2.8. Use of the Services for serving video or a disproportionate percentage of pictures, audio files, or other non-HTML content is prohibited, unless purchased separately as part of a Paid Service or expressly allowed under our Supplemental Terms for a specific Service. If we determine you have breached this Section 2.8, we may immediately suspend or restrict your use of the Services, or limit End User access to certain of your resources through the Services.
And I am wondering what kind of subscription would be appropriate for my use-case if any, that would allow me to host the files on Backblaze B2, and serve these files through Cloudflare.
(Cloudflare would also be hosting our API, so it’s not just video files that we want to serve via Cloudflare.)
For what is is worth, I've been using their Free Tier to serve over 10k requests per hour with ~60% html / ~40% media (mainly pictures, very few videos) and I've never had a problem.
However, for a CDN for video content YMMV. Perhaps you should try contacting them directly to have a written estimate.
I basically have something similar to your use-case. While I'm on the free tier for all my domains, the supplemental agreement, at least in the US version, removes the specific limitation to serve primarily html/text content if you enable Cloudflare Workers, which also has a free tier, applies on an account and not domain level, and has no requirement for you to serve your content through their KV system. I ended up paying for my CF Workers subscription for a different project on my account but the system worked fine prior to that. I served 111GB of a combination of m4a and png files in data from my B2 bucket via CF in the past month.
Is it possible that this person's free-plan traffic was routed through a POP that was under attack? Then, upon upgrading to Pro the routing rules for the account were modified to avoid under-attack data centers.
I think it might also be worth considering whether returning 503 is the right response when rate limited. 5xx indicates server failure, a "429 Rate Limited" is a more appropriate response.
4xx is a client error and 429 indicates that particular client is making too many requests. 5xx is a server error and 503 says the service itself is unavailable irrespective of the client’s actions. In this particular case it’d be unavailable for contractual rather than technical reasons, but it’s still the right status
Please keep us posted. From a naive perspective, what happened has a flavor of "protection fees." A detailed explanation would help quickly dispel this perspective.
Why don't you send an email to the customer if there is an error while serving content by cloudflare? It can be a daily digest of the errors of the day.
If the problem is on cloudflare's side then the cloudflare user doesn't even know about it if it's an intermittent error.
My application[1] with similar rpm as OP received 503 status last month for about an hour first time in 1.5 years, I assumed it to be an one-off issue at CF end. It happened while I was using it and so I'm now concerned if it's a regular affair for my users, most of whom are not from my geographical location(data-center).
I'm a free plan user because my application is not monetized and obviously would switch to paid plan once it starts generating money. CF free plan has been indispensable for many such free projects.
UptimeRobot(Free) doesn't show anything fishy, except for CF error 521 occasionally. Again UptimeRobot requires paid version to check for HTTP status message, guess I'll have to do it or setup a self hosted(recommendations?) solution to monitor HTTP status of CF Free application.
If CF does perform rate limiting for such low rpm, I would prefer to be intimated as even third rate shared hosting providers do it and I hope CF wouldn't put their reputation on the line for something like this.
I understand you can't share a customer's account details.
But can we at least get an update if this is the intended behaviour of the CF free plan accounts (so people know to upgrade) or is it an one-off incident?
I’d look into idle connection handling, I’ve had problems in the past with cloudflare and it’s long open sockets to origin being quietly closed at origin, breaking requests to cloudflare.
That does seem strange, I never experienced this myself. Also, I don't understand the use case for Cloudflare here, can you elaborate on "I’ve been using Cloudflare free plan for Abot"
This article seems fairly scant on details to be honest. I understand how they came to the conclusion, but there's still quite a lot of conjecture here.
I feel like the main problem is the inability to properly debug your problem without paying while the plan essentially necessitates a degree of self-diagnostic when things go wrong. I also serve an xml feed via CF and also use it as a caching service in front of my Backblaze B2 bucket, and on the free tier by virtue of the files I'm serving far fewer unique users (less than half of the author's last 30 days) but a far greater amount of data (111GB). I'm not certain if the author should jump from what's clearly a vague and uncommitted conjecture from the Cloudflare tech about where the 503 error comes from straight to "they are definitely throttling me", since I've had the opposite experience and have seen 503s in all sorts of situations unrelated to throttling, but with a tiny sample size and no adequate diagnostic tooling it's also impossible for me or anyone to say that he's not being throttled, except CF themselves. I realize that it's their business model but it seems counter-intuitive that the service they sell seems partly "better ability for you to fix your problems yourself", sort of the opposite of say, managed/unmanaged servers. It'd make more sense, from a customer's point of view, that if I'm paying for a service, there should be service in both senses of the word - the product and the servicing of issues if possible, but not making diagnostics more transparent makes little sense because it prevents people who want to use the service from continuing to use the service, presumably part of CF's goal.
what a load of horseshit, this is my free plan 30 day stats, 2 order magnitude more request, a order magnitude less cache hits, no request limit in sight
edit: changed bull crap to the more appropriate horseshit in response to this unsubstantiated-bordering-conspiracy blog post, because if the former is already enough to triggers the community downvoting brigades, no point in showing any restraint
Very few, we're a regional level startup, the CF map doesn't show it well because it's per state and not per pop on the free account, but basically all hits in logs are marked MXP or FCO
I'm starting to lose faith in cloudflare lately - we're on the business plan and there's been some really strange UI bugs in the dashboard where it looks like state just isn't reflected properly (Access and Load Balancing being ones we see quite a bit)
Summary of things that we've had issues with lately:
- Caching of a specific route just stopped working.
I log a support ticket (nightmare to find in the dashboard) and miraculously it starts working again after the ticket gets responsded to.
- Some of our staff got locked out of part of our app because we exceeded the 5 free users of the "Access" plan. Upgraded to the 50 user plan in the new Teams part and it still didn't work. Contacted support, fixed again but no explanation. (multiple day turnaround on tickets)
We're invested in their tech a lot and I love workers - they really take some big tasks off us. If Azure come up with something compelling we'll probably switch though (assuming I can make sense of Azure's billing and product naming strategy)
> There's no scarier company online than Cloudflare.
Really? I don't think you're aware of what's actually out there.
Let's put it this way: Cloudflare has a straightforward business model. The fact that you can pay them is comforting. The big shady companies that you cannot even pay, yet still make money "somehow", are more scary.
Today there was a topic on HN about customers who complain about limiting free tiers. That’s a great example of it, especially from someone who provides paid SaaS product.
Free tier is awesome for small websites, keeping domains and many other things. But if you run business who earns, then just pay for that god damn thing, and stop ranting. :-)
Someone came on HN to report that a free-tier product is failing in a way that’s undetectable and doesn’t make sense as described by the products’ marketing materials, and the resolution is to pay (a small amount) for the upgraded version. One of the Cloudflare engineers* immediately responds to the article saying he’s looking into things because this shouldn’t be happening. Everything seemed to be going so well in this story. I only continued reading the comments to find the first one where someone turns this into a story of blame for the poster being cheap, and your post was it.
It’s not the case with everyone, OP could wait until supports get back to him and investigates everything, and then push a pice on that. Bug -> Contact Support -> Wait for answer & solutions -> Happy or not ending.
What if he accuses them without justification? Which is probably a case here.
From where I sit he is entirely justified, and the only reason problem is getting fixed is precisely because he finally decided to warn others about this problem.
They did get back to him. What evidence do you have that they were still investigating anything? They answered his questions and the answers were no good.
How do you know for sure that you didn't? One of the main takeaways from the original article is that these errors are invisible from server side (your own stack) because it's the traffic is being throttled before it gets to your own servers.
Unless you have very good and consistent client side availability metrics you probably wouldn't notice the 2%-3% drop.
I have nothing but good things to say about Cloudflare's free services. However my usage is limited to small static sites, which are free of surprises.
Could this be explained by something like “there was some bug causing this person’s config to be in a weird state (on the cloudflare side), and changing the payment tier merely gave the system a kick that got it out of the bad state? After all, the person payed $20pm not because they thought it would give a better reliability but because they wanted better support for a weird issue. I’m not really very appalled by the idea that free users don’t get great support.
2. Data collection (after all if you are not paying you are the product) There is monetary value in collection, observation, and tracking of a vast amount of internet traffic.
3. Education. If you get people just starting out in their careers using your product or service for they hobby, personal project, etc then when they have a business need for some like your product or service they will just naturally choose you. Adobe and MS has been doing this for decades with low or no cost education licensing
Sounds weird. I'm not saying to trust Cloudflare blindly, but I used a social media site (very "reddit-like") my friend made and kept for over a year on free plan with no issues. Around 20-80 people online were actively creating content, private messages, browsing, upvoting, with live notifications with websockets and so on, it was a quite busy little site for a year or so). There never was issues with couldflare and it basically allowed the site to run on two smallest VPS machines mentioned friend could find.
Also why is author giving trust in a free service for his paid service? CF is not that expensive.
I was doing a couple of POSTs a minute on a paid plan, and I saw a random 2-3% of request to be failing they never hit my backend, speaking to support wasn't helpful at all, that's when we moved away from Cloudflare, I wouldn't base my business on it.
You can blame Cloudflare for not meeting their promises on free tier. But you can't blame them for "hiding" the errors. You are responsible to monitor your service, and it's good practice to have that happen externally from your environment. If you have a public API, monitor it publicly. Things like Pingdom and synthetic monitoring exist for this reason.
I've always seen free plans as a way to get started. A great way for a startup to get things up before they get money rolling in. Then as you grow, you outgrow the free plan but by then you have the money to upgrade.
This is highly unlikely to be as simple as the author conjects, we're talking about a service that processes an enormous amount of traffic and if what the author suggests is true, would someone else not have noticed by now? It's certainly possible that different infrastructure is used by free/paid plans, and perhaps this specific site was hosted on an unhealthy instance. But we don't have any external data points here to analyse - only those reported by Cloudflare, along with anecdotal reports.
Edit: I see the title has now been changed, at least we're reducing clickbait.
I thought about responding to some comments here but I feel like this point deserves a comment on its own: not to defend Cloudflare, but charging at least $19 / month, having 100 customers (as he said on the post) and not using a proper fully featured WAF to protect an enterprise product is... amateurish?
Also, if the infrastructure is in AWS, CloudFront would cost cents if he really does have "4 requests per minute" with full integration and logs.
I have the feeling that with the advent of cloud solutions, very big companies depend on these really small and very useful tools that disregard almost entirely a number of best practices and standards. The Solarwinds incident is going to happen a lot more, that's for sure.
Regarding Cloudflare: I'm fine with the free product not being great, but hiding the logs is not understandable. I bet a lot more people would upgrade if they knew.
Cloudflare doesn't offer a WAF in the free plan, only a DDoS attack mitigation service. This means it doesn't go into the application level (i.e. no SQL Injection protection or XSS). They do, however, offer a filtering capability.
You can see more in the Cloudflare page [1], they explain well what it is. None of the features they talk about come in the free version.
127 comments
[ 2.9 ms ] story [ 188 ms ] threadAdmittedly the cloudflare free plan does say it's for things "that aren’t business-critical", but what does that actually mean in terms of resource quota and expected uptime?
Isn't that quite reasonable? Especially if you're scared if your site goes down.
[0] https://www.cloudflare.com/en-gb/plans/
you can (and should) put monitors on the outside of your infrastructure as well as inside. I've multiples hitting the login page and robots file where I work, and never got an unaccounted 503
one has to start somewhere reasonable and realistic first.
I’m working on such an app, and am hoping to use Cloudflare + Backblaze B2, because of the bandwidth alliance. https://www.cloudflare.com/en-gb/bandwidth-alliance/
However, the Terms of Service say:
> 2.8 Limitation on Serving Non-HTML Content
> The Services are offered primarily as a platform to cache and serve web pages and websites. Unless explicitly included as part of a Paid Service purchased by you, you agree to use the Services solely for the purpose of (i) serving web pages as viewed through a web browser or other functionally equivalent applications, including rendering Hypertext Markup Language (HTML) or other functional equivalents, and (ii) serving web APIs subject to the restrictions set forth in this Section 2.8. Use of the Services for serving video or a disproportionate percentage of pictures, audio files, or other non-HTML content is prohibited, unless purchased separately as part of a Paid Service or expressly allowed under our Supplemental Terms for a specific Service. If we determine you have breached this Section 2.8, we may immediately suspend or restrict your use of the Services, or limit End User access to certain of your resources through the Services.
https://www.cloudflare.com/en-gb/terms/
And I am wondering what kind of subscription would be appropriate for my use-case if any, that would allow me to host the files on Backblaze B2, and serve these files through Cloudflare.
(Cloudflare would also be hosting our API, so it’s not just video files that we want to serve via Cloudflare.)
However, for a CDN for video content YMMV. Perhaps you should try contacting them directly to have a written estimate.
You'll have to get an Enterprise account for that (it's no longer starting at $5K/mo, you can get custom accounts setup to match your needs).
Meaning that it might be possible to get what we need at a suitable price? $5K/mo is way over our budget until we have actual users and revenue.
If so then that is great. Thanks for the tip :)
https://www.cloudflare.com/products/cloudflare-stream/
https://community.cloudflare.com/t/cloudflare-workers-live-v...
TLDR; He's retracted his claims
If the problem is on cloudflare's side then the cloudflare user doesn't even know about it if it's an intermittent error.
I'm a free plan user because my application is not monetized and obviously would switch to paid plan once it starts generating money. CF free plan has been indispensable for many such free projects.
UptimeRobot(Free) doesn't show anything fishy, except for CF error 521 occasionally. Again UptimeRobot requires paid version to check for HTTP status message, guess I'll have to do it or setup a self hosted(recommendations?) solution to monitor HTTP status of CF Free application.
If CF does perform rate limiting for such low rpm, I would prefer to be intimated as even third rate shared hosting providers do it and I hope CF wouldn't put their reputation on the line for something like this.
[1]https://needgap.com
But can we at least get an update if this is the intended behaviour of the CF free plan accounts (so people know to upgrade) or is it an one-off incident?
I would not want to use a CDN with some hidden limits. Especially not if I don't even get informed when rate limited.
I can imagine them rate limiting me in a way that I will never notice. Like only limiting requests from some other country or continent.
What is a good CDN for a site that has about a million visitors per month? And how much would one have to expect to pay for it?
Same applies for AWS 'Free tier' database usage. Nothing free about it practically.
https://i.imgur.com/UOlW836.png
edit: changed bull crap to the more appropriate horseshit in response to this unsubstantiated-bordering-conspiracy blog post, because if the former is already enough to triggers the community downvoting brigades, no point in showing any restraint
Summary of things that we've had issues with lately:
- Caching of a specific route just stopped working. I log a support ticket (nightmare to find in the dashboard) and miraculously it starts working again after the ticket gets responsded to.
- Some of our staff got locked out of part of our app because we exceeded the 5 free users of the "Access" plan. Upgraded to the 50 user plan in the new Teams part and it still didn't work. Contacted support, fixed again but no explanation. (multiple day turnaround on tickets)
We're invested in their tech a lot and I love workers - they really take some big tasks off us. If Azure come up with something compelling we'll probably switch though (assuming I can make sense of Azure's billing and product naming strategy)
In this case I suspect the monetization algo kicked in. Considering it just produced $40/month, it did well.
Really? I don't think you're aware of what's actually out there.
Let's put it this way: Cloudflare has a straightforward business model. The fact that you can pay them is comforting. The big shady companies that you cannot even pay, yet still make money "somehow", are more scary.
However today and few days ago too it seems requests going through Cloudflare are just timing out. So we finally move to paid plan
We're monitoring both endpoints (CF and the origin)
So that's another reason to subscribe since it's 20$ month. (AWS and other SaaS cost us an order of magnitude more)
Any case, what I meant to say, is that free plan works fairly well even at higher volumes.
If you expect more reliability, should definitely subscribe.
the advanced cache statistics (will help us reduce traffic sent to the S3 origin, in one case, and reduce bandwidth cost)
Traffic is served from the closest location to the user (instead of the bigger central locations, where they can serve traffic cheaper)
Plus paid suppport, for sure
Free tier is awesome for small websites, keeping domains and many other things. But if you run business who earns, then just pay for that god damn thing, and stop ranting. :-)
ETA: * The CTO of Cloudflare
What if he accuses them without justification? Which is probably a case here.
He's already doing that. Even if it turns out that he happens to be right, it's coincidence.
I'm running around 40 domains on Cloudflare Free plan. About 20 of these domains have some traffic.
How do you know for sure that you didn't? One of the main takeaways from the original article is that these errors are invisible from server side (your own stack) because it's the traffic is being throttled before it gets to your own servers. Unless you have very good and consistent client side availability metrics you probably wouldn't notice the 2%-3% drop.
I guess HN users might have caught up to the rest of the internet on how to get clicks.
It's not a recent phenomenon.
Bug resolved. Reason: not enough coffee.
Seriously these systems are complicated and it's easy to confuse correlation for causality.
It does seem to be more Marketing if you want to lock in customers before they get too big to switch.
1. Marketing...
2. Data collection (after all if you are not paying you are the product) There is monetary value in collection, observation, and tracking of a vast amount of internet traffic.
3. Education. If you get people just starting out in their careers using your product or service for they hobby, personal project, etc then when they have a business need for some like your product or service they will just naturally choose you. Adobe and MS has been doing this for decades with low or no cost education licensing
Also why is author giving trust in a free service for his paid service? CF is not that expensive.
Then use another service or pay for it.
This is highly unlikely to be as simple as the author conjects, we're talking about a service that processes an enormous amount of traffic and if what the author suggests is true, would someone else not have noticed by now? It's certainly possible that different infrastructure is used by free/paid plans, and perhaps this specific site was hosted on an unhealthy instance. But we don't have any external data points here to analyse - only those reported by Cloudflare, along with anecdotal reports.
Edit: I see the title has now been changed, at least we're reducing clickbait.
Also, if the infrastructure is in AWS, CloudFront would cost cents if he really does have "4 requests per minute" with full integration and logs.
I have the feeling that with the advent of cloud solutions, very big companies depend on these really small and very useful tools that disregard almost entirely a number of best practices and standards. The Solarwinds incident is going to happen a lot more, that's for sure.
Regarding Cloudflare: I'm fine with the free product not being great, but hiding the logs is not understandable. I bet a lot more people would upgrade if they knew.
[1] https://www.cloudflare.com/learning/ddos/glossary/web-applic...