Render nuked my entire account with no notice
This weekend I began the process to move some of my infra to Render, including a static website, Postgres database and Redis. I switched the static website over but had some issues restoring my database from SQL backups (Render's Postgres appears to timeout connections after a certain amount of time).
After spending a few hours attempting to import all my data, I was logged out from dashboard and all my services were abruptly taken offline with no notice.
When trying to login again, it just throws me back to the login screen and sleuthing the network requests I can see I am now getting "Unauthorized" responses.
My production website was abruptly taken offline with the message "This service has been suspended by its owner." (I didn't suspend it)
For clarity, I was on a paid plan, had a credit card added, and all of these services were paid. I have received no communication, now I have to migrate back to AWS.
I can only assume they saw I was using the database heavily (normal during a restore) and decided I was abusing something? I absolutely did not violate any of their terms nor was I doing anything shady. It's just a static website and a Node service. Nothing crazy.
I just wanted to provide a warning to others because this seems egregious.
118 comments
[ 2.9 ms ] story [ 33.2 ms ] threadI even went through their live-video face verification and passport scan and they didn't budge on this.
What’s the point of these draconian identity measures if criminals are abusing others through your service anyway?
The less strict they make their identity checks - would abuse increase, decrease or stay the same do you think?
Meanwhile, what have they done to catch and ban malicious actors who passed verification long ago and have been abusing other services on the internet from the safety of Hetzner's network for months or years?
While I do run a mail server with them I've never had any reports so I can't say anything on that but I'd assume theyre equally or even more strict with email abuse reports they get.
Although I’d be sore too if I were turned away in the signup process, I’d just as soon learn early that they don’t want my business, before I invest more time and effort getting moved in.
By then, it felt very shaky to depend on such a company, so we switched server to Digitalocean.
Hetzner are cheap though.
Edit: we did also pay, but they never acknowledged that first payment. Then we somehow had an ”unpaid” invoice which we couldn’t find a way to pay for.
I prepaid some invoices, they didn't activate the service because of a technical bug (if you change the bank card, services get cancelled, and not renewed even if you have prepayment, it just goes as unpaid).
They said that as a "goodwill gesture", they are willing to activate my access, and keep all the prepaid money as compensation.
Well, that's my guess, because actually they just replied with 3 dots "..." when I said it was not normal.
Btw, if you are Ukrainian, you can have the "privilege" to prepay for your services instead of getting banned.
https://de.linkedin.com/posts/dennydo_product-overview-activ...
And they are "nice", they are "not planning any sanctions against Ukraine" for now.
>We recently did some routine reviews of our customer accounts. We noticed some suspicious information in your account.
>We have some concerns regarding this information and we have decided to close your account.
>We do not share details about why certain accounts appear suspicious. Publishing this information would make it easier for people to create fake accounts and abuse our services.
>We apologize for the inconvenience.
"My AWS bill was $100k last month" is the new "I'm a manager with 10 direct reports".
In other words, a community (hacker news, some subreddit, or perhaps an independent site) would invite people to post problems that ought to be easily solved by customer support except that the company is failing at customer support. Because problems tend to repeat themselves, the people who spend time on this forum would be able to identify (and sometimes provide workarounds for) the majority of cases -- all at no cost to the company that was failing at customer support.
The remaining cases could be ranked by the community according to how egregious they were. Then the company that was failing at customer support could assign a tiny amount of actual attention to look at just the top few issues sorted by the community.
It’s bad enough companies started outsourcing deciding which features to implement. I don’t want them also outsourcing deciding which cases to support.
Unless we find a way to turn it into some [AITA HN], so that it’s not just drama but also a bit of introspection.
HN threads also rank very highly on Google so even for future users researching the service may gain some value from it.
Like I was interested in trying out fly.io for a long time but after reading so many recent threads about their mistreatment of users I've decided to defer.
These threads are really very useful.
On the other hand, they are often heavily subsided by aforementioned VCs in the name of building momentum. So if you use them cautiously with all this in mind, you can often treat it as an informal piece of redistributative taxation.
The other consideration is DX where these startup infras shine. They make it easy to deploy and that can be tempting. But I am not convinced it is that much easier. Learn the slightest amount of devops (a weekend) and you can script what they did for your usecase. Use github actions or whatever but not too much! Just get that to call a bash script that does what you want.
I think there is a lot of value of trying to keep off locked in paths. But it is a delicate path to tread. And for startups growth is everything. But a few seconds of thought into tech choices and not just jumping into every freemium shiny tech is probably a good idea.
Always happy to hear alternative suggestions but I really just love being able to git push and leave it to CI to deploy easily.
We offer a similar experience to Render/Heroku but we do it on dedicated VM instead of a giant cluster with all the customers. So this means full isolation of your workloads. And you can deploy anywhere (112 regions supported and also BYOVM)
We support 17 managed DB (including Postgres of course) and also 233 open source software.
What hardware are you using? Asking since Intel just got hit real hard with Downfall - another round of speculative execution.
So to mitigate Downfall, we rely on cloud providers infra.
It's still way more isolated than a giant cluster hosting all the apps and routing becoming a single point if failure.
If you don't even have rudimentary stuff like message broker/database you're not an AWS/Azure alternative.
I run some smaller stuff on Hetzner. Have for the last 5-10 years and never had any issues. If Hetzner added managed Postgres, something like Cloud Run, and maybe PubSub I’d be moving over a bunch more stuff.
Most of those 3rd party companies will happily sell you managed database service that are run on Hetzner. So this part of it is sort of a non-issue for a lot of organisations if the cost are low enough.
Heck, even if you have your own inhouse operations department going for you, why wouldn't you be able to run your own stuff on Hetzner? That's what people did in the transition from having your iron in the basement and buying it from Microsoft/Amazon. The only reason we left the "own cloud" options was cost, and if Azure exceeds the alternative. Or the EU makes it too cumbersome to use Azure, then there won't be much of a reason to stay.
I tried to register twice but got denied before I even got the chance to complete registration.
`After reviewing your updated customer information, we have decided to deactivate your account because of some concerns we have regarding this information. Therefore, we have cancelled all your existing products and orders with us.`
No support, no details whatsoever, I'm not sure if they don't want to serve customers outside EU or something else at play, since I didn't even get the chance to finish registering.
I've read similar stories on Reddit, where you needed to deposit $20, but I didn't even get to that step. They seem too strict.
I might have better luck getting hired there than open an account with them.
For context, I live in the EU and send them a copy of my passport when they requested it. Still got denied. I asked them what else they would need, but they said their decision was final. Oh well.
I just went with OVH instead. No hassle, and I haven't had a single issue with them.
Might be worth investigating.
> I tried to register twice but got denied before I even got the chance to complete registration.
Am in EU, had the same happen to me.
Tried creating an account when I was in university, they just suspended it for no reason, even before the process was fully completed. I even sent them copies of my documents to verify my identity, yet they chose not to do any business with me.
A few years later creating an account succeeded and I've been a customer of theirs since - no idea what changed, but I'm not going to move over all of my stuff to them after that experience, or at least keep all of my backups (that I need to restore my sites on any hosting provider) local to my homelab with rsync or something like that.
Then again, maybe being stringent like that works well for them, for whatever reason. That said, their prices are better than pretty much every other platform out there, at least for basic VPSes. Maybe Contabo can compete with them directly, though.
It’s funny, if someone posts a Hetzner horror story here, of which there are plenty, I’m sure someone will recommend Render in the comments. Lots of services are cool until they decide to hit you.
Some reasons why I wouldn't use it for critical infrastructure:
- It doesn't have high availability options (if a server goes goes down, your service goes down)
- It doesn't handle backups
It is great for excellent for anything non critical though
I help run another PaaS, and I’m very curious about your production evaluation criteria. Would you be willing to share?
For context, our PaaS has been focused on running reliable and scalable production workloads for about a decade. We don’t really do hobbyist use cases, sadly. But instead we prioritize actual businesses with production use cases.
If you share your evaluation criteria, I would love to test Aptible against it. I can share the results with you if you’d find it interesting, but frankly I just want the results for my team and me. I’d be so grateful, and happy to return the favor in any way I can. If you prefer, you can email me at henry AT aptible.com
Hit up here if render told you to stick it. But I also often see these end up with "I fucked up but want to yell at someone"
Wary of Render is what you (and I) should be.
There should be no case where a site is taken offline, without an attempt to contact the client first. Ever.
Anything else means the provider doesn't care about your uptime, and thus, should never ever be trusted for anything ever.
By posting here, at least one other person has said they had the same problem and others are suggesting competitors. It’s useful to those of us who are looking for a hosting service.
And like you say, anurag shows up here occasionally to say things like “the company is doubling down on making the cloud delightful”. If they want to use this forum for promotion, then it seems only fair for people who have been hurt by the shitty behavior of the company to come here to complain.
A day ago, in response to another failure at Render, anurag was on HN saying “it's helpful to overshare in these situations”. Well, now they have another opportunity to overshare. What looks like a pattern of bad behavior may have a more reasonable explanation.
This is obviously overzealous abuse management, and we will fix our systems so this kind of thing never happens to legitimate users.
To OP and others who may have been affected, I'm deeply sorry.
• Will you pause the resource utilization, but still allow log-in?
• Will you give notice?
Does "render" (whatever they are) own their own infra ?
If so, this is a weird response since, really, who cares what the retail monthly spend is projected to be when, on your end, it's just some ineffable, tiny portion of your electricity and transit bill.
Right ?
Unless, of course, "render" (whatever they are) is itself hosted on an underlying cloud.
Let me guess: you don't own your own infra ?
This would have been fine if these startups grew on the merits (like DO). But they want to run infrastructure like they run a social network. No.