> By 2024, the legacy Python service had a history of reliability and performance issues. Ownership and maintenance of this service had become more cumbersome for all involved teams. Due to this, we decided to move forward into modern and domain-specific Go microservices.
Using Python for a backend system to "scale" really is just pure cope and was unscalable in the first place and in the long run as Reddit just found out. They knew they needed a lot more than just fake optimizations from an interpreted language to improve the performance and a Golang rewrite unsurprisingly solved those issues.
This once again clearly shows that other than in the prototyping stage of an MVP, it really makes no sense to scale with backends written in these interpreted languages in 2025.
Switching to safe, highly performant mature languages such as Golang tells you that it not only generally improves performance, but correctly handles concurrency which Python has always struggled at, especially at scale which is why in Reddit's case, the race conditions now revealed themselves more clearly before the rewrite.
Here we go, someone's throwing around the S word again. Based on my experience, every time someone mentions "scalability" in a web application context I smell red flags.
Now we don't have details on what the comments service in Reddit entails - maybe it does indeed do a lot of CPU-intensive processing, in which case moving to Golang will definitely help.
But maybe it's also just a trivial "read from DB, spit out JSON", in which case the bottleneck will always be the DB, and "scalability" is just an excuse to justify the work.
The fact this is part of a move off a "legacy" system to "modern" "microservices" suggests there's a huge amount of developers having fun and are incentivized to justify continuing getting paid to have fun replacing a perfectly functioning system, rather than an actual hard blocker to scalability that can't be solved in a simpler way like by throwing more hardware at it.
Reddit was founded in 2005, Go was released in 2007.
They picked the tech that was available and mature at the time, and enabled them to scale for 20 years (to 100M+ DAUs + IPO) - seems like a pretty good choice to me.
You know which other platform was built on Python? Youtube.
Python isn't a bad choice if you're building [certain kinds of] billion-dollar businesses.
> What was unexpected was the underlying differences in how Go and Python communicate with the database layer. Python uses an ORM to make querying and writing to our Postgres store a bit simpler. We don’t use an ORM for our Golang services at Reddit, and some unknown underlying optimizations on Python’s ORM resulted in some database pressure when we started ramping up our new Go endpoint. Luckily, we caught on early and were able to optimize our queries in Go.
A few weeks ago I was evaluating Hibernate for Java again, since my product is expected to support 2 different databases next year. In the end I decided to keep the codebase ORM free, because it’s much easier to me to directly debug our SQL queries than trying to find what and why Hibernate does exactly. I think I’m done using ORMs for the foreseeable future.
yeah - largely a reason why people went to nosql. due to the performance hit you get with ORMs. in ruby-verse you can see this active-record vs sequel(a better orm) the performance gap.
now there's better tools like sqlc etc that have orm ergonomics without the performance hit.
That’s actually poorly written on their part. I did assume they were using an ORM Python library, not that ORM is somehow implicit in Python. But after rereading it, I can see how someone without any Python experience might think otherwise.
9 comments
[ 2.1 ms ] story [ 28.1 ms ] threadUsing Python for a backend system to "scale" really is just pure cope and was unscalable in the first place and in the long run as Reddit just found out. They knew they needed a lot more than just fake optimizations from an interpreted language to improve the performance and a Golang rewrite unsurprisingly solved those issues.
This once again clearly shows that other than in the prototyping stage of an MVP, it really makes no sense to scale with backends written in these interpreted languages in 2025.
Switching to safe, highly performant mature languages such as Golang tells you that it not only generally improves performance, but correctly handles concurrency which Python has always struggled at, especially at scale which is why in Reddit's case, the race conditions now revealed themselves more clearly before the rewrite.
Now we don't have details on what the comments service in Reddit entails - maybe it does indeed do a lot of CPU-intensive processing, in which case moving to Golang will definitely help.
But maybe it's also just a trivial "read from DB, spit out JSON", in which case the bottleneck will always be the DB, and "scalability" is just an excuse to justify the work.
The fact this is part of a move off a "legacy" system to "modern" "microservices" suggests there's a huge amount of developers having fun and are incentivized to justify continuing getting paid to have fun replacing a perfectly functioning system, rather than an actual hard blocker to scalability that can't be solved in a simpler way like by throwing more hardware at it.
They picked the tech that was available and mature at the time, and enabled them to scale for 20 years (to 100M+ DAUs + IPO) - seems like a pretty good choice to me.
You know which other platform was built on Python? Youtube.
Python isn't a bad choice if you're building [certain kinds of] billion-dollar businesses.
http://www.aaronsw.com/weblog/rewritingreddit
A few weeks ago I was evaluating Hibernate for Java again, since my product is expected to support 2 different databases next year. In the end I decided to keep the codebase ORM free, because it’s much easier to me to directly debug our SQL queries than trying to find what and why Hibernate does exactly. I think I’m done using ORMs for the foreseeable future.
now there's better tools like sqlc etc that have orm ergonomics without the performance hit.
at times just use nosql