HN Explorer

Ask HN: Where can I see many examples of real companies' software architecture?

469 points by PeledYuval ↗ HN
I want to broaden my horizon regarding how things are solved in the real world. Other than some very high-profile companies (like Netflix, github) and companies that I've worked at, it's hard for me to find easily digestible (20-60 mins) examples of actual working architecture of differently sized companies from different business verticals.

105 comments

[ 4.8 ms ] story [ 172 ms ] thread
mikojan
Twitch recently open-sourced all their software.
oooooooooooow
You forgot to add unwillingly, but otherwise correct
Nextgrid
Also Patreon and Microsoft (Bing). I’m sure there are other lesser-known ones that might be interesting too.

Oh and I’m pretty sure I’ve seen GitHub Enterprise too.

andy_ppp
I guess searching YouTube is best for these things.

https://martinfowler.com/ but I'm not sure if he touches the real world sometimes, it all feels very academic rather than pragmatic.

I wonder if you'll find "good" outcomes though, it seems to most startups or companies bumble their way to an architecture that works for them. It might not be correct but it might be best way to build a company without architecting everything too much up front.

pc86
Most of MF's articles read like someone extremely intelligent but who last actually wrote code or worked on a real system in the late 90's.
protomolecule
What made you think that?

Just curious, last time I happened to read anything from Thoughtworks was quite a long time ago.

ArtWomb
AWS Summit is approaching. I usually find other teams, even nominal competitors, or hulking behemoths of industry, to be quite proud of what they've built, and generous with their battle tested knowledge. All you have to do is reach out and ask ;)
nuclearnice1
Great plan.

Additionally, there is a “This is my architecture” series on YouTube where AWS interviews users.

https://youtube.com/playlist?list=PLhr1KZpdzukdeX8mQ2qO73bg6...

https://aws.amazon.com/architecture/this-is-my-architecture/

jon-wood
Be careful with things like this - it is ultimately an AWS marketing channel, so they’re not going to bring on people who say “we tried running everything on Lambda and it turned out to be a deployment nightmare”. The very best way to do this is find a tech meet-up around the sort of thing you do, and then go for the after event drinks. Get to know people, chat with them, and find out all the many ways architectures can shoot you in the foot.
[deleted]
(comment deleted)
xs83
Slideshare can give you some insights from various companies, most tech presentations discuss something around their architecture!
baq
look at public org charts, they'll define the architecture.

only half joking

[deleted]
(comment deleted)
fsloth
Conway's law - I don't think it's a joke at all.

Like most things in software engineering, it's qualitative and empirical - but also has very strong potential to function as a supporting "first principles" theory for so many things.

Conway - "How committees innovate"

http://www.melconway.com/Home/pdf/committees.pdf

I think this paper has a fantastic corollary in Peter Naur's "Programming as theory building" which triumphantly explores the implications of institutional knowledge in long term software maintenance. https://pages.cs.wisc.edu/~remzi/Naur.pdf

wsostt
A certain amount of information can be gleaned from job descriptions from a company’s careers page.
alhirzel
You can get a good view on some architectures from AOSA, thought it may not be as focused on web as you're looking for:

http://aosabook.org/en/index.html

A good example is Scalable Web Architecture and Distributed Systems by Kate Matsudaira:

http://aosabook.org/en/distsys.html

DougMellon
Wow, those all look fantastic. Thank you for sharing!
buescher
Thanks! I didn't know about the AOSA books. There are some really nice examples of top-level software documentation in them.
cfors
High Scalability has some good articles around this.

http://highscalability.com/all-time-favorites/

jedberg
Keep in mind that the author of that blog doesn't actually talk to anyone at the company they are writing about, they just collect articles around the internet and public statements and piece it together from that.

For example one of the most popular article on that site (which is part of their book now) is the article on Netflix. A lot of that was cribbed directly from my talks, but they never reached out to me to even check it over, and as such missed a lot of nuance and detail, things I didn't cover in my talks.

Same thing for the article about reddit -- also cribbed a lot from my talks.

It's a fine overview, but light on specifics. I've reached out a few times and some things have been corrected after the fact, but I don't know if the other articles have been reviewed.

So my point is, be warned that the articles on that site are not primary sources but are derived from them.

evanelias
Other times, they would directly talk to a single employee, but get skewed or misleading information based entirely just on that one employee's POV.

Their post about Tumblr's architecture [1] focused a lot about JVM-based services, HBase, etc which in reality was only ever used for a tiny subset of the backend. The huge section on "Cell Design for Dashboard Inbox" was especially ridiculous: the systems described there were literally a mix of complete vaporware and failed/canceled projects that never even got close to production.

As an early Tumblr engineer, I was really upset to read this nonsense. I spent several months of my life working very long hours to successfully scale the existing (PHP/MySQL) dashboard activity feed architecture in 2011-2012. It continued to be used as-is for many years after this interview, with lower latency and much lower cost than the proposed hbase/scala cell replacement.

And of course, engineering candidates being interviewed would always ask about this hbase cell architecture thing that they read about in High Scalability...

[1] http://highscalability.com/blog/2012/2/13/tumblr-architectur...

lemedro
https://runninginproduction.com is a podcast about software architecture examples.
nickjj
Thanks for linking that.

The purpose of each episode is for anyone to walk away having a reasonable understanding of why and how a company built and deployed their app with XYZ technologies without needing to know anything up front. There's over 100 different companies / individuals who were on the show.

I tried to make it as efficient as possible to get these details. There's a lot more detail than a few bullet points but it doesn't get super lost in the woods with a million low level details that's specific to 1 company. It's basically an hour or 2 conversation for each episode where we cover everything from building to deploying their app, lessons learned, etc..

ben30
http://highscalability.com/

Looks like it hasn’t been updated for a while though.

jkingsbery
+1... I haven't read through it in a while, but I read it frequently earlier in my career and have found that really valuable. In the cases where I've been asked about situations that I had no first-hand experience in interviews, it's been helpful to draw on knowledge from that reading. Being able to say, "well, Company X got to scale Y using technique Z" sounds more compelling than taking guesses.
hamasho
> Other than some very high-profile companies (like Netflix, github) ... it's hard for me to find easily digestible (20-60 mins) examples

So I think this doesn't meet your requirements, but I like Tech Dummies Narendra L's YouTube videos [0]. He introduces big tech companies' systems in 30-60min videos and it's not difficult to understand.

[0] https://www.youtube.com/playlist?list=PLkQkbY7JNJuBoTemzQfjy...

swyx
so like.. not to be too cynical but how does he know his representation is correct or at least not misleading? a lot of youtuber content is just made up.
hamasho
Yeah... It's dangerous to accept random YouTubers' content as fact mindlessly.

I'm not sure all he says are correct, but at least he uses the target companies' engineer blogs, external articles, and some open-sourced part of systems (and list them in the video's detail section). His main targets are often big techs like Twitter, Uber, and Netflix, so I guess such documents are often available.

jryantaylor
Narendra's content is awesome, but I think you're right to be skeptical. His content is more focused on how to answer system design interview questions about how the companies operate.
geoduck14
Go to google.com, search "pile of spaghetti", click "images"
rglullis
I'd really want to have that, but for small companies/services that benefited from avoiding the trends to get some competitive advantage.

Something like "IT Architecture for the Forbes 500-thousand"