I remember reading they worked very hard to make their app available on many, many types of phones. Which is weird because they regularly announce they drop support for certain phone OS.
I was there from 2011 to 2019. We had pretty clear numerical guidelines for continuing support of a platform or OS. If the numbers are close, things like upgradability or difficulty of support make a difference: if all the devices running OS 1.1 can actually be upgraded to 1.2 and 1.2 fixes important things, then dropping support for OS 1.1 can happen with more users still there.
If builds need to be signed by the OS maker and they won't sign any more builds, user count doesn't matter (sorry Nokia S40 and iPhone 3G).
We did make some longer lasting builds for platforms we were ending support for, but there needs to be an end. Older clients don't support newer features and we didn't want to have a network where that was the status quo. I don't know what the current expirations are, but mandatory updates a couple times a year helps keep the feature support similar.
Whatsapp used to charge 1 eur or usd per year per user.
Assuming a 20% vat, that’s $800M in yearly revenue. If you spend $500k per engineer per year, you’re still left with $775M to handle infrastructure and all other expenses.
Whatsapp would have been sustainable without fb. That’s the regret.
Technically, but GP is allocating over 95% of post-tax revenue to everything other than engineer salaries, which is probably extremely generous.
WhatsApp had 200M users in February 2013[1], which was a year before they were acquired by Facebook. Perhaps they wouldn't have gotten to 1B users without Facebook, but I'm sure they still would've grown in the last ~8 years, and fewer users likely results in lower costs. My guess is they would still be doing very well.
Even before the sale, most of the customers are non paying. They are offering the service for free in countries with largest userbase(India and Brazil I think). I really doubt it had reached 50 million paying at any point in time.
Exactly. Facebook bought WhatsApp’s users like cattle on auction. WhatsApp guys just needed the right price offer. Now with pseudo tech posts like the current they are trying to mask it’s public image behind some ancient tech innovation from it’s past…
WA Management was misled by Facebook about what FB wanted to do with WA. I agree that you'd have to pretty naive to believe FB but it's not like WA founders sold solely because of money.
WhatsApp founder Brian Acton left 850 million USD on the table when he left Facebook prematurely in protest of Facebook's lies about the vision they had for WhatsApp after its acquisition. After that he donated 50 million USD to the Signal foundation.
> If you don't sell to us, we'll build our own and offer it for free
Weren't they already doing that? Facebook Messenger officially launched in 2011 (essentially an upgrade to Facebook's built-in chat functionality which existed before that) and was free, yet Whatsapp continued to grow.
I've used Whatsapp for a few years when they had this pricing model. Use 1 year for free and pay after. I've never paid anything and nobody who I know needed to pay to keep using WhatsApp.
They could also easily have had different pricing based on location. Requiring EU and US users to pay €5 per year, while have perhaps just €0,25 in Asia, and free in Africa.
If you can't get someone to pay €5/$5 per year for a service they use every single day, then there's something really wrong in how we as users think about the service we use every day.
> If you can't get someone to pay €5/$5 per year for a service they use every single day, then there's something really wrong in how we as users think about the service we use every day.
Which is absolutely the case with most people and recurring payments, even with one time payments. If Signal costed money (even 1-2$), I could not have convinced even 10% of the people I got over to Signal - recurring or one time.
Whatsapp had 700m+ monthly users in early 2015 [1] before they scrapped their subscription fee in early 2016 [2]. Whatsapp was free to use in some jurisdictions for the first year, so we can't assume there was $700m in revenue, but there was certainly enough to pay for 50 engineers you would think.
Whatsapp only had a subscription fee for android users, and I don't know anybody who ever paid for it. They would keep giving you more and more free time; free subscription time would never run out. I imagine it ran out for some users, but I don't know what the criteria was. Perhaps for users in certain countries? Or for users with lots of iphone friends?
On the other hand, iphone users had to pay for the app upfront to download it off the app store.
Whatsapp is built on a massive, mature, global system specifically designed for sending messages. It's sad that it needs to exist and even more sad that it has to "make money" for someone. Aren't we already paying for our internet connexion/phone bill? Why do we need to be used by Whatsapp just to utilise it for its most basic purpose?
Does it? Dumbphones from 30+ years ago can still send SMS messages just fine with zero software updates. Maybe once a decade or so that could have been enhanced with longer messages, pictures etc. But overall the messaging problem really should have been solved by now.
Yes, it does. SMS aren't encrypted. And messaging is closer to being solved than ever before. If you want to use a "dumbphone" with SMS you can do that.
while WhatsApp is the largest I am aware of (it's one of the leading platforms Worldwide anyway), there are quite a few, sometimes in the form of Elixir:
This article does very little to explain how they did that. It basically states the type of tech they used (Erlang, FreeBSD and SoftLayer), and something about not trying to over engineer stuff. The title also makes it sound like 500 engineers would make it easier to scale up WhatsApp, which does not make too much sense either.
Thanks, it does seem like choosing a language that was made for doing things in parallel to begin with was a good idea. To be honest, I've always felt that programming in functional languages creates less bugs, but perhaps that's just my imagination.
> The title also makes it sound like 500 engineers would make it easier to scale up WhatsApp, which does not make too much sense either.
It's something I think we have all seen a lot of times - that by the time a company is serving 1 billion users it has quickly expanded out it's engineering team hugely, and because of that additional abstraction is required and the complexity/LOC skyrockets.
> it has quickly expanded out it's engineering team hugely
But makes little sense (as a developer/engineer myself) to think that growth in users, requires a ton of new developers/engineers. We are not tattoo artists, our job can scale indefinitely if set up properly, i.e. all you should need to scale further (to 1BN or 7BN) should be enough money to buy more hardware; which WhatsApp/Facebook clearly has.
I think that makes sense if the application doesn’t need to also grow much functionally during the expansion, but I believe the more common pattern is more users = more functionality to manage those users, support teams, global legal compliance and accommodate more niche phone systems = more engineers.
You're assuming that their use case & technology stack & application scale cleanly horizontally and they're not spending all their time fighting fires.
True in this instance, but far from a universal truth.
"this team of 7 engineers is responsible for formatting the text content of a chat, including alignment and sizes of emojis and gifs"
"this team of 4 engineers is responsible for formatting the date of a message"
"this team of 7 engineers is responsible for the overall formatting of a chat"
"this team of 5 engineers is responsible for the formatting of the non-chat pars of the application, settings, profile page"
"this team of 4 ux persons is responsible for aligning the non-technical parts of formatting and the user experience over all parts of the application"
"this crossfunctional team of 5 is responsible for creating a framework to let the configuration of formatting be disconnected from the actual implementation of the formatting"
"this team of 7 QA engineers will aid in manual verification of changes and bugfixes but will also automate test cases for formatting in the entire application"
"this supporting team of 4 will develop automation tools and enable the formatting teams to collaborate in a high speed agile context"
The technology stack isn't even that relevant. There have been a few different articles about WhatsApp and their 1B users with only 50 engineers. The key in previous articles was that WhatsApp hired really smart people.
Having extremely talented engineers and a rather small scope is what allowed them to grow til 1B users, with a minimum staff. Yet people seem to focus on the technology, because that's more easily replicated, and easier to accept, in my opinion.
> Yet people seem to focus on the technology, because that's more easily replicated, and easier to accept, in my opinion.
I think you are correct. It also works as some sort of advertisement. Others feel like they made the right tech choices, if they choose the same tech as WhatsApp or Spotify. Completely forgetting that they probably need to be somewhat proficient in Erlang to get the benefits from the language etc.
I guess that "hire extremely good developers" is less sexy than just choose Rust/Erlang etc...
From my experience it's always a good idea to choose a language that people would have to learn by themselves, i.e. none of the school languages (Java, C, C++, C#, Python, Javascript etc).
In that way you only attract people who do sit down and learn new languages by themselves in their spare time, which filters out people with less interest in programming at least.
But if you choose technology for its intrinsic merits you attract engineers with taste, who have the drive and luxury to care about their work on a different level.
There are plenty more people who can't be arsed to or don't have the time or simply don't find it valuable enough to adopt "niche" technology, despite technical merits.
I don't think it has anything to do with smarts, but rather with priorities and human nature. Most people follow the mainstream because they favor stability and convention, those who break out in different directions favor autonomy and freedom.
It's not just less sexy, it leads to frustrated readers/viewers of your article/blog post/conference talk etc. If someone asked how to build a better system for their company and the answer they got was "first spend 5-10 years becoming a better programmer", they will be less happy than if you sell them on the idea that adopting Rust/Erlang/etc will magically make the hard task easier.
I think the technology stack they choose is highly relevant to their success.
The choice of Erlang was crucial: it's basically built for communications and has all pesky issues like version updating, resilience to failure and scalability already taken care of by nature of its architecture and framework.
Of course, these 50 engineers were smart, most in that specialised space are, but I strongly doubt that they could have achieved the reliability and scalability of what made the success of WhatsApp with a PHP or Ruby back-end without more complexity, more ressouces and more hardware (like what FB and Twitter had to go through).
Well true, but my point is that you're not going to be able to scale as successfully as WhatsApp, just by using the same technology stack, if your problem is different.
You're right that Erlang was/is the right choice for WhatsApp, and it was most likely picked as the language of choice, because of the smart people working there. It's the same with FreeBSD, a failing startup isn't going to be able to layoff 50% of its engineers just by switching from Linux.
Erlang was picked for a particular reason. WhatsApp uses a prebuilt open-source chat solution, an XMPP server written in Erlang, ejabberd. This thing was first released in 2003 (old). Already a smart move to get started.
I’m pretty sure they hired Erlang developers to dig into ejabberd internals and optimize certain things. They didn’t just decide to become an Erlang shop out of the blue.
It wasn’t Erlang that was the initial right choice, it was using xmpp and ejabberd that was the root reason. Erlang just happened to be a consequence of that.
I will contest your attribution of ‘smart’ with Erlang. These types of correlations are generally bias fitting. It justifies ‘smart’ being correlated with any and all niche languages, eg ‘so and so likes Haskell, so they must be smart’. No good.
It’s better to attribute ‘smart’ with the pragmatic decision they made to simply use a pre-existing chat server solution that already has the capability to scale. Harder to assess this as smart since there’s no ‘signaling’ here, you have to objectively assess if it was the right tech (which it seems like it was). Way less vanity in this assessment as opposed to what I already pointed out, how your Haskell or Rust devs must be particularly smarter, as opposed to say PHP or JavaScript devs who are considered dumber. I don’t buy it, I need to see more than just your affiliations.
So, I reject your initial post contending ‘The technology stack isn't even that relevant.’ It was precisely the tech decisions that mattered, and the right people to make such decisions. Chicken and egg scenario, I’ll concede that.
In any case, one does not simply pick a old chat server written in Erlang out of the blue - this decision was critical. How many over-funded tech teams would try to do this from scratch in Go? Plenty, and that whole team would easily be full of ‘smart’ people.
> I’m pretty sure they hired Erlang developers to dig into ejabberd internals and optimize certain things. They didn’t just decide to become an Erlang shop out of the blue.
Eh, I was there since October 2011, and we didn't hire any people who knew Erlang until much later. All of the early server engineers (including me) learned it on the job. By the time I left in 2019, I think we hired two people to the server team with previous Erlang experience; it's hard to find people with it, and while it might have been nice, it's not important.
It's possible we had some consulting possibly before I was hired, but I don't remember seeing any evidence of that; OTOH, I do remember setting up and working with a FreeBSD consultant and Moxie when he was consulting on end to end. From my understanding, when things started bottlenecking, Rick Reed was hired to fix bottlenecks; which he had been doing at Yahoo! for many years and had worked with Jan and Brian there.
FWIW; WhatsApp the service started as just a text status, built on PHP and MySQL, but people were using it to chat, so the founders went looking for a chat server to use rather than building one from scratch. I don't know the decision process, but ejabberd was then powering Facebook chat at the time. (Of course, Facebook abandoned Erlang, they said because they couldn't find people with Erlang experience to hire)
Anyway, by the time I got there, I was told that the chat server had been mostly refactored over time and while a lot of names remained the same, and some of the basic architecture was the same, it wasn't ejabberd anymore. Mostly I worked on things that weren't chatd, and I don't think I've seen ejabberd code, so I can't verify, but it seems likely, as we customized the protocol, auth, offline messaging, contacts, session handling, etc.
Erlang is a tremendously right fit for a chat server, and hot code loading is almost necessary when you have hundreds of thousands or millions of connections per chat machine and want to push small changes. Of course, changes to BEAM itself, or the OS kernel take restarts, so you still need to do those from time to time.
Absolutely. A relatively small, highly efficient team that invests in their knowledge, skill and processes is the cause of correct technology choices and not the other way around.
The article isn't really "an article" but it's an email newsletter I send out. I'm quite surprised to see it on the frontpage of HN lol.
So what you're reading is one of my emails.
It's meant to be more of a summary with additional links to sources where people can read more if they're interested (emailing people long-form articles doesn't work well as a content format from what I've seen - even ignoring the length limit that email has).
I'll add in some more links for additional reading (like the high scalability post) to give some more detail for people interested.
OK, I guess any post on this site is dissected a bit too much - looking for insight that was sometimes never even meant to be there. It is usually a good thing, as we expect anything making it to the front page of HN to contain some profound knowledge. Sometimes a popular title referring to an interesting topic can gain a lot of up-votes as well, and then the criticism can be harsh :-)
Some of these points are a bit suspicious, and I bet don't tell "the whole story" the way they're intended. For instance, the statement about not investing in automation unless absolutely necessary or the point about hot-swapping Erlang code.
Would be good to actually hear from one of these engineers for a discussion on the paint points of working on WhatsApp back then.
I would expect there to be certain scaling. Specially when you move to more severs and more redundancy. Cloud solves part, but still not everything. If single person can support 10k or 100k users. You might still need more than that one to move to million or hundred million.
Also some features like language support and such get more complicated when you want to widen userbase from just English speaking ones.
I asked if there was a correlation here, using an if statement as an answer doesn't answer it haha. You're now asking the same question I was, just in-line
"The simpler product makes it much easier to maintain and scale."
This is good news for the developers, the product owners and the customers.
I still can't understand why today people choose a (Javascript) stack of build systems, ton of dependencies, and all kinds of exotic tech that is the latest and most hyped. As a developer you need to support this in the future. It might be nice to build today but it will be a nightmare later.
I have seen days go to waste because of Docker misconfigurations, problems with dependency versions, outdated dependencies not available anymore, bugs deep down in an unknown dependencies, and so on. Personally I want to build systems. I don't want to spend my day debugging all kinds of weird problems.
The (old) WhatsApp teams sounds like a great workplace.
Sure. And so do I. Specifically, I don't want to rebuild systems that have already been built for me which I can use as dependencies. And I don't want to spend all my time troubleshooting bugs, either - I'll take a well-maintained repository on GitHub with thousands of users bug-testing it for me over some janky thing some guy on another team threw together a few quarters ago before he quit.
Simple isn't always better. Reusable patterns are. WhatsApp followed pretty standard erlang design patterns.
Rails and Django to some extent force a standard pattern, they're not really low dependency systems(although they are compared to most node projects). But that pattern to some extent makes sure that you can hand over the project to the next dev and he'll be able to make sense of it.
Somehow node.js has become what PHP used to be.
Whatever happened to interoperability? How are there hundreds of queuing system that use redis and rabbitmq under the hood, but you can only process things in python, javascript or ruby? The data structures are considered private.
So if you want to process your code in python you have to use the python message queue, if you want to process it in javascript you have to find a node mq. How does that make sense?
Want to do business intelligence on your javascript based system now? Gotta write javascript code. Welcome to debugging the same kind of memory leak and processing issues every other queuing system has had to go through.
I agree, but this assumes you actually know what you are building. This isn't usually the case for a startup.
I'm sure whatsapp wanted a million users, but they didn't originally know that would happen. If whatsapp later turned out to be about serving fewer users with more features, this becomes a story about premature optimization.
Do we really think Javascript is exotic? Hasn't it proved itself over many years now?
That doesn't mean it can't have issues like anything else. And because of the massive scale of the language you'll see a lot of shit. It's a language and ecosystem like anything else - it's what you do with it that matters.
I wouldn't say JavaScript is exotic, it's more to do with what is being done with it.
Traditionally it was used to enhance HTML pages with some DOM manipulation but it's now being used for a million other things too.
So historically, if your JavaScript code broke your page would still render as it was just html but now there are all sorts of build processes and long chains that use JavaScript to create the html in the first place... I'd consider that the exotic bit.
Traditional JS running in the browser manipulating DOM elements is very much a foundational aspect of the web and won't deteriorate with age (with the exception of perhaps deprecated functions in the far-off future) but all these js libraries and tools and build processes with massive dependency-chains are what's being referred to.
It's been used as a general purpose programming language for over a decade. It is a 'boring technology' nowadays and hardly exotic.
Of course, you could decide to do something exotic with it (edge rendering, data programming, whatever) but that's not a problem with the language but people wanting to stretch themselves...
JavaScript has matured massively over the past 10 years. I'm not a Node developer myself, but JS + TS is a very palatable combination to me. I'd pick it over many other languages.
10 years ago I would not have touched JS unless I was forced.
I'd be interested to know which ones, because I can't think of one advantage that Javascript has - aside from the ubiquity of browsers. If we're comparing any other aspect of it though, I just don't know what advantage it has (seriously).
Functions are first class. Async i/o by default. Familiar syntax (I'm sure Haskell or Clojure are better languages, but they sure take some time to get used to. You can be fairly productive in js in very little time). There are packages for anything you can think of, no need to reinvent the wheel most of the times.
And I even like it's single threaded nature. Being able to keep a process waiting without needing to spin a new thread with all that implies is very convenient.
Edit: Original commenter had already mentioned typescript.
So what's the deal with TS? I think on top of adding much needed type safety to Javascript, TS is also one of the best type systems you can ask for.
Firstly, thank you for answering. I think most people would be too defensive (devs are a touchy lot:) to bother or just dismiss my sincere question. I'm going to disagree though (quite gently, I hope:)
> Functions are first class.
Always a good thing to have in a language but I struggle to think of a language that doesn't have this now. Java has it, C#, Python… if a language is being developed and didn't have it then now it does, right? Perhaps C doesn't - I checked, functions may be passed as arguments but does not allw nested functions, but it has callbacks so a type of closure is possible… hard to tell. Still, I'd say not using C was an advantage!:)
> Async i/o by default
I'm not sure which languages this gives an advantage over?
> Familiar syntax
I don't think this is an advantage, most languages are much of a muchness, and there are some serious footguns that still lay around in JS. Is `==` familiar? No, that's a footgun. Automatic semicolon insertion (ASI) rules? No, they're another footgun. Are fat arrow operators familiar? Doubtful. The new backticks? They're the other way round to what I'm used to.
Certainly though, it can be picked up quickly enough.
> There are packages for anything you can think of
This is definitely an advantage over some newer languages, but over those in major use, it's not. Still, the way packages are installed and handled is… eye-opening. That wheel has been reinvented several times, and it still looks wonky.
> I even like it's single threaded nature
Sometimes that's a good thing, true, but it's not an advantage to have it set that way all the time.
> TS is also one of the best type systems you can ask for
I haven't used it, tbh, but I would say that an optional type system (it's optional in that you don't have to use TS, I don't know if it's optional once you start using TS, that would be better) is a definite advantage over some languages. It's not baked in though so that's half a point.
I just don't see it. I think the driving force of the move to Javascript everywhere was because devs were tired of learning at least three languages (JS, something, SQL) and not being very good in all three (usually poor at SQL) and thinking they'd get more control by kicking out the DB guy who'd stop their bad ideas (because they didn't know SQL) and they could become an expert in one language to rule them all. Companies love it because of fungibility and more new devs tend to start with Javascript than anything else.
Perhaps I've just got used to keeping several languages in my head?
> I think the driving force of the move to Javascript
> everywhere was because devs were tired of learning at
> least three languages [...] Companies love it because
> of fungibility [...]
>
> Perhaps I've just got used to keeping several languages in my head?
Another possibility that doesn't involve you being smarter than everybody in the room is that many people that were capable of learning several languages saw that the majority of companies wouldn't care for this and instead would prefer to hire for a fungible language. They specialised in this language and then used their free time to learn other skills so they could become more valuable in the marketplace.
You wrote a lot of words about how others were "tired of learning" and "not being very good" and "kicking out the DB guy who'd stop their bad ideas" and then finished it off by congratulating yourself on keeping several languages in your head. I can be rude but you should think about how this would be perceived by others.
No, you should try to keep your temper, you could’ve provided your objections without resorting to rudeness. As such, I feel that I’ll simply ignore your point of view.
Somehow I suspect this isn't the first time you've ignored other people's points of view and felt righteous about it. But, the way you act, must give the people around you a lot of entertainment so I'd certainly not want you to change!
Mate, there are published guidelines for the site. If you don't want to follow them, that is up to you. People aren't even supposed to be snarky but they are, sometimes I am - we're all human and can be a bit spiky at times - but outright rudeness isn't defensible. Of course I'm going to ignore you because of it.
> But, the way you act, must give the people around you a lot of entertainment so I'd certainly not want you to change!
I wanted to point out to you that there are other reasons for people choosing different choices to you which don't involve them being "tired of learning" and "not being very good".
You described yourself as being "used to keeping several languages in [your] head" as if this was an achievement that others couldn't muster.
I still feel that your derision towards others and self-congratulatory tone were deserving of rudeness and snark.
Anyway, I apologise for being rude since it was hurtful. I thought your comment was bad and let you know this by directly accusing you of exactly what you were doing, instead of finding some non-confrontational way of saying it.
Typescript has a pretty cool type system. A mixture of dynamic and static typing. You can be as dynamic as with js (or python), or stricter than Java (you can state a reference cannot be null for instance), or anything in between (and have dynamic parts and static parts in the same program).
I haven't used Typescript but I've generally heard very good things about it. I didn't know you can mix and match, that might give me a bit more of kick to include it in a project, thanks.
Whether or not it's an advantage of Javascript… I'm not sure. I mean, if someone wrote a transpiler for Java that overlayed stricter typing, would that really be an advantage of Java? Not sure, but of course, worth considering when choosing a language for a project.
These issues that you described can happen with any language, ask any programmer or system engineer with long enough experience they’ll have stories with weird problems.
Regardless which language you pick, there will always be these kinds of issues.
> These issues that you described can happen with any language
While you're not wrong, I do strongly feel it depends on the language and architecture chosen. I'm a Go developer these days, and there's a big mindset to e.g. avoid dependencies, and for those dependencies to avoid including even more dependencies, keeping things fairly lightweight and with a low 'attack surface'. I mean I personally wouldn't mind a stricter type system and native enum support and some other things, but for now, I enjoy how basic it is, whilst avoiding the footguns that C/C++ brings.
Seems redundant to have a Express backend + REST API, why can't you just build your REST API on the Express backend?
And also, React is complex, hardly anyone seems to know the internals. The API interface might be easy to learn, but "simple" is not something that you should label React with.
I'm not sure I understand why people build websites like that. My theory would be that usually at some point you have to go lower in the stack (learn how Linux/Node/V8 work or stuff like that). You can avoid going lower in the stack, but the complexity has to go somewhere, and that's what we see here.
Well, WhatsApp had to support exotic cases like BlackBerry, Symbian OS (Nokia), and Windows Phone. I will say, dealing with JS stacks, seems easy in comparison to mobile fragmentation from back then.
I don't see how this is related. Server-side you write and expose an API. Then you have 1-3 developers for each platform that write code that consumes that API. Those developers don't need to know anything about the internals of the server.
But this speaks to good architecture decisions, right?
If they tried to make a shared framework that ran the same codebase everywhere and then port it onto different platforms (cough facebook) then you don't get these nice isolated issues, for example.
Yes, if you don't write a native app for every platform the result will be rubbish. That's not a "good architecture decision", that is common sense. If they don't do it that way it's to save money, because as we all know Facebook is strapped for cash. (Talking about Facebook and Instagram here, because as far as I know WhatsApp is completely native on every platform)
This then also means a well defined server API without undocumented behaviour that individual implementations depend on. I stand by my assertion that the good scaling and separable builds is a sign of good architecture.
I wish that WhatsApp could make a public API. I'm very thankful that Facebook is integrating WhatsApp into their services, because that gives me hope that a true web interface will be developed.
For me, it takes between 10 hours (worst case) to 5 minutes (best case) to open WhatsApp.
My iPhone 4S is too old, so opening WhatsApp gives a message "This version of WhatsApp ha..., Update WhatsApp, Your update will be free of cha...". Clicking Update WhatsApp opens the App Store, and clicking Update results in another error message: "This application requires iOS 10.0 or later."
So I only use WhatsApp through BlueStacks Android emulator on my personal laptop. That's what leads to the 10 hour worst-case response time: a full working day, including commute. The 5 minute best-case is because BlueStacks takes a very long time to open, and runs my laptop fans really loud.
This is not only a pain point with WhatsApp, it's also a problem with WeChat, KakaoTalk, LINE, even 9gag (for posting videos).
Meanwhile, email continues to work just fine on my decade-old phone, mbasic.facebook.com is particularly speedy, iMessage still works, and Hacker News is great :)
Hey, if you do prefer that, I too have a broken phone, but the new multi-device beta let's you to have whatsapp in your computer with a powered-off phone. That works for me.
But this would be client side only, right? These issues would also only occur in the corresponding app repository, when building a separate native app for all platforms. The backend for all of these apps would be the same, not the frontend.
Honest question, is specializing in a JS stack bad in terms of career prospects? I've been using express + react/vuejs professionally for a couple of years and I wonder if it'll be in demand in the next 10-15 years.
You’re fine — don’t listen to JS haters on Hacker News. They pine for the simpler days of sites that primarily use HTML and CSS, with maybe a little bit of JS sprinkled in. But consumers are increasingly demanding an app-like experience from the web, which requires JS frameworks.
As for Express, JS on the backend will be popular as long as JS on the frontend is popular. JS backends have their flaws, but there’s a lot of value in using the same language in the browser and server.
> But consumers are increasingly demanding an app-like experience from the web
This seems tendentious. Which average user was looking for these changes? Can you point to a site that shifted to an "app-like experience" and became successful because of it?
Google Maps was successful from the start, and in my view, has become worse over time. Docs started off "app-like", though I haven't used it in such a long time it may be different now.
This is a strange way to phrase the question. Sites that had to shift to app-like experiences are hard to find, because nowadays pretty much every web app is an app-like experience from the beginning and is created with a JS framework. The shift happened years ago.
Barring informational sites like blogs and news publications, it’s actually more challenging to come up with new web products that are NOT app-like in nature, and that do not use any kind of JS framework. Craigslist may be one of the few big ones and even it is losing market share to FB marketplace, which is an app-like experience.
If I start with 1990s ebay, does it become app-like when I add the ability to zoom images without a pageload? When I add a WYSIWYG listing editor? When I let people drag and drop images into their listings? When I add JS infinite scrolling to search results? When I add AJAX search autocomplete?
Or do I have to go as far as Google Docs, re-implementing copy/paste functions, taking over the mouse wheel, and adding my own text highlighting and zoom implementations?
My personal "line" is when links won't work without js and urls aren't written in the location bar. It makes a site quite useless without js. I know that progressive enhancement is supposed to be a thing but I've yet to see it outside of tutorials (browsers not properly supporting PWAs could be part of that, but I doubt it).
I think it's a "general trend" rather than a website by website trend. I think that 20 years ago most of the discussion online took place with things that looked like websites (forums, mailing lists). These days, most of it takes place in places that look like apps (Twitter, Facebook).
Yes, but you'll find that a lot of work becomes maintenance of older systems, and less new and shiny things.
I do think the 'craze' of new frameworks popping up left and right quieted down some years ago, and things have roughly settled on React, Vue, maybe Angular (which lost a lot of its shine after the Angular 2 announcement fiasco), etc.
For a new application, I picked React + Go because I'm confident that ten years down the line it will still be maintainable - although, Go moreso than the front-end, which doesn't feel nearly as solid and stable.
Same stack as what I chose for my personal projects. This is just my opinion, JavaScript on the server was a horrible idea. There are far better languages to reach for when writing API’s and other server related stuff. I’m curious how you are handling authorization/authentication (assuming you are developing api’s)?
But there are great things happening. The wider community has solutions for everything. (For example here's the "reactive forms are not strongly typed" issue [0] that showcases both the good and the bad. The need for this feature has clearly emerged in 2016. People stepped up and a PR was created, but ... basically no signal from the Angular team. Of course using a wrapper was an easy workaround with a distinctly sour taste in the mouth. Then finally something happened and an Angular team member now seems to be working on it in "full steam ahead" mode.)
I recently had to maintain a large React + NestJS application and I was seriously considering organizing a terrorist cell to go back in time and ...
JS is everywhere though. You even have parasitic languages that compile to JS (ClojureScript, TypeScript, etc). You got Node and also Deno for back-end stuff with support for multithreading. Concequently, JS is among the most popular, and most used languages.
I do admit the ecosystem needs to find a way to have more stability, because NPM outdated package tree nightmare is pretty bad, but I guess that's what you get with an industry that moves so fast, so in that case it's up to you as a developer to not use too many dependencies.
Anyway this is all to say I don't think JS is going anywhere for the next 10 years.
No, it's great. But keep up with the "ecosystem". Learn a bit of frontend, backend and testing too. (Then keeping it fresh won't be a problem.) And we shall see where it goes in ~5-10-... years.
> I still can't understand why today people choose a (Javascript) stack of build systems, ton of dependencies, and all kinds of exotic tech that is the latest and most hyped.
So the same cannot be said for "newer" languages like Rust, Go, and other's whose ecosystems and paradigms aren't completely fleshed out yet?
This comment reeks of someone who is looking from the outside in when it comes to building stuff in JS. Most comments I see criticizing JS stacks are so superficial and demeaning that's its obvious the commenters have little to no experience working in JS.
Why do you compare it to Rust though? JS is as old as Java, however its ecosystem seems to be much less mature (at least from outside). I haven't touched Java for 15 years, but I am fairly confident that people still use Apache Ant as build system. Every time I try to peak into JS world, it appears similar to tiktok trends in terms of how quickly people move from one thing to another.
JS is as old as java, but they're both talking about the build process of JS.
in the beginning, JS wasn't really transcoded/compiled. The first time i personally found out about that was sometime after 2010 with coffeescript, not sure if there were preceeding examples.
and your claim that people move around is really false. React has been the defacto standard for a pretty long time at this point.
yes, other UI libraries/frameworks exist, but reacts marketshare has been extremely dominant since it displaced jquery/ember etc
Express isn't a framework though, maybe a "micro-framework". The nearest competitor to real frameworks like ASP.NET or Spring is Nest.js and it's definitely not got the same longevity as those stalwarts.
> I am fairly confident that people still use Apache Ant as build system
As in: there are still (older) projects around that haven't (yet) migrated away from Ant? Sure.
As in: Ant is still the go-to build system or at least still commonly used? No, not at all. When I create a new Java project in IntelliJ for example, I can choose between Maven and Gradle as the build system. Ant isn't even offered as an option.
It's even worse from the inside. The last 2 companies I have worked at have had significant amounts of JS code (even the majority of code) and it's inevitably became unmaintainable mess through a combination of lack of solid frameworks to instill structure and trying to apply it outside of its domain of the browser.
The last 3-5 years have convinced me that JS isn't appropriate outside of a browser almost ever. I am sure if you think hard enough you can think of cases where it's superior to some other lang but in general it's a very poor choice for almost anything that isn't DOM manipulation.
Worst was definitely attempting to diagnose an off-heap memory leak due to C extensions. Naturally the JS folk gave up and dumped the problem on my lap so I proceeded to do my usual "C guy" stuff and was amazed at just how bad stuff like node-gyp and friends are and just how fragile everything is. I found the leak and patched it and all was "ok" again but just peeking inside those layers makes you deeply uncomfortable with the runtime in production.
The rest of the problems can probably be attributed to lower quality developers but point remains. Things like lack of structure leading to insane architectures, pushing for microservices without understanding the tradeoffs because they didn't want to work on "legacy" JS code that was built with last years hipster tech rather than this years, etc.
These problems are endemic to to culture and ecosystem which IMO are inseparable from a language/tool in practice despite what we want to believe in theory.
I don't refuse to work with JS but I definitely make my concerns abundantly clear and I generally don't hold back with "I told you so" when it inevitably bites people in the ass.
JavaScript is from 1996, it's 25 years old, and apps built with it are used by billions. With regards to the backend, it has a vibrant full stack ecosystem, it's performant, simple, and easy to hire talent for.
It's okay not to enjoy JavaScript (I'd always prefer to go with Dart or Rust myself), it's okay thinking that other languages (and ecosystems) are better equipped to solve some problems, but calling it unproven and exotic is very surprising to me.
Is the implication that FreeBSD is less mature now, intended? I'm assuming not. The BSDs have grown in popularity significantly in the last couple years.
No, but there are signs of distress, IMO. I think their decline in usage has resulted in a decline in port maintainers, which is a bigger deal now that systemd is the default on Linux for so many things.
In the 32 bit days it wasn’t hard to get most Linux binaries to run, now it’s virtually impossible for an end user to do on their own.
Pretty much nothing important depends on systemd, for the simple reason that it’s proprietary and non-portable. Thus, even if there is some code that depends on it, it’s optional.
Also, Linux binary compatibility - the capability to run unmodified Linux binaries - works much better now than ever before.
See whilst obviously the JS ecosystem is quite volatile and can be trend-based, I think there are benefits - the language breeds innovation. Innovation doesn't always make the correct choices, and some tech-conservativism is important in many types of companies. But if our ancestors had just been lazy and just stuck with what was known and good, we'd never even had computers in the first place.
One is that the future is unknown. Your goal might be to keep the product simple and scale to 1bn users, but you don't really know if it will play out like this.
Maybe you plateau at 1m users with the initial idea and pivot the product somehow. Now the app does dating, social media or specialised communications for highway construction teams.
Tradeoffs are easier to make in hindsight. Whatsapp is a good demonstration of what you can gain with good trade offs. Do less but well. Whatsapp did trade some things off though. Their web version came late, is feature poor and a little clunky. Same for a lot of features, compared to similar apps ATT. IDK what exactly can be traced to the stack, but their decisions around user identity are similar. Phone numbers only. One device at a time. They gained simplicity, but lost options.
It's easy to see the benefits in hindsight. Real time is harder. Play that game again and it might turn out different. Most apps that set out to have 1bn never get close. At this point, scalability doesn't matter and tradeoffs made for maximum scalability don't seem as wise.
I think the Unix philosophy should apply here. The fact that WhatsApp did one thing well and it had a good business model (was it $1 per year?) is the important bit.
When you say, “Now the app does dating…” I think the right move is to scrap the project because that’s a colossal fuckup. Unless you have Microsoft cash to have multiple colossal fuckups in a row, don’t add another dating app to your messaging app. Ergo, less is more.
WhatsApp probably started with passion and a solid vision. The Zuckerberg gave them an offer they couldn’t refuse. Anyone will take 19 billion for a basket full of Indian users.
Another thing WhatsApp did well is they targeted ALL phones. They didn’t abandon their users like the fang-bangers (sorry been watching True Blood— FAANG is an annoying acronym so they are hereby fang-bangers).
>Another thing WhatsApp did well is they targeted ALL phones. They didn’t abandon their users like the fang-bangers (sorry been watching True Blood— FAANG is an annoying acronym so they are hereby fang-bangers).
That was clearly the killer "feature." Whatsapp is synonymous with communication on the developing world because of this. I remember when I was introduced to it fairly early on in Brazil and someone claiming that yeah, anyone no matter the OS could get on it. I couldn't believe it, to be honest, it felt like what iMessage was starting to look like... but for everyone? I can't imagine what it would've been like to support so many things, but clearly it was a lot of sweat that paid off extremely handsomely.
I'm not saying that philosophy is bad, just that reality is complicated.
"Scrap the project and move on" works in some contexts, not others. The way startups/products actually work, often, is evolutionary. If your texting idea didn't work, but you see a chance to pivot into something... are you really going to just fire everyone and tell investors "sorry?"
That said, the "one thing well" philosophy really does have big engineering advantages. You can't have everything. I'm just raising the "retrospectives" warning.
In any case, the "$1 per year" was never a real business model. They never even got around to actually charging it... because anything that limits the usership of a messaging app will sink it. It's the opposite of "support everything" strategy that made them successful.
Yeah I agree with the idea of pivoting. I suppose to clarify:
* Pivoting would leverage existing technology built in the process of initial concept. Which to me is the equivalent of scrapping the initial idea (while salvaging the generally-useful IP/technology).
* Adding a bunch of tangential features to a product to increase revenue is a colossal fuckup scenario (maybe the language is a bit over dramatic).
For instance, Google is great at search; gmail is cool; docs was innovative (albeit limited); and then… https://killedbygoogle.com/
Unfortunately, after seeing this time and time again it’s tough for me to get behind mainstream tech. I loved the old Microsoft/Nokia phones; and the Zune. You can tell a lot or love went into the design/engineering but then projects just get axed by corporate interests.
Meanwhile, you can by a mechanical device or appliance from 1950s and it’ll still work just fine.
Again, I don't disagree with you... just think reality makes it messy.
Pivoting via "scrap & salvage" is pretty tough for a startup. The tech behind whatsapp was evidently good, but the IP behind a messaging app is probably not enough to give you an edge. Users are.
Made up scenario: whatsapp loses the SMS replacement game. They have millions of users, but not a billion. Meanwhile, they find that a subset of users like to use whatsapp for dating (or customer support, etc. doesn't matter). They pivot to focus on those customers, and evolve into something else.
This might be a (drama noted) colossal fuckup scenario in an engineering sense. A tractor that you are now converting into a ship.
Evolving is definitely a worse way of engineering than starting with the intention of designing a ship, with neatly defined tonnage, speed and size requirements. Instead, it takes a miracle to implement basic ship features like floating.
Evolving is how a lot of actual software gets invented. Spreadsheets were intended for accountants. They weren't meant to be used as a database, incident report generators, a casual programming environment, or a HR tool. It became those things by evolving.
It happened that way because inventing UIs is hard, and evolving into them happened to work. It's still true that "colossal fuckup scenarios" arise because of this approach. Excel programmer spent decades making excel better at things it's architecture wasn't good at. It's ugly and messy, but life is sometimes ugly and messy.
Flexibility is valuable. Knowing the spec in advance is valuable. Very valuable. They're in conflict with each other to some extent
I think it was 79 euro cents for a lifetime at first (this thread brought back memories of my installing WhatsApp on my phone while I was asleep), then it became 89 euro cents per year (except for the users who had been grandfathered in), and then came the Facebook acquisition and they made it free for everyone.
"Personally I want to build systems. I don't want to spend my day debugging all kinds of weird problems."
Don't we all, I guess we all crave that snap, click, build -feeling Lego gave us. But in my experience building a system means debugging all kinds of weird problems. At least, when I start something, I usually for a long time feel like I'm just hopping from weird problem to weird problem. But, TBH, those problems don't seem weird anymore as experience grows.
I still can't understand why today people choose a (Javascript) stack of build systems, ton of dependencies, and all kinds of exotic tech that is the latest and most hyped. As a developer you need to support this in the future. It might be nice to build today but it will be a nightmare later.
You know that JavaScript developers don't see JavaScript as weird or exotic, right?
I think one thing to consider is that WhatsApp is probably by nature a scalable application. It is in the end, simply a client app that sends data to other client apps. So by nature, you scale in parallel very well, there are no 1-all communications. So unlike something like facebook where the data all needs to go to a central server and be polled by an unknown (but potentially arbitrary large) number of clients, in this case you have a huge number of point to point communications. It also means your server application can be largely stateless.
I'm not saying that it's an easy problem to solve, but it is a much easier problem to solve than a lot of other applications.
To add to this, not only is it a simpler problem, but they stood on the shoulders of giants that already solved it. They used ejabberd, a full-featured xmpp server written in Erlang. Imagine half your job already done for you. Granted, they heavily customized it by this point, but they did a great job picking the right off the shelf solution to begin with.
Why do you assume that WhatsApp uses point-to-point connections?
My assumption is that most devices cannot talk to each other directly and require central servers.
I could be wrong but I don't think he was saying "point-to-point communication" in the technical aspect of the term (i.e. peer-to-peer). Rather, just making the point that they're not storing tons of data on their server that has to be polled on demand by an unknown amount of clients forever. Even if there's a server between the users, once they receive a message and pass it onto whoever necessary, their job is done and they don't need to worry about that data any more.
> It is in the end, simply a client app that sends data to other client apps
That's not the case. Consider the "one tick" messages - they must be on a server somewhere. One tick means they've left your phone but aren't yet on (all) the recipients' phones
Servers proxy the message yes but once all recipients (maybe a dozen clients in the worst cases) receive the message, the server can drop the data once and for all. No need to optimize for harder problems like search, caching, or recommender systems.
I'm honestly not that surprised about the size. In my experience, efficiency and the number of engineers have not necessarily correlated.
For example, I used to work for a company with about 500 employees (relatively large by German standards) and now work for a competing company in the same industry with less than 150 employees. The core engineering team only consists of a dozen engineers.
The output is noticeably larger. The number of new features and especially the stability is significantly better. It is noticeable that due to the small team, each project has one or two "heroes". They are responsible for most of the commits and know the system inside out.
These "heroes" have been working on the same or similar systems for over 30 years in some cases and have experienced a lot in their careers. As such, they can bring experience and advice that is just out of reach for others.
Given the choice, I would always prefer a small team over a large team.
Having worked in a range of companies, there is definitely an inverse correlation between company size and speed, especially for individual teams. At 500 people you're well into enterprise level organisation, where a lot of processes are put in place to minimize the amount of damage a single person can do to the organisation. That also puts a speed limit on those efficient and knowledgeable contributors.
Some companies spend a lot of time & energy trying to implement operating models that try to overcome this but actually end up putting in place more decision processes & oversight. Sometimes the real solution is to reduce control and let the efficient teams lead the way. But then what would all that management do to fill their days...
I think this is more correlated to maturity than size. It's just that size is highly correlated to maturity.
I've been in large companies where I was the admin/owner of their AWS account. And I've worked at companies with 200 people where I have to submit tickets to get anything created in AWS. The major difference was how mature the product was.
I work on a suite of products over 50 years old that have been incumbent for about that long and still have a submit tickets to get just about anything done.
My current employer has a nice balanced approach to this: everything is a code review. I may need a gatekeeper to approve my change, but I can always submit it.
It’s also safer this way. Not even the proper owners should routinely be on production shells or tools with live write access. They also go through peer review with each other. Once the infrastructure is in place for that, it makes sense to open up.
As long as the core system is built on solid engineering, this definitely makes sense.
From my experience in a deadline-constrained environment, things end up being slapped together until they "just work". Adding new features gets more and more difficult and if you're still constrained by deadlines, then the solution is to hire more, which is effectively a brute-force solution to meet the deadlines. Output per employee gets smaller, but total output should grow by some marginal amount.
Absolutely. This is also when bigger and more pervasive bugs enter the conversation. Decisions that kicked the can 2 years ago are now a big problem and no one team can fix it.
Heroes are a red flag for me, as a manager, trying to build a resilient team. Which is not to say that you can't have a range of experience levels. But what happens when those heroes are unavailable? Maybe they're out on vacation, or medical leave, or they eventually retire or leave the organization because they're tired of propping it up? Heroes tend to be a single point of failure.
You're right up to a point, but I've seen far worse red flags. Like companies that can never acquire these heroes in the first place because no one competent would work there that long.
As somebody else mentioned, these companies end up regularly throwing millions at consultant projects that always fail. In other words, they pay a hefty premium for shitty temp "heroes" that give them less than employee heroes would have given them.
You see this a lot in the traditional finance sector, where managers don't appreciate tech workers and relentlessly fuck themselves over trying to save a dime.
Yeah, somewhat counterintuitively treating tech as a cost center will inevitably lead to wasted funds. But hey, that just makes incumbents die faster and give way to organizations treating tech as an investment. Which are the ones you want to work for as a tech person anyway.
If you have good trust and communication within a team, the scenarios you describe are surmountable.
Every strength is a weakness, and every weakness is a strength. IME, heroes are no more a red flag than pretending that good engineers are interchangeable. It depends on the context.
The expected outcomes of these teams are different, and that’s OK. If you’re a very small company and don’t have a couple heroes, you won’t build anything important. If you’re a very big company, the heroes that built it left years ago, and you need resiliency more than new heroes (unless the business is going sideways and needs saving).
This is definitely a potential risk. However, you have this risk with small teams (less than 10 developers) even without hero roles. The smaller the number of developers relative to the complexity of the product, the higher the probability of a single point of failure.
However, you can reduce the risk through good documentation, good engineering practices and so on. As long as you are aware that the team is partly made up of heroes, you can prepare accordingly. So as a manager, I can try to ensure that a few heroes keep productivity high while making sure that the rest of the team understands the basics of the system through things like code review and the like. In the worst case, a teammate can then at least get up to speed quickly.
The worst team is the one with a single point of failure. All hell breaks loose if they're unavailable. This makes me wonder why managers have teams like this?
There is one type of team which is even worse than one with a single point of failure: those teams without anyone at all capable of fixing the system. Given that alternative, having a single point of failure is better than having nothing at all. It would of course be even better to have multiple capable employees, but you cannot always have the team you want with the hiring budget you have.
Not all managers have a primary concern of "trying to build a resilient team".
Sometimes, the #1 goal is building a high-performance and/or high-quality product. Lack of team resilience, (temporary) downtime, risk of project failure due to an insufficient bus factor...they're all not good things, but they can be compromised in the pursuit of that goal.
If you want high resilience as well as high performance and also high quality, well, you should go work at NASA, and hope that the Senate works out their budget issues...
Being replaceable tends to make work less satisfying. All the places I've worked at that followed your advice had the most churn and the least productivity/ROI on engineering $ spent.
Being replaceable tends to make work less satisfying.
In my experience the opposite is true. My personal goal on any project is to make myself replaceable. There's nothing I find more tedious than having to work on the same thing for years because no one else can take over from me.
You’re talking about something else, where you are an key employee building a valuable system that doesn’t need you. The other person is talking about a company hiring you to fit into a system that never needed you.
I would hate to be stuck on the same product or set of features, but I very much enjoy getting to a point where I know all of the systems, how they relate, and roughly where all the functionality lies and who's worked on what. It takes a lot of time to develop that experience and it's so valuable, and it seems so strange that companies don't want to select for this and we're dominated by a culture of job hopping every 2-3 years because it's the only way to maintain appropriate pay. Then companies wonder why everything is over budget and never on time
> My personal goal on any project is to make myself replaceable.
So you admit that you aren't replaceable? If the company mandated that you should always be replaceable then you wouldn't need that goal, it would always be fulfilled. And working in a way that makes you always replaceable isn't fun.
Edit: What we learned from your comment is that making yourself replaceable is fun, but being replaceable isn't fun, since as you say you work to become replaceable so you can start working on something new rather than to stay replaceable forever.
I think like most things there is a balance. The sweet spot may be I can go hard on the project but if I feel like I need some time off - there is the ability to step back and have the work continue.
You hire more "heroes". Pay what they're worth. Make them happy.
Or you just pay an order of magnitude on hosting, performance consults, bandwidth, etc. etc. etc. And STILL won't EVER get to the same result, because you can't always pay (I'd say almost never) your way into a stable and performant solution. Look at all the others.. Nothing works as well as whatsapp.
Good luck hiring 500 engineers who have no idea what happens throughout the whole stack.
I'd rather pay what you save on cloud providers and snowflakes to them. (not even taking into account the result of these "heroes".. the business value / dominance whatsapp has because of this)
You can hire 1000 people who can't draw, or maybe just a little bit, or you can pay an artist and get the result you'll otherwise never get.
> You hire more "heroes". Pay what they're worth. Make them happy.
i dont think its as easy as you make it sound. these "heroes" generally are as productive because they dont sync and already have a clear idea of what they want to achieve/how to implement what they wish.
if you add a second person like that to the same team you run a massive risk of having two competing ideas which creates a lot of friction.
The difference between hero or primadonna can be environmental. When there are multiple experts for a department/tech stack, you get heroes. If the bus number is one, a hero risks converting to a primadonna.
I find if you hire heroes with different skill sets you can avoid this dilemma. Or don't hire two Supermen. Hire Batman and Superman instead. But if a hero converts to a primadonna then they were never a hero. On the other hand if you're skimping a hero of the equipment and resources they told you they needed from day one, you're the a***.
I agree. I'd add that having multiple experts in competition with each other (due to bad culture, bad communication or even just bad personality) also has the potential of turning heroes into primadonnas.
In my experience, insufferable mid-level management can bring out the worst in the best people, and then accuse them of being primadonnas because they are absolutely incapable of any self-reflection themselves.
At my company, “heroes” are given their own projects with them given full creative control over every detail, including the people they work with. I think I’ve read somewhere on HN to never let any two person do the same thing, to avoid the conflicts you mentioned.
Benefit of this is that these heroes can each break new grounds, and their BKM shared amongst themselves making the team extremely productive.
I work as a contractor, thus I have read a lots of code from various different types of projects and the conclusion is always the same, if they had spent more money on skill to begin with I wouldn’t be needed.
5 heros on vacation for 7 days, and coming back and finishing the work in next 5 days is more comfortable position than 50 clueless engineers claiming work will be over in 4 days.
Many/most teams don't have heroes at all, they only have noobs :) I.e. most people stay on a team for a year or three and change jobs afterwards - there's no chance of accumulating knowledge. Nobody is sure how things work and why they are the way they are.
I believe many managers are fine with this, because the team is producing something (i.e. "velocity" is higher than zero), but are oblivious to the fact that team could be way more productive if the knowledge was actually retained in people's heads and not just lousy attempts at documentation.
I don't know if they are oblivious to it - but it might not be possible to pay people enough to get them to stay due to pay bands set by HR etc.
OP mentions that he is in Germany - there it might be more possible as SWE doesn't pay as well as it does in the USA and there are fewer companies so it might be feasible to pay a lot more than the competition, whereas in the USA that's unlikely to be viable outside of companies like Netflix etc.
I understand your point, but what managers fail to understand is that someone usually "carries", i.e. is actually responsible for the success of the product. I have even seen it be a manager...once.
> Heroes are a red flag for me, as a manager, trying to build a resilient team
Heroes are your most powerful asset, but you have to use them responsibly. The best thing you can do when handed a 10x unicorn developer is to try to document 100% of the things they say & do, and also make it a requirement that the hero mentor others some % of the week. E.g. For 1-2 hours every Friday you force them to hold a "no stupid questions" session.
My first interaction with this management philosophy was when our partner AOL replaced experienced SRE serving our project with 3 far less experienced people for the same price. What used to take an hour now literally would take weeks on the bright side for the manager he had way more reports now.
I tried to clarify in my top comment that I wasn't saying we should avoid having experienced people on a team, but based on your response, maybe a little more clarification is needed.
When I think about a "hero" in terms of team dynamics, it's a person who is consistently relied upon to save everybody else's butt, often by doing things that are flashy but not particularly healthy, such as working long into the night to meet an unrealistic deadline. When you have this sort of Superman figure who's willing to swoop in and save the day, the problem isn't so much their level of experience but the unhealthy level of dependency that's placed on their shoulders.
I'm all for having highly skilled, highly experienced engineers who are productive themselves and can further increase productivity by helping unblock others on their team. And I agree with you that replacing them with some number of juniors without the same institutional knowledge can be disastrous. But if your team becomes so dependent on heroics that they can't stay afloat otherwise, then when that hero gets hit by a bus or just quits to take a position elsewhere, you're screwed.
> But what happens when those heroes are unavailable? Maybe they're out on vacation, or medical leave, or they eventually retire or leave the organization because they're tired of propping it up?
I don't know why people insist on using examples like this instead of the more obvious one: What if they are hit by a bus?
Any of the other examples have trivial solutions in the event of an emergency: call them and maybe offer them some money. If they are very stubborn then go to their home and beat them with a wrench. But you cannot get information from a dead person no matter how much money or how big a wrench you have, and people die suddenly every single day. That's the worst case you need to be making contingencies for.
Having a "hero" at a 5 person startup can mean life when death was inevitable. Having that at a 500 person org likely means another team is left cleaning up crap.
It's really not hard to be a "10x engineer" if you disregard tech-debt and error-handling. I was part of a very productive, complementary duo where my partner churned out a lot of code that only catered to the happy path, and I'd clean it up/"productize" it.
I actually enjoy that sort of work, but didn't receive as many accolades as the guy "churning out features quickly", even though he'd break the build and block the entire company at least once a week before I paired up with him.
Once I saw how the sausage is made, I'm a lot more sceptical of the "10x" label, either there's an invisible support system, or the code base had a short half-life. Any other scenario is a set-up for disaster.
Managers prefer a team reliably working at 0.1x speed rather than have it work at 1x speed 95% of the time. Yes, sometimes people leave and you will have less velocity for a while, but people pick it up and you go back to high velocity. To fix that you ensure the team always works at slow velocity, that way it wont get slowed down since it was already performing as if you lacked those heroes from the start.
I worked at a place that made a group of loosely-related websites. Basically different spins/themes on the same products. There were 4 of us on the development team. There were 10 websites. Some new, some in maintenance mode.
I quit after a few years, and about 3-4 years later they went on a hiring spree. Tripled the number of developers. Refreshed some products, discontinued some, and some went into maintenance mode. They still had around the same 10 websites, but 12 developers.
The products didn't get better. The products didn't get made any faster. They didn't have any less bugs. My wife actually works there now, and when she tells me about the issues they have I laugh a little bit.
It's all the same problems we had back then. Invites don't work. Teams doesn't work. Uploading profile pictures has issues. Entering data doesn't work. Registration is broken.
I had a somewhat similar experience with a large tech company. Grew from <100 to 600 in a few years, but in the meantime they weren't even able to update the homepage. Not even the content changed, in almost three years. Everything was convoluted and over-engineered. The whole thing was rewritten from scratch twice, and when I left there were plans for a third rewrite from scratch. COVID destroyed the company: there were zero customers. But the website still struggled to stay online. That's when I decided to leave.
Why is this being downvoted? Whatsapp is notoriously "just" a highly-optimized fork of ejabberd [0]. ejabberd is written in Erlang/OTP which enables quasi-horizontal scaling and amazing failure recovery, along with incredible uptime.
What's different about Whatsapp compared to a traditional XMPP experience is:
- it uses a single centralized server, without federating
- it uses phone numbers as identifiers instead of nicknames
- it has a tightly integrated (closed-source) client
So you can not only "build your own WhatsApp", but also federate with others doing the same, which is pretty cool and prevents a single actor from abusing the whole network.
Doesn't anybody remember how literally Whatsapp's founder wrote on one of the ejabberd mailing lists with something like "I just installed the server, please help me configure authentication" or the like?
Retrospectively that has always colored my perception of how relevant technical aspects are for a successful startup (i.e. not much). Marketing (and viral, user-capturing & monopolist practices) are (sadly) much more important.
> how relevant technical aspects are for a successful startup (i.e. not much)
That's why i'm very interested in human and political aspects of free software. Successful FLOSS projects usually have a strong community backing them, and strong connections between developers and the community so that the technology isn't too disconnected from practical needs and UX concerns.
I think non-profits and cooperatives building upon free-software solutions are a better alternative model. For example, framasoft.org (french NGO for libre culture/software) has been instrumental in developing new solutions like Peertube, and in kickstarting a new wave of hosting coops (chatons.org federation).
In the IM world specifically, i'm very interested about Snikket.im project. It aims to build a complete client/server distribution of XMPP software tailored for specific user expectations.
If you look at https://www.infoq.com/presentations/whatsapp-scalability/ though it gives a very different picture. Sure, the founding team may have tried to prototype without knowing full technical details, but they knew enough to choose software that could fundamentally scale towards the future, and they rapidly brought on fantastically talented developers with a hunger for optimization. You can't choose the right technical people without technical chops; that being said, in many cases the right technical people are those who know how and when (and when not) to stand on the shoulders of giants!
It doesn't look like "they knew enough to choose software that could fundamentally scale". It looks more like they choose the first option that was available to them. Apple also choose XMPP for Facetime as did a million other companies that serve similar scales of simultaneous clients.
They may have hired people later on (though this entire article is kind of showing that no, they actually didn't hire that much technical people). But by the time they even hired the first Erlang engineer they had already won over the European market. And for the record it was not precisely due to the quality of either their Android or iOS android apps -- the J2ME and Symbian clients had quite a following and I would even bet were more used.
Original Whatsapp was a disaster and this did not prevent them from gaining marketshare. You could basically login as anyone just by having their phone number (which maps to their JID) and defeating their XMPP obfuscation layer (some XML encoding, which was easily done due to the easily RE Java clients).
It was only later on (2011ish?) that they started getting somewhat serious, which coincided with them, already swimming in users (and money?), starting to get more evil (with a more hardcore stance against 3rd party clients).
Not at all, I have a bunch of friends on my private server and getting someone an XMPP client and account takes less than a minute. No phone number or email required.
Sure, everyone out there isn't going to be on my server - but for the close friends I care about - it's been flawless.
I decided to delete all proprietary messaging apps and use XMPP exclusively in Janurary this year. It took some time to explain but now most of my friends and family can now be reached via XMPP.
The client is another story, but for the server they forked ejabberd which is a reliable, established XMPP server. So they probably had "fast-cheap-secure" on the server-side, because 6 years had been spent ironing out ejabberd bugs before the first version of Whatsapp came out.
My understanding is that vulns in Whatsapp so far were client-side, but i'm interested if i missed some on the server side.
I bet these engineers got only a tiny tiny fraction of the billions of revenue.
It always fascinates me how in the engineers' mind it reconciles that their work brings corporations they work for billions and yet they have to rent a tiny apartment for a substantial portion of their salary and otherwise lead a pretty average life. Then going to lavish offices witnessing how the money they could make use of is wasted on vanity.
That's the founding principle of capitalism: private property.
People living someplace don't own it, and people working some field don't own it. Some "owner" owns the land and means of production and extracts value from people doing the actual work.
Yes, it's a deeply broken system. A slightly better system revolves around self-organized workers coop where everyone gets equal pay and there are no shares to hold (or everyone owns the same amount). An even better system abolishes money and private property so that people can live meaningful/useful lives without worrying about imaginary numbers ruining their entire existence.
This just makes more clear that we didn't try enough creating an open-source instant messaging app that respects our privacy as it should be. It's probably much more difficult to do the same thing as WhatsApp did because of the dependency we created around it but I think we just need the right initiative to do so.
It is hard to get the general public to care about privacy when they post pictures of every meal and share every thought and emotion they have with no filter.
It’s hard to get the general public to care about inferior things pushed on them by patronising people sneering at them. How is a photo of some food important to privacy enough to be the first thing you teach for? Seems more like punching down at people you see as inferior because photographing meals is “so lower class, right guys?”.
The many comments alluding to the technical challenge of building a chat app with 50 engineers being relatively easy are forgetting that scaling said chat app to 1 billion users is absolutely not easy and a function of more than just the engineering quality and technical challenge. There's MUCH more to a successful product than engineering...
User count is fairly meaningless. How much was that in daily active / weekly active users? How many of those were bot / spammer accounts being “seasoned” to then sell to abusers?
There was a joke, or maybe anecdote, going around the Israeli high tech industry when Google purchased Waze
When it came time to get an overview of the codebase being purchased. The Google Maps car navigation iOS team had over 200 engineers. The Waze iOS team had 2
off topic but I recently started working with an Israeli tech company and I'm amazed by the culture they have, what about Israel produces such high levels of entrepreneurship?
> The Google Maps car navigation iOS team had over 200 engineers.
I don't really believe that. I worked at Google and the frontend teams were usually just a couple of engineers per target client, I doubt any projects had 200 entire engineers just for a single target client. Not even big projects like gmail has nearly that many.
Edit: If I were to guess Google maps would have 200 engineers in total, or at least 200 figure would be the entire team of frontend engineers for every target client, and then many more working in other roles. 1 engineer per million users is a pretty good rule of thumb for Google products, and a large majority of those will work on backend systems.
What is amazing to me is that they did scale to 400 million monthly active users with less than 50 engineers... In 2014.
That is: on hardware from seven years ago and older. Which is an eternity in tech. Since then we've seen incredible new hardware come out (EPYC 2 in 2018 for example).
One can wonder: up to how many MAU could a service scale today, on today's hardware, with a small team? And what about tomorrow's hardware...
Seven years can be an eternity in tech, but is it an eternity counting from 2021? I mean, AWS launched in 2006, Netflix expanded to Europe in 2012, Docker had their initial release in 2013 and so on, so none of these major events are that "new" anymore. Obviously things have changed (a lot) during the past seven years, but have they changed as much as they did between 2007 and 2014?
What I'm saying is that for the general case of "scaling with few engineers", it seems to me that the tools available matter a lot more than the specifics of what hardware is available at the time. Launching an international service "from scratch" with a small team and a modest budget just wouldn't have been possible 20 years ago, but not because CPUs were too slow.
intuitively I would agree, but the point of my comment was that intuition might be a poor guide if we want to understand what kind of team and software architectures "succeed".
if you rewind some years microservices was also the word of the day (and probably there were some iconic projects picked to justify...)
They not only used FreeBSD for their servers, their CEO at the time Jan Koum donated $1 million dollars to FreeBSD foundation too. Talk about giving back!
The only thing they failed at was making money. I really wished they had. They knew what Whatsapp meant to the world. Look at WA now.
The founders hearts were in the right place. Brian Acton created Signal Foundation and Jan Koum like I said also have done good things.
They also tried the paid version for a while if you remember. And they were also dead against ads on the app. Unless I missed something or if I am wrong somewhere (At which point I would love to be corrected), Selling was the plan all along doesn't seem like a right observation.
>The founders hearts were in the right place. Brian Acton created Signal Foundation and Jan Koum like I said also have done good things.
I'm not saying they are bad people, and selling is not a bad thing either. I'm just pointing out that selling was the plan all along.
The WhatsApp story that gets told regularly (like in this instance) is a story of how such few employees and such small infrastructure is enough to build a massive product. You know what that translates to? Low opex. If they wanted to, WhatsApp would've been quite profitable if they wanted but instead they chose to sell to the highest bidder.
Again, that's not a bad thing, but also, I'm sure there were PLENTY of people that were interested in buying a billion user platform with 99% user retention. Why did they sold to Facebook? Only they know, but back then Facebook already had a tarnished reputation, so they definitely didn't do it "because of their mission and values".
This is a good point of view and this is entirely possible. But from an employee POV, FB was touted the best place to work. Things like that would've been a reason to select.
There is another chance that FB was seen as the lesser evil compared to Google then, who was also in the bidding for Whatsapp IIRC. My younger self would've chosen FB instead of Google for sure.
Not to mention, people wouldn't have guessed how horrible FB would turn out to be.
Edit: Funnily enough, after I wrote this comment, It became clear that money was definitely a factor.
It was free for the first year iirc, and then a dollar a year. A dollar a year is way more than needed to cover the cost of hosting, and most would spend that to avoid paying for SMS.
They ended up with a billion or two users. $1-2 billion with the subscription is good for a company of a few dozen employees. They could have branched into all sorts of value-add stuff in the future if they wanted to, all without tracking and such even. A simple payments or shopping interface a la Instagram Stores could have done the trick.
WhatsApp was successful only because they had no intention of ever making money. Simple lightweight app, no tracking, no ads, no upsell. Yet they were funded by VCs who would want a return on their investment at some point.
But if they started charging even 1$/user/year, most of the 450M users might not be on the platform. There are other free platforms offered with similar functionality. Security/Privacy is not a concern for a lot of that 450M users.
Because complexity kills. Something that has yet to be learned by the new generation of developers. While disciples of FAANG create a gazillion microservices that they spend most of their time debugging in production ("observability"), smaller teams just get shit done.
Since "monolith" has become a dirty word, it would be instructive to remember that some of the major companies out there are still light on distributed systems, which we all knew even back in the olden days of the internet were an absolute killer of efficiency and productivity. Yes, "microservices" is just another word for it - believe it or not, there was life before Node and Docker.
Whatsapp, Dropbox, Instagram, StackOverflow - all existed or still exist as one easy-to-maintain application. But go ahead, "be like Google".
Messaging is a weird app to focus on mau/dau. There have been many messaging protocols which scale to hundreds of millions of users, and there is prior art of large not particularly valuable chat systems like aim, gchat etc.
No one is going to use a consumer messaging system which charges money or shows ads. Which brings to mind substantial questions on the business model others are using the messaging platform for e.g. data collection.
461 comments
[ 3.1 ms ] story [ 333 ms ] threadIf builds need to be signed by the OS maker and they won't sign any more builds, user count doesn't matter (sorry Nokia S40 and iPhone 3G).
We did make some longer lasting builds for platforms we were ending support for, but there needs to be an end. Older clients don't support newer features and we didn't want to have a network where that was the status quo. I don't know what the current expirations are, but mandatory updates a couple times a year helps keep the feature support similar.
Some things just can't really make money unless they do it in scummy ways (or we haven't found a viable alternative or a way to protect them).
Public instant messengers, public forums, public file hosting, public image hosting.
It's like Microsoft Office these days. If your home license expires they keep threatening you they'll disable it. For years and years.
Assuming a 20% vat, that’s $800M in yearly revenue. If you spend $500k per engineer per year, you’re still left with $775M to handle infrastructure and all other expenses.
Whatsapp would have been sustainable without fb. That’s the regret.
WhatsApp had 200M users in February 2013[1], which was a year before they were acquired by Facebook. Perhaps they wouldn't have gotten to 1B users without Facebook, but I'm sure they still would've grown in the last ~8 years, and fewer users likely results in lower costs. My guess is they would still be doing very well.
[1] https://en.wikipedia.org/wiki/WhatsApp
And they would have been out of business sooner or later. There are enough people wanting to save that one dollar.
Business accounts, facilitating p2p payments (minus a scummy new coin), etc.
Sustainability would have been possible in all sorts of ways even though massive levels of profits (in any company) inevitably means being scummy.
Weren't they already doing that? Facebook Messenger officially launched in 2011 (essentially an upgrade to Facebook's built-in chat functionality which existed before that) and was free, yet Whatsapp continued to grow.
If you can't get someone to pay €5/$5 per year for a service they use every single day, then there's something really wrong in how we as users think about the service we use every day.
Which is absolutely the case with most people and recurring payments, even with one time payments. If Signal costed money (even 1-2$), I could not have convinced even 10% of the people I got over to Signal - recurring or one time.
Plan B: ummm
[1] https://www.businessinsider.com/whatsapp-passes-700-million-...
[2] https://blog.whatsapp.com/making-whats-app-free-and-more-use...
On the other hand, iphone users had to pay for the app upfront to download it off the app store.
2) A lot of people would be rightly suspicious to use govt systems for their own private data.
I would say the Elxir is based on Erlang about as much as Scala is based on Java.
- Discord
- Pinterest
- Spotify
- Klarna
- Riot games (the messaging service)
- PepsiCo (for their billion making e-commerce)
- Toyota (for the connected platform)
I framed it this way intentionally. Without a doubt the most successful Erlang apps are telecoms related.
whatsapp being a closed system I don't understand how this disparity is possible.
It's something I think we have all seen a lot of times - that by the time a company is serving 1 billion users it has quickly expanded out it's engineering team hugely, and because of that additional abstraction is required and the complexity/LOC skyrockets.
But makes little sense (as a developer/engineer myself) to think that growth in users, requires a ton of new developers/engineers. We are not tattoo artists, our job can scale indefinitely if set up properly, i.e. all you should need to scale further (to 1BN or 7BN) should be enough money to buy more hardware; which WhatsApp/Facebook clearly has.
True in this instance, but far from a universal truth.
"this team of 4 engineers is responsible for formatting the date of a message"
"this team of 7 engineers is responsible for the overall formatting of a chat"
"this team of 5 engineers is responsible for the formatting of the non-chat pars of the application, settings, profile page"
"this team of 4 ux persons is responsible for aligning the non-technical parts of formatting and the user experience over all parts of the application"
"this crossfunctional team of 5 is responsible for creating a framework to let the configuration of formatting be disconnected from the actual implementation of the formatting"
"this team of 7 QA engineers will aid in manual verification of changes and bugfixes but will also automate test cases for formatting in the entire application"
"this supporting team of 4 will develop automation tools and enable the formatting teams to collaborate in a high speed agile context"
Having extremely talented engineers and a rather small scope is what allowed them to grow til 1B users, with a minimum staff. Yet people seem to focus on the technology, because that's more easily replicated, and easier to accept, in my opinion.
I think you are correct. It also works as some sort of advertisement. Others feel like they made the right tech choices, if they choose the same tech as WhatsApp or Spotify. Completely forgetting that they probably need to be somewhat proficient in Erlang to get the benefits from the language etc.
I guess that "hire extremely good developers" is less sexy than just choose Rust/Erlang etc...
In that way you only attract people who do sit down and learn new languages by themselves in their spare time, which filters out people with less interest in programming at least.
But if you choose technology for its intrinsic merits you attract engineers with taste, who have the drive and luxury to care about their work on a different level.
There are plenty more people who can't be arsed to or don't have the time or simply don't find it valuable enough to adopt "niche" technology, despite technical merits.
I don't think it has anything to do with smarts, but rather with priorities and human nature. Most people follow the mainstream because they favor stability and convention, those who break out in different directions favor autonomy and freedom.
The choice of Erlang was crucial: it's basically built for communications and has all pesky issues like version updating, resilience to failure and scalability already taken care of by nature of its architecture and framework.
Of course, these 50 engineers were smart, most in that specialised space are, but I strongly doubt that they could have achieved the reliability and scalability of what made the success of WhatsApp with a PHP or Ruby back-end without more complexity, more ressouces and more hardware (like what FB and Twitter had to go through).
You're right that Erlang was/is the right choice for WhatsApp, and it was most likely picked as the language of choice, because of the smart people working there. It's the same with FreeBSD, a failing startup isn't going to be able to layoff 50% of its engineers just by switching from Linux.
I’m pretty sure they hired Erlang developers to dig into ejabberd internals and optimize certain things. They didn’t just decide to become an Erlang shop out of the blue.
It wasn’t Erlang that was the initial right choice, it was using xmpp and ejabberd that was the root reason. Erlang just happened to be a consequence of that.
https://www.ejabberd.im/
I will contest your attribution of ‘smart’ with Erlang. These types of correlations are generally bias fitting. It justifies ‘smart’ being correlated with any and all niche languages, eg ‘so and so likes Haskell, so they must be smart’. No good.
It’s better to attribute ‘smart’ with the pragmatic decision they made to simply use a pre-existing chat server solution that already has the capability to scale. Harder to assess this as smart since there’s no ‘signaling’ here, you have to objectively assess if it was the right tech (which it seems like it was). Way less vanity in this assessment as opposed to what I already pointed out, how your Haskell or Rust devs must be particularly smarter, as opposed to say PHP or JavaScript devs who are considered dumber. I don’t buy it, I need to see more than just your affiliations.
So, I reject your initial post contending ‘The technology stack isn't even that relevant.’ It was precisely the tech decisions that mattered, and the right people to make such decisions. Chicken and egg scenario, I’ll concede that.
In any case, one does not simply pick a old chat server written in Erlang out of the blue - this decision was critical. How many over-funded tech teams would try to do this from scratch in Go? Plenty, and that whole team would easily be full of ‘smart’ people.
Eh, I was there since October 2011, and we didn't hire any people who knew Erlang until much later. All of the early server engineers (including me) learned it on the job. By the time I left in 2019, I think we hired two people to the server team with previous Erlang experience; it's hard to find people with it, and while it might have been nice, it's not important.
It's possible we had some consulting possibly before I was hired, but I don't remember seeing any evidence of that; OTOH, I do remember setting up and working with a FreeBSD consultant and Moxie when he was consulting on end to end. From my understanding, when things started bottlenecking, Rick Reed was hired to fix bottlenecks; which he had been doing at Yahoo! for many years and had worked with Jan and Brian there.
FWIW; WhatsApp the service started as just a text status, built on PHP and MySQL, but people were using it to chat, so the founders went looking for a chat server to use rather than building one from scratch. I don't know the decision process, but ejabberd was then powering Facebook chat at the time. (Of course, Facebook abandoned Erlang, they said because they couldn't find people with Erlang experience to hire)
Anyway, by the time I got there, I was told that the chat server had been mostly refactored over time and while a lot of names remained the same, and some of the basic architecture was the same, it wasn't ejabberd anymore. Mostly I worked on things that weren't chatd, and I don't think I've seen ejabberd code, so I can't verify, but it seems likely, as we customized the protocol, auth, offline messaging, contacts, session handling, etc.
Erlang is a tremendously right fit for a chat server, and hot code loading is almost necessary when you have hundreds of thousands or millions of connections per chat machine and want to push small changes. Of course, changes to BEAM itself, or the OS kernel take restarts, so you still need to do those from time to time.
Author of the article here.
Totally agreed. Apologies for that!
The article isn't really "an article" but it's an email newsletter I send out. I'm quite surprised to see it on the frontpage of HN lol.
So what you're reading is one of my emails.
It's meant to be more of a summary with additional links to sources where people can read more if they're interested (emailing people long-form articles doesn't work well as a content format from what I've seen - even ignoring the length limit that email has).
I'll add in some more links for additional reading (like the high scalability post) to give some more detail for people interested.
Thanks a lot for the feedback.
Would be good to actually hear from one of these engineers for a discussion on the paint points of working on WhatsApp back then.
I think this XKCD applies: https://xkcd.com/1205/
Also some features like language support and such get more complicated when you want to widen userbase from just English speaking ones.
I asked if there was a correlation here, using an if statement as an answer doesn't answer it haha. You're now asking the same question I was, just in-line
This is good news for the developers, the product owners and the customers.
I still can't understand why today people choose a (Javascript) stack of build systems, ton of dependencies, and all kinds of exotic tech that is the latest and most hyped. As a developer you need to support this in the future. It might be nice to build today but it will be a nightmare later.
I have seen days go to waste because of Docker misconfigurations, problems with dependency versions, outdated dependencies not available anymore, bugs deep down in an unknown dependencies, and so on. Personally I want to build systems. I don't want to spend my day debugging all kinds of weird problems.
The (old) WhatsApp teams sounds like a great workplace.
Sure. And so do I. Specifically, I don't want to rebuild systems that have already been built for me which I can use as dependencies. And I don't want to spend all my time troubleshooting bugs, either - I'll take a well-maintained repository on GitHub with thousands of users bug-testing it for me over some janky thing some guy on another team threw together a few quarters ago before he quit.
Simple is better in my opinion.
Rails and Django to some extent force a standard pattern, they're not really low dependency systems(although they are compared to most node projects). But that pattern to some extent makes sure that you can hand over the project to the next dev and he'll be able to make sense of it.
Somehow node.js has become what PHP used to be.
Whatever happened to interoperability? How are there hundreds of queuing system that use redis and rabbitmq under the hood, but you can only process things in python, javascript or ruby? The data structures are considered private.
So if you want to process your code in python you have to use the python message queue, if you want to process it in javascript you have to find a node mq. How does that make sense?
Want to do business intelligence on your javascript based system now? Gotta write javascript code. Welcome to debugging the same kind of memory leak and processing issues every other queuing system has had to go through.
I'm sure whatsapp wanted a million users, but they didn't originally know that would happen. If whatsapp later turned out to be about serving fewer users with more features, this becomes a story about premature optimization.
Replace this with:
> with thousands of users just like me that once picked it as a dependency and then forgot about the project
You answered your own question. Because it's "exotic" and "hyped".
My fast reading brain with not enough caffeine read that as 'toxic'.
But the sort of people choosing a JS stack are quite quick to move on. Make a mess, slap a new achievement on CV and dash.
That doesn't mean it can't have issues like anything else. And because of the massive scale of the language you'll see a lot of shit. It's a language and ecosystem like anything else - it's what you do with it that matters.
Traditionally it was used to enhance HTML pages with some DOM manipulation but it's now being used for a million other things too.
So historically, if your JavaScript code broke your page would still render as it was just html but now there are all sorts of build processes and long chains that use JavaScript to create the html in the first place... I'd consider that the exotic bit.
Traditional JS running in the browser manipulating DOM elements is very much a foundational aspect of the web and won't deteriorate with age (with the exception of perhaps deprecated functions in the far-off future) but all these js libraries and tools and build processes with massive dependency-chains are what's being referred to.
Of course, you could decide to do something exotic with it (edge rendering, data programming, whatever) but that's not a problem with the language but people wanting to stretch themselves...
https://i.redd.it/h7nt4keyd7oy.jpg
Yes, the ecosystem is pretty mature, we have things like ESlint, TypeScript, Babel, TC39, and so on.
10 years ago I would not have touched JS unless I was forced.
I'd be interested to know which ones, because I can't think of one advantage that Javascript has - aside from the ubiquity of browsers. If we're comparing any other aspect of it though, I just don't know what advantage it has (seriously).
So what's the deal with TS? I think on top of adding much needed type safety to Javascript, TS is also one of the best type systems you can ask for.
> Functions are first class.
Always a good thing to have in a language but I struggle to think of a language that doesn't have this now. Java has it, C#, Python… if a language is being developed and didn't have it then now it does, right? Perhaps C doesn't - I checked, functions may be passed as arguments but does not allw nested functions, but it has callbacks so a type of closure is possible… hard to tell. Still, I'd say not using C was an advantage!:)
> Async i/o by default
I'm not sure which languages this gives an advantage over?
> Familiar syntax
I don't think this is an advantage, most languages are much of a muchness, and there are some serious footguns that still lay around in JS. Is `==` familiar? No, that's a footgun. Automatic semicolon insertion (ASI) rules? No, they're another footgun. Are fat arrow operators familiar? Doubtful. The new backticks? They're the other way round to what I'm used to.
Certainly though, it can be picked up quickly enough.
> There are packages for anything you can think of
This is definitely an advantage over some newer languages, but over those in major use, it's not. Still, the way packages are installed and handled is… eye-opening. That wheel has been reinvented several times, and it still looks wonky.
> I even like it's single threaded nature
Sometimes that's a good thing, true, but it's not an advantage to have it set that way all the time.
> TS is also one of the best type systems you can ask for
I haven't used it, tbh, but I would say that an optional type system (it's optional in that you don't have to use TS, I don't know if it's optional once you start using TS, that would be better) is a definite advantage over some languages. It's not baked in though so that's half a point.
I just don't see it. I think the driving force of the move to Javascript everywhere was because devs were tired of learning at least three languages (JS, something, SQL) and not being very good in all three (usually poor at SQL) and thinking they'd get more control by kicking out the DB guy who'd stop their bad ideas (because they didn't know SQL) and they could become an expert in one language to rule them all. Companies love it because of fungibility and more new devs tend to start with Javascript than anything else.
Perhaps I've just got used to keeping several languages in my head?
No one made a claim like that and your rudeness is uncalled for.
There's just no excuse for it.
> But, the way you act, must give the people around you a lot of entertainment so I'd certainly not want you to change!
You see, you can do it!
You described yourself as being "used to keeping several languages in [your] head" as if this was an achievement that others couldn't muster.
I still feel that your derision towards others and self-congratulatory tone were deserving of rudeness and snark.
Anyway, I apologise for being rude since it was hurtful. I thought your comment was bad and let you know this by directly accusing you of exactly what you were doing, instead of finding some non-confrontational way of saying it.
Whether or not it's an advantage of Javascript… I'm not sure. I mean, if someone wrote a transpiler for Java that overlayed stricter typing, would that really be an advantage of Java? Not sure, but of course, worth considering when choosing a language for a project.
Regardless which language you pick, there will always be these kinds of issues.
While you're not wrong, I do strongly feel it depends on the language and architecture chosen. I'm a Go developer these days, and there's a big mindset to e.g. avoid dependencies, and for those dependencies to avoid including even more dependencies, keeping things fairly lightweight and with a low 'attack surface'. I mean I personally wouldn't mind a stricter type system and native enum support and some other things, but for now, I enjoy how basic it is, whilst avoiding the footguns that C/C++ brings.
And also, React is complex, hardly anyone seems to know the internals. The API interface might be easy to learn, but "simple" is not something that you should label React with.
Pretty sure he means to set up the REST api in Express with a few simple routes.
JS is not complex, and pretty vanilla choice. It’s what people add on top of it that gives it a bad name. I’ll link to this guy so you can understand:
https://kentcdodds.com/blog/how-i-built-a-modern-website-in-...
^ This is the problem, not JavaScript.
Too bad I can't even remember the last time I jumped on a project with less than 80 dependencies.
If they tried to make a shared framework that ran the same codebase everywhere and then port it onto different platforms (cough facebook) then you don't get these nice isolated issues, for example.
All these requirements are like saying "having a good experience using this app requires an app that doesn't crash all the time". Well, duh?
For me, it takes between 10 hours (worst case) to 5 minutes (best case) to open WhatsApp.
My iPhone 4S is too old, so opening WhatsApp gives a message "This version of WhatsApp ha..., Update WhatsApp, Your update will be free of cha...". Clicking Update WhatsApp opens the App Store, and clicking Update results in another error message: "This application requires iOS 10.0 or later."
So I only use WhatsApp through BlueStacks Android emulator on my personal laptop. That's what leads to the 10 hour worst-case response time: a full working day, including commute. The 5 minute best-case is because BlueStacks takes a very long time to open, and runs my laptop fans really loud.
This is not only a pain point with WhatsApp, it's also a problem with WeChat, KakaoTalk, LINE, even 9gag (for posting videos).
Meanwhile, email continues to work just fine on my decade-old phone, mbasic.facebook.com is particularly speedy, iMessage still works, and Hacker News is great :)
https://faq.whatsapp.com/web/download-and-installation/how-t...
Docker itself can provide a stable backbone that allows you to run repeatable and predictable builds.
As for Express, JS on the backend will be popular as long as JS on the frontend is popular. JS backends have their flaws, but there’s a lot of value in using the same language in the browser and server.
This seems tendentious. Which average user was looking for these changes? Can you point to a site that shifted to an "app-like experience" and became successful because of it?
Barring informational sites like blogs and news publications, it’s actually more challenging to come up with new web products that are NOT app-like in nature, and that do not use any kind of JS framework. Craigslist may be one of the few big ones and even it is losing market share to FB marketplace, which is an app-like experience.
If I start with 1990s ebay, does it become app-like when I add the ability to zoom images without a pageload? When I add a WYSIWYG listing editor? When I let people drag and drop images into their listings? When I add JS infinite scrolling to search results? When I add AJAX search autocomplete?
Or do I have to go as far as Google Docs, re-implementing copy/paste functions, taking over the mouse wheel, and adding my own text highlighting and zoom implementations?
I do think the 'craze' of new frameworks popping up left and right quieted down some years ago, and things have roughly settled on React, Vue, maybe Angular (which lost a lot of its shine after the Angular 2 announcement fiasco), etc.
For a new application, I picked React + Go because I'm confident that ten years down the line it will still be maintainable - although, Go moreso than the front-end, which doesn't feel nearly as solid and stable.
What is (was?) problematic with Angular is the toxic leadership: https://medium.com/@jeffbcross/jeffs-letter-to-the-angular-t...
But there are great things happening. The wider community has solutions for everything. (For example here's the "reactive forms are not strongly typed" issue [0] that showcases both the good and the bad. The need for this feature has clearly emerged in 2016. People stepped up and a PR was created, but ... basically no signal from the Angular team. Of course using a wrapper was an easy workaround with a distinctly sour taste in the mouth. Then finally something happened and an Angular team member now seems to be working on it in "full steam ahead" mode.)
I recently had to maintain a large React + NestJS application and I was seriously considering organizing a terrorist cell to go back in time and ...
[0] https://github.com/angular/angular/issues/13721
I do admit the ecosystem needs to find a way to have more stability, because NPM outdated package tree nightmare is pretty bad, but I guess that's what you get with an industry that moves so fast, so in that case it's up to you as a developer to not use too many dependencies.
Anyway this is all to say I don't think JS is going anywhere for the next 10 years.
So the same cannot be said for "newer" languages like Rust, Go, and other's whose ecosystems and paradigms aren't completely fleshed out yet?
This comment reeks of someone who is looking from the outside in when it comes to building stuff in JS. Most comments I see criticizing JS stacks are so superficial and demeaning that's its obvious the commenters have little to no experience working in JS.
in the beginning, JS wasn't really transcoded/compiled. The first time i personally found out about that was sometime after 2010 with coffeescript, not sure if there were preceeding examples.
and your claim that people move around is really false. React has been the defacto standard for a pretty long time at this point.
yes, other UI libraries/frameworks exist, but reacts marketshare has been extremely dominant since it displaced jquery/ember etc
Most contention I have seen is around attempting to apply it outside the browser. That path only seems to lead to misery.
As in: there are still (older) projects around that haven't (yet) migrated away from Ant? Sure.
As in: Ant is still the go-to build system or at least still commonly used? No, not at all. When I create a new Java project in IntelliJ for example, I can choose between Maven and Gradle as the build system. Ant isn't even offered as an option.
The last 3-5 years have convinced me that JS isn't appropriate outside of a browser almost ever. I am sure if you think hard enough you can think of cases where it's superior to some other lang but in general it's a very poor choice for almost anything that isn't DOM manipulation.
Worst was definitely attempting to diagnose an off-heap memory leak due to C extensions. Naturally the JS folk gave up and dumped the problem on my lap so I proceeded to do my usual "C guy" stuff and was amazed at just how bad stuff like node-gyp and friends are and just how fragile everything is. I found the leak and patched it and all was "ok" again but just peeking inside those layers makes you deeply uncomfortable with the runtime in production.
The rest of the problems can probably be attributed to lower quality developers but point remains. Things like lack of structure leading to insane architectures, pushing for microservices without understanding the tradeoffs because they didn't want to work on "legacy" JS code that was built with last years hipster tech rather than this years, etc. These problems are endemic to to culture and ecosystem which IMO are inseparable from a language/tool in practice despite what we want to believe in theory.
I don't refuse to work with JS but I definitely make my concerns abundantly clear and I generally don't hold back with "I told you so" when it inevitably bites people in the ass.
Erlang has been released in 1986 and has been a well known language for message systems.
When I think about exotic I think about some new tech that sound really nice but is still very unproven.
It's okay not to enjoy JavaScript (I'd always prefer to go with Dart or Rust myself), it's okay thinking that other languages (and ecosystems) are better equipped to solve some problems, but calling it unproven and exotic is very surprising to me.
Is the market share for BSDs growing?
Even if the PlayStation is growing, I am not aware of BSD growing in desktop, server, embedded devices, etc.
In the 32 bit days it wasn’t hard to get most Linux binaries to run, now it’s virtually impossible for an end user to do on their own.
Also, Linux binary compatibility - the capability to run unmodified Linux binaries - works much better now than ever before.
See whilst obviously the JS ecosystem is quite volatile and can be trend-based, I think there are benefits - the language breeds innovation. Innovation doesn't always make the correct choices, and some tech-conservativism is important in many types of companies. But if our ancestors had just been lazy and just stuck with what was known and good, we'd never even had computers in the first place.
One is that the future is unknown. Your goal might be to keep the product simple and scale to 1bn users, but you don't really know if it will play out like this.
Maybe you plateau at 1m users with the initial idea and pivot the product somehow. Now the app does dating, social media or specialised communications for highway construction teams.
Tradeoffs are easier to make in hindsight. Whatsapp is a good demonstration of what you can gain with good trade offs. Do less but well. Whatsapp did trade some things off though. Their web version came late, is feature poor and a little clunky. Same for a lot of features, compared to similar apps ATT. IDK what exactly can be traced to the stack, but their decisions around user identity are similar. Phone numbers only. One device at a time. They gained simplicity, but lost options.
It's easy to see the benefits in hindsight. Real time is harder. Play that game again and it might turn out different. Most apps that set out to have 1bn never get close. At this point, scalability doesn't matter and tradeoffs made for maximum scalability don't seem as wise.
When you say, “Now the app does dating…” I think the right move is to scrap the project because that’s a colossal fuckup. Unless you have Microsoft cash to have multiple colossal fuckups in a row, don’t add another dating app to your messaging app. Ergo, less is more.
WhatsApp probably started with passion and a solid vision. The Zuckerberg gave them an offer they couldn’t refuse. Anyone will take 19 billion for a basket full of Indian users.
Another thing WhatsApp did well is they targeted ALL phones. They didn’t abandon their users like the fang-bangers (sorry been watching True Blood— FAANG is an annoying acronym so they are hereby fang-bangers).
That was clearly the killer "feature." Whatsapp is synonymous with communication on the developing world because of this. I remember when I was introduced to it fairly early on in Brazil and someone claiming that yeah, anyone no matter the OS could get on it. I couldn't believe it, to be honest, it felt like what iMessage was starting to look like... but for everyone? I can't imagine what it would've been like to support so many things, but clearly it was a lot of sweat that paid off extremely handsomely.
I'm not saying that philosophy is bad, just that reality is complicated.
"Scrap the project and move on" works in some contexts, not others. The way startups/products actually work, often, is evolutionary. If your texting idea didn't work, but you see a chance to pivot into something... are you really going to just fire everyone and tell investors "sorry?"
That said, the "one thing well" philosophy really does have big engineering advantages. You can't have everything. I'm just raising the "retrospectives" warning.
In any case, the "$1 per year" was never a real business model. They never even got around to actually charging it... because anything that limits the usership of a messaging app will sink it. It's the opposite of "support everything" strategy that made them successful.
"Sell to Zuck" was always the plan.
* Pivoting would leverage existing technology built in the process of initial concept. Which to me is the equivalent of scrapping the initial idea (while salvaging the generally-useful IP/technology).
* Adding a bunch of tangential features to a product to increase revenue is a colossal fuckup scenario (maybe the language is a bit over dramatic).
For instance, Google is great at search; gmail is cool; docs was innovative (albeit limited); and then… https://killedbygoogle.com/
Unfortunately, after seeing this time and time again it’s tough for me to get behind mainstream tech. I loved the old Microsoft/Nokia phones; and the Zune. You can tell a lot or love went into the design/engineering but then projects just get axed by corporate interests.
Meanwhile, you can by a mechanical device or appliance from 1950s and it’ll still work just fine.
Pivoting via "scrap & salvage" is pretty tough for a startup. The tech behind whatsapp was evidently good, but the IP behind a messaging app is probably not enough to give you an edge. Users are.
Made up scenario: whatsapp loses the SMS replacement game. They have millions of users, but not a billion. Meanwhile, they find that a subset of users like to use whatsapp for dating (or customer support, etc. doesn't matter). They pivot to focus on those customers, and evolve into something else.
This might be a (drama noted) colossal fuckup scenario in an engineering sense. A tractor that you are now converting into a ship.
Evolving is definitely a worse way of engineering than starting with the intention of designing a ship, with neatly defined tonnage, speed and size requirements. Instead, it takes a miracle to implement basic ship features like floating.
Evolving is how a lot of actual software gets invented. Spreadsheets were intended for accountants. They weren't meant to be used as a database, incident report generators, a casual programming environment, or a HR tool. It became those things by evolving.
It happened that way because inventing UIs is hard, and evolving into them happened to work. It's still true that "colossal fuckup scenarios" arise because of this approach. Excel programmer spent decades making excel better at things it's architecture wasn't good at. It's ugly and messy, but life is sometimes ugly and messy.
Flexibility is valuable. Knowing the spec in advance is valuable. Very valuable. They're in conflict with each other to some extent
True, but one can do simpler things with JavaScript, too.
And a word on exotic tech: Well, some of it is really good in helping small dev shops achieve scale of development, deployment, and maintenance.
Don't we all, I guess we all crave that snap, click, build -feeling Lego gave us. But in my experience building a system means debugging all kinds of weird problems. At least, when I start something, I usually for a long time feel like I'm just hopping from weird problem to weird problem. But, TBH, those problems don't seem weird anymore as experience grows.
You know that JavaScript developers don't see JavaScript as weird or exotic, right?
I'm not saying that it's an easy problem to solve, but it is a much easier problem to solve than a lot of other applications.
Sending messages to a known (small) number of people is a solved problem for a long time. Email for example.
With WhatsApp, it is even easier than email because WhatsApp is a closed system, and you can optimize.
They used an uncommon setup? Okay cool. How does this help me? or any other company?
I'm not sure what I should learn from this story?
The only thing is: Feature creep is bad.
That's not the case. Consider the "one tick" messages - they must be on a server somewhere. One tick means they've left your phone but aren't yet on (all) the recipients' phones
For example, I used to work for a company with about 500 employees (relatively large by German standards) and now work for a competing company in the same industry with less than 150 employees. The core engineering team only consists of a dozen engineers.
The output is noticeably larger. The number of new features and especially the stability is significantly better. It is noticeable that due to the small team, each project has one or two "heroes". They are responsible for most of the commits and know the system inside out.
These "heroes" have been working on the same or similar systems for over 30 years in some cases and have experienced a lot in their careers. As such, they can bring experience and advice that is just out of reach for others.
Given the choice, I would always prefer a small team over a large team.
Some companies spend a lot of time & energy trying to implement operating models that try to overcome this but actually end up putting in place more decision processes & oversight. Sometimes the real solution is to reduce control and let the efficient teams lead the way. But then what would all that management do to fill their days...
I've been in large companies where I was the admin/owner of their AWS account. And I've worked at companies with 200 people where I have to submit tickets to get anything created in AWS. The major difference was how mature the product was.
It’s also safer this way. Not even the proper owners should routinely be on production shells or tools with live write access. They also go through peer review with each other. Once the infrastructure is in place for that, it makes sense to open up.
From my experience in a deadline-constrained environment, things end up being slapped together until they "just work". Adding new features gets more and more difficult and if you're still constrained by deadlines, then the solution is to hire more, which is effectively a brute-force solution to meet the deadlines. Output per employee gets smaller, but total output should grow by some marginal amount.
As somebody else mentioned, these companies end up regularly throwing millions at consultant projects that always fail. In other words, they pay a hefty premium for shitty temp "heroes" that give them less than employee heroes would have given them.
You see this a lot in the traditional finance sector, where managers don't appreciate tech workers and relentlessly fuck themselves over trying to save a dime.
Every strength is a weakness, and every weakness is a strength. IME, heroes are no more a red flag than pretending that good engineers are interchangeable. It depends on the context.
The expected outcomes of these teams are different, and that’s OK. If you’re a very small company and don’t have a couple heroes, you won’t build anything important. If you’re a very big company, the heroes that built it left years ago, and you need resiliency more than new heroes (unless the business is going sideways and needs saving).
However, you can reduce the risk through good documentation, good engineering practices and so on. As long as you are aware that the team is partly made up of heroes, you can prepare accordingly. So as a manager, I can try to ensure that a few heroes keep productivity high while making sure that the rest of the team understands the basics of the system through things like code review and the like. In the worst case, a teammate can then at least get up to speed quickly.
Sometimes, the #1 goal is building a high-performance and/or high-quality product. Lack of team resilience, (temporary) downtime, risk of project failure due to an insufficient bus factor...they're all not good things, but they can be compromised in the pursuit of that goal.
If you want high resilience as well as high performance and also high quality, well, you should go work at NASA, and hope that the Senate works out their budget issues...
I think that your point stays valid there because reality is far far away from ideal case.
Oh, they can also blame the single point of failure for being stupid and lazy. That tends to work a few times before their boss catches on.
In my experience the opposite is true. My personal goal on any project is to make myself replaceable. There's nothing I find more tedious than having to work on the same thing for years because no one else can take over from me.
So you admit that you aren't replaceable? If the company mandated that you should always be replaceable then you wouldn't need that goal, it would always be fulfilled. And working in a way that makes you always replaceable isn't fun.
Edit: What we learned from your comment is that making yourself replaceable is fun, but being replaceable isn't fun, since as you say you work to become replaceable so you can start working on something new rather than to stay replaceable forever.
I'm sure I don't know how to achieve it though.
Or you just pay an order of magnitude on hosting, performance consults, bandwidth, etc. etc. etc. And STILL won't EVER get to the same result, because you can't always pay (I'd say almost never) your way into a stable and performant solution. Look at all the others.. Nothing works as well as whatsapp.
Good luck hiring 500 engineers who have no idea what happens throughout the whole stack.
I'd rather pay what you save on cloud providers and snowflakes to them. (not even taking into account the result of these "heroes".. the business value / dominance whatsapp has because of this)
You can hire 1000 people who can't draw, or maybe just a little bit, or you can pay an artist and get the result you'll otherwise never get.
Ehh, that's very debatable. Signal, Threema, and yes even Telegram (regardless of E2E enc, etc.) work well and are reliable.
i dont think its as easy as you make it sound. these "heroes" generally are as productive because they dont sync and already have a clear idea of what they want to achieve/how to implement what they wish.
if you add a second person like that to the same team you run a massive risk of having two competing ideas which creates a lot of friction.
Benefit of this is that these heroes can each break new grounds, and their BKM shared amongst themselves making the team extremely productive.
I’ve read this in Peter Thiel’s “Zero to one”.
I work as a contractor, thus I have read a lots of code from various different types of projects and the conclusion is always the same, if they had spent more money on skill to begin with I wouldn’t be needed.
I don't disagree with the rest of your post but Telegram does in fact work as well if not better than whatsapp.
I believe many managers are fine with this, because the team is producing something (i.e. "velocity" is higher than zero), but are oblivious to the fact that team could be way more productive if the knowledge was actually retained in people's heads and not just lousy attempts at documentation.
OP mentions that he is in Germany - there it might be more possible as SWE doesn't pay as well as it does in the USA and there are fewer companies so it might be feasible to pay a lot more than the competition, whereas in the USA that's unlikely to be viable outside of companies like Netflix etc.
Heroes are your most powerful asset, but you have to use them responsibly. The best thing you can do when handed a 10x unicorn developer is to try to document 100% of the things they say & do, and also make it a requirement that the hero mentor others some % of the week. E.g. For 1-2 hours every Friday you force them to hold a "no stupid questions" session.
When I think about a "hero" in terms of team dynamics, it's a person who is consistently relied upon to save everybody else's butt, often by doing things that are flashy but not particularly healthy, such as working long into the night to meet an unrealistic deadline. When you have this sort of Superman figure who's willing to swoop in and save the day, the problem isn't so much their level of experience but the unhealthy level of dependency that's placed on their shoulders.
I'm all for having highly skilled, highly experienced engineers who are productive themselves and can further increase productivity by helping unblock others on their team. And I agree with you that replacing them with some number of juniors without the same institutional knowledge can be disastrous. But if your team becomes so dependent on heroics that they can't stay afloat otherwise, then when that hero gets hit by a bus or just quits to take a position elsewhere, you're screwed.
I don't know why people insist on using examples like this instead of the more obvious one: What if they are hit by a bus?
Any of the other examples have trivial solutions in the event of an emergency: call them and maybe offer them some money. If they are very stubborn then go to their home and beat them with a wrench. But you cannot get information from a dead person no matter how much money or how big a wrench you have, and people die suddenly every single day. That's the worst case you need to be making contingencies for.
Having a "hero" at a 5 person startup can mean life when death was inevitable. Having that at a 500 person org likely means another team is left cleaning up crap.
I actually enjoy that sort of work, but didn't receive as many accolades as the guy "churning out features quickly", even though he'd break the build and block the entire company at least once a week before I paired up with him.
Once I saw how the sausage is made, I'm a lot more sceptical of the "10x" label, either there's an invisible support system, or the code base had a short half-life. Any other scenario is a set-up for disaster.
I think that's one of the conclusion of the "mythical man-month" book.
Often inversely correlated...
I quit after a few years, and about 3-4 years later they went on a hiring spree. Tripled the number of developers. Refreshed some products, discontinued some, and some went into maintenance mode. They still had around the same 10 websites, but 12 developers.
The products didn't get better. The products didn't get made any faster. They didn't have any less bugs. My wife actually works there now, and when she tells me about the issues they have I laugh a little bit.
It's all the same problems we had back then. Invites don't work. Teams doesn't work. Uploading profile pictures has issues. Entering data doesn't work. Registration is broken.
What's different about Whatsapp compared to a traditional XMPP experience is:
- it uses a single centralized server, without federating
- it uses phone numbers as identifiers instead of nicknames
- it has a tightly integrated (closed-source) client
So you can not only "build your own WhatsApp", but also federate with others doing the same, which is pretty cool and prevents a single actor from abusing the whole network.
[0] https://www.infoq.com/presentations/whatsapp-scalability/
Retrospectively that has always colored my perception of how relevant technical aspects are for a successful startup (i.e. not much). Marketing (and viral, user-capturing & monopolist practices) are (sadly) much more important.
That's why i'm very interested in human and political aspects of free software. Successful FLOSS projects usually have a strong community backing them, and strong connections between developers and the community so that the technology isn't too disconnected from practical needs and UX concerns.
I think non-profits and cooperatives building upon free-software solutions are a better alternative model. For example, framasoft.org (french NGO for libre culture/software) has been instrumental in developing new solutions like Peertube, and in kickstarting a new wave of hosting coops (chatons.org federation).
In the IM world specifically, i'm very interested about Snikket.im project. It aims to build a complete client/server distribution of XMPP software tailored for specific user expectations.
They may have hired people later on (though this entire article is kind of showing that no, they actually didn't hire that much technical people). But by the time they even hired the first Erlang engineer they had already won over the European market. And for the record it was not precisely due to the quality of either their Android or iOS android apps -- the J2ME and Symbian clients had quite a following and I would even bet were more used.
Original Whatsapp was a disaster and this did not prevent them from gaining marketshare. You could basically login as anyone just by having their phone number (which maps to their JID) and defeating their XMPP obfuscation layer (some XML encoding, which was easily done due to the easily RE Java clients).
It was only later on (2011ish?) that they started getting somewhat serious, which coincided with them, already swimming in users (and money?), starting to get more evil (with a more hardcore stance against 3rd party clients).
Sure, everyone out there isn't going to be on my server - but for the close friends I care about - it's been flawless.
Now guess which one they skipped
My understanding is that vulns in Whatsapp so far were client-side, but i'm interested if i missed some on the server side.
What could possibly go wrong? :)
Thanks for the links!
It always fascinates me how in the engineers' mind it reconciles that their work brings corporations they work for billions and yet they have to rent a tiny apartment for a substantial portion of their salary and otherwise lead a pretty average life. Then going to lavish offices witnessing how the money they could make use of is wasted on vanity.
People living someplace don't own it, and people working some field don't own it. Some "owner" owns the land and means of production and extracts value from people doing the actual work.
Yes, it's a deeply broken system. A slightly better system revolves around self-organized workers coop where everyone gets equal pay and there are no shares to hold (or everyone owns the same amount). An even better system abolishes money and private property so that people can live meaningful/useful lives without worrying about imaginary numbers ruining their entire existence.
Therefore, honestly I'm not really surprised to see 50 engineers capable of building a product around it.
When it came time to get an overview of the codebase being purchased. The Google Maps car navigation iOS team had over 200 engineers. The Waze iOS team had 2
I don't really believe that. I worked at Google and the frontend teams were usually just a couple of engineers per target client, I doubt any projects had 200 entire engineers just for a single target client. Not even big projects like gmail has nearly that many.
Edit: If I were to guess Google maps would have 200 engineers in total, or at least 200 figure would be the entire team of frontend engineers for every target client, and then many more working in other roles. 1 engineer per million users is a pretty good rule of thumb for Google products, and a large majority of those will work on backend systems.
That is: on hardware from seven years ago and older. Which is an eternity in tech. Since then we've seen incredible new hardware come out (EPYC 2 in 2018 for example).
One can wonder: up to how many MAU could a service scale today, on today's hardware, with a small team? And what about tomorrow's hardware...
This is fascinating.
What I'm saying is that for the general case of "scaling with few engineers", it seems to me that the tools available matter a lot more than the specifics of what hardware is available at the time. Launching an international service "from scratch" with a small team and a modest budget just wouldn't have been possible 20 years ago, but not because CPUs were too slow.
Team does X, does not achieve scale, nobody knows about it
Team does X, achieves scale, X is the new sliced bread
if you rewind some years microservices was also the word of the day (and probably there were some iconic projects picked to justify...)
The only thing they failed at was making money. I really wished they had. They knew what Whatsapp meant to the world. Look at WA now.
https://freebsdfoundation.blogspot.com/2014/11/freebsd-found...
Huh? They're all million/billionaires now. That part of the plan seems to have worked.
They also tried the paid version for a while if you remember. And they were also dead against ads on the app. Unless I missed something or if I am wrong somewhere (At which point I would love to be corrected), Selling was the plan all along doesn't seem like a right observation.
I'm not saying they are bad people, and selling is not a bad thing either. I'm just pointing out that selling was the plan all along.
The WhatsApp story that gets told regularly (like in this instance) is a story of how such few employees and such small infrastructure is enough to build a massive product. You know what that translates to? Low opex. If they wanted to, WhatsApp would've been quite profitable if they wanted but instead they chose to sell to the highest bidder.
Again, that's not a bad thing, but also, I'm sure there were PLENTY of people that were interested in buying a billion user platform with 99% user retention. Why did they sold to Facebook? Only they know, but back then Facebook already had a tarnished reputation, so they definitely didn't do it "because of their mission and values".
There is another chance that FB was seen as the lesser evil compared to Google then, who was also in the bidding for Whatsapp IIRC. My younger self would've chosen FB instead of Google for sure.
Not to mention, people wouldn't have guessed how horrible FB would turn out to be.
Edit: Funnily enough, after I wrote this comment, It became clear that money was definitely a factor.
IIRC they had 450M users when they sold to FB. Their plan was to charge $1/user/yr. A $450M ARR company run by 50 people would have done fine.
Since "monolith" has become a dirty word, it would be instructive to remember that some of the major companies out there are still light on distributed systems, which we all knew even back in the olden days of the internet were an absolute killer of efficiency and productivity. Yes, "microservices" is just another word for it - believe it or not, there was life before Node and Docker.
Whatsapp, Dropbox, Instagram, StackOverflow - all existed or still exist as one easy-to-maintain application. But go ahead, "be like Google".
Messaging is a weird app to focus on mau/dau. There have been many messaging protocols which scale to hundreds of millions of users, and there is prior art of large not particularly valuable chat systems like aim, gchat etc.
No one is going to use a consumer messaging system which charges money or shows ads. Which brings to mind substantial questions on the business model others are using the messaging platform for e.g. data collection.