I can assure that your experience isn't representative; probably not in "big" enough spaces. It's deeply ironic, but the bigger you go, the more you can easily see luck as a factor (e.g. daddy's money)
The hard work I'm referring to is all the effort to get to place where you have even be "lucky" or serendipitous.
For example, professional networking is a tremendous amount of work. But in meeting, bonding and potentially working with those people opens so many possibilities for serendipity / luck what have you. Sure there are those with Daddy's money, but more importantly the relationships, networks and resources that Daddy has are more important than just the cash.
Nepotism, sure, but that's how the society has pretty much always worked.
Working hard works better if you are lucky in the first place. You need to work much harder than the lucky ones to leverage opportunities if you were born in a third world country, or when you are physically disabled, or simply if your parents aren't wealthy.
I increasingly think idea is much much less than 1%. For whatever crazy idea, there are at least 100 people who have thought the exact same idea and 10 people actually tried the one.
This article leaves out the big "D": Distribution.
Google was a far superior product at the time they launched. They captured lightning in a bottle and participated in - and accelerated - a big inflection point of internet adoption; the web went from nerds to mainstream as the younger, web-native generation grew up.
The technology is relevant insofar as it gave them an insurmountable lead at the time they had it. Re-creating the initial search technology itself pales in comparison to re-creating the initial conditions that allowed Google to reach millions, and then billions, of internet users in a relatively short timeframe.
I worked for InstallShield for a while, and had customers routinely say, "I could build a better installer in a weekend than InstallScript / Windows Installer / Whatever". I always just sort of rolled my eyes, due to the complexity I was aware of across the (many) features of any given installation engine.
It's a long story as to why, but I actually ended up building an installer engine in Powershell some years later. I can now officially say all those people are morons, and you definitely should not in this day and age be rolling your own installer engine.
Maintaining state for uninstall is a big one which comes to mind.
Reference counting (which could sort of be rolled into the above item), to make sure you're not removing something someone else is using, if you install stuff to a shared location.
Silent installation. Yes, this is an afterthought for a lot of devs, but a lot of techs expect it for rolling out software.
Extensibility / Customization at install time. Also ties into the previous one (how to configure the install with no UI running).
Resume-after-reboot functionality, to update files which may have been locked.
Publishing accurate information into Add/Remove Programs (or "Programs and Features" as they call it these days).
Handling elevation in cases where the user isn't Admin (or post Vista - the installer is launched under the non-admin token).
"Repair" functionality, which requires maintaining the currently installed state and configuration.
Support for packing up and deploying redistributables (maybe even download support).
You start adding all that stuff up, and you realize - you can't bang that out in a weekend. It's a LOT of work actually.
Ooh, I'm just going to jump on this chance, because I wondered for ages:
What is "repair" for?
What exactly does it do, and what's the intended use case?
I rarely do much on Windows these days, but my impression was that the age of random applications dumping DLL and OCX files into system32 was fortunately long over, so programs stepping over each other should be almost nonexistent at this point.
It's basically a reinstall where the user's configuration and preferences are preserved. As to why - perhaps a file got corrupted (rare, these days). But often what happens is another program overwrites a shared file to one that is incompatible with your application. Or the user watches a video that tells them about this secret hack that makes your computer run faster...
Story time regarding DLL naming from the old days - Windows 95 would only keep one copy of a DLL in memory (the days when having 8mb of RAM was amazing). We had developed an application that worked perfectly, except on this one salesman's laptop - where it would only work occasionally. What I found was they had installed "Tiger Woods Golf" on their machine and it had a DLL with the exact same name as ours. If the salesman played the game before running our application, Windows said "Oh, I already have that DLL. No need to load it again." and our app would blow up.
* Getting real deep with detecting OS/instruction set and edge cases
* Constantly validating permissions on every directory and file
* Constantly verifying checksums on everything put in place
* Concurrency controls to make sure the user didn't launch the installer twice, or the system wasn't live running when being reinstalled.
* Dependency verification was it's own rats nest of problems
* Uninstalling
Easier problems:
* Logging
* Status tracking (except for really large files things get weird...)
* Aborting/Cancelling install
Logging is easier - but deceptively so! Most problems have hidden depth, and logging is certainly a great example.
So - do you want to log variables for debugging purposes once installers get to a certain level of complexity? Great. Now you're logging people's usernames and passwords, and you have to add some functionality to not do that.
One time I discovered the hard way why you need to exclude A: and B: drives when scanning for disks for a new installation; our demo disc would hang on any computer old enough to have a floppy drive.
Turns out many players still had floppies. Whoops.
I only put "correct" in scare quotes because it is kinda lame that this is still required. When was the last time anyone was happy that they got an "Abort, Retry or Cancel" dialog?
I’m guessing this was either after the early 2000s or your software was not available in Japan.
In the ‘80s and ‘90s one of the most popular PCs in Japan was the NEC PC-9800 series. These were x86 machines but were not IBM PC compatible. MS-DOS and Windows supported them, but didn’t hide all the differences from IBM PC compatibles from apps.
In particular, the boot drive was A: so a system with two hard drives and a floppy would usually have A: and B: be hard drives and C: be floppy.
That really confused a lot installers that assumed hard drives always started at C: and that A: and B: were always floppies if they existed.
Why use an installer at all in this day and age? A lot can be customized in the browser nowadays before download. Or tweaked after download/execution. Or use a build script (if technical audience). Or just isn't needed.
I think the last installer I used was probably the Linux Mint/Ubuntu live CD installer?, But other than that I don't think I've seen an installer for a couple years. Albeit I'm in the Linux ecosystem now so maybe that's why
Are you talking about OS installer? Cause this is application installer. You likely used an installer today but didn't notice it.
The model on Windows is that you ran installer program that installed the application. Installers had to do a lot more stuff than drop the application in the right place like change registry entries or write files in other place. InstallShield wrote the installation engine and build tool.
Windows moved to model of .msi packages with .exe wrapped. InstallShield turned into an .msi build tool. Macos has .pkg installer for things that can install .app. Linux has .dpkg or .rpm. All of them have the installer pre-installed.
I'm just talking about an installer in general, a wizard handling user customization and environment specific casing. Seems like the only place you might want something like that in the modern age would be an OS installer. Except I'm less familiar with the Windows ecosystem nowadays
raises hand Similar only we created a Java Swing GUI since it was multiplatform.... So you'd install the JRE (it's multi-platform right?), then you'd install Postgres (what?) then run our installer to put code where it needed to go and configure the database and application settings.
The testing cycle was the biggest pain, this was right as VirtualBox had become a thing so at least starting with a fresh system was slightly easier. (The entire company had 4 engineers + some researchers).
I think I was two years post-college as were most of my other colleagues... I'm fairly certain that product never made it out of initial trial (I left the company before it finished).
The install of postgres was relatively trivial, it did however raise many licensing questions and I don't think we even had a lawyer on retainer to talk through it. This also made it weird because of license reasons, we couldn't actually download and agree to the license because that would be our company 'selling postgres' as part of the package so we needed the client to do that step.
I can't remember exactly why HSQL/H2 wasn't used but I think it had to do with fault tolerance and redundancy and what we could do with postgres replication was preferred over what we would have otherwise used SQLite for.
If the application is a single .exe or something that works fine when you unzip a .zip wherever, then the installer functionality needed is almost trivial.
My limited experience is that most complexity in installers comes from the system not helping making it comfortable. Like something not having a simple to use version number, but having to check for 5 registry keys instead, or some such.
With all due respect, I was involved in advanced Windows projects and always thought that InstallShield was s** and beyond its complexity it has an advantage because of business reasons more than technical ones: https://en.wikipedia.org/wiki/InstallShield
Microsoft promoted it strongly, and mysteriously they never come up with a decent installation support, even for basic desktop apps.
Answering the topic here: it is the business more than the product. With exceptions, as usual.
InstallShield was the only game in town at first. So it was amazing from that perspective.
Windows installer came along and was a lot better. The InstallScript engine carried with it a lot of legacy tech-startup technical debt.
Fun fact: InstallShield became a general purpose packaging tool at some point which spit out *.msi packages (and had a bunch of extensions to Windows Installer engine via custom actions). So it was pretty awesome from that perspective.
In this day and age Windows Installer itself has a lot of legacy design cruft from the era it was designed for - the late 90's.
So you're not wrong, but based on the era InstallShield was more or less crappy depending.
I would like to add that was incredible that MSI were the installation standard and Visual Studio, which is the core of MS development, couldn't generate them! Yes, you can generate vanilla MSIs but are almost never what you want except for trivial installations.
Sorry, but I want to add one more thing: the upgrade logic of MSIs is completely wrong, you need a lot of tricks to do an upgrade.
Upgrades were not that bad if you had a document in one place that explained how they worked (MSFT did not have this concentration of documentation - they are not the best at knowledge management).
Patches though... Fuck. I think I could still produce a document that explains everything you need to know about patching but it is a lot more complicated than regular upgrades.
At the same time, my career has moved to Cloud stuff now. Is my life better? Definitely not.
I've written some installers using Installshield and later the WiX toolset, and the typical application developer has no appreciation on how difficult it is. "Just copy over the binaries, it's not hard."
Well, when your customer is an enterprise who wants to deploy your product hands-off onto five thousand machines, and expects a clean uninstall when they want it removed later - yes, it's that hard. Writing installers really is it's own sub-speciality in software development.
Post 2008 or so, there was a very good question of why anyone would use anything other than *.msi on windows. But most Installshield users by then were building those packages with InstallShield by then.
These days, lightweight apps and dev tools are still preferred to use *.msi or *.exe based formats instead of windows store apps. Partly because it is quick and dirty to do so, and cheap using things like WIX and NSIS. But InstallShield still has a legacy base among less sophisticated users; it is easier than WIX and more robust than NSIS and the like.
Windows Store package formats are superior in most ways.
And yet for some reason pretty much all the software I install on my mac is literally "drag and drop into applications"?
Or homebrew which is really just dependency management + standardized directories for files which is what it should be. Installshield was a solution for an entirely artificial problem.
There are many tradeoffs the major OS's made over the years.
There is a reason Mac had such tiny market share for so long, especially among enterprises. Installation was among pros and cons, but I would advise you to come off your high horse and understand the rest of it. Some reasons technical, some not. For a long time enterprises shunned Mac for extremely pragmatic reasons.
I've never had a Mac and have no particular experience with it, what are the extremely pragmatic reasons? I know there are a few like HFS+ only allowing one file to be read at a time under-the-hood, but I doubt minor performance issues are the key reason for enterprise choices.
These is as generic a response as the article itself, and is just as unhelpful since any of the two extremes fit: it could very well be that those tradeoffs were mostly bad decisions, or it could also be the only proper way to do it, and Mac's easy way is still wrong
It used an *.xml file to store all the things which needed to happen to setup the system. File copies, registry entries, installing dependencies of various types, running scripts, configure windows features, IIS sites and directories and app pools and what-have-you, some other misc. stuff I forget now. Also would restart the computer if a restart return code was received, and would provision a user it would set up for auto-logon to resume install progress after the restart.
Each type of entry in the config file had a module which knew:
- How to check if the entry was already done and in a good state
- How to uninstall cleanly the thing (not actually needed; this was for system imaging)
- Usage of variables that would dynamically configure the system
- Variables also could conditionalize what gets installed
At the end of the day it was somewhere between a true installer engine and Powershell DSC (DSC would come about a few years later). This installer is probably no longer in use, as you can imagine more robust ways of solving this need exist now.
It was a heck of a lot of fun to build and watch it working though!
Writing a general install engine like that is indeed hard. If that’s what people claimed they could do in a weekend more than an eye roll would be justified, such as hails of laughter. Such a reaction would be justified even if they said they could do it in a month or two.
However, if someone just claimed they could write a better installer for one specific program of theirs only that was better for that one specific program than an installer built with one of those general engines, that is not so unreasonable except for the in a weekend part.
I once worked for a DOS and Windows utility software company that wrote an installer for each product, and I was usually the guy that ended up doing the installer and it wasn’t difficult. If however they had decided to try to make an installer engine product for other companies to use I would have worked hard to not get assigned to that project.
I worked for a company that was making windows software when I was 17, and writing an installer for one of their apps was given to me as a small project.
Obviously he's right on this, at least mostly right. The zero interest era fueled a VC-dominated boom where headcount was erroneously used as a proxy for scale by a lot of places. And I think the dream of a lot of people saying "let's build a better twitter" or whatever is assuming a utopian ideal where that "better twitter" is not chasing a multi-billion dollar valuation but rather trying to sustain itself for public benefit. And really, that's not crazy since Wikimedia Foundation claims 220 FTEs running one of the world's most vital services and doing so based purely on donations. That leads to other reason build a better twitter/google/facebook is so hard and that is acquiring and retaining the millions of users needed to make your platform valuable. Even a perfect product can't guarantee that will happen.
I often wonder this too, but... I think making "another Google" is not the best example.
I think it's hard to argue (although some people clearly did) that making a search engine is easy. You need to index all of Internet, cache it maybe too, rank stuff, filter out spam, work over different languages, do something about images, parse a lot of (broken) HTML, maybe run some js to figure out what a dynamic page should show, then serve it up super quickly... google is faster than pinging a local (!) NFS server at my work.
But then I remember a comparison on HN about how booking.com is O(50) people and Airbnb is O(2000) total. Or I remember the total headcounts at Twitter, an app that lets you post very short messages.
Or LinkedIn, a vast company with a very subpar UX, basically an inferior clone of a dozen of other social networking sites, only kept afloat by its network/monopoly effect. I mean, searching for people by name barely works! You're better off using Google to look for people on LI that its own search box!
No doubt it's just my cognitive limitation, like I can't visualize the vastness of the Cosmos, but this is a close second.
Making a search engine that is good is actually a reasonable task as shown here on HN. It doesn't seem to take very many people, either. Google has set the bar really low.
By contrast, making a search engine that is monetizable and good seems to be an impossible task.
> I often wonder this too, but... I think making "another Google" is not the best example.
100% agreed.
They really should have used Twitter as an example. Any competent full-stack developer could probably have a working clone of Twitter running in a weekend.
The challenge I think is scaling it to millions of daily active users.
But, even then, Twitter has been around for over 10 years. The scaling problem seems to have been solved. I would think that these days, Twitter engineering would just be a whack-a-mole of odd obscure bugs. I can't think of any useful new features it's gotten in the last two years. Why do they still need so many engineers?
This is one of those _things_ that I suspect causes many greybeards to grimace.
What's a tweet? We can haggle this up and down so cut me some slack. 200 bytes? 50 bytes? gzipped -90%, 5 bytes.
Holy shit what absurd amount of traffic could a 1gb fiber connection handle?
oh huh 200 million tweets a second. That's silly. How wrong could I be? -- then you run down the numbers and you realize that no matter how wrong you are, a twitter-clone-for-most-people oughta be cheap. The network effects and fuzzy human reasons are their moat, nothing else.
One of those fuzzy human reasons is how amazon EC2 eats your ass. Just gobbles it from stem to stern.
They have pictures and video now. You also have to sort them in real time, check for blocked, mutes, etc, and each tweet is amplified to followers so one tweet requires e.g. 700 copies or 1M copies depending on follower count.
The text part is basically solved at this point, but not quite that easy. I hear dealing with blocks is one of the slow parts.
HN doesn't host pictures or video. I come here well before I do twitter, reddit, facebook, youtube or anywhere else on the internet commons.
I hazard there's a verdant, halcyon prairie between the two zones. The common zone of maximum extraction, and the hypothetical 200$-a-month-operating-costs text only twitter that can service 5% of the entire planet.
When I look at Twitter, I think first about the business model, and then I think about the app. I think, today, it's a lot easier to build Twitter the user facing app, even at scale.
Where I see your engineering headcount blow up is in moderation, legal and advertising; which I think everyone skips over when doing their mental math. Campaign management, targeting, reporting, and optimization on such a large corpus of data is where I struggle to see even a team of 5 building that in a couple of weekends.
OK, but you aren’t just pushing raw ASCII across the wire, you’re getting those results out of a database.
For a weekend clone, someone will probably just use an RDBMS (a very reasonable choice). I’ll be generous and say they can retrieve a given tweet in sub-msec time, say 10 usec. Suddenly, you can only send 100,000 tweets/sec, ignoring all other overhead (which of course you can’t).
I’m sure there is unoptimized code and some unnecessary bloat, but fundamentally there are limits on data retrieval that are very, very hard to get close to.
It’s not about features, it’s about their algorithms.
Think about TikTok: It’s essentially just an app that lets you scroll through videos, yet it became breathtakingly popular just due to the fact that they seem to know exactly which videos you may want to see.
Also, for Twitter, it may be easy to clone the "public facing" product which is basically a list of filtered tweets from the DB with probably some tweaks for the scale.
But the point is that this product is not what makes the company money.
If you only consider this, sure it’s a easy to clone product, backoffice for the advertisers, algorithms to maximize engagement, external integrations, marketing, PRs, partnerships are the thing that brings money home.
Of course if there was no Twitter, I imagine you could earn some money/be profitable with just the public product and some basic backend for ads publishers. But you’ll probably encounter a revenue bottleneck and start researching fresh income which you can do by … becoming Twitter, the big company.
That appears to be an iceberg problem. While the public UI of Twitter is simple and is often used as the basis of a starter project in training courses, there's also likely to be a lot of backend admin tools for things like moderation, advertising, sales, etc (not to mention as others have the scaling: at this level it's not so much a "solved" problem but one that requires constant monitoring, tweaking, and preventing newly-minted CEOs from literally pulling the plug on running servers).
I never claimed to be a competent full-stack web developer.
I'm an AppSec engineer. I don't write a lot of code, and when I do, it's entirely scripts and occasionally modifying existing code (both front- and back-end, but more often back-end) to implement security fixes.
That said, I could probably hobble it together. It'd be ugly as sin with hand-written Web 1.0 style HTML and likely server-side rendering since I hate writing front-end code, but it really wouldn't be hard as long as you're keeping it simple - Just text posts (No video/pictures, though maybe that wouldn't be too hard to add?), no algorithm for engagement (just chronological ordered tweets), no analytics to determine what's trending. Obviously no mobile app.
The NSA actually created clones of Twitter, Facebook, and ironically enough, tumblr back in maybe 2014 or so? In order to be able to deploy classified versions on the JWICS network that people could use for cross-agency discussion that can't happen on the Internet. I don't think it took them much more than a weekend to do all three and they're maintained by a only a few people working in their spare time.
The obvious differences being the scale, not really having to worry about the complexities of running your own data centers and overlay networks or whatever, and the fact there is no advertising, no sales, so none of the clutter that comes from trying to optimize that as well as defeat your user's attempts to subvert being tracked, nor any tracking of what you're doing off-site.
The signup process is really nice, though. You don't need individual accounts on each site. They pull your identity automatically from your IC PKI client cert and create the account for you when you first use it.
I work from home now, so I'm not on there any more, but they called the Twitter clone "eChirp" and the Facebook one "Tapioca." They also clone Wikipedia and Stack Overflow, but those are just read-only mirrors that get updated once a day.
Cool. Now, how are you going to report CSAM to NCMEC, or human trafficking to law enforcement as per Sesta-Fosta?
And then how are you going to ensure you don't provide content rabbit holes that can be harmful to mental health. Then detect credible cases of people threatening suicide to get a welfare check run.
That's just one of the things you don't see on the surface, there are hundreds more functions like that, that you'll very quickly realise you need to do so you don't get fined out of existence and/or end up with execs going to jail.
I have no idea about true inner workings of LinkedIn but the surface of LinkedIn you and I use is not the entire product and not the part that generates revenue. Off the top of my bead, they offer HR/recruiting capabilities, training capabilities, etc. They have a sales force around those, etc.
That's not to say they are or aren't staffed right, just that the part we are conscious of daily isn't the entire thing.
That's nonsense. Booking.com with it's subsidiaries was about 18000 or so people when I left in 2017. About 1800 of that were in engineering The vast majority of staff were working either on the supply or customer service side.
I don't know about booking.com, but when Facebook acquired Instagram for $1B they only had 13 employees. WhatsApp had 55 employees when they were bought (again by Facebook) for $19B.
> then I remember a comparison on HN about how booking.com is O(50) people and Airbnb is O(2000) total
One thing to ask yourself - is the company chasing growth? A company like AirBnB or Twitter could absolutely run on a much smaller team if they agreed to mostly abandon growth. They could focus on reducing tech debt, make architectural decisions that reduced flexibility in the name of reliability and lower maintenance, and then eventually lay off a huge portion of their staff.
Instead they have a growth thesis - that growth requires investment, having to favor velocity over unit economics, etc. The higher the growth target, the more inefficiencies you take on.
> "big company MUST have lots of valid underlying work/complexity/effort"
Almost 99% of the cases this is true. The question is whether this complexity is actually valuable or just accidental one accrued from path dependency.
Lots of works are actually valid in that if someone doesn't do their job then those big corps will lose more money than their salary. Of course, the whole situation is far from the optimal and they could make it much more efficient with a magic wand but usually we don't have that so it takes years to improve the situation.
Of course, someone (like Elon) might believe that they have such magic wand, but Twitter is losing more money than ever so probably it was not the case...
This is getting towards the topic of bullshit jobs. (I'm going to rip off Graber's book here.) There are several "invalid jobs", including:
Flunkies - people hired to make their boss look important, because said boss has a lot of staff in their department. Their purpose is ultimately managerial infighting, which is zero-sum.
Duct tapers - people whose job is to inefficiently work around problems caused by solvable social issues, e.g. a person whose job is printing out emails because $EXEC is old and refuses to learn how computers work.
Box-tickers - people whose job isn't to do something, but to be seen doing something, so that company can claim they're trying to do X when they don't give a shit about actually accomplishing X.
I'm skipping the others, because I don't understand them well enough to explain them in my own words.
The author took probably the dumbest and most egregious instance of this ("I can replace google with lucene!") and completely sidesteps the legit discussion that often happens in the face of this absurdity. Why does it take hundreds of facebook engineers to make instagram completely nonfunctional, when the original company had created a nice product with a dozen engineers? Seems like the answer is in the ulterior motives of the company, and the centralization and decline of the internet in general. But it still feels absolutely absurd on the outside looking in and the underlying reasons are somewhat interesting, though usually not technical.
There are reasonable discussions of this sort, but the example OP goes on about is kind of a straw man. Just because anyone can go on the internet and say anything ("I'll build google in a weekend!") doesn't mean it needs a whole rebuttal that takes it seriously.
The answer for IG seems somewhat easy. It takes as much work for a UX designer to design the French translation as the [very minor language] translation. Or financial disclosure requirements in Germany vs Andorra. Each employee serves fewer people (assuming you start in the biggest regions).
The same logic applies to bugs on more and more obscure setups, or more and more niche features.
It takes raw talent or many hands to make something completely nonfunctional. With a small team someone might accidentally do something coherent. In a large team there’s no danger of that. Unless one or two people somehow wrangle control of the whole thing.
Building something _different_ faster/cheaper than a competitor is not really the same as rebuilding the same thing with a fraction of the people. In the past when I've seen the claim "I could rebuild this in a weekend" it's literally calling into question why so many engineers were needed to build _that thing_, not something else.
More specifically, new entrants (and especially so in new categories) regularly get the luxury of extended time to iron out issues, create a better UX, find all the edge cases, invest in internationalization, distribute to multiple platforms, integrate with other systems, address security issues, and more. These tend to be exactly the kind of things that make recreating the incumbent products hard.
If OpenAI continues on its current trajectory, it can easily end up with a team equal to or larger in size than all of Google working on search.
> OpenAI has made a better product than Google with less time and engineers
Based on the theoretical foundation from Google, the framework built by Facebook, hardware produced by NVIDIA and cloud infrastructure run by Microsoft? This doesn't make a great example.
Doesn’t that apply for every software product? Google didn’t start out by designing their own hardware, building their own datacentres, and implementing their own frameworks did they? Those things come later for optimisation.
Is this sarcasm? Twitter has been less stable and has been making more dumb choices since Elon took over, and they've lost at least half of their revenue so far. They've lost literal billions.
I don’t think that’s fair. Twitter was already operating at a loss and wasn’t going to win at performance advertising. A change in business strategy to something like payments seems reasonable for the long term.
Twitter lost advertising because its a bad outlook, not because of technical issues.
Twitter still works. Sure, it has issues, but still works. It definitely was a bloated company, and if Elon could keep his autism under reigns, he could have came out with reducing the company, keeping the same advertising income, and being more cash positive.
> if Elon could keep his autism under reigns, he could have came out with reducing the company, keeping the same advertising income, and being more cash positive
That's all your imagination - we can make and support any claim that way. The actual facts we have point the other way.
Let's stop making Musk look like a victim. Nobody blames autism when Tesla's sales increase.
It is very possible, in fact likely, that gutting a ton of people directly involved in ad-related work - sales, press management, content monitoring, etc - had a negative impact on advertising revenue.
Yes, but that doesn't preclude the fact that Twitter had 7500 employees prior to layoffs, without anything really to show for it. He could have still trimmed a lot, while keeping the relevant people in place.
Hasn't revenue dropped by something like 50%? It's reputation is in tatters - it's an example of what not to do. Hasn't Musk lost a ton of money on it?
It's too early to say whether this is an example of what not to do.
Elon hasn't realized any losses yet, while on paper the company may be worth less that game isn't over for him.
He hasn't done himself many favors by stirring up controversy with his own statements, but it isn't surprising that revenue for an ad-based business takes a hit when the brand is raked through the coals by a large enough group of people making noise. That could be one step towards it failing, or it could be one of many dips while the company is not rebranding itself and changing their business model.
The first moat is usually capital expenses. Look at AWS - they're built mainly on open source software so it is obviously trivial to re-implement a lot of their core functionality. But it is hard to compete with the sheer amount of capital.
The software aspects make a big difference too, but it is actually quite easy to re-implement software. Something like git for example was built rapidly (3 months?) after a few years of using other VCS to learn what it needed to do.
In my experience, there's a huge difference between "software that works," and "software that ships."
Most of my test harnesses are full-fat applications. They are written in a ship fashion, but they don't have the polish and TLC that I lavish on the ship stuff.
I can write a test harness app in a day or two. It will be a fully-functional, accessible, localizable app, with a non-trivial feature set.
However, if I write something to ship, it may take a couple of months (or longer).
An example is the app I'm writing now. I wrote the "heart" library in about a month and a half. It has a test harness that runs every feature, and the test harness was the largest part of the job.
However, the ship app is coming to beta, about a year later. It knocks the test harness into a cocked hat.
There is definitely an aspect of raw time that needs to be put into development to make a prod version of software robust, but that still doesn't account for the bloat.
The real issue for the bloat is simple lack of engineering skill. Most "engineers" have very little clue of what goes on under the hood when their code is running, and really only rely on learned patterns to complete their job. So many times I have been given code for a service that when run locally doesn't work, with the explanation that you have to test it in beta stage - the reason this happens is because nobody who started writing that code has any idea of what goes on outside of the boilerplate that is provided for them to fill in.
Have you ever seen a modernist piece of art sell for a bunch of money? There's almost always someone who offers "I could do that".
But they cannot. And they will not. It is a statement made to demean others and elevate the self.
I'm often reminded of a famous anti-Dropbox comment somewhere, but a lot of the time it's about the right product at the right time with the right marketing.
I think people on both sides of this debate are being obtuse. As we've seen from the rounds and rounds of layoffs over the last 18 months, most big companies were (probably) bigger than they needed to be to function (TBD on X). However it's silly to point to something like Instagram and think "they built something great with 12 ppl, what are all these other thousands of people even doing?" An app used by 1B+ people is more than 10X more complicated than an app that is used by 100M people.
> This reminds me of a common fallacy we see in unreliable systems, where people build the happy path with the idea that the happy path is the “real” work, and that error handling can be tacked on later. For reliable systems, error handling is more work than the happy path. The same thing is true for large services -- all of this stuff that people don't think of as “real” work is more work than the core service.
I guess the simple answer would be: No, first seek to understand.
Given this principle, we would mostly realize that even if it was that simple, the business is still years ahead because those people don't just come to work to drink coffee.
However, I'd like to point out that there are rare cases when some newcomers can replace existing businesses because they found some technology that simply replaces years of work.
When Cuil [1] did a search engine back in 2008-2010, one of the ex-Google people there told me that the core team that did the Google search engine was under 100 people. It's become more complex since then, of course, but the core search engine isn't using up most of Google's work force.
If you did it today, you'd probably want a fairly literal-minded search engine front-ended by a large language model to make it more user-friendly. That might be easier than trying to make the core search engine appear semi-intelligent.
I don't know if "add engineers until the cost of adding an engineer equals the revenue gain plus the cost savings at the margin" is a good idea. A lot of companies did that before/during the pandemic, and after that they had to layoff thousands of people. Under the assumption that your market will always increase it makes sense tho, but that's a huge assumption.
142 comments
[ 4.2 ms ] story [ 133 ms ] threadThe difference between being able and doing is infinite.
10% idea, 90% execution. Writing code over a weekend is the “idea” only. The rest of the work takes years of disciplined execution.
The world is full of people that worked really, really hard on something and failed to achieve the specific goal they set out for.
Success is complicated.
For example, professional networking is a tremendous amount of work. But in meeting, bonding and potentially working with those people opens so many possibilities for serendipity / luck what have you. Sure there are those with Daddy's money, but more importantly the relationships, networks and resources that Daddy has are more important than just the cash.
Nepotism, sure, but that's how the society has pretty much always worked.
Hindsight is 20/20. And there's plenty more to product than the build.
Put another way, and I say this often: "Making it look easy is very very hard."
If only more fools understood this, there'd be less fools.
Google was a far superior product at the time they launched. They captured lightning in a bottle and participated in - and accelerated - a big inflection point of internet adoption; the web went from nerds to mainstream as the younger, web-native generation grew up.
The technology is relevant insofar as it gave them an insurmountable lead at the time they had it. Re-creating the initial search technology itself pales in comparison to re-creating the initial conditions that allowed Google to reach millions, and then billions, of internet users in a relatively short timeframe.
It's a long story as to why, but I actually ended up building an installer engine in Powershell some years later. I can now officially say all those people are morons, and you definitely should not in this day and age be rolling your own installer engine.
Reference counting (which could sort of be rolled into the above item), to make sure you're not removing something someone else is using, if you install stuff to a shared location.
Silent installation. Yes, this is an afterthought for a lot of devs, but a lot of techs expect it for rolling out software.
Extensibility / Customization at install time. Also ties into the previous one (how to configure the install with no UI running).
Resume-after-reboot functionality, to update files which may have been locked.
Publishing accurate information into Add/Remove Programs (or "Programs and Features" as they call it these days).
Handling elevation in cases where the user isn't Admin (or post Vista - the installer is launched under the non-admin token).
"Repair" functionality, which requires maintaining the currently installed state and configuration.
Support for packing up and deploying redistributables (maybe even download support).
You start adding all that stuff up, and you realize - you can't bang that out in a weekend. It's a LOT of work actually.
https://groups.google.com/g/alt.games.half-life/c/15MUk17ppH...
* How would we install all these toolbars?
* How would we change them in flight based on characteristics we can get from elevated permissions during install time?
* How can we bundle adware with the installer so it works without an internet connection?
* How can we subvert the Microsoft Store signing algorithm to add arbitrary unsigned payloads at the end of the executable?
* How can we run unsanboxed internet-connected javascript with elevated administration privileges and direct access to Windows API? [0]
* INCOHERENT QUESTIONS ABOUT METRICS?!
[0]: https://en.wikipedia.org/wiki/HTML_Application
What is "repair" for?
What exactly does it do, and what's the intended use case?
I rarely do much on Windows these days, but my impression was that the age of random applications dumping DLL and OCX files into system32 was fortunately long over, so programs stepping over each other should be almost nonexistent at this point.
Story time regarding DLL naming from the old days - Windows 95 would only keep one copy of a DLL in memory (the days when having 8mb of RAM was amazing). We had developed an application that worked perfectly, except on this one salesman's laptop - where it would only work occasionally. What I found was they had installed "Tiger Woods Golf" on their machine and it had a DLL with the exact same name as ours. If the salesman played the game before running our application, Windows said "Oh, I already have that DLL. No need to load it again." and our app would blow up.
So - do you want to log variables for debugging purposes once installers get to a certain level of complexity? Great. Now you're logging people's usernames and passwords, and you have to add some functionality to not do that.
Turns out many players still had floppies. Whoops.
https://learn.microsoft.com/en-us/windows/win32/api/errhandl...
I only put "correct" in scare quotes because it is kinda lame that this is still required. When was the last time anyone was happy that they got an "Abort, Retry or Cancel" dialog?
In the ‘80s and ‘90s one of the most popular PCs in Japan was the NEC PC-9800 series. These were x86 machines but were not IBM PC compatible. MS-DOS and Windows supported them, but didn’t hide all the differences from IBM PC compatibles from apps.
In particular, the boot drive was A: so a system with two hard drives and a floppy would usually have A: and B: be hard drives and C: be floppy.
That really confused a lot installers that assumed hard drives always started at C: and that A: and B: were always floppies if they existed.
I think the last installer I used was probably the Linux Mint/Ubuntu live CD installer?, But other than that I don't think I've seen an installer for a couple years. Albeit I'm in the Linux ecosystem now so maybe that's why
The model on Windows is that you ran installer program that installed the application. Installers had to do a lot more stuff than drop the application in the right place like change registry entries or write files in other place. InstallShield wrote the installation engine and build tool.
Windows moved to model of .msi packages with .exe wrapped. InstallShield turned into an .msi build tool. Macos has .pkg installer for things that can install .app. Linux has .dpkg or .rpm. All of them have the installer pre-installed.
The testing cycle was the biggest pain, this was right as VirtualBox had become a thing so at least starting with a fresh system was slightly easier. (The entire company had 4 engineers + some researchers).
I think I was two years post-college as were most of my other colleagues... I'm fairly certain that product never made it out of initial trial (I left the company before it finished).
That sounds horrible. Why didn't you use file based HSQLDB or H2 (or any other pure java sql DB)?
I can't remember exactly why HSQL/H2 wasn't used but I think it had to do with fault tolerance and redundancy and what we could do with postgres replication was preferred over what we would have otherwise used SQLite for.
If the application is a single .exe or something that works fine when you unzip a .zip wherever, then the installer functionality needed is almost trivial.
My limited experience is that most complexity in installers comes from the system not helping making it comfortable. Like something not having a simple to use version number, but having to check for 5 registry keys instead, or some such.
Microsoft promoted it strongly, and mysteriously they never come up with a decent installation support, even for basic desktop apps.
Answering the topic here: it is the business more than the product. With exceptions, as usual.
Windows installer came along and was a lot better. The InstallScript engine carried with it a lot of legacy tech-startup technical debt.
Fun fact: InstallShield became a general purpose packaging tool at some point which spit out *.msi packages (and had a bunch of extensions to Windows Installer engine via custom actions). So it was pretty awesome from that perspective.
In this day and age Windows Installer itself has a lot of legacy design cruft from the era it was designed for - the late 90's.
So you're not wrong, but based on the era InstallShield was more or less crappy depending.
Sorry, but I want to add one more thing: the upgrade logic of MSIs is completely wrong, you need a lot of tricks to do an upgrade.
Patches though... Fuck. I think I could still produce a document that explains everything you need to know about patching but it is a lot more complicated than regular upgrades.
At the same time, my career has moved to Cloud stuff now. Is my life better? Definitely not.
Well, when your customer is an enterprise who wants to deploy your product hands-off onto five thousand machines, and expects a clean uninstall when they want it removed later - yes, it's that hard. Writing installers really is it's own sub-speciality in software development.
https://en.m.wikipedia.org/wiki/InstallShield
MSFT released MSI 1.0 in 1999.
https://en.m.wikipedia.org/wiki/Windows_Installer
Post 2008 or so, there was a very good question of why anyone would use anything other than *.msi on windows. But most Installshield users by then were building those packages with InstallShield by then.
These days, lightweight apps and dev tools are still preferred to use *.msi or *.exe based formats instead of windows store apps. Partly because it is quick and dirty to do so, and cheap using things like WIX and NSIS. But InstallShield still has a legacy base among less sophisticated users; it is easier than WIX and more robust than NSIS and the like.
Windows Store package formats are superior in most ways.
Or homebrew which is really just dependency management + standardized directories for files which is what it should be. Installshield was a solution for an entirely artificial problem.
There is a reason Mac had such tiny market share for so long, especially among enterprises. Installation was among pros and cons, but I would advise you to come off your high horse and understand the rest of it. Some reasons technical, some not. For a long time enterprises shunned Mac for extremely pragmatic reasons.
Each type of entry in the config file had a module which knew:
- How to check if the entry was already done and in a good state
- How to uninstall cleanly the thing (not actually needed; this was for system imaging)
- Usage of variables that would dynamically configure the system
- Variables also could conditionalize what gets installed
At the end of the day it was somewhere between a true installer engine and Powershell DSC (DSC would come about a few years later). This installer is probably no longer in use, as you can imagine more robust ways of solving this need exist now.
It was a heck of a lot of fun to build and watch it working though!
I was always very impressed by it.
However, if someone just claimed they could write a better installer for one specific program of theirs only that was better for that one specific program than an installer built with one of those general engines, that is not so unreasonable except for the in a weekend part.
I once worked for a DOS and Windows utility software company that wrote an installer for each product, and I was usually the guy that ended up doing the installer and it wasn’t difficult. If however they had decided to try to make an installer engine product for other companies to use I would have worked hard to not get assigned to that project.
"Microsoft could whip up paint.net in 1 day."
https://news.ycombinator.com/item?id=37559256#37560257
I think it's hard to argue (although some people clearly did) that making a search engine is easy. You need to index all of Internet, cache it maybe too, rank stuff, filter out spam, work over different languages, do something about images, parse a lot of (broken) HTML, maybe run some js to figure out what a dynamic page should show, then serve it up super quickly... google is faster than pinging a local (!) NFS server at my work.
But then I remember a comparison on HN about how booking.com is O(50) people and Airbnb is O(2000) total. Or I remember the total headcounts at Twitter, an app that lets you post very short messages.
Or LinkedIn, a vast company with a very subpar UX, basically an inferior clone of a dozen of other social networking sites, only kept afloat by its network/monopoly effect. I mean, searching for people by name barely works! You're better off using Google to look for people on LI that its own search box!
No doubt it's just my cognitive limitation, like I can't visualize the vastness of the Cosmos, but this is a close second.
Making a search engine that is good is actually a reasonable task as shown here on HN. It doesn't seem to take very many people, either. Google has set the bar really low.
By contrast, making a search engine that is monetizable and good seems to be an impossible task.
100% agreed.
They really should have used Twitter as an example. Any competent full-stack developer could probably have a working clone of Twitter running in a weekend.
The challenge I think is scaling it to millions of daily active users.
But, even then, Twitter has been around for over 10 years. The scaling problem seems to have been solved. I would think that these days, Twitter engineering would just be a whack-a-mole of odd obscure bugs. I can't think of any useful new features it's gotten in the last two years. Why do they still need so many engineers?
What's a tweet? We can haggle this up and down so cut me some slack. 200 bytes? 50 bytes? gzipped -90%, 5 bytes.
Holy shit what absurd amount of traffic could a 1gb fiber connection handle?
oh huh 200 million tweets a second. That's silly. How wrong could I be? -- then you run down the numbers and you realize that no matter how wrong you are, a twitter-clone-for-most-people oughta be cheap. The network effects and fuzzy human reasons are their moat, nothing else.
One of those fuzzy human reasons is how amazon EC2 eats your ass. Just gobbles it from stem to stern.
The text part is basically solved at this point, but not quite that easy. I hear dealing with blocks is one of the slow parts.
I hazard there's a verdant, halcyon prairie between the two zones. The common zone of maximum extraction, and the hypothetical 200$-a-month-operating-costs text only twitter that can service 5% of the entire planet.
Where I see your engineering headcount blow up is in moderation, legal and advertising; which I think everyone skips over when doing their mental math. Campaign management, targeting, reporting, and optimization on such a large corpus of data is where I struggle to see even a team of 5 building that in a couple of weekends.
OK, but you aren’t just pushing raw ASCII across the wire, you’re getting those results out of a database.
For a weekend clone, someone will probably just use an RDBMS (a very reasonable choice). I’ll be generous and say they can retrieve a given tweet in sub-msec time, say 10 usec. Suddenly, you can only send 100,000 tweets/sec, ignoring all other overhead (which of course you can’t).
I’m sure there is unoptimized code and some unnecessary bloat, but fundamentally there are limits on data retrieval that are very, very hard to get close to.
Think about TikTok: It’s essentially just an app that lets you scroll through videos, yet it became breathtakingly popular just due to the fact that they seem to know exactly which videos you may want to see.
But the point is that this product is not what makes the company money.
If you only consider this, sure it’s a easy to clone product, backoffice for the advertisers, algorithms to maximize engagement, external integrations, marketing, PRs, partnerships are the thing that brings money home.
Of course if there was no Twitter, I imagine you could earn some money/be profitable with just the public product and some basic backend for ads publishers. But you’ll probably encounter a revenue bottleneck and start researching fresh income which you can do by … becoming Twitter, the big company.
I'm an AppSec engineer. I don't write a lot of code, and when I do, it's entirely scripts and occasionally modifying existing code (both front- and back-end, but more often back-end) to implement security fixes.
That said, I could probably hobble it together. It'd be ugly as sin with hand-written Web 1.0 style HTML and likely server-side rendering since I hate writing front-end code, but it really wouldn't be hard as long as you're keeping it simple - Just text posts (No video/pictures, though maybe that wouldn't be too hard to add?), no algorithm for engagement (just chronological ordered tweets), no analytics to determine what's trending. Obviously no mobile app.
The obvious differences being the scale, not really having to worry about the complexities of running your own data centers and overlay networks or whatever, and the fact there is no advertising, no sales, so none of the clutter that comes from trying to optimize that as well as defeat your user's attempts to subvert being tracked, nor any tracking of what you're doing off-site.
The signup process is really nice, though. You don't need individual accounts on each site. They pull your identity automatically from your IC PKI client cert and create the account for you when you first use it.
I work from home now, so I'm not on there any more, but they called the Twitter clone "eChirp" and the Facebook one "Tapioca." They also clone Wikipedia and Stack Overflow, but those are just read-only mirrors that get updated once a day.
And then how are you going to ensure you don't provide content rabbit holes that can be harmful to mental health. Then detect credible cases of people threatening suicide to get a welfare check run.
That's just one of the things you don't see on the surface, there are hundreds more functions like that, that you'll very quickly realise you need to do so you don't get fined out of existence and/or end up with execs going to jail.
That's not to say they are or aren't staffed right, just that the part we are conscious of daily isn't the entire thing.
Does anyone have a link to this?
BKNG: Employees: 23,100, Revenue(2022): 17.1B, Net Income(2022): 3.1B
ABNB: Employees: 6,811, Revenue(2022): 8.4B, Net Income(2022): 1.9B
https://www.google.com/finance/quote/BKNG:NASDAQ
https://www.google.com/finance/quote/ABNB:NASDAQ
I can be critical of Google at times but I'll never actually properly come to terms with how Google's search index is still so fast.
One thing to ask yourself - is the company chasing growth? A company like AirBnB or Twitter could absolutely run on a much smaller team if they agreed to mostly abandon growth. They could focus on reducing tech debt, make architectural decisions that reduced flexibility in the name of reliability and lower maintenance, and then eventually lay off a huge portion of their staff.
Instead they have a growth thesis - that growth requires investment, having to favor velocity over unit economics, etc. The higher the growth target, the more inefficiencies you take on.
Luck does happen, sometimes people miss seemingly obvious ideas.
Almost 99% of the cases this is true. The question is whether this complexity is actually valuable or just accidental one accrued from path dependency.
Of course, someone (like Elon) might believe that they have such magic wand, but Twitter is losing more money than ever so probably it was not the case...
Flunkies - people hired to make their boss look important, because said boss has a lot of staff in their department. Their purpose is ultimately managerial infighting, which is zero-sum.
Duct tapers - people whose job is to inefficiently work around problems caused by solvable social issues, e.g. a person whose job is printing out emails because $EXEC is old and refuses to learn how computers work.
Box-tickers - people whose job isn't to do something, but to be seen doing something, so that company can claim they're trying to do X when they don't give a shit about actually accomplishing X.
I'm skipping the others, because I don't understand them well enough to explain them in my own words.
There are reasonable discussions of this sort, but the example OP goes on about is kind of a straw man. Just because anyone can go on the internet and say anything ("I'll build google in a weekend!") doesn't mean it needs a whole rebuttal that takes it seriously.
edit: just realized this was 2016. a purer time.
The same logic applies to bugs on more and more obscure setups, or more and more niche features.
Stable Diffusion has made a state of the art software with less time and engineers than OpenAI
Let's stop talking about hypotheticals, the examples are many
Google has its flaws; but this is manifestly false _even_ if you restrict "Google" to just Search.
Both are fantastic, but not the same.
More specifically, new entrants (and especially so in new categories) regularly get the luxury of extended time to iron out issues, create a better UX, find all the edge cases, invest in internationalization, distribute to multiple platforms, integrate with other systems, address security issues, and more. These tend to be exactly the kind of things that make recreating the incumbent products hard.
If OpenAI continues on its current trajectory, it can easily end up with a team equal to or larger in size than all of Google working on search.
Based on the theoretical foundation from Google, the framework built by Facebook, hardware produced by NVIDIA and cloud infrastructure run by Microsoft? This doesn't make a great example.
Twitter lost advertising because its a bad outlook, not because of technical issues.
Twitter still works. Sure, it has issues, but still works. It definitely was a bloated company, and if Elon could keep his autism under reigns, he could have came out with reducing the company, keeping the same advertising income, and being more cash positive.
That's all your imagination - we can make and support any claim that way. The actual facts we have point the other way.
Let's stop making Musk look like a victim. Nobody blames autism when Tesla's sales increase.
Well, they had 3x or 4x more income and market value than Twitter with fewer employees. That's quite a lot to show for it.
A massive social campaign was ran to make twitter seem like a hateful place which advertisers should stay away from.
If that hasn't happened, for the most part twitter runs just as well as it did prior.
https://www.lemonde.fr/en/les-decodeurs/article/2022/12/19/c...
https://apnews.com/article/elon-musk-technology-donald-trump...
Elon hasn't realized any losses yet, while on paper the company may be worth less that game isn't over for him.
He hasn't done himself many favors by stirring up controversy with his own statements, but it isn't surprising that revenue for an ad-based business takes a hit when the brand is raked through the coals by a large enough group of people making noise. That could be one step towards it failing, or it could be one of many dips while the company is not rebranding itself and changing their business model.
The software aspects make a big difference too, but it is actually quite easy to re-implement software. Something like git for example was built rapidly (3 months?) after a few years of using other VCS to learn what it needed to do.
In my experience, there's a huge difference between "software that works," and "software that ships."
Most of my test harnesses are full-fat applications. They are written in a ship fashion, but they don't have the polish and TLC that I lavish on the ship stuff.
I can write a test harness app in a day or two. It will be a fully-functional, accessible, localizable app, with a non-trivial feature set.
However, if I write something to ship, it may take a couple of months (or longer).
An example is the app I'm writing now. I wrote the "heart" library in about a month and a half. It has a test harness that runs every feature, and the test harness was the largest part of the job.
However, the ship app is coming to beta, about a year later. It knocks the test harness into a cocked hat.
The real issue for the bloat is simple lack of engineering skill. Most "engineers" have very little clue of what goes on under the hood when their code is running, and really only rely on learned patterns to complete their job. So many times I have been given code for a service that when run locally doesn't work, with the explanation that you have to test it in beta stage - the reason this happens is because nobody who started writing that code has any idea of what goes on outside of the boilerplate that is provided for them to fill in.
This is a fantastic point.
However, I'd like to point out that there are rare cases when some newcomers can replace existing businesses because they found some technology that simply replaces years of work.
If you did it today, you'd probably want a fairly literal-minded search engine front-ended by a large language model to make it more user-friendly. That might be easier than trying to make the core search engine appear semi-intelligent.
[1] https://en.wikipedia.org/wiki/Cuil
The principals were then re-hired by Google.
That is shocking!