Ask HN: Why are big companies so much less efficient?

You've seen those graphs about how lines of communication explode combinatorically. Once an organization hits ~20 people, if they're all communicating directly and keeping up to date with everything they need to know, everyone could be spending every minute of their day communicating and not getting any other work done.

In order to cope with the communication overhead, larger corporations add management hierarchies that maintain linear communication "vertically" and smaller group sizes for "horizontal" communication. They also add processes that decrease or streamline the amount of communication necessary. Management and process are necessary to keep the company from grinding to a halt under the weight of its own communication overhead.

However, management and process even when done well will add a certain amount of overhead and slow things down. Less than they would if every engineer had to check with everyone in Sales and Marketing before deploying any change in order to make sure that a deploy wouldn't disrupt a major campaign. More than you and your two co-founders sitting next to one another and asking "hey, I'm gonna deploy the Foo feature now - is that cool?" And when done poorly, management and process slow things down even more.

However, regimentation can become less efficient if it gets in the way of change, and big orgs are known to be slow to change.

Big companies often end up buying small ones precisely to extract the efficiency benefits you cite.

Perhaps a good question is : could big companies' R&D be made more efficient?

In answering this you'll probably run into the question "who gives a crap about the things you want to be more efficient?" Most people working in big companies are focused on other stuff such as how to avoid being fired by their psychotic boss, how to get a promotion, how to get a good performance rating, how to get a better job elsewhere, and so on. Only sometimes are their motives aligned with efficiency and things like delivering good products.

Each process brings an overhead - someone needs to create the process, then some number of people need to manage the process, another number of people to document and explain the process to others and finally a group of really unfortunate people who will be forced to live with it.

All that happens is that the more people there are that need to use a process, the more people are needed to manage it. If the use-cases outgrow the process then someone else needs to come along and make a new process, and then even more people will need to come along and advertise the merits of this wonderful new process and to explain to everyone else how it works and why they should be using it. Suddenly you have more processes and more people to drive them.

You hire some more people because you realise that all these processes are all well and good but actually they aren't making you any money. All that happens is that these people are also inundated with processes and therefore are less efficient, because suddenly you're having to actually justify your work internally or to go through some kind of process and time wasted doing that is not making the company any money.

What happens eventually is that most of the people working at a big company aren't actually working towards the goal of the company whatsoever. They're just there to hand-crank a process. None of these people are actually revenue-generating, and the people who are actually there to be revenue-generating are slowed down or otherwise hindered by process.

The problem is self-perpetuating. Oh, no, we aren't making any money. How can we improve our processes to make more money? The cycle continues.

When I confront it with the times I worked for a small company... You had no choice but to work hard. If you didn't do it, it wasn't done. Deadlines were pushed forward, everybody was unhappy. There was no margin for anything, so I had to work overtime very often. These two approaches are incomparable. (Of course you can easily find opposite examples, especially in big consulting firms, so take this with a grain of salt.)

The larger the company, the greater the share of all the mental energy of the total mental energy is spent internally competing for position.

Height is not the same as output (and H / V is not the same as efficiency), but this model has some similarity to the real world engineering output. Just my 2c.

You're basically a huge asshole if you book a meeting at 3pm because you'll make all the moms and dads late for picking up their kids. And you can't book any meetings before 10am because more often than not, it's tough to get everyone in earlier because they have to drop their kids off.

Aside from that, new parents are often glued to their phones waiting for updates from whoever is taking care of their children. They are not fully engaged in meetings and I often watch their attention drift to the daycare nanny cam. I have a coworker who receives hourly updates from her mother who is taking care of her young child. Sure enough, every hour there is something that needs to be corrected and thus, a 15 min phone call ensues.

So here I am, a single software developer, who has to wait a week for our sprint planning meeting because everyone else on the team has put up roadblocks that prevent it from happening. I literally will sit here this week doing nothing because of this. This is a regular occurrence.

This isn't a complaint really, I will want these same luxuries when I am a parent, but I can't help but notice that 90% of my team would really rather be elsewhere.

If you're a startup founder, you eat what you kill. There is no degree of separation at all.

If you're working at a 10-person startup, you report directly to the founder. 1 degree of separation.

At a 50 person startup, you probably report to someone who reports to the founder. 2 degrees of separation.

At a place like Microsoft, you're probably 7-8 degrees removed from the CEO, who himself reports to the board-of-directors, who report to a decentralized mob of shareholders.

At every degree of separation, there is an amplification of:

- distortion of incentives

- "Not my job"

- "that decision is above my paygrade"

- "What's in it for me?"

I don't think individual productivity can ever be as high in mega-corps, the way they are in startups and smaller teams. Hence why mega-corps are generally best at milking existing cash-cows, whereas startups are much better at actually driving innovations.

this is because efficiency isnt a goal , impact is. you might be efficient enough to iterate 10 times a month. but that is not what a customer wants. planning goes a long way compared to just doing something. as software Engineers we often mistaken efficiency as "getting shit done". which is great but there are too many instances when it is shit. the bigger the org and the user base / affected population, the things you do has an impact . i've known users to shift loyalty due to small mistakes that companies do. as a dev deeply connected to the impact of your product, you realize that decisions make a big difference. you plan ahead and mitigate probable failures. these require time.A good product can go a longer way than an efficient team with no direction. this does not imply that everything takes time. but repercussions of a bad decision can be lasting in a big organisation. you have a large number of stakeholders involved who would want to and can help you achieve your goal. this is however not always good. the greater amount of red tape involved can lead to competitors gaining an edge and also several cases of "i fucked up". the bottom line being. efficiency isnt merely moving fast. its about building things which matter and having an impact. if you look at efficiency in that manner, then big orgs aren't any less efficient than the smaller ones since their reach is much greater. perspective matters

eg. If a large company is 10x more inefficient it simply hires 11x more people.

Startups are more like predators, you need highly skilled individuals capable of leading a pack and killing frequently. This skill is rare and hard to execute / scale. (Notice how when a pack/pride gets too large it splits) Large companies are more like a herd, most individuals can figure out how to eat grass and run away when a predator appears, since the predators pick off the weak individuals anyway, they are actually doing a service to the herd organization which lacks the predatory nature to kill the weaker members / deny them resources. Herds can grow almost infinitely limited only by grassland / water.

It is a systems tradeoff.

You can hire 5 10X people, but if one of them leaves (10X invariably do), you would have a harder time replacing, on-boarding, ramping up, culture-indoc, etc. another 10X person, assuming you can find one. In the mean time, this could potentially mean anything from a chaotic breakage of process or severe team reorg.

Whereas if you hire 50 1X people, if 5 leave, the system will likely still run at reduced capacity while you backfill if you have good process (not all big companies do, but those that survive in competitive industries tend to).

Good process often wins over a concentration of pure talent over the long run.

Anyone can start a new product with 1/10th of the features of a competitor and feel that large corporations are slow and inefficient.

Hammering out all of the edge cases and flushing out a product is often what takes a lot of time and manpower. Not to mention dealing with a larger number of customers who probably use said product.

There are success stories of small shops who automate everything, or who offer a product that is so simple (not a knock, actually a complement) that it can be run by a small team. Complexity is the enemy to shops like this, and I think few know how to avoid it in the long run.

One thing to wonder is if it's at all important to measure how much gets done, 'per engineer'. How can you possibly measure that and why bother?

One can be very productive, at doing something nobody will ever use. Another can be 50% as productive, doing something that'll be used by billions of people. One can be miserable and productive, another can be happy and 50% as productive.

These things matter - individual productivity in isolation, not so much.

By narrowing your question down to single individual performance, you've eliminated the possibility of getting a meaningful answer, other than 'revise your question'.

When you combine risk management with certain corporate cultures and market verticals, you get a combination of the following:

Risk aversion

Lack of technical knowledge in the management layer

Technology viewed as overhead vs. a value provider

These three things are deadly to an ambitious corporate IT project. Unskilled management can't appreciate requirements, manage scope nor understand trade-offs very well. The business wants guarantees on budget and timelines. Budgets are fixed, and management rarely wants to ask for more since ROI may be hard, if not impossible to quantify.

So what do you end up with? The standard "over-promised and under-delivered technology" solution that are endemic across so many enterprises.

[edit - formatting and spelling]

* Existing codebases need to be maintained. Small companies tend to be younger, which naturally comes with fewer existing systems that require maintenance, and those systems that do exist are generally smaller. The time spent keeping a large production system up and running feels wasted, but it's absolutely not, especially when you consider the alternative is letting the thing fall over and losing the revenue, users, etc. Contrast this against a smaller company where more projects are fresh and satisfying to write.

* Lots of work centers around adding features instead of greenfield projects with no history. Most of the code is already written, much of the infrastructure is already in place, and oftentimes the problem is similar to one that was solved previously. You end up writing less code to get more results, as opposed to a greenfield project where you write tons of code to even arrive at a baseline.

* Larger companies almost certainly have more irons in the fire than smaller ones. While on an individual level it's annoying to have to wait for input from someone who has half a dozen projects on their plate at once, on a company level this makes sense: when you have to put fifty features into production, you don't organize each one individually, you create a pipeline composed of people and run your work through it.

* As a result, projects at larger companies have more stakeholders, which means you need to get agreement from more parties before you can move forward. Also, you're likely not the only thing on those peoples' radars, meaning you'll have to fight for their attention.

* Larger and older companies tend to encapsulate their lessons learned into this pipeline. I'll hazard a guess that this is what people mean when they talk about "institutional failures" led to damages like Well Fargo or Facebook/CA, etc: no individuals failed, instead the system put into place to pipeline their work was faulty. The "human pipeline" of a larger organization was almost certainly built up in response to a long series of nonpublic failures. More quality in the form of fewer such failures, at the expense of higher latency.

* Politics is a thing. Larger companies almost uniformly have career ladders, with the aim of giving people a way to advance their career besides "take a CTO position somewhere else and come back at a L+1 18 months later." This means at least some of engineers' time will be spent marketing themselves for promotion rather than writing code. This often means choosing and prioritizing their projects by biggest expected impact. This has consequences if that engineer is a stakeholder in some other engineer's project and they disagree on the importance of that project to their respective careers.

* Finally, there's a human component to this. In my experience, the sorts of people who join smaller organizations and get tons of work done tend to be younger and less tied down. You're not going to be working at a 6-days-a-week 8am-8pm startup with a wife and children. You're going to be looking to put in your time and head off to live your life. Meanwhile your management is still going to demand big things from you, so you end up becoming efficient out of necessity.

I'll leave you with the upside to this so you don't come away thinking large organizations are hellish slave pits: The impact of your work is larger simply by virtue of your being at a large company. I won't go into details, but I work in a group that puts in mont...

To give a personal example from a FAANG, where I work at, web developers don't have to worry much about a lot of the ops infrastructure - a lot of that work is offloaded to another org whose sole responsibility is to have a robust setup for the whole company. We don't have to reinvent the wheel for things like CI/CD, and such nice features like deploy on a pull request basis. The main inefficiency I see is that large organizations take time to make the decision of whether something is the right thing to build. Once the decision has been made, the implementation happens much faster than what small companies can match that I've seen - my own employer moves frightfully fast compared to any startup I've worked for.

From personal experience, I see small companies constantly struggle with trying to standardize these things, and often solve these problems in a way that causes long term iteration pain. Oftentimes the engineers aren't good enough to solve many of these problems, and the short-termism of feature development strangles a lot of proper solution building, which actually leads to a lot of inefficiency.

A lot of times, small companies also often take shortcuts that bigger companies cannot take. For example, internationalization and accessibility may be minimum requirements at some bigger companies, whereas small companies may not be hamstrung with supporting those fully for their customers, if at all.

Second, as others have mentioned is coordination. Now introducing a new process, feature, or whatever to clients or internally means creating the requirements, the documentation, working with sales or client services on timing and release, working with QA on test scenarios, working with training to get it into training programs, etc.

There's a couple reasons, but the biggest is that less established companies and their investors tend to have a higher risk tolerance, while well established companies tend to be more risk averse; consequently, the small companies that survive get more done per engineer, on average, because they've taken a higher risk / higher reward approach. (Google often gets mocked as inefficient for taking multiple parallel stabs at the same problem, and, sure, a company which just did whichever one turns out to get validated by the market would be more efficient—but one which did only one of the losing ones would fail and be forgotten.)

Another issue that is that bigger firms are often devoting considerable resources to solving non-technical problems (like “securing enterprise and government contracts”), which often are very rewarding in terms of profit, can often be rewarding in terms of impact, but often compromise efficiency in a narrow technical development sense.

https://corbt.com/posts/2017/10/28/small-companies-and-the-p...

TL;DR is that I feel much more productive in the smaller company, in large part because there were SO MANY stakeholders at Google that it took a very long time to decide on a course and move forward with it. I think that experience probably generalizes to most large organizations.