I would like to think there will always be a frontier, and there will always be people driven by their curiosity to seek the frontier out. Some of them will be called 10x engineers, some visionaries, some pioneers.
This seems to be pretty specious reasoning which has at its heart the principle that acquisitions are the final goal, not a functioning enterprise. I think the acquiring companies would tend to disagree.
This article fails to mention that our expectations for software have increased. We expect more from apps than ever before. This has lead to new specialities such as Front End Engineers and UX staff. Surely we can build more with less with current technologies as compared to the past but we also expect more. If our expectations outpace technological improvements this could lead to spending more time/money on developing apps.
Uniformity isn't so great for people who want to differentiate their products. (Not saying that they should want to differentiate their products visually, but they almost certainly do want this).
This seems to be ignoring entire classes of software. I can't imagine the enterprise development world will be taken over with easy to snap together software packages any time soon.
Because enterprises are infinitely complex and diverse and there will never be a single piece of software that can run all the businesses in the world with a couple configuration setting tweaks.
The article suggests that, as super-human programmers build ever-better platforms/languages/tools, those building apps/whatever on top of those will be less differentiated in terms of programming ability, because it will be so easy to develop stuff anyway.
I'm not sure this is true. Are 10x programmers really 10x because they can grapple with technology stacks, or because they're really smart and experienced at problem solving? Programming will always be about problem solving, no matter how easy you make the final implementation of the design.
Specific skills (e.g. working with a certain framework) will go in and out of style, but problem solving will always be in demand.
Also, maybe I'm naive, but I just don't see the software-as-legos future as feasible. There will always be the need for customization and ever-more-advanced functionality.
If you spend a few years at university (I have two degrees), you learn things like sorts, automata, proofs, and calculus.
On the other hand, most of the junk I deal with day-to-day as a working software developer turns more on my knowledge of tools like Chef/IntelliJ, software libraries (the stdlib of various languages, the Java/.NET BCLs), build systems, git/github, and how to do a proper code review.
For most stuff today, I would highly prefer the person with more tools experience. Granted, there are some problems people who aren't "10x developers" could never solve (e.g. writing linux) but for most stuff industrial software devs today are doing, it just doesn't matter. You just need to write the code, it needs to be maintainable, and it needs to be done as quickly as possible.
You compare two people who have different skills. Presumably either one could learn the skills of the other. Why choose the one with the easiest skills to learn?
Its funny I've always thought that Twitter could really be run by 100 folks (heck I've said 40) and be hugely profitable and it a lot of ways more efficient. Instead they've got 2300 employees to allow folks stream 140 character tweets.
It's already happening. There's an incredible difference (in personality, tooling, skills) between your typical full-stack web developer and kernel programmer, as there is between the kernel programmer and chip designer. They're each difficult in their own way, but one definitely has a lower barrier of entry.
I had this vision 5 years ago, for sort of an operating system for social apps. Where a regular user could build their own app by just installing some plugins, dropping a chatroom and some other components on a page, paying a developer to wire them up, and paying a designer to design the theme. And there would be a marketplace of components built by more experienced developers.
Like mobile apps, social apps require an operating system that solves common problems in a standard way:
* User signup and secure auth, login
* Should work with emails, mobile #, facebook
* Accessible from desktop, tablets, mobile
* Contacts integration / friendlists
* Real time collaboration from any device
* Offline notifications via fb, sms, app notifications etc.
* Access control and privacy - as expressive as possible
* Native apps on iPhone and iOS via Cordova
* Invitations and one-click account creation
* Encourage best practices for scaling, web standards etc.
* Installer to upgrade apps and plugin versions cleanly
etc etc.
All that stuff should have standard solutions which you can override. The apps and plugins should all interoperate.
And finally, since I really care about decentralization, it would be cool if this whole operating system could be as decentralized as Wordpress is for blogs, as opposed to say facebook. That would also solve all the scaling problems that facebook and other centralized social networks have.
That last part - decentralizing the social network - took the most effort, as I had to solve problems of security, identity, privacy, as well as app and plugin versioning and signing across a distributed network. Just like iOS is for mobile apps, and Bitcoin is for distributed contract apps, the Q Platform is if you want a social layer for your app, from user accounts to everything else, without having to write it from scratch.
How come every time I post about the platform I've spent 3 years building someone here decides to vote it down? I'm just curious. Is it really something you don't want to be spoken about?
I think this got downvoted due to the way you describe it.
My experience with HN is that it's a very hard-edged, no-shit crowd. People here expect things to be described with simple, crisp language, no jargon (especially not computer/science/technology terms with precise meanings), and a minimum of hand-waving. It's a bit hard on the ego, but it'll make you a better communicator.
In the first sentence of your post, you described your thing as a "social operating system" or something like this -- this immediately sets off my bullshit alarm. "Operating system" is a very specific thing to software people, but for whatever reason, non-technical people love using this term to talk about something perceived to be big and complicated.
In general, try to cut the fat. If it's a library of software components you're making, say that. If it's a Ruby gem, or an app, or some kind of a specification, just describe it for what it is. You don't gain much here trying to hype it up.
Other things I notice:
- You say you can "override" -- how? Using what?
- What is a "social app", anyway? Aren't all apps "social" today? This descriptor (social) adds very little value. Github, facebook, twitter, instagram, linkedin, could all be considered "social" apps.
- "Social" is a very overused buzzword in SF. Everyone here is building bullshit "social" companies and all the devs here get like 10 job ads/day for some dumb social network for cats.
- Again, "interoperate". How? Via a REST API? Server-side includes? Be specific.
- Don't say "it's come a long way". Stop telling. Show.
- "Decentralized". Again, what does this mean? It can be hosted anywhere? The software is widely distributed? What is a "distributed network"? Aren't all networks "distributed"?
> As software becomes a high-impact, low-skill trade, we decouple the technical ability and experience needed to write tricky software from the ability to solve problems for people.
It's one thing to know that capital secretly delights over the commoditization of labor. It's quite another to watch a16z gush so openly about a future where we can be tossed aside.
It seems this claim has been made forever. COBOL was supposed to make programmers unnecessary. So were rules engines. And code generators and an endless list of other technologies.
I agree that Gerstenzang is premature. But it'll happen eventually. If it doesn't happen by making programming easy it will happen by flooding the industry with enough competent programmers to have the same effect (depressed wages, high unemployment, etc). See: non-software engineering occupations (excepting doctors, who have managed to turn the risks inherent in their profession into a legally mandated union).
Demand for engineers is increasing much more rapidly than non-software engineering occupations. Soon every company will need an software engineer of some sort.
Great programmers move up the chain as well. If your skills are not being valued by your company as it is setup in a way that it needs only average engineers, you can move on and create your own company now.
If you are a great engineer, then you are a walking capital by yourself.
If somebody with an idea, and no skills, hires you, then it probably needs at least 300-500k (even for a smallish prototype) to make it happen. If you are a full stack engineer, and pair up with a buddy, you can make it happen, with a lot less. If you are a great engineer, you are the capital!
Programming talent and skills do not translated into business talent and skills. And vice versa. It does not even translate into an idea doable by just two guys that can sell well and is not easily doable by other two guys. How many such products does exist?
> If you are a great engineer, you are the capital!
It only seems that way because engineering is currently the most capital-heavy part of bringing a softwarebusiness to market. Once that ceases to be the case (which may take a while, but it's the future SG was gushing about) then as long as there is a single scarce or mutually-exclusive commodity necessary for success its price will ratchet upwards until it is out of reach for most engineers (after risk-adjustment).
Pertinent example: the implosion of the indie app dev market that's been all over HN this past week.
But once the Engineering component has been hammered down as much as server costs were in the previous generation ('Sun pizza boxes with Oracle' turned into AWS on OSS), what's the next largest expense line?
If the next most expensive thing is the marketing spend, so that the company gets the required exposure in overcrowded app stores (for example), then the VCs are painting themselves into a Hollywood hit role - at best.
Of course there's an underlying BS premise that "in that wonderful future humanity would be freed from the burden of non-essential programming work, and would move on to higher level problem solving" -- but in real life they could not care less if humanity actually moves forward or just rich companies have to pay less while millions suffer and fall from middle class.
People talk as if everyone can become a creator, which it ain't so (partly because of the personal skills needed, partly because there's no economy where "everybody is a creator" -- for the same reason why not everybody can be a millionaire at the same time (money loses it's value then). Any large scale economy needs few sellers/creators and many buyers).
So what this trend does, it empower a few programmers/creators (ok, more than the past few large capital owners), and makes all the other programmers redundant or devalued.
In theory, yes. In practice, most people aren't entrepreneurs. Try convincing someone living in the Bay Area to walk from their 200K/yr plus stock units from FB/Google/etc. and you'll find out the reality: what really separates capital from labor is risk tolerance.
Labor = show up, do your job, get paid. Capital = put money at risk without any guarantee of getting anything back, let alone earning a profit. Capital makes the rules because half of America is two paychecks away from bankruptcy (I read this somewhere credible) and can't/won't take any risk.
It seems to me that with crowd-funding and big data, the people who have jobs picking winning companies out from a lineup will be obsolete before the engineers building the crowd-funding and big data platforms will be.
> the people who have jobs picking winning companies
Remember, capital has TWO strategies for earning money:
1. Investing in new ideas, hoping they yield a positive ROI
2. Collecting dividends on ideas that have already panned out
#1 might conceivably be replaced by algorithms, but I doubt it (has HFT eaten Wall St?). #2 is inherent in our economic system and will never become obsolete.
I cannot believe that this post by Sam Gerstanzang, a partner, speaks for a16z. I certainly hope it doesn't get much traction internally and there have been a few internal notes explaining to him the number of ways he is wrong.
What I don't understand is how someone who is so obviously wrong about the software craft and isn't even coherent within a short blog post can be a partner at one of the best VC firms. I sincerely hope he's there because of nepotism.
What's wrong with a future where some software devs can be tossed aside? We have been excited to disrupt all other areas of labor. Disruption is good, it means progress, right?
Troubleshooting and problem-solving are specific skills
that can be completely unrelated to software, even tricky
software.
It's great when both problem-solving and outstanding engineering exist within the same individual, in some positions even essential.
Realistically, problem-solving is the more uncommon natural ability of the two. If it were not, there would be a lot less problems with software by now.
I'm failing to follow the logic here. The initial point is that "The potential impact of the lone software engineer is soaring". However, the author then takes that to mean that engineers will be less valuable in the future?
This analogy seems apt to me, and I think it makes the logical leap more obvious:
Fifty years ago, if you wanted to design a new model of car, you had to get plywood and clay, and work hundreds of hours of to get even the most basic model completed. Now, you can open up a CAD program and begin making real decisions immediately. What previously took a large team to build, a single designer can now do. (The analogous logical leap to what the OP seems to be saying is) Soon, we won't need designers at all! A company will come out with a new car model simply by snapping design-legos together.
In reality, removing inefficiency made designers more valuable. If you were the best car designer 50 years ago, you had to spend a significant amount of time molding clay. Today, you spend almost all of your time creatively finding solutions to problems. I see the value of individual engineers increasing significantly as we remove cruft in the creation process.
Replace "software engineer" with "person building a product" in that first quote.
The value of engineering skills as they are defined today is falling, but the value of a single person with an idea and the drive to implement it is soaring.
Back when I was a physicists every two bit crank used to go on about how ideas were more important than knowledge.
Now I'm hearing the same in programming. And it's just as wrong here. Picking the right algorithm and stack means the difference between a website that can handle 2,000 people and one that can handle 2,000,000. How many websites start off with a bad idea, get popular and then need to hire hundreds of engineers to put ban aid after band aid on a fundamentally terrible idea?
Facebook and google have both had to spend millions of man hours to fix the fact they were optimized for millions of users and billions of web pages and now have to deal with x1000 times those. And the only reason why they were that prepared in the first place was because extraordinary people wrote them. The average developer who bangs out a half thought out colour by numbers website would be lucky to have it not crash before it reaches a thousand users a day.
On the flip-side, the scalability doesn't matter if no one comes. The efficiency of the implementation doesn't matter if the wrong idea is implemented.
Sometimes skimping on the engineering is better than skimping on the idea. That's what MVP and quick-to-market are about, correcting the idea so you can architect a solution to the RIGHT problem.
I think the core of the argument is that most software problems are becoming easier, which means the talents of the very best just aren't applicable or especially useful.
If something is easy enough, anybody can do it, and do it nearly as well and as fast as the best. For example, nobody can flip burgers 10x faster than average. Maybe 2x, but not 10x.
If something is difficult, then many people won't be able to do it at all. Most of the rest may only be able to do it very slowly and with great difficulty. The very best will make it look easy, even though it's not.
When something becomes 10 times easier, that doesn't mean the best become 10 times better or faster at it. They may become a little faster, but the real gains are at the low end: The average and worst performers become significantly better. Further, many people who just can't do the thing become able to accomplish it.
This has always been the case. Software people in the 70s and 80s already had the distinction between "systems" and "application" programmers -- the former understood to be concerned with making the pipes and plumbing, vs. the applications guys concerned with the end users. Applications have always been like legos.
I work as a systems developer but often do consulting projects as an application developer, and it's a totally different mindset. Writing software for end users, I rarely even remotely think about CS, algorithms, pointers, recursion, numerical stability, memory allocators, etc, whereas I do care intimately about the tools I'm using, much like I suspect a carpenter cares about what drill he's got. On the other end of the spectrum, I can hardly go a day writing traditional "systems" software without considering things like what data structure MongoDB/Postgres uses to hold indices, and gigantic, complex consensus problems (distributed systems, e.g. how DNS behaves).
I hope we start to treat these two endeavors differently because they require totally different mindsets and education. A typical app developer need know no more about how OSes work, than a plumber needs to understand the metallurgy used to make pipes.
I see the overall point you're making, but it's not true programming applications is always (or even often) like building with Lego pieces. It's just that many of us have to write mindless web apps which mostly consist of CRUDs. Those definitely are like working with Lego.
CS, algorithms, pointers, recursion, numerical stability, etc, etc, are all alive and well in the world of applications programming. Just not in the world of run-of-the-mill CRUDs.
Sure. And to be clear, I in no way think applications is easier. In fact, applications, as they're closer to end-users, have to deal with all kinds of socially-imposed whimsy like people with four names, tax rules, interest rate calculations, and 100 other things that make my head hurt to consider.
The Lego approach has been the holy grail of software development at least back into the 80's with OOP. Yet it still requires programmers who know about data structures, algorithms, etc. to write software. Even if they're just cobbling together libraries, there is still a lot of additional functionality required--not just 'glue' code.
All we have now are improvements in efficiency (great, mind you), with operational improvements like IaaS and development improvements with the great OSS that we have now. It will require a sea change in the field of comp sci. to achieve Lego-like orders of magnitude improvements.
> The same software foundation (open source software, development tools like Github, infrastructure as a service provided by the likes of Digital Ocean, and more) that allowed Whatsapp and Imgur to scale, means that experience and skill writing software become less important.
The author focuses a lot on ease of scaling, but the bigger challenge has always been building applications (experiences) that need scale. If you build it, they will not come.
Let's compare writing software to writing English. (Writing is kinda an individual sport) Almost everyone can write a short story in English, but the 10x writers really really stand out. Writing isn't a perfect analogy - we generally think of programming as a skill (right and wrong, boolean, either you can or can't) and writing as an art (shades of gray, non-boolean skill).
The assumption is that more people being able to make things with code would make the 10x engineer less valuable - but I think it makes them more valuable. In a noisy world - only the truly great can stand out. And to really make our future great, we are going to need a lot more than CRUD web apps.
I could be wrong - but I don't think this is as cut and dry as the author claims.
Teams are not getting smaller. The original Macintosh team was not much larger than the teams he mentions and probably had much more technological impact.
The web has been around for a quarter century now and still rather trivial websites that mostly could be replaced by tools sell for some $50k. Because even those basic tools we had 25 years to build are not there yet.
Software engineering is actually becoming proper engineering, at least in Europe where much of it is focused more on industry. Crashes are mostly unacceptable now. Embedded software is sometimes formally proved.
Finally, as everyone who actually develops software knows, we need to cut many needed features to ship in time. Think how incredibly long it still takes to implement what you can imagine: It takes many months. This is a good indication of how far we are actually from combining things in a lego-like fashion, from making dreams come true quickly because it is so easy.
One could argue the exact opposite: Engineers are needed now more than ever. A skilled engineer (what does this even mean in a context so wide?!) can run a website like PinBoard, serving millions of users, from the a jungle in Thailand, on a crappy 56k using a 250 USD ChromeBook.
Of course it's possible but how many people do actually manage to do it successfully?
And while having AWS, Github and you-name-it makes this possible (scaling was the big issue apparently), AWS and Github can not be run by a lonely engineer, you need an army of them.
This article is wildly off base, particularly when you look at the examples the article cites.
WhatsApp was able to scale to so many users while keeping the engineering team small precisely because they hired great engineers who wrote an extremely efficient system in Erlang. (Does he think commodity coders know Erlang? Really!?) They heavily tuned their architecture and built efficient paradigms which kept developer productivity & performance from degrading as their user base grew. [1] Commodity engineers don't write that sort of code.
Yes, we increasingly have tools which allow mediocre coders to get stuff done. (For example, Ruby on Rails.) But the inevitable byproduct of letting mediocre coders loose on a stack they only partially understand is a mess of spaghetti code which takes even more developers to maintain. This is already what you see with non-tech corporations who employ dozens/hundreds of developers to maintain Drupal/RoR/Java apps which are significantly simpler in scale & complexity than WhatsApp or Instagram. If you want to use tools well, you need to hire people who understand the tools.
Yes, the tooling has improved dramatically. A 10x engineer of yesteryear might actually be a 100x engineer today. But if you give a bad engineer modern tools, it just allows them to fail faster.
I might be able to build a house with legos, but I expect to be spending a lot more time repairing it than if I'd just hired a carpenter in the first place.
If I remember correctly from a high scalability post about WhatsApp it was also on bare metal. So more complexity to manage then a simple AWS setup where you can tear up/down in seconds.
>WhatsApp was able to scale to so many users while keeping the engineering team small precisely because they hired great engineers who wrote an extremely efficient system in Erlang. They heavily tuned their architecture and build efficient paradigms which kept developer productivity & performance from degrading as their user base grew. [1] Commodity engineers don't write that sort of code.
The problem is they did it with only a few (16?) "great engineers". That's the premise of the article -- our world today might need 100,000 great engineers to solve great problems. Tomorrow 1000 might be enough.
As for the "near great" they would too be less needed, and commodity engineers would be the equivalent of burger flipers.
>Yes, the tooling has improved dramatically. A 10x engineer of yesteryear might actually be a 100x engineer today. But if you give a bad engineer modern tools, it just allow.
The first yes, the second not really. A bad engineer can get enormously more done with modern tools than he could 10 or 30 years ago. Essentially a bad engineer today can build stuff that took high level senior engineers to build -- and do it in less time too. It might be a little crude, but it will work.
E.g a bad engineer can automate a workflow that would have taken expert engineers, custom software and several man-years to do 20 years ago, and can do it in a day.
And because it's so abstracted, building something like that needs little influence from his skills, so his "badness" doesn't show up much.
The problem with the article's conclusion (and title) is that they don't stop with 100,000 -> 1,000 engineers.
"Tomorrow, that billion-dollar startup acquisition might not need an engineer at all."
The article's claim is not that startups are going to keep having smaller and smaller teams of higher-value engineers, they're saying that they're going to have zero engineers. Which I don't see at all from the evidence provided.
Well, they might technically have an engineer, but it could be some average joe, using a "21st century" equivalent of Hypercard -- building a whole startup with some composable cloud components and some logic.
After all, the basic skills for a successful startup, even today, are not hardcore engineering skills, but idea, business, finances, etc.
I mean, most startups bought for multi billion dollars work on BS trivial engineering problems (mostly: make a social web service scalable), compared to what talented engineers do elsewhere (in research, in space exploration, in robotics, in DSP, etc).
Heck, wasn't Facebook started with crappy PHP code a 15 year old could write (and a 20 year old did wrote)? Or, for a smaller scale example, Derek Shivers (which I admire) sold his CDBaby startup for a nice sum, and that was also started and based on amateurish code at best that he wrote himself.
> The problem is they did it with only a few (16?) "great engineers". That's the premise of the article -- our world today might need 100,000 great engineers to solve great problems. Tomorrow 1000 might be enough.
The premise of the article seems to be that we no longer need great engineers at all. Or that they only need to work at infrastructure companies (not application companies like WhatsApp). The very example he chose shows that's empirically untrue.
The fact that a small team of amazing engineers can accomplish huge things (given some ops support) is not new. The original Macintosh team was also tiny.
There's no evidence that we will ever stop needing great engineers, particularly as more and more applications require huge scale while maintaining agility.
>The premise of the article seems to be that we no longer need great engineers at all.
Well, not really, because somebody will have to build all those elaborate abstraction layers people will use.
The core idea is that we'll need much fewer great engineers -- which is another way of saying we won't need "great engineers" at all. Read "not need them at all" liberally (very few), not literally (nobody).
After all if job demand goes from 1,000,000 people to 1000 people (arbitrary numbers), it's like it's the end of that job, even if technically people are still employed (there are after all horse cabbies and film lab technicians and letterpress printers employed still today).
> Well, not really, because somebody will have to build all those elaborate abstraction layers people will use.
It's almost as though you completely ignored my next sentence, which clarified that great engineers continue to be valued outside of abstraction/infrastructure companies.
Well, I don't find it valid to be frank. They might be valued, but they are valued less and less, and will be valued even less in the future.
An enterprise that used to need 100 engineers, with the cloud, SaaS, PaaS etc, can even today make do with 1/5 of that. Is there any trend showing companies needing MORE software engineers?
I would say yes. More and more stuff is getting done by computer. The majority of IT staff are still building / maintaining in house applications as far as I am aware, not building teh next Facebook / Whatsapp / etc.
This is absurd. How many people where creating their own languages 20 years ago? Everyone and their grand ma is building new languages on top of javascript. The expertise level of software engineers is going up! 20 years ago, how many people could build a VM from the ground up? Have you visited github lately? Sure, an average engineer can do much more today, and the reason is because of these great engineers, and they are not fewer by any means.
>How many people where creating their own languages 20 years ago?
20 years ago? Far less. That means writing a language is getting more commoditized -- and we work in a higher, easier, level of abstraction. That's what TFA says about other spheres of development and deployment too.
>Sure, an average engineer can do much more today, and the reason is because of these great engineers, and they are not fewer by any means.
Neither I, nor the article, said there are fewer great engineers. Only that fewer great engineers are (and fewer still will be) needed. Both as a percentage of all great engineers, and in absolute numbers.
I can't substantiate it, but there were in fact a lot of languages being created through the 60's to 90's. One could argue the language implementors of old were quiet daring, push boundaries much more than today (see APL and ICON as examples). sed and awk arose in an era of language explosion, make, sql, the list goes on.
It's a problem if you are an engineer. It's the kind of "going homeless" fuckery.
It's also a problem, longer term, if you are an entrepreneur. Initially it starts as "lower costs", but with an impoverished middle class, you soon find there are not that many people to buy your fancy products and services.
Automation is only good is it's combined with a scheme for the surplus of labor -- e.g less hours work week, creation of new kind of jobs, etc. Without that is a huge creator of poverty and inequality, and eventually the collapse of a market economy.
Whatsapp also makes design decisions that I would call 'engineer-easy'. Their iOS app is pretty much all default system widgets and behavior for example, which is significantly less code to maintain than something that was customized heavily.
On the server side they don't retain conversations, have multi-session usage (many clients, one user), etc which simplifies what is required.
I haven't used whatsapp in a while so things might of changed, but before they were acquired by Facebook, that was definitely their direction.
The implications of this are pretty interesting. People will only use software that solves actual problems they have. Software development is now more accessible to more people because of all these frameworks and tools. It only follows that the people most familiar with actual problems will be able to create the most used software, and those people aren't necessarily going to be software engineers by trade. The question "How long before we have a billion-dollar acquisition offer for a one-engineer startup?" is crazy, but likely not too far in the future, but even more interestingly, "How long before we have a billion-dollar acquisition offer for a zero-engineer startup?" might not be that far off either.
The author makes some good observations but I feel he is undervaluing post-foundation engineering skills.
Sometimes the reason why certain companies end up with so many engineers due to the lack of those so-called "10x engineers" -- specifically, those who know how to build robust, scalable systems while coordinating with the needs of product development.
I know some of the engineers who have worked for those lean, highly scaled shops and I know for a fact that they know their shit extremely well - and they've poured hundreds of hours into carefully scaling their infrastructure. They could not be replaced by your average full-stack engineer off the street as they'd immediately be in over their head and would have to spend the next few years learning all the hard-won knowledge of the more experienced engineer.
Gerstenzang is watching the latest complexification of small shops into huge industries and projecting it onto the entire field. This is very silly reasoning and it ignores history.
Transported back in time, he could say the same thing watching the small, passionate computer shops of the 1970s becoming huge organizations as customer money and investment capital flow in, as more engineers are hired, as efficiencies and industry-wide standardization set in. He could say that the emergence of low-skill labor in the mature industry was the end of engineering, that there was nothing more to do.
But there will always be a new layer to this game. Once the previous layer has walked its arc from hobbyist to small shops to huge industries and maturity, there emerges a ragtag group that takes advantage of the new environment.
Gerstenzang does identify and appreciate this crucial factor, he just seems to view it as an endpoint and not a beginning.
Of course, an AI that can do real, human-level engineering might throw my own entire line of reasoning out of the window.
The article makes the error of assuming that a 10x engineer is 10x because he has arcane knowledge about the intricacies of low level programming. This may be true for some layers of development. In my experience, for application programming, this isn't the "10x skill".
10x engineers are the guys that can grasp the complexities of large systems, enabling them to change them with minimal effort. When prompted with the need to implement a new use case, they understand the whole system or most of the system and thus they can plot the minimal change path to achieve the new functionality. They don't code faster. They code less, leveraging work that is already done.
If you have fewer engineers working on a system, because of better abstraction layers, then you only need the 10x guys. These are the ones that can drive the powerful machines.
What we are witnessing isn't the demise of the 10x engineer. It's the demise of the code monkey.
A billion dollars for a one engineer social media company? Certainly possible, but then you'd have to ask yourself what you are paying for. Perhaps it's not just (or even mostly) the engineering; in which case I'd say the author slightly misses the mark here.
This is certainly not to say that I believe more engineers/engineering == better product or more value, far from it. Only that devaluation of skill and experience suggests that the problem being addressed might not be primarily focused on engineering tasks. And that perhaps the gargantuan sums we see being paid for some small collection of companies are calculated somewhat differently than we might be used to.
Even if I'm wrong about that, heralding the death of the highly skilled engineer is more than a little premature either way. I tend to think that problems will continue to present themselves to challenge our best tools, regardless of how much engineer quality of life improves. If that's not happening in social media (which I highly, highly doubt), then that can probably be considered a separate issue entirely.
110 comments
[ 4.3 ms ] story [ 169 ms ] threaddisclaimer, that's my blog! ;)
I'm not sure this is true. Are 10x programmers really 10x because they can grapple with technology stacks, or because they're really smart and experienced at problem solving? Programming will always be about problem solving, no matter how easy you make the final implementation of the design.
Specific skills (e.g. working with a certain framework) will go in and out of style, but problem solving will always be in demand.
Also, maybe I'm naive, but I just don't see the software-as-legos future as feasible. There will always be the need for customization and ever-more-advanced functionality.
If you spend a few years at university (I have two degrees), you learn things like sorts, automata, proofs, and calculus.
On the other hand, most of the junk I deal with day-to-day as a working software developer turns more on my knowledge of tools like Chef/IntelliJ, software libraries (the stdlib of various languages, the Java/.NET BCLs), build systems, git/github, and how to do a proper code review.
For most stuff today, I would highly prefer the person with more tools experience. Granted, there are some problems people who aren't "10x developers" could never solve (e.g. writing linux) but for most stuff industrial software devs today are doing, it just doesn't matter. You just need to write the code, it needs to be maintainable, and it needs to be done as quickly as possible.
So 5 years ago I started my first open source platform, http://phponpie.com
Since then it's come a long way, and it turned into a real platform: http://platform.qbix.com
Like mobile apps, social apps require an operating system that solves common problems in a standard way:
etc etc.All that stuff should have standard solutions which you can override. The apps and plugins should all interoperate.
And finally, since I really care about decentralization, it would be cool if this whole operating system could be as decentralized as Wordpress is for blogs, as opposed to say facebook. That would also solve all the scaling problems that facebook and other centralized social networks have.
That last part - decentralizing the social network - took the most effort, as I had to solve problems of security, identity, privacy, as well as app and plugin versioning and signing across a distributed network. Just like iOS is for mobile apps, and Bitcoin is for distributed contract apps, the Q Platform is if you want a social layer for your app, from user accounts to everything else, without having to write it from scratch.
Yeah, so that's what I built. Link to the open source: http://github.com/EGreg/Q
My experience with HN is that it's a very hard-edged, no-shit crowd. People here expect things to be described with simple, crisp language, no jargon (especially not computer/science/technology terms with precise meanings), and a minimum of hand-waving. It's a bit hard on the ego, but it'll make you a better communicator.
In the first sentence of your post, you described your thing as a "social operating system" or something like this -- this immediately sets off my bullshit alarm. "Operating system" is a very specific thing to software people, but for whatever reason, non-technical people love using this term to talk about something perceived to be big and complicated.
In general, try to cut the fat. If it's a library of software components you're making, say that. If it's a Ruby gem, or an app, or some kind of a specification, just describe it for what it is. You don't gain much here trying to hype it up.
Other things I notice:
- You say you can "override" -- how? Using what?
- What is a "social app", anyway? Aren't all apps "social" today? This descriptor (social) adds very little value. Github, facebook, twitter, instagram, linkedin, could all be considered "social" apps.
- "Social" is a very overused buzzword in SF. Everyone here is building bullshit "social" companies and all the devs here get like 10 job ads/day for some dumb social network for cats.
- Again, "interoperate". How? Via a REST API? Server-side includes? Be specific.
- Don't say "it's come a long way". Stop telling. Show.
- "Decentralized". Again, what does this mean? It can be hosted anywhere? The software is widely distributed? What is a "distributed network"? Aren't all networks "distributed"?
It's one thing to know that capital secretly delights over the commoditization of labor. It's quite another to watch a16z gush so openly about a future where we can be tossed aside.
If you are a great engineer, then you are a walking capital by yourself.
If somebody with an idea, and no skills, hires you, then it probably needs at least 300-500k (even for a smallish prototype) to make it happen. If you are a full stack engineer, and pair up with a buddy, you can make it happen, with a lot less. If you are a great engineer, you are the capital!
It only seems that way because engineering is currently the most capital-heavy part of bringing a softwarebusiness to market. Once that ceases to be the case (which may take a while, but it's the future SG was gushing about) then as long as there is a single scarce or mutually-exclusive commodity necessary for success its price will ratchet upwards until it is out of reach for most engineers (after risk-adjustment).
Pertinent example: the implosion of the indie app dev market that's been all over HN this past week.
If the next most expensive thing is the marketing spend, so that the company gets the required exposure in overcrowded app stores (for example), then the VCs are painting themselves into a Hollywood hit role - at best.
Of course there's an underlying BS premise that "in that wonderful future humanity would be freed from the burden of non-essential programming work, and would move on to higher level problem solving" -- but in real life they could not care less if humanity actually moves forward or just rich companies have to pay less while millions suffer and fall from middle class.
People talk as if everyone can become a creator, which it ain't so (partly because of the personal skills needed, partly because there's no economy where "everybody is a creator" -- for the same reason why not everybody can be a millionaire at the same time (money loses it's value then). Any large scale economy needs few sellers/creators and many buyers).
So what this trend does, it empower a few programmers/creators (ok, more than the past few large capital owners), and makes all the other programmers redundant or devalued.
Labor = show up, do your job, get paid. Capital = put money at risk without any guarantee of getting anything back, let alone earning a profit. Capital makes the rules because half of America is two paychecks away from bankruptcy (I read this somewhere credible) and can't/won't take any risk.
Remember, capital has TWO strategies for earning money:
1. Investing in new ideas, hoping they yield a positive ROI
2. Collecting dividends on ideas that have already panned out
#1 might conceivably be replaced by algorithms, but I doubt it (has HFT eaten Wall St?). #2 is inherent in our economic system and will never become obsolete.
What I don't understand is how someone who is so obviously wrong about the software craft and isn't even coherent within a short blog post can be a partner at one of the best VC firms. I sincerely hope he's there because of nepotism.
It's great when both problem-solving and outstanding engineering exist within the same individual, in some positions even essential.
Realistically, problem-solving is the more uncommon natural ability of the two. If it were not, there would be a lot less problems with software by now.
This analogy seems apt to me, and I think it makes the logical leap more obvious:
Fifty years ago, if you wanted to design a new model of car, you had to get plywood and clay, and work hundreds of hours of to get even the most basic model completed. Now, you can open up a CAD program and begin making real decisions immediately. What previously took a large team to build, a single designer can now do. (The analogous logical leap to what the OP seems to be saying is) Soon, we won't need designers at all! A company will come out with a new car model simply by snapping design-legos together.
In reality, removing inefficiency made designers more valuable. If you were the best car designer 50 years ago, you had to spend a significant amount of time molding clay. Today, you spend almost all of your time creatively finding solutions to problems. I see the value of individual engineers increasing significantly as we remove cruft in the creation process.
The value of engineering skills as they are defined today is falling, but the value of a single person with an idea and the drive to implement it is soaring.
Great engineering is no less valuable than it once was, but pure "idea guys" can do stuff now that they couldn't do before.
You might mean "nontechnical people can do a functioning MVP" which they couldn't before :)
Now I'm hearing the same in programming. And it's just as wrong here. Picking the right algorithm and stack means the difference between a website that can handle 2,000 people and one that can handle 2,000,000. How many websites start off with a bad idea, get popular and then need to hire hundreds of engineers to put ban aid after band aid on a fundamentally terrible idea?
Facebook and google have both had to spend millions of man hours to fix the fact they were optimized for millions of users and billions of web pages and now have to deal with x1000 times those. And the only reason why they were that prepared in the first place was because extraordinary people wrote them. The average developer who bangs out a half thought out colour by numbers website would be lucky to have it not crash before it reaches a thousand users a day.
Sometimes skimping on the engineering is better than skimping on the idea. That's what MVP and quick-to-market are about, correcting the idea so you can architect a solution to the RIGHT problem.
They'll use a combination of A/B testing and genetic algorithms to reach an equilibrium between what customers want and what physics will allow.
If something is easy enough, anybody can do it, and do it nearly as well and as fast as the best. For example, nobody can flip burgers 10x faster than average. Maybe 2x, but not 10x.
If something is difficult, then many people won't be able to do it at all. Most of the rest may only be able to do it very slowly and with great difficulty. The very best will make it look easy, even though it's not.
When something becomes 10 times easier, that doesn't mean the best become 10 times better or faster at it. They may become a little faster, but the real gains are at the low end: The average and worst performers become significantly better. Further, many people who just can't do the thing become able to accomplish it.
I work as a systems developer but often do consulting projects as an application developer, and it's a totally different mindset. Writing software for end users, I rarely even remotely think about CS, algorithms, pointers, recursion, numerical stability, memory allocators, etc, whereas I do care intimately about the tools I'm using, much like I suspect a carpenter cares about what drill he's got. On the other end of the spectrum, I can hardly go a day writing traditional "systems" software without considering things like what data structure MongoDB/Postgres uses to hold indices, and gigantic, complex consensus problems (distributed systems, e.g. how DNS behaves).
I hope we start to treat these two endeavors differently because they require totally different mindsets and education. A typical app developer need know no more about how OSes work, than a plumber needs to understand the metallurgy used to make pipes.
CS, algorithms, pointers, recursion, numerical stability, etc, etc, are all alive and well in the world of applications programming. Just not in the world of run-of-the-mill CRUDs.
All we have now are improvements in efficiency (great, mind you), with operational improvements like IaaS and development improvements with the great OSS that we have now. It will require a sea change in the field of comp sci. to achieve Lego-like orders of magnitude improvements.
The author focuses a lot on ease of scaling, but the bigger challenge has always been building applications (experiences) that need scale. If you build it, they will not come.
The assumption is that more people being able to make things with code would make the 10x engineer less valuable - but I think it makes them more valuable. In a noisy world - only the truly great can stand out. And to really make our future great, we are going to need a lot more than CRUD web apps.
I could be wrong - but I don't think this is as cut and dry as the author claims.
The web has been around for a quarter century now and still rather trivial websites that mostly could be replaced by tools sell for some $50k. Because even those basic tools we had 25 years to build are not there yet.
Software engineering is actually becoming proper engineering, at least in Europe where much of it is focused more on industry. Crashes are mostly unacceptable now. Embedded software is sometimes formally proved.
Finally, as everyone who actually develops software knows, we need to cut many needed features to ship in time. Think how incredibly long it still takes to implement what you can imagine: It takes many months. This is a good indication of how far we are actually from combining things in a lego-like fashion, from making dreams come true quickly because it is so easy.
One could argue the exact opposite: Engineers are needed now more than ever. A skilled engineer (what does this even mean in a context so wide?!) can run a website like PinBoard, serving millions of users, from the a jungle in Thailand, on a crappy 56k using a 250 USD ChromeBook.
Of course it's possible but how many people do actually manage to do it successfully?
And while having AWS, Github and you-name-it makes this possible (scaling was the big issue apparently), AWS and Github can not be run by a lonely engineer, you need an army of them.
WhatsApp was able to scale to so many users while keeping the engineering team small precisely because they hired great engineers who wrote an extremely efficient system in Erlang. (Does he think commodity coders know Erlang? Really!?) They heavily tuned their architecture and built efficient paradigms which kept developer productivity & performance from degrading as their user base grew. [1] Commodity engineers don't write that sort of code.
Yes, we increasingly have tools which allow mediocre coders to get stuff done. (For example, Ruby on Rails.) But the inevitable byproduct of letting mediocre coders loose on a stack they only partially understand is a mess of spaghetti code which takes even more developers to maintain. This is already what you see with non-tech corporations who employ dozens/hundreds of developers to maintain Drupal/RoR/Java apps which are significantly simpler in scale & complexity than WhatsApp or Instagram. If you want to use tools well, you need to hire people who understand the tools.
Yes, the tooling has improved dramatically. A 10x engineer of yesteryear might actually be a 100x engineer today. But if you give a bad engineer modern tools, it just allows them to fail faster.
I might be able to build a house with legos, but I expect to be spending a lot more time repairing it than if I'd just hired a carpenter in the first place.
[1] http://highscalability.com/blog/2014/2/26/the-whatsapp-archi...
The problem is they did it with only a few (16?) "great engineers". That's the premise of the article -- our world today might need 100,000 great engineers to solve great problems. Tomorrow 1000 might be enough.
As for the "near great" they would too be less needed, and commodity engineers would be the equivalent of burger flipers.
>Yes, the tooling has improved dramatically. A 10x engineer of yesteryear might actually be a 100x engineer today. But if you give a bad engineer modern tools, it just allow.
The first yes, the second not really. A bad engineer can get enormously more done with modern tools than he could 10 or 30 years ago. Essentially a bad engineer today can build stuff that took high level senior engineers to build -- and do it in less time too. It might be a little crude, but it will work.
E.g a bad engineer can automate a workflow that would have taken expert engineers, custom software and several man-years to do 20 years ago, and can do it in a day.
And because it's so abstracted, building something like that needs little influence from his skills, so his "badness" doesn't show up much.
"Tomorrow, that billion-dollar startup acquisition might not need an engineer at all."
The article's claim is not that startups are going to keep having smaller and smaller teams of higher-value engineers, they're saying that they're going to have zero engineers. Which I don't see at all from the evidence provided.
After all, the basic skills for a successful startup, even today, are not hardcore engineering skills, but idea, business, finances, etc.
I mean, most startups bought for multi billion dollars work on BS trivial engineering problems (mostly: make a social web service scalable), compared to what talented engineers do elsewhere (in research, in space exploration, in robotics, in DSP, etc).
Heck, wasn't Facebook started with crappy PHP code a 15 year old could write (and a 20 year old did wrote)? Or, for a smaller scale example, Derek Shivers (which I admire) sold his CDBaby startup for a nice sum, and that was also started and based on amateurish code at best that he wrote himself.
The premise of the article seems to be that we no longer need great engineers at all. Or that they only need to work at infrastructure companies (not application companies like WhatsApp). The very example he chose shows that's empirically untrue.
The fact that a small team of amazing engineers can accomplish huge things (given some ops support) is not new. The original Macintosh team was also tiny.
There's no evidence that we will ever stop needing great engineers, particularly as more and more applications require huge scale while maintaining agility.
Well, not really, because somebody will have to build all those elaborate abstraction layers people will use.
The core idea is that we'll need much fewer great engineers -- which is another way of saying we won't need "great engineers" at all. Read "not need them at all" liberally (very few), not literally (nobody).
After all if job demand goes from 1,000,000 people to 1000 people (arbitrary numbers), it's like it's the end of that job, even if technically people are still employed (there are after all horse cabbies and film lab technicians and letterpress printers employed still today).
It's almost as though you completely ignored my next sentence, which clarified that great engineers continue to be valued outside of abstraction/infrastructure companies.
An enterprise that used to need 100 engineers, with the cloud, SaaS, PaaS etc, can even today make do with 1/5 of that. Is there any trend showing companies needing MORE software engineers?
>How many people where creating their own languages 20 years ago?
20 years ago? Far less. That means writing a language is getting more commoditized -- and we work in a higher, easier, level of abstraction. That's what TFA says about other spheres of development and deployment too.
>Sure, an average engineer can do much more today, and the reason is because of these great engineers, and they are not fewer by any means.
Neither I, nor the article, said there are fewer great engineers. Only that fewer great engineers are (and fewer still will be) needed. Both as a percentage of all great engineers, and in absolute numbers.
It's also a problem, longer term, if you are an entrepreneur. Initially it starts as "lower costs", but with an impoverished middle class, you soon find there are not that many people to buy your fancy products and services.
Automation is only good is it's combined with a scheme for the surplus of labor -- e.g less hours work week, creation of new kind of jobs, etc. Without that is a huge creator of poverty and inequality, and eventually the collapse of a market economy.
On the server side they don't retain conversations, have multi-session usage (many clients, one user), etc which simplifies what is required.
I haven't used whatsapp in a while so things might of changed, but before they were acquired by Facebook, that was definitely their direction.
Sometimes the reason why certain companies end up with so many engineers due to the lack of those so-called "10x engineers" -- specifically, those who know how to build robust, scalable systems while coordinating with the needs of product development.
I know some of the engineers who have worked for those lean, highly scaled shops and I know for a fact that they know their shit extremely well - and they've poured hundreds of hours into carefully scaling their infrastructure. They could not be replaced by your average full-stack engineer off the street as they'd immediately be in over their head and would have to spend the next few years learning all the hard-won knowledge of the more experienced engineer.
Transported back in time, he could say the same thing watching the small, passionate computer shops of the 1970s becoming huge organizations as customer money and investment capital flow in, as more engineers are hired, as efficiencies and industry-wide standardization set in. He could say that the emergence of low-skill labor in the mature industry was the end of engineering, that there was nothing more to do.
But there will always be a new layer to this game. Once the previous layer has walked its arc from hobbyist to small shops to huge industries and maturity, there emerges a ragtag group that takes advantage of the new environment.
Gerstenzang does identify and appreciate this crucial factor, he just seems to view it as an endpoint and not a beginning.
Of course, an AI that can do real, human-level engineering might throw my own entire line of reasoning out of the window.
10x engineers are the guys that can grasp the complexities of large systems, enabling them to change them with minimal effort. When prompted with the need to implement a new use case, they understand the whole system or most of the system and thus they can plot the minimal change path to achieve the new functionality. They don't code faster. They code less, leveraging work that is already done.
If you have fewer engineers working on a system, because of better abstraction layers, then you only need the 10x guys. These are the ones that can drive the powerful machines.
What we are witnessing isn't the demise of the 10x engineer. It's the demise of the code monkey.
This is certainly not to say that I believe more engineers/engineering == better product or more value, far from it. Only that devaluation of skill and experience suggests that the problem being addressed might not be primarily focused on engineering tasks. And that perhaps the gargantuan sums we see being paid for some small collection of companies are calculated somewhat differently than we might be used to.
Even if I'm wrong about that, heralding the death of the highly skilled engineer is more than a little premature either way. I tend to think that problems will continue to present themselves to challenge our best tools, regardless of how much engineer quality of life improves. If that's not happening in social media (which I highly, highly doubt), then that can probably be considered a separate issue entirely.