Keep it simple, use boring tech, etc. We know. The whole issue is what is boring, what angle we wish to view simplicity/complexity from. These pretty much always boil down to “what I like”. What I like is simple and boring, your stuff is shiny and overengineered.
He gave examples that would be more accurately described as "tried and true" not "your stuff" — the code that has been battle tested in the field for decades.
It's not hard to find things that virtually everyone agrees are complicating and shouldn't be used in a project unless they have clear benefits. Metaprogramming. Macros. Multiple inheritance. Multiple languages.
The silver pants unicyclists know this too, they just lack impulse control.
I think this gets to the next level of the problem. Some people (usually earlier in their career) really get attracted to weird shiny stuff.
But in reality, discussions I have about arguing for simplicity have two opposing sides both basing on tried and true technologies in their opinion.
I find a good solution is to have reviewers and outsiders who can give feedback, and generally trust them when they recommend avoiding something. (But I have seen the cases where this exact approach would have killed a crazy idea that became a huge success story later on)
Almost everyone wants simple. But simplicity is subjective. My simple can be a touch screen and yours can be buttons and we would both be correct.
The whole article is genius (and funny) and rings true from many years experience in the biz.
> I’m imagining the archetypical unicycle-juggler buying a shop full of solid printers and weird CNC machines and forgetting to buy cutters, hacksaws, files and machinist squares. As if files and machinist squares are beneath them in current year.
Of all the things I could have commented on, I have to call out this last line simply because I only just saw a YouTuber last week 3D-print a part for a project he was working on: the part was the exact size and shape of a 4-inch long piece of standard PVC pipe you could pick up from Home Despot for a few bucks and saw the length you wanted in a few seconds.
In the last few months I’ve printed a few parts that could be bought OTS because I don’t want to go into a store in the era of COVID.
More generally, I’ve also printed stuff you can buy OTS because it can be less labor than going to the store, and occasionally cheaper. Also, sometimes it’s just more fun.
I suspect the author has a lot more exposure to the kinds of people who want to write their next web server basically in category theory than most people actually do, and so thinks that kind of person is much more common than they actually are.
But I still like the take a lot, particularly because of the perfect truth of the line "[Go] is a boring garbage collected thing that looks like C for grownups, or Java not designed by 90s era object oriented nanotech fearing imbeciles. "
There was an article here a few days ago about"incompetent altruism" or something like that. The commonality is whether you're laser focused on solving the problem, versus being obsessed with yourself as the solver.
The bit about us not being smart enough for some of the technologies also rings really true. Thinking of a coworker who was "smart" enough to reach for threads right away, and then inevitably spent the next 2 years of any project dealing with thread issues in production.
Exactly. Over the years I've learned to keep my self-indulgence to my hobby projects. When I'm coding for money, I try very hard to set aside my ego and do the best thing for the users. As much as I am personally motivated by novelty and curiosity, when I'm on the clock I'm a lifetime member of the Boring Technology Club: http://boringtechnology.club/
"Everyone knows that debugging is twice as hard as writing a program in the first place. So if you're as clever as you can be when you write it, how will you ever debug it?"
It's definitely true, there's a frequent temptation in software engineering to be drawn to novel technologies.
That said, the idea of different data types in C could have been considered a fancy addition by BCPL developers.
The key is to use critical thinking to determine what developments may be useful longer-term, and also to watch out for any (typically accidental) flaws exist in the novelties that are being proposed - sometimes it takes years, but those can come back to bite us.
It may not make business sense to exert effort shifting to new technologies straight away, but encouraging people to learn the strengths, weaknesses and properties of them can be valuable nonetheless.
I had a few job interviews last year where it was clear the people trying to hire me had plenty of programming talent on board already. They were just looking for a technically competent adult who could look at what was going on and figure out why all the fancy work that was getting done was not doing what the rest of the company needed and expected.
One company had several different teams of programmers that were great at building things, but they were all going in different directions with different tech stacks (two in Go, one in Rust, and one in Python) and completely different data models for the company's basic business domain. When a directive came down from above that two systems HAD to be integrated, because a customer paying for one product wanted the other one as well, as sales and product had been hoping and planning since before the products were built, the engineers gave them an estimate of four months to do the integration. The business knew that meant 8+ months. How many programmers did they have to get this disjointed? Less than twenty.
Lots of other companies have similar stories. I disagree that it is about lack of financial compensation. These companies were paying good money, and their engineers were living comfortably and abusing the relationship, whether wittingly or unwittingly.
Anybody who reads HN can whip up an impressive, Netflix-blog-worthy design for any problem their company has. If their company is unlucky, they might even have the energy and raw hacker ability to get it mostly working. Where we're failing as a profession is in educating people that this is poor engineering, that it's bush league bullshit. It isn't impressive. It doesn't show any ability except a certain amount of shallow raw intelligence, which is necessary and valuable, but which is more like raw material to build ability out of than ability itself.
In other forms of engineering, the culture is very clear: simplicity is an achievement. Solving an easy problem with a big, expensive solution is not an achievement. It's unskillful. The best engineers come up with solutions that make you see the problem as an easy one. Solving easy problems with difficult solutions proves you aren't qualified to take on the problems that need those solutions. How do we establish this as a standard belief in software engineering as well?
A lot of what you describe is attributable to resume-driven development, in my opinion. Complex solutions are used to insinuate that the engineers who implemented them are solving complex problems. This in turn improves their marketability by insinuating they've got the razzle dazzle and trendiness under their belt that justifies $500k a year.
Sometimes it's also not resume-driven development, at least not intentionally. Sometimes it's because the engineers just want to work on overcomplicated shit. For example I have my personal blog running on half a dozen AWS features - S3, Athena, Cloudfront, Cloudwatch, Lambda, SNS, SES, ...
I don't need any of that for a simple blog, and I write on it like once a year. And the site is still slow as shit (relatively) even though it's static because I haven't optimized the font and script loads to be non-blocking. But it was fun to set up!
There is also an element of each project optimizing to its own minimum. If each project selects the best language, tech stack, hosting stack, etc, for itself, it creates an overarching complexity across projects.
Fixing this requires lots of communication and leadership. Each project may need to select sub-optimal components (like language) for itself, but in doing so it can remove overall complexity.
What you describe you did with your blog is, in some sense, resume-driven development: you probably learned stuff in the process, which might be useful in a future job situation.
The difference is that nobody paid you for doing all that unnecessary stuff. Doing unnecessary things for learning is good, but not what a company is paying you for, usually.
I agree about the resume-driven development aspect, and I think it's great to use a personal project as an outlet for experimentation! Too many people do stuff like that at the expense of unnecessary complexity in production systems.
I'll grant that collectively an engineering team might decide to overengineer a simple, noncritical service to get production experience with a technology or two that they are targeting for future adoption. Nothing wrong with that, and there's nothing wrong with making room in the schedule for engineering experiments to test out new technologies.
Managers and the rest of the business, for their part, need to understand the need for experimentation so it can be done honestly and not under the guise of other work.
Folks with the outside perspective often attribute some intent to a situation, but if they went through it, it would be obvious and simple.
I think engineers try to do the best they can. It's just old guys might do it the mainframe way, slightly younger unix, then windows, then linux.
Or folks who've been with the culture do it that way, and new folks from outside the company bring new college grad or last-company-i-worked-for culture along with them.
And folks who try something new suffer from the 'fog of war' and your "the docker solution" that got everything started so well makes you a real expert in nested docker configurations and firewall issues. :)
I think part of the difference in mentality comes from the difference in material constraints.
If a mechanical engineer designs something that needs a ton of fancy moving parts, those moving parts significantly and directly impact their budget (I assume), because they're going to have to be purchased or made for each widget that gets manufactured. They also unavoidably impact physical properties of the object, like weight and dimensions.
Whereas a programmer can spin out as much code as they want, basically without material consequence. There is no constant gravitational force keeping them in check, requiring them to justify each piece of complexity as they go. It's ethereal, weightless, and its disadvantages usually only surface years down the line.
In my experience, the costs are felt immediately, because there is more code and more complexity. Virtually every cost in software development goes up when there is more code and more complexity. There are more bugs. It takes longer to diagnose and fix bugs. It takes longer to add new functionality. Performance is less predictable, and performance problems are harder to diagnose. Onboarding new programmers is slower. Fewer members of the team can do critical work on the code. Routine library upgrades are more likely to impact the code and take longer when they do.
Code is a liability, in other words. It may not be tangible, but you can think of it as weight that the team has to carry everywhere it goes.
You're totally missing my point, which is that all of those costs are abstract and protracted. They aren't felt as you're literally typing the characters; they can't be listed as concrete numbers in a budget. If an engineer is using a CAD tool and they add a part, they can see the space that's no longer available for other parts. Or they can see that the product would have to be made physically larger to accommodate it. There is concrete feedback to contend with during the development process, not just a vague notion of future reliability.
As you ask for simplicity, I think this can be succinctly summed up as a lack of engineering leadership.
I know having completely independent teams is the in thing but there needs to be some high level design and coordination. I'm not even against different teams using different tools but the integration story has to be there from day zero if it's a business requirement.
Ultimately there needs to be an engineer who has both the power and knowledge to coordinate design decisions.
I agree with this 100%, and I think the people providing this leadership should be in management. Software engineers need to be managed by someone who can understand and evaluate their work, because of what you said and also because other aspects of management benefit from the ability to cast a critical eye over the work being done as well.
EDIT: Going further, I think the people trying to hire me as an engineer to fix their problems should have replaced their front-line management with managers who understood code, so they had some clue what their engineers were doing wrong and how to fix it. The idea that engineers should be able self-manage is a rare best case scenario, not a best practice that can be replicated across the industry.
In other forms of engineering, the culture is very clear: simplicity is an achievement. Solving an easy problem with a big, expensive solution is not an achievement. It's unskillful. The best engineers come up with solutions that make you see the problem as an easy one. Solving easy problems with difficult solutions proves you aren't qualified to take on the problems that need those solutions. How do we establish this as a standard belief in software engineering as well?
My experience with things like foreign automotive components suggsts this is not exactly true in general... but it's true that monsterously complex solutions seem to be far more common in the software world.
German sport / luxury cars are renowned for excessive "because we can" capability, and the complexity to go with it.
A cynic would say the complexity does not even have to bring extra capability; frequent and costly maintenance is an important part of the status signaling.
This is diametrically opposed to e.g. Toyota's simple, traditional, robust, and scalable transportation appliances.
The German car situation in Europe is a bit different, several popular explanations:
- Lower spec versions are available and common; the brands themselves are not inherently special or upmarket
- This means mechanics who work on the brands do not automatically command a premium
- Parts are not subject to import duties and supply chain cannot get away with as much markup (after Brexit who knows?)
- Culture of better compliance with scheduled maintenance
- Depending on location, better maintained roads and milder weather
Also they have invested a lot in reliability in the 2010s and some of the reputation is a holdover from previous generation models. Problems will still be outrageously expensive to fix compared to Japanese cars but are not so disproportionately likely to occur.
My dad briefly owned a BMW as his midlife crisis car. The bi-xenon headlights looked better than any I had ever seen. But when one burned out it was $1500 between the part and the labor to remove practically the whole front end. This is considered a characteristic, “classic BMW” kind of problem.
I almost hate to say it, but the answer to your last question might lie in instituting the kind of standards, credentialing, mentorship and licensing that exist in the "real" engineering fields. Stuff that will correctly be called out as unfortunately really "hampering individual freedom" (including the freedom to be a jackass) and being "unnecessary" and "bureaucratic" and (quite intentionally and without quotation marks) gatekeeping.
Civil engineering is sort of the oldest and ultimate example, where a bad design puts people's lives at risk. In deference to said lives, senior engineers in that field review every drawing or calculation that goes out the door, and they also mentor the junior engineers, who have to be graduates of accredited schools, and also can't get certified until they've spent a certain amount of time being mentored and passed the exams, etc.
The mindset you're talking about is basically all about craftsmanship or elegance of a solution. Formal mentorship or apprenticeship programs do a great job of maintaining and transmitting that. School isn't really enough, and certain fields have recognized that fact.
In software however, and I'm not sure whether this is the problem or the symptom, the number of developers apparently (I was told recently) doubles approximately every 5 years. Which means the field is always overwhelmed with new people, full of energy but not experience, who want to do something cool and exciting, but haven't been, for example, burned by their own past decisions to try to do something cool and exciting. They see only upside. Which is valuable in itself, but they need more direction than they're getting, from the older engineers who have the war stories and who aren't especially concerned with being cool (which c'mon let's face it, is essentially a mating display) and who are boring.
I think traditional engineering standards would enable over-engineering rather than discouraging it. In fact, I think engineering standards would be every architecture astronaut's dream, because it would be an excuse to architect every small company's software to handle Netflix load. "This is an X type problem, and the standard says the data for X type problems has to be stored in a distributed event log." We would have to flip the standards on their head and write them to say, you are not allowed to use Kafka until you have a problem meeting certain requirements. Projections from our sales pipeline say we can only use technologies certified for less than ten requests per second.
Can you imagine a bureaucratic oversight committee rejecting a CQRS design with streaming everything and telling an engineer, "Seriously, WTF. This is a config console. It will have less than 300 users two years from now even if you live up to that insane hockey stick graph they showed the investors. Just build a basic CRUD app backed by RDS. Use the same front and back end frameworks you use for everything else. The engineering standard says you can't afford to waste time on this because there's a lot of other work you're going to need to do this year, and some of it might turn out to be hard, even if you don't know what it is yet." Can you imagine that?
Math and physics also have a strong cultural orientation towards simplicity, which seems to be universally embraced. If you solve a homework problem in half a page, and the professor calls on somebody else to put their proof on the board, and they walk up with two pages covered front and back, you don't think, "Wow, they're smart. Their solution is so much more complicated; it must be better." Conversely, if you and all your friends took three pages to solve a problem, and somebody else presents a solution that fits on one chalkboard, you're in awe. I really don't know what the difference is that makes the culture of software different.
I think if it's being enforced by standards or licensure, it's too late and too external. It needs to be internal and cultural, at a gut level, to aspire to do more with less. In software, the primary measure of cost, the definition of "less" in "do more with less," is the intellectual load on the software development team. (We have that in common with mathematics.) Do more with dumber. How do we learn to value that as a profession?
this is a funny and thought provoking article. There's a bit of texture that I feel is missing, and I feel personally attacked, so I'm going to throw up a weak defense of category theory web servers.
Back in the day, in perl I'd bless regexes and use AUTOLOAD to provide the class implementations at runtime, as any good unicycle rider would. Changing the Rube Goldberg machine is hard. grep won't really help. I'd resort to even dirtier tricks to change behavior.
Later I worked in java. grep would work, but even better, I could make a class private, create an interface and just chase compiler errors. So much nicer than hunting down the right line of code to hijack in perl. It's pretty good, and I think go provides a similar level of support, but there's still a bunch of room for Class.forName(foo); //haha! grep will never find me! and it's cool, cause you'll only find out about it at runtime.
Anyway Haskell is neat because you can throw it all in a do in IO. hack away for days. When it's time to change, just change it. the compiler will help. The compiler will tell you everything you need to know to scope down to the finest level you want. It's super easy to be way too fancy up front, but any mistakes can be fixed.
Anyway, Haskell is probably never the right choice but it's the only thing I know of that lets you easily change the design whenever you feel like it.
It's Saturday, and I've thought about this too much. Time to put on my silver pants and go unicycling.
Tools should be chosen to match the use case. Haskell is a good choice when you've got:
1) a problem that requires a very high-assurance system with very low tolerance for bugs, errors, and failures, and
2) a large complex system where refactoring and other maintenance-over-time activities are made materially more manageable by Haskell's strong type system and other tools,
3) a need for large-scale parallelization that a purely functional language like Haskell facilitates, and
4) a world class team of engineers who can become productive in it relatively quickly, support new hires doing so too, and won't foreseeably be outsourced to some low cost overseas sweat shop.
Rust is also worth considering under those criteria, in addition to embedded systems, hard/soft real-time or constant-time systems, etc.
But obviously not all software projects have all those criteria. Other languages like Go, Python, Javascript, Java-ecosystem, .Net, etc may be ideal in other circumstances. You just have to match the tools for the job.
One thing I've noticed in my career is that those who possess a certain kind of 'wow' factor are the ones who are categorised as the 'stars' and thus progress quickly through the technical ranks.
I've tended to find that these people don't write particularly good code (not usually bad, just mediocre) and tend to lack ability in software engineering - writing deeply unrobust code that falls far short of their perceived 'rockstar' qualities.
I can't tell you the number of conversations I've had with managers or even colleagues where their untouchable genius is extolled to the heavens.
Combine this with the fact that those who raise issues and foresee problems are often viewed poorly (possibly even MORE so if those problems actually end up happening).
Ultimately you end up with a situation where those who advocate for solid engineering are, while respected, never really raised above the low levels of the career ladder and those who embrace woo rise up.
> I’m imagining the archetypical unicycle-juggler buying a shop full of solid printers and weird CNC machines and forgetting to buy cutters, hacksaws, files and machinist squares.
What are you doing? What are the constraints? What is the problem?
Do you need innovation? Do you need plumbing?
Can you lift your eyes and see the value in a radically new way? Does it look like what you are already doing, or does a beginner's mind really suggest a new approach?
The author sounds like a crabby old guy who doesn't want to change. Probably an artifact of the topic. But it *is* a real problem- people get locked into fancy-chasing. This introduces risk into the project delivery: team based (knowledge) and tech (not adequately debugged).
I've worked at length in "fancy" languages and some of the fancy tech. My conclusion is you have a limited amount of investment you can do. Some of this is team based (language learning, keeping the employees happy), some is in product, some is in tech stack (FlashyFancypants DB and SparklePants.JS). Good tech leadership includes picking out what to take risks on, judging appropriate ROI. Note this holds true for open source, not just business projects!
I work in a Clojure shop, so I interact with the rewards and risks of this strategy right now. My last team/company, we chose Go for the team language. It wasn't as unilaterally good as some might hope. Nor was it as unilaterally as bad! It was just another language.
The “just solve the problem” attitude also gets used as an excuse to ship the worst code that can get through user acceptance testing.
The worst bridge that can stand up to the load is good, cost efficient engineering. The worst code is not. One, because bad code hides bugs that you probably don’t have rigorous enough testing to find. Two, because there will be new demands on that code in the future, and bad code is harder and riskier to change. You might look like a responsible adult when you ship it today, but three years later the new maintainers will know you were a child making a mess. And probably end up doing a rewrite, or at least needing to.
All this “woo” with languages and frameworks and services and whatever else, is the search for higher quality code.
There is a certain myopia about new things vs. getting better at design with old things. There is a certain blindness to iterative improvement, e.g. Rust is much sexier than modern C++ as a solution to the problems of C++98. But rather than condescending to this you can channel the energy into more responsible directions.
I work in a team where many people have zero aesthetic sensibility about code. Not like “I know this is ugly but we have to get it done today” or “I know this is ugly but it’s the best we could come up with,” just zero sense that it’s ugly. Grass is not greener.
Trying to explain that code is more than "well, it works!" is especially hard. Maybe its just the part of programming that's more an art than a science?
programming is a systems science - it is qualitative and constraint-based, not metric-based. It's an optimisation problem where you have to optimise for multiple concerns - maintainability, extensibility, scope, testability, and so on. Overengineering in a way that doesn't help any of these is falling at the first hurdle.
20% content, 80% sneering and self-pleasuring about inferior other people. It's not that solid trustable systems don't exist, or are a bad idea, or chasing the new-shiny doesn't happen, but they're not really what the post is about.
> underpaid compensated in self esteem [..] manager saying “dude yer so fookin smart” [...] brainlet payment system. [..] benefit to boring languages is people concentrate on the problem not interesting complexities of tool
Missing the point raised earlier: "the problem" being solved is that work is boring; Woo isn't added "for its own sake", it's added because being an interchangable cog in a relentless productivity optimization machine isn't what humans exist for, no matter how much employers and the author would like that to be so.
> people who feel like every line of code has to be innovative
Employees don't get paid or promoted for saying "this is good enough, let's take 18 months off", but they do for adding a new widget or showing off to the boss. Broken incentives lead to people following them.
> like bike-shedding, but more perverse
It's pleasure! Spit! Disgusting, sinful and shameful, and the people who enjoy it are inferior. Get back to the grindstone!
> you are presumably solving real world problems which someone pays for
"Impress my manager", "buy some food", "survive this stupid doomed project nobody wants that doesn't have the political backing or funding to get to its impossible goals" are problems people solve for money. Jobs where people solve real world problems seem a small subset of jobs.
> about as necessary and in demand as riding to work in silver pants on a unicycle
The way someone commutes to work has so little to do with anything that it's a big reinforcement this is about image, not engineering.
> developers [..] are often child-like in their enthusiasm for new and shiny things
Everyone knows that companies are at their best when petrified, stagnant, resistant to change and unwilling to entertain new ideas, right?
> Engineers are different: they’re trying to solve a problem
No-true-Scotsman so that "Engineer" now means person who tries to solve a problem (only using existing tools and techniques), and people titled or qualified as "engineer" who aren't trying to solve problems, or anyone who is trying use a new approach to solve a problem or doesn't have the same constraints and goals are "not real engineers".
> Engineers understand it’s OK to solve the problem with ephemeral, trashy, but fast-to-market solutions
Nothing the world loves more than falling bridges, burning skyscrapers. A moment ago the important thing was to build stable, long-term, easily maintained systems with boring technologies (a classic description of "engineering"). Now Engineers are building ephemeral trashy minimal viable products? Is trashy and ephemeral "engineering" now? (No, the point is who to mock and shame, and who is superior).
> if the product manager is going to change it all next week
Could be related to why the idiot in the clown shoes is enjoying building it in Haskell, knowing that it won't need to be long-term maintainable because it doesn't matter anyway. No, it must be because they're inferior to the blog post author, who is all about solving real world problems like making sure software engineers know their status is beneath him.
> Engineers also plan for the future [..] critical infrastructure that lives [..] depend on
Engineers do the right thing in the right circumstances, and you can tell they are engineers post-hoc when it worked? Handwave away any critical infrastructure which failed as "not built by engineers", e.g. Boeing must have put no engineers on the Maxxx, you can tell because it ki...
> long term software engineering looks [like] this. Stuff you can trust looks like a giant one-piston semidiesel
Does it? Oracle's JVM - is it a giant simple on-piston semidiesel or untrustworthy? It seems complex and trusted to me. Linux kernel, so simple such that a fisherman can fix it in the dark and cold and wet by burning pile of logbooks? AMD64 instruction set, trivial or untrustworthy? Stuff you can trust doesn't /look/ like anything in particular, it behaves in a trustworthy way long enough to get a repuation.
> Duct tape programmers don’t give a shit what you think about them
It makes more sense to say unicycle riders don't give a shit what you think about them, since they're the ones actually demonstrating non-conformity through their actions despite social pressure to stop doing that.
> the perverseness and destructiveness of people who try to get fancy for no reason, nor do I think JWZ was a “duct tape programmer” -he was an engineer.
Gone from saying duct tape programming is good, to defending someone respectable as "not a duct tape programmer" as if it was bad, then applying the True Scotsgineer label retrospectively - it must have been true engineering(tm) because it didn't not-ship, QED.
> OCaML, APL and Lisp [..] small teams can deliver
Small teams have tried to deliver same in many languages and processes, we see the survivors and say "this tool must be why". That needs a large evolutionary body-count of attempts in other languages which failed, and many attempts in OCAML/APL/LISP which delivered nothing. If nobody had ever used O/A/L because they weirdo unicycling languages, there would be no knowledge of whether they could do this. Rust and Golang came out about a decade ago, author is clear that Go is for engineers and Rust for unicyclers, but both came out of respectable teams that built complex software maintained for years, with good reasons (memory safety, not "add Monads to C++"); at what point in the past ten years did it become clear to the author that Go is engineering and Rust is unicycling? Was it clear that O/A/L were going to shake out well in 60 years, after their first 10 years?
> For mission critical [..] hit by a bus [..] deep sheeyit
Like the boat engines, there are visible examples where "deep shit" happened - e.g. crippling ransomware attacks, systems being offline for days, huge data leaks - and a month later there are no visible consequences; PR puts out an apology statement, bonuses are unaffected, business as usual. If your startup run by a true engineer using Golang gets hit by a bus it could be the end of the company too. If IBM has a team using Haskell and one of them dies, the project is swallowed up and there isn't any deep sheeyit that anyone outside the immediate customer blast radius will notice.
> Using the latest tool is even worse. 99 times out of 100, the latest woo in machine learning is not an actual improvement over existing techniques. 100% of the time it is touted as a great revolution because it beat some other technique … on a carefully curated data set. Such results are trumpeted by the researcher because …. WTF else do you expect them to do? They just spent a year or two developing a new technique; the professor is trying to get tenure or be a big kahuna, and the student is trying to get a job by being expert in the new technique. What are they going to tell you? That their new technique was kind of dumb and worthless?
It's like you understand incentives, but pretend you don't when it gives you an opportunity to sneer at people you want us to think you're superior to. Why don't you see that the same incentives lead to people introducing NewLang because the upside is a potential huge status boost?
> just quietly used by engineering types who find them helpful
probably get used by engineering types who find them unhelpful and wish...
There are vast swathes of counter-examples to the article, to the point where I don't think that, given the business framing of the OP, the article actually holds water outside of "some people glue useless things on and it's a pain to deal with", which is sort of... known already?
He kinda rags on Haskell. I am wondering what people on HN think of pandoc, a Haskell project by someone outside the software industry for converting to/from .docx and other formats. I like it because it can be used as a single binary. Is there some other preferred way amongst so-called "software engineers" to do the sort of conversions pandoc does. Python libraries, maybe?
Pandoc is fine, but it is literally the most popular OSS Haskell project, and it’s a super obscure document converter. The most popular OSS projects in Go are Docker and Kubernetes (maybe a strike against Go? Lol) and Rust has this thing called “Firefox”. Haskell has been around for twenty years and by any objective measure has made almost no direct impact on OSS. It’s a thing programmers do for fun on the weekend instead of Sudoku. It had some influence on Rust and Swift, but all of the software written in Haskell could disappear tomorrow and no one would notice.
Yes, and? The point is that Firefox substantially depends on Rust and is the most prominent OSS Rust project. People would notice if FF went away one day. No one would notice if pandoc disappeared until they got some weird files they needed to convert.
This article would make a lot more sense if it was criticizing something more prevalent (e.g. JavaScript library churn, etc.)
Point taken, but I see no reason to straw man Haskell and Rust. They have their uses and new successful features they explore get ported back into the boring languages. This is how technological development works. For a non-programming example, anti-lock brakes were used in Concorde jets before they were used in automobiles.
Also, one must consider that Java was a 'shiny' language once.
Honestly, if OP knows of many companies that are picking Haskell or Rust over the likes Java or Golang, please post links. I'll happily apply.
Java was never a shiny language. It was many things. It was, most particularly, a ticket out of the world of the Microsoft sharecropper, needing to learn a whole new Microsoft Framework every second year, and discovering over again each time all the shit that didn't work, that couldn't be made to do what it was for, and cobbled-together ways to get around it instead.
I managed to never need to code on Microsoft until a bit around 2010, so missed the appeal.
I was amazed to discover that, even in 2010, nobody who had coded for MS their whole lives knew any better than I did how to get an MS OS to do any sensible thing. I would ask, and they would start rooting, trying random things until something seemed to work. The idea of a documented, stable, standard library component that actually did what it said without crashing was an absolute revelation, but the opposite of every shiny thing imposed on them before.
This reads like a paraphrase of David Graeber's "On the Phenomenon of Bullshit Jobs" but with chan-speak loan words like 'brainlet' mixed in for flavor. Is the author trying to suggest that software development is a skilled trade but that most jobs in the trade are bullshit jobs best outsourced to Elbonia?
57 comments
[ 3.2 ms ] story [ 131 ms ] threadThe silver pants unicyclists know this too, they just lack impulse control.
But in reality, discussions I have about arguing for simplicity have two opposing sides both basing on tried and true technologies in their opinion.
I find a good solution is to have reviewers and outsiders who can give feedback, and generally trust them when they recommend avoiding something. (But I have seen the cases where this exact approach would have killed a crazy idea that became a huge success story later on)
Almost everyone wants simple. But simplicity is subjective. My simple can be a touch screen and yours can be buttons and we would both be correct.
> I’m imagining the archetypical unicycle-juggler buying a shop full of solid printers and weird CNC machines and forgetting to buy cutters, hacksaws, files and machinist squares. As if files and machinist squares are beneath them in current year.
Of all the things I could have commented on, I have to call out this last line simply because I only just saw a YouTuber last week 3D-print a part for a project he was working on: the part was the exact size and shape of a 4-inch long piece of standard PVC pipe you could pick up from Home Despot for a few bucks and saw the length you wanted in a few seconds.
More generally, I’ve also printed stuff you can buy OTS because it can be less labor than going to the store, and occasionally cheaper. Also, sometimes it’s just more fun.
So edgy.
But I still like the take a lot, particularly because of the perfect truth of the line "[Go] is a boring garbage collected thing that looks like C for grownups, or Java not designed by 90s era object oriented nanotech fearing imbeciles. "
The bit about us not being smart enough for some of the technologies also rings really true. Thinking of a coworker who was "smart" enough to reach for threads right away, and then inevitably spent the next 2 years of any project dealing with thread issues in production.
https://en.wikiquote.org/wiki/Brian_Kernighan
That said, the idea of different data types in C could have been considered a fancy addition by BCPL developers.
The key is to use critical thinking to determine what developments may be useful longer-term, and also to watch out for any (typically accidental) flaws exist in the novelties that are being proposed - sometimes it takes years, but those can come back to bite us.
It may not make business sense to exert effort shifting to new technologies straight away, but encouraging people to learn the strengths, weaknesses and properties of them can be valuable nonetheless.
One company had several different teams of programmers that were great at building things, but they were all going in different directions with different tech stacks (two in Go, one in Rust, and one in Python) and completely different data models for the company's basic business domain. When a directive came down from above that two systems HAD to be integrated, because a customer paying for one product wanted the other one as well, as sales and product had been hoping and planning since before the products were built, the engineers gave them an estimate of four months to do the integration. The business knew that meant 8+ months. How many programmers did they have to get this disjointed? Less than twenty.
Lots of other companies have similar stories. I disagree that it is about lack of financial compensation. These companies were paying good money, and their engineers were living comfortably and abusing the relationship, whether wittingly or unwittingly.
Anybody who reads HN can whip up an impressive, Netflix-blog-worthy design for any problem their company has. If their company is unlucky, they might even have the energy and raw hacker ability to get it mostly working. Where we're failing as a profession is in educating people that this is poor engineering, that it's bush league bullshit. It isn't impressive. It doesn't show any ability except a certain amount of shallow raw intelligence, which is necessary and valuable, but which is more like raw material to build ability out of than ability itself.
In other forms of engineering, the culture is very clear: simplicity is an achievement. Solving an easy problem with a big, expensive solution is not an achievement. It's unskillful. The best engineers come up with solutions that make you see the problem as an easy one. Solving easy problems with difficult solutions proves you aren't qualified to take on the problems that need those solutions. How do we establish this as a standard belief in software engineering as well?
Sometimes it's also not resume-driven development, at least not intentionally. Sometimes it's because the engineers just want to work on overcomplicated shit. For example I have my personal blog running on half a dozen AWS features - S3, Athena, Cloudfront, Cloudwatch, Lambda, SNS, SES, ...
I don't need any of that for a simple blog, and I write on it like once a year. And the site is still slow as shit (relatively) even though it's static because I haven't optimized the font and script loads to be non-blocking. But it was fun to set up!
Fixing this requires lots of communication and leadership. Each project may need to select sub-optimal components (like language) for itself, but in doing so it can remove overall complexity.
The difference is that nobody paid you for doing all that unnecessary stuff. Doing unnecessary things for learning is good, but not what a company is paying you for, usually.
I'll grant that collectively an engineering team might decide to overengineer a simple, noncritical service to get production experience with a technology or two that they are targeting for future adoption. Nothing wrong with that, and there's nothing wrong with making room in the schedule for engineering experiments to test out new technologies.
Managers and the rest of the business, for their part, need to understand the need for experimentation so it can be done honestly and not under the guise of other work.
I think engineers try to do the best they can. It's just old guys might do it the mainframe way, slightly younger unix, then windows, then linux.
Or folks who've been with the culture do it that way, and new folks from outside the company bring new college grad or last-company-i-worked-for culture along with them.
And folks who try something new suffer from the 'fog of war' and your "the docker solution" that got everything started so well makes you a real expert in nested docker configurations and firewall issues. :)
If a mechanical engineer designs something that needs a ton of fancy moving parts, those moving parts significantly and directly impact their budget (I assume), because they're going to have to be purchased or made for each widget that gets manufactured. They also unavoidably impact physical properties of the object, like weight and dimensions.
Whereas a programmer can spin out as much code as they want, basically without material consequence. There is no constant gravitational force keeping them in check, requiring them to justify each piece of complexity as they go. It's ethereal, weightless, and its disadvantages usually only surface years down the line.
Code is a liability, in other words. It may not be tangible, but you can think of it as weight that the team has to carry everywhere it goes.
I know having completely independent teams is the in thing but there needs to be some high level design and coordination. I'm not even against different teams using different tools but the integration story has to be there from day zero if it's a business requirement.
Ultimately there needs to be an engineer who has both the power and knowledge to coordinate design decisions.
EDIT: Going further, I think the people trying to hire me as an engineer to fix their problems should have replaced their front-line management with managers who understood code, so they had some clue what their engineers were doing wrong and how to fix it. The idea that engineers should be able self-manage is a rare best case scenario, not a best practice that can be replicated across the industry.
My experience with things like foreign automotive components suggsts this is not exactly true in general... but it's true that monsterously complex solutions seem to be far more common in the software world.
A cynic would say the complexity does not even have to bring extra capability; frequent and costly maintenance is an important part of the status signaling.
This is diametrically opposed to e.g. Toyota's simple, traditional, robust, and scalable transportation appliances.
- Lower spec versions are available and common; the brands themselves are not inherently special or upmarket
- This means mechanics who work on the brands do not automatically command a premium
- Parts are not subject to import duties and supply chain cannot get away with as much markup (after Brexit who knows?)
- Culture of better compliance with scheduled maintenance
- Depending on location, better maintained roads and milder weather
Also they have invested a lot in reliability in the 2010s and some of the reputation is a holdover from previous generation models. Problems will still be outrageously expensive to fix compared to Japanese cars but are not so disproportionately likely to occur.
My dad briefly owned a BMW as his midlife crisis car. The bi-xenon headlights looked better than any I had ever seen. But when one burned out it was $1500 between the part and the labor to remove practically the whole front end. This is considered a characteristic, “classic BMW” kind of problem.
Civil engineering is sort of the oldest and ultimate example, where a bad design puts people's lives at risk. In deference to said lives, senior engineers in that field review every drawing or calculation that goes out the door, and they also mentor the junior engineers, who have to be graduates of accredited schools, and also can't get certified until they've spent a certain amount of time being mentored and passed the exams, etc.
The mindset you're talking about is basically all about craftsmanship or elegance of a solution. Formal mentorship or apprenticeship programs do a great job of maintaining and transmitting that. School isn't really enough, and certain fields have recognized that fact.
In software however, and I'm not sure whether this is the problem or the symptom, the number of developers apparently (I was told recently) doubles approximately every 5 years. Which means the field is always overwhelmed with new people, full of energy but not experience, who want to do something cool and exciting, but haven't been, for example, burned by their own past decisions to try to do something cool and exciting. They see only upside. Which is valuable in itself, but they need more direction than they're getting, from the older engineers who have the war stories and who aren't especially concerned with being cool (which c'mon let's face it, is essentially a mating display) and who are boring.
Can you imagine a bureaucratic oversight committee rejecting a CQRS design with streaming everything and telling an engineer, "Seriously, WTF. This is a config console. It will have less than 300 users two years from now even if you live up to that insane hockey stick graph they showed the investors. Just build a basic CRUD app backed by RDS. Use the same front and back end frameworks you use for everything else. The engineering standard says you can't afford to waste time on this because there's a lot of other work you're going to need to do this year, and some of it might turn out to be hard, even if you don't know what it is yet." Can you imagine that?
Math and physics also have a strong cultural orientation towards simplicity, which seems to be universally embraced. If you solve a homework problem in half a page, and the professor calls on somebody else to put their proof on the board, and they walk up with two pages covered front and back, you don't think, "Wow, they're smart. Their solution is so much more complicated; it must be better." Conversely, if you and all your friends took three pages to solve a problem, and somebody else presents a solution that fits on one chalkboard, you're in awe. I really don't know what the difference is that makes the culture of software different.
I think if it's being enforced by standards or licensure, it's too late and too external. It needs to be internal and cultural, at a gut level, to aspire to do more with less. In software, the primary measure of cost, the definition of "less" in "do more with less," is the intellectual load on the software development team. (We have that in common with mathematics.) Do more with dumber. How do we learn to value that as a profession?
Back in the day, in perl I'd bless regexes and use AUTOLOAD to provide the class implementations at runtime, as any good unicycle rider would. Changing the Rube Goldberg machine is hard. grep won't really help. I'd resort to even dirtier tricks to change behavior.
Later I worked in java. grep would work, but even better, I could make a class private, create an interface and just chase compiler errors. So much nicer than hunting down the right line of code to hijack in perl. It's pretty good, and I think go provides a similar level of support, but there's still a bunch of room for Class.forName(foo); //haha! grep will never find me! and it's cool, cause you'll only find out about it at runtime.
Anyway Haskell is neat because you can throw it all in a do in IO. hack away for days. When it's time to change, just change it. the compiler will help. The compiler will tell you everything you need to know to scope down to the finest level you want. It's super easy to be way too fancy up front, but any mistakes can be fixed.
Anyway, Haskell is probably never the right choice but it's the only thing I know of that lets you easily change the design whenever you feel like it.
It's Saturday, and I've thought about this too much. Time to put on my silver pants and go unicycling.
1) a problem that requires a very high-assurance system with very low tolerance for bugs, errors, and failures, and
2) a large complex system where refactoring and other maintenance-over-time activities are made materially more manageable by Haskell's strong type system and other tools,
3) a need for large-scale parallelization that a purely functional language like Haskell facilitates, and
4) a world class team of engineers who can become productive in it relatively quickly, support new hires doing so too, and won't foreseeably be outsourced to some low cost overseas sweat shop.
Rust is also worth considering under those criteria, in addition to embedded systems, hard/soft real-time or constant-time systems, etc.
But obviously not all software projects have all those criteria. Other languages like Go, Python, Javascript, Java-ecosystem, .Net, etc may be ideal in other circumstances. You just have to match the tools for the job.
I've tended to find that these people don't write particularly good code (not usually bad, just mediocre) and tend to lack ability in software engineering - writing deeply unrobust code that falls far short of their perceived 'rockstar' qualities.
I can't tell you the number of conversations I've had with managers or even colleagues where their untouchable genius is extolled to the heavens.
Combine this with the fact that those who raise issues and foresee problems are often viewed poorly (possibly even MORE so if those problems actually end up happening).
Ultimately you end up with a situation where those who advocate for solid engineering are, while respected, never really raised above the low levels of the career ladder and those who embrace woo rise up.
In 2020 we call this a "makerspace".
What are you doing? What are the constraints? What is the problem?
Do you need innovation? Do you need plumbing?
Can you lift your eyes and see the value in a radically new way? Does it look like what you are already doing, or does a beginner's mind really suggest a new approach?
The author sounds like a crabby old guy who doesn't want to change. Probably an artifact of the topic. But it *is* a real problem- people get locked into fancy-chasing. This introduces risk into the project delivery: team based (knowledge) and tech (not adequately debugged).
I've worked at length in "fancy" languages and some of the fancy tech. My conclusion is you have a limited amount of investment you can do. Some of this is team based (language learning, keeping the employees happy), some is in product, some is in tech stack (FlashyFancypants DB and SparklePants.JS). Good tech leadership includes picking out what to take risks on, judging appropriate ROI. Note this holds true for open source, not just business projects!
I work in a Clojure shop, so I interact with the rewards and risks of this strategy right now. My last team/company, we chose Go for the team language. It wasn't as unilaterally good as some might hope. Nor was it as unilaterally as bad! It was just another language.
The worst bridge that can stand up to the load is good, cost efficient engineering. The worst code is not. One, because bad code hides bugs that you probably don’t have rigorous enough testing to find. Two, because there will be new demands on that code in the future, and bad code is harder and riskier to change. You might look like a responsible adult when you ship it today, but three years later the new maintainers will know you were a child making a mess. And probably end up doing a rewrite, or at least needing to.
All this “woo” with languages and frameworks and services and whatever else, is the search for higher quality code.
There is a certain myopia about new things vs. getting better at design with old things. There is a certain blindness to iterative improvement, e.g. Rust is much sexier than modern C++ as a solution to the problems of C++98. But rather than condescending to this you can channel the energy into more responsible directions.
I work in a team where many people have zero aesthetic sensibility about code. Not like “I know this is ugly but we have to get it done today” or “I know this is ugly but it’s the best we could come up with,” just zero sense that it’s ugly. Grass is not greener.
> underpaid compensated in self esteem [..] manager saying “dude yer so fookin smart” [...] brainlet payment system. [..] benefit to boring languages is people concentrate on the problem not interesting complexities of tool
Missing the point raised earlier: "the problem" being solved is that work is boring; Woo isn't added "for its own sake", it's added because being an interchangable cog in a relentless productivity optimization machine isn't what humans exist for, no matter how much employers and the author would like that to be so.
> people who feel like every line of code has to be innovative
Employees don't get paid or promoted for saying "this is good enough, let's take 18 months off", but they do for adding a new widget or showing off to the boss. Broken incentives lead to people following them.
> like bike-shedding, but more perverse
It's pleasure! Spit! Disgusting, sinful and shameful, and the people who enjoy it are inferior. Get back to the grindstone!
> you are presumably solving real world problems which someone pays for
"Impress my manager", "buy some food", "survive this stupid doomed project nobody wants that doesn't have the political backing or funding to get to its impossible goals" are problems people solve for money. Jobs where people solve real world problems seem a small subset of jobs.
> about as necessary and in demand as riding to work in silver pants on a unicycle
The way someone commutes to work has so little to do with anything that it's a big reinforcement this is about image, not engineering.
> developers [..] are often child-like in their enthusiasm for new and shiny things
Everyone knows that companies are at their best when petrified, stagnant, resistant to change and unwilling to entertain new ideas, right?
> Engineers are different: they’re trying to solve a problem
No-true-Scotsman so that "Engineer" now means person who tries to solve a problem (only using existing tools and techniques), and people titled or qualified as "engineer" who aren't trying to solve problems, or anyone who is trying use a new approach to solve a problem or doesn't have the same constraints and goals are "not real engineers".
> Engineers understand it’s OK to solve the problem with ephemeral, trashy, but fast-to-market solutions
Nothing the world loves more than falling bridges, burning skyscrapers. A moment ago the important thing was to build stable, long-term, easily maintained systems with boring technologies (a classic description of "engineering"). Now Engineers are building ephemeral trashy minimal viable products? Is trashy and ephemeral "engineering" now? (No, the point is who to mock and shame, and who is superior).
> if the product manager is going to change it all next week
Could be related to why the idiot in the clown shoes is enjoying building it in Haskell, knowing that it won't need to be long-term maintainable because it doesn't matter anyway. No, it must be because they're inferior to the blog post author, who is all about solving real world problems like making sure software engineers know their status is beneath him.
> Engineers also plan for the future [..] critical infrastructure that lives [..] depend on
Engineers do the right thing in the right circumstances, and you can tell they are engineers post-hoc when it worked? Handwave away any critical infrastructure which failed as "not built by engineers", e.g. Boeing must have put no engineers on the Maxxx, you can tell because it ki...
Does it? Oracle's JVM - is it a giant simple on-piston semidiesel or untrustworthy? It seems complex and trusted to me. Linux kernel, so simple such that a fisherman can fix it in the dark and cold and wet by burning pile of logbooks? AMD64 instruction set, trivial or untrustworthy? Stuff you can trust doesn't /look/ like anything in particular, it behaves in a trustworthy way long enough to get a repuation.
> Duct tape programmers don’t give a shit what you think about them
It makes more sense to say unicycle riders don't give a shit what you think about them, since they're the ones actually demonstrating non-conformity through their actions despite social pressure to stop doing that.
> the perverseness and destructiveness of people who try to get fancy for no reason, nor do I think JWZ was a “duct tape programmer” -he was an engineer.
Gone from saying duct tape programming is good, to defending someone respectable as "not a duct tape programmer" as if it was bad, then applying the True Scotsgineer label retrospectively - it must have been true engineering(tm) because it didn't not-ship, QED.
> OCaML, APL and Lisp [..] small teams can deliver
Small teams have tried to deliver same in many languages and processes, we see the survivors and say "this tool must be why". That needs a large evolutionary body-count of attempts in other languages which failed, and many attempts in OCAML/APL/LISP which delivered nothing. If nobody had ever used O/A/L because they weirdo unicycling languages, there would be no knowledge of whether they could do this. Rust and Golang came out about a decade ago, author is clear that Go is for engineers and Rust for unicyclers, but both came out of respectable teams that built complex software maintained for years, with good reasons (memory safety, not "add Monads to C++"); at what point in the past ten years did it become clear to the author that Go is engineering and Rust is unicycling? Was it clear that O/A/L were going to shake out well in 60 years, after their first 10 years?
> For mission critical [..] hit by a bus [..] deep sheeyit
Like the boat engines, there are visible examples where "deep shit" happened - e.g. crippling ransomware attacks, systems being offline for days, huge data leaks - and a month later there are no visible consequences; PR puts out an apology statement, bonuses are unaffected, business as usual. If your startup run by a true engineer using Golang gets hit by a bus it could be the end of the company too. If IBM has a team using Haskell and one of them dies, the project is swallowed up and there isn't any deep sheeyit that anyone outside the immediate customer blast radius will notice.
> Using the latest tool is even worse. 99 times out of 100, the latest woo in machine learning is not an actual improvement over existing techniques. 100% of the time it is touted as a great revolution because it beat some other technique … on a carefully curated data set. Such results are trumpeted by the researcher because …. WTF else do you expect them to do? They just spent a year or two developing a new technique; the professor is trying to get tenure or be a big kahuna, and the student is trying to get a job by being expert in the new technique. What are they going to tell you? That their new technique was kind of dumb and worthless?
It's like you understand incentives, but pretend you don't when it gives you an opportunity to sneer at people you want us to think you're superior to. Why don't you see that the same incentives lead to people introducing NewLang because the upside is a potential huge status boost?
> just quietly used by engineering types who find them helpful
probably get used by engineering types who find them unhelpful and wish...
There are vast swathes of counter-examples to the article, to the point where I don't think that, given the business framing of the OP, the article actually holds water outside of "some people glue useless things on and it's a pain to deal with", which is sort of... known already?
Less than 10% of it:
https://4e6.github.io/firefox-lang-stats/
Also there were several Rust developers laid off from Mozilla a few months back, including the Servo team:
https://www.reddit.com/r/rust/comments/i7stjy/how_do_mozilla...
Point taken, but I see no reason to straw man Haskell and Rust. They have their uses and new successful features they explore get ported back into the boring languages. This is how technological development works. For a non-programming example, anti-lock brakes were used in Concorde jets before they were used in automobiles.
Also, one must consider that Java was a 'shiny' language once.
Honestly, if OP knows of many companies that are picking Haskell or Rust over the likes Java or Golang, please post links. I'll happily apply.
Java was never a shiny language. It was many things. It was, most particularly, a ticket out of the world of the Microsoft sharecropper, needing to learn a whole new Microsoft Framework every second year, and discovering over again each time all the shit that didn't work, that couldn't be made to do what it was for, and cobbled-together ways to get around it instead.
I managed to never need to code on Microsoft until a bit around 2010, so missed the appeal.
I was amazed to discover that, even in 2010, nobody who had coded for MS their whole lives knew any better than I did how to get an MS OS to do any sensible thing. I would ask, and they would start rooting, trying random things until something seemed to work. The idea of a documented, stable, standard library component that actually did what it said without crashing was an absolute revelation, but the opposite of every shiny thing imposed on them before.