73 comments

[ 3.0 ms ] story [ 133 ms ] thread
>Once you’re settled in, 95% of programming isn’t hard. It doesn’t involve algorithms, hard design problems, performance or storage constraints. It isn’t worth blogging about.

Once you're settled in ... having dealt with architectural design in a way that allows you to continue doing something with the code for 20 years or more.

OK, sure, 100% of programming isn't hard. But if you think that the hard part is only 5%, you're not doing the right job.

I think you're right, but in a different way than I understand that you mean.

Whatever project I'm on I try my best to take on the hardest tasks. This allows me to code against things that are actually a challenge. Large refactorings, risky features, hard we-arn't-sure-if-this-is-possible tasks. This makes 40% of what I do hard. But it isn't because I'm writing software for ICMBs or satellites. I'm the dude that takes on the tough stuff and makes it work, and I think more developers would do themselves a favour if they did the same thing. Maybe you don't need to quit your job in order to change the ratio. If there is work at your present place that seems hard then volunteer for it. It will make you a better software developer.

> every time that someone tries to use JavaScript objects for data, they write a security vulnerability by way of prototype pollution

I would love to hear more on this, as it makes little sense as written. You never touch prototypes when using data objects, is the author talking about constructors pre-ES6? Or the fact you can shadow methods from the Object prototype? I don't remember seeing any exploits based on that (vs "that’s, really often, a problem").

Fair point, I should have cited that. From snyk: https://www.google.com/search?hl=en&q=prototype%20pollution%...

You'll see prototype pollution CVEs in such libraries as jQuery, Lodash, handlebars, ajv (and transitively, request). A pretty good set of modules that are heavily used.

Scroll through the latest vunerabilities in the database and you'll see prototype pollution popping up very reliably https://snyk.io/vuln?type=npm

May I ask for clarification?

Is my understanding correct in saying that this is a peculiarly JavaScript issue because you can overwrite language constructs on the object prototype?

Whereas in other languages I can't affect the language core constructs?

Most languages have one kind of "object" used for data and another kind used for instances of classes associated with OO programming. JavaScript traditionally uses one kind for both (traditionally, in that there is now an alternative, Map, but it's underused https://macwright.com/2017/03/13/maps-not-strictly-better.ht... )

In other languages, the data kind of object - dicts in python, hashes in ruby - allows you to associate any key with any value. In JavaScript, the hybrid kind of object that's used for both, allows you to associate keys with values, but has some keys that are special, like __proto__, that override language constructs and cause chaos and vulnerabilities.

I was wondering about this, too. An attacker who can run arbitrary code in your JavaScript environment can overwrite properties on base object types. But they have to be able to run arbitrary code already to do this. Running untrusted code is always a problem, in any language or environment, so I guess this would just be a way to take advantage of it. But you already assume the browser client is potentially compromised, since you never know what the user will do with it. And if you have an attacker running their own code in your server process, you're already hosed. So I'm not sure why this more of a big deal.
Concretely, at least on Node:

    const myData = {};
    myData[rawUserInput()].key = value;

This is vulnerable if rawUserInput() returns `__proto__`: `key` on the prototype may be set to value if `myData` had `__proto__` set. If the hacker controls `key` and `value` then they can effectively modify `key` field on all objects.

https://github.com/HoLyVieR/prototype-pollution-nsec18/blob/...

A common way is as described by that paper ^

Recursive merge functions, unless if they check for this vulnerability, are often susceptible, since JSON.parse defines __proto__ on its return values.

Indeed, though you'll only manage to break or DDOS the server by doing that.

It seems that the Ghost vulnerability explained in the paper relied on the handlebars engine calling the equivalent of `eval()` on a string value, in the global scope, plus a path traversal vulnerability allowing loading a template from node_modules.

Are there are any other examples of this being exploited in a meaningful way? Even if you end up passing raw user input to key and value, user payloads will never be able to define a function, so the possibilities are very limited. I think having every NPM library that does object assignment using a user-provided key be marked as 'vulnerable to prototype pollution' is quite different from it 'being a problem very often' in practice. Happy to be shown otherwise.

It could be serious.

Let's say your company's code is open-source, and the attacker knows there is code somewhere like this:

    let state = getState(); // returns empty {} if user not authenticated

    if (!state.userIsAuthenticated) {
       respond(401);
    }

    showBankAccount();

If the hacker is able to set `Object.prototype.userIsAuthenticated` then the auth check is now bypassed.

So I think the break / DDOS is pretty serious here.

They're referring to how in js, the for...in loop includes properties from the object's prototype.

There used to be much concern around the idea that you might load some third party js that adds thing to the base Object prototype, which will then be enumerated with for..in loops on all your data objects.

I always thought this was a stupid vulnerability because if you're executing malicious javascript, you are already screwed in a much more direct way

[Edit: i was mixing this up with a slightly different vulnerability that is a bit sillier than the one they are reffering to]

I kind of disagree. When programming isn't hard, it is because it is not done right, and the coder uses repetition instead of higher level abstraction.

Regardless of the programming task, it is usually hard to find the best way to do it, and not trivial to implement it correctly. This is demonstrated by the fact that most code sucks.

> When programming isn't hard, it is because it is not done right, and the coder uses repetition instead of higher level abstraction.

Aren't higher level abstractions supposed to make programming easier?

And yes, for the vast majority of programmers programming is easy thanks to existing high level abstractions, and it's usually the business domain that is hard.

They are supposed to ... but finding correct abstractions is hard. After a year or two abstractions usually are not correct anymore. That is why "favor composition instead of inheritance" is important because hierarchy of inheritance is fragile abstraction and if you compose things it is easier to swap one thing with other if they have the same interface.
IMO, abstractions aren’t easy, but programming with the wrong abstraction is significantly worse than with the right abstraction.

I think that because everyone starts with few visible abstractions, they get the sense that adding abstractions adds ease. Removing or changing them is often just as valuable.

When I read what you wrote it seems like the opposite is true. When programming isn't hard it is done right. Using higher level abstractions is not "golden bullet". If it is easier to have repetition and it easier to do find and replace and maintain and change code, why even make an abstraction?

Just like true developers write all their code in notepad and real developers etch bits on disk with magnetic needles.

Had the same reaction.
The hard thing about programming is making it easy ( readable)
The first part of your comment reads like a talking point a meeting hopping noncoding archetect would use.

Simple repeatable solutions are better and more maintainable than constantly abstracting things to a higher level. I write systems and code with operations and developer ease in mind, not some academic idea of abstraction.

> I write systems and code with operations and developer ease in mind, not some academic idea of abstraction.

I'll be quite honest, if this is the same sentiment behind my experience at work (convoluted tangles of conditionals everywhere, 100+ line functions doing simple things - just to name a few), abstraction isn't just some academic idea.

I work with people who follow the path of least resistance and leave everyone else with the path of most resistance when we have to figure out what the hell is wrong with their code and fix the glaring oversights and errors they left behind.

(comment deleted)
> The obsession with citing facts in paranoid literature that Hofstadter mentions is no longer a thing: modern paranoia is untethered.

The paranoid now cite "facts" that are stated somewhere online. They still cite them, though, as though they definitively settled the case.

My one friend writes code for satellites. My other friend is a security vulnerability researcher working on the Linux kernel. I write code to process terabytes, potentially petabytes of data consistently and performant. I don't really know people who do frontend professionally in my personal life but I can't imagine keeping a webapp serving millions of people on many different devices performantly is easy.

My point is it really depends on the scale or depth of the work you're doing. It's pretty easy to keep things running smoothly for a smaller company or one not doing hardware etc etc but I think this really misses the mark for a lot of work, there's a scale or depth where you can't get away with not using what the author says is only used in interviews.

What you described here falls within 5% of all programming jobs.The rest 95% aren't that demanding.
I would guess that most satellite and big data code falls into the non-hard category. In my experience working with things that sound similarly challenging, it's usually the case that one person or a small group of people do most of the "hard" part early on and then a much larger team spends years doing relatively simple work expanding on the foundation they built.
I think there are a lot of other issues with that and don't think your experiences reflect the norm, or at the least they certainly run counter to mine so far.
In my experience with being on a big data team is that most days you don't have to do anything all that hard but it always comes up eventually.
In my experience the stuff that comes later usually gets harder to do right. The first few features are pretty easy - the code base is small, the rules and state are simple, and there aren't any major performance issues. Later on features start coming into conflict, there's more complexity to be managed, and you've used up all of your latency budget.

How often do you hear that you just need to rewrite everything at that point? How often does that payoff?

I just don't think you're correct. I think that idea is why we have companies on their third rewrites and most codebases look like sphagetti with performance issues abound. I think many domains are much more demanding but the people working on them don't know any better to avoid the pitfalls.
In my experience, he/she is correct. I do a lot of "plumbing". Take data from one source (database, API, vendor), massage it (map/filter/reduce/tweak/translate), send it back to client.

Lots of frontend work is using the same paradigm (human input triggers events that interact with components to manipulate state and make network requests) over and over for different screens.

In the 80s it was called "application development" and it was the vast majority of jobs in the software industry. Database, forms software, user data entry, data presentation.

The web has just replaced the technological components, but the basic model is more or less identical (and arguably still hasn't reached the same level of sophistication that the forms software of, say, 1988 had (though this is certainly arguable)).

I don't think programming for satelites / kernel is more demanding than modern frontend, for example the Linux kernel did not change that much in the last 15 years, it's pretty much the same old C with the same tools.

On paper it looks cool and complicated, in reality it's not that hard. Lot of fields in CS don't move that much especially things based on C / asm, kernel, embeded ect ...

This doesn’t seem true to me. You can definitely work with the traditional kernel and C model, but we have eBPF, GPU computing, unikernels, C++20 and Rust… and I’m not a low-level guy, that’s just off the top of my head. Things do evolve.
I’m lost. What does Rust have to do with the Linux kernel?

I agree that most kernel programming isn’t computer-science hard, but its peopleware-hard: you need to maintain compatibility and get signoff and step very carefully forward because any change can affect so many disparate groupa.

I like the article title and agree, the other points the author make are valid but seem like randomly picked issues.
Glad I wasn't the only one who thought that.
> notes on thoughts, in sort of a Matt Levine-esque style

Matt Levine is a columnist for Bloomberg.com. His columns are typically a random collection of thoughts on current events, from the point of view of economics.

He says at the start that he just kept some notes for the last month and was publishing them all
> 95% of programming isn’t hard

> It’s easy to screw up the 95%, though

To me, those just don't seem like compatible assertions. If something isn't easy to get right, what else do I call it but "hard"?

I've been walking for a good number of years now, yet every now and again I'll trip on something, or bump into a wall. I wouldn't call walking hard.
I would call walking hard - most animals are just very good at it.

No programmer is good enough to build a walking algorithm that is 1% as good.

TLDR: People are hard, computers are easy.

The only hard part about programming, and computers in general, is other people.

The OC mentions JavaScript, something insightful about data and objects. Just imagine: where would JavaScript be today without people like Brendan Eichmann?

I imagine other industries are more or less the same wrt the work vs the people.

The author should ask himself if he is doing 5% writing (and thinking) or 95% writing.

An honest contemplation of this question would likely result in a very different set of ideas and likely a different presentation format to go along with it.

I commit my full force of being to craft the highest quality posts on my personal blog for you, this is really 1% stuff.
Thank you for your effort. It clearly shows both in your work which is prolific and creative as well as in your writing.
> Once you’re settled in, 95% of programming isn’t hard. It doesn’t involve algorithms, hard design problems, performance or storage constraints.

It sounds like OP thinks the hard parts are algorithm design and architecture. I wouldn't even really consider those programming. They're things you need to do before you can start programming.

> 5% or less of programming is hard: it’s the stuff that interview questions refer to. You might need to think about algorithms, might need to try out multiple strategies before deciding on the one.

The author is correct that most programs don't need complex or clever leetcode type algorithms, but IMO program architecture is just as difficult to get right for any program longer than a thousand lines.

What percentage of the population is even intellectually/psychologically capable of programming at a professional level? My guess is significantly less than half. That's not a we're smart and they're not. There's many jobs I'm psychologically ill-equipped to perform. For programming to be easy you have to have just the right amount of autistic/systemizing brain type, which appears to be significantly genetic. Everything in programming is abstract, so you have to be able to hold very large completely abstract systems in your brain. That's a different skill/aptitude than, for example, holding all the working parts of an internal combustion engine in your mind.

Programming is "easy" for only a tiny percentage of the population. That's just regular programming problems, not particularly difficult concepts/tasks I see many posts referring to.

I disagree with a few points here.

1. Most people could be trained to code. The most important trait in a programmer is persistence. Momentum built from solving past problems is a powerful force.

2. The idea of needing to be atypical in order to be successful is damaging and wrong. What matters is practice.

I understand what you are trying to get at, but there a lot of very smart people outside of the realm of software development. The ideas you are perpetuating raise the barrier to entry.

The blank slate theory has been thoroughly debunked at this point. I explicitly said it wasn't about being smart vs not smart.
I'm not arguing for that. Of course there are genetic differences that will make some people better at some things.

But to say that half the population bus not capable of programming? That is elitist nonsense. Especially given the breath of the field.

It is not. On the contrast I'd say thinking that more than ~10% of the population is in principle able to program is based on an elitist filter bubble.

Abstract thinking is really not that easy. Just try to go to a shopping mall and teach people simple math or programming concepts.

Have you actually done the experiment you propose? It sounds like a fun challenge honestly.

I have taught at least a dozen people to code. A good number had no idea what was going on for a while, but left knowing how to handle the basics.

Coding, math, and related subjects are made harder than needed by focusing on the abstract too much. Most of the concepts taught in a typical computer science degree are not difficult once you strip away the terminology.

I'm also not sure how you see these opinions stemming from an elitist filter bubble. I know what I know because of how I've been educated. Anyone who put in the same amount of work (within a reasonable margin) would know the same things.

>That is elitist nonsense.

You appear to have hallucinated a version of what I've said and are arguing against that.

Maybe? It seems like saying at least half of the population is not capable of your job is elitist.

I could be misunderstanding something about what you're saying.

There are many jobs for which most people are not psychologically compatible. Off the top of my head, here's a few: long haul trucking, EMT, police officer, public school teacher...

Is that elitist?

I pretty much agree here. I am an example of someone who simply does not have the genes for it, so to speak. I have loved programming since I was a little boy but am horrible at it. I can learn a language's basic syntax just fine (and usually in very little time), and have written many little tools/small apps that I have personally found helpful in automating certain aspects of my life and work. So I certainly value what little bit I can do. But every time I've tried to study a serious book on data structures and algorithms, it's been a disaster. I simply lack basic problem-solving skills. I try but I'm just not good at it. I mean, I can usually come up with working code to do whatever I've ever wanted to do, and I'm certainly thankful for that since it's ok if my personal projects aren't terribly efficient. But professional programming would never be a possibility for me. I'd get laughed out of an interview b/c of the convoluted/entirely-too-complicated ways I solve problems. I just wish I had a good sense of what algorithms/data structures to use where. That would certainly help me a lot! But I'm just not sure I "have what it takes", if you know what I mean, to develop that skill.
> I can usually come up with working code to do whatever I've ever wanted to do

Been working as a programmer for 25 years, and I’ve never read (or considered reading, or felt the need to read) a book on data structures or algorithms.

Also, I’ve seen plenty of people who are employed as programmers who simply cannot come up with working code. I don’t see how you can write working code without problem-solving skills, as by definition you’re solving a problem by writing working code..?

(I don’t really have a point, sorry, this just resonated with me)

Yeah, my wording wasn't great, so your confusion with my post is my fault for not being clearer. I obviously can solve problems to some degree otherwise I wouldn't be able to even create the simple tools I've made to solve some problems in my work. What I guess I'm trying to say is that I don't solve them WELL or efficiently. I'll use a DS/algorithm that, sure, technically works and gets the job done but is by no means the most efficient way to do it. Someone who has the common problem-solving sense I lack would have solved the problem more efficiently, more quickly and with more attractive code. That's what I was aiming for.
>I just wish I had a good sense of what algorithms/data structures to use where. That would certainly help me a lot! But I'm just not sure I "have what it takes", if you know what I mean, to develop that skill.

I wouldn't worry about it. I had a guy I mentored who couldn't code his way out of a paper bag in spite of having had a programming job for 5+ years at that point. I taught him how to program and while he did improve significantly, he realized he wasn't going to make it as a programmer. Now he works in infosec where he's largely administering software with occasional scripting type stuff. He gets to do the scripting/programming bits he likes and makes a ton of money. He's actually my boss now. There's a lot of this kind of work that I think is really rewarding. I hope you find a good fit for you.

There was a story that i'm having trouble finding now, where researchers figured out a test for who would succeed in a compsci program.

It didn't matter how many questions you got right on the test, but rather, if you answered the questions in a consistent matter. even if you were wrong, as long as you were wrong consistently, you'd do well.

If you saw a question, and then a very similar problem 3 pages later, and answered them wildly differently, you wouldn't make it.

the conclusion was that if you were building patterns in your brain, or using logic to build up answers, you were good to go.

if you do find it please post it. that is an entirely fascinating idea.
I really don’t think that’s true at all. Just think of all the spreadsheets out there in the world: they aren’t programming by the strictest definition, but they have most of the components.

To me, programming is simply an acquired skill. The vast majority of the population could do it.

Creating a solution to a defined problem is one thing.

Understanding what the problem is ... that's something else.

> you have to be able to hold very large completely abstract systems in your brain. That's a different skill/aptitude than, for example, holding all the working parts of an internal combustion engine in your mind.

Is it really different though? “Why isn’t this engine working?” [traces through combustion path to find problem] doesn’t require ‘holding all the steps in you mind’ but it does require knowing them and how they interact, whereas “how do I make this engine more efficient?” does require a mental model of the whole engine and how the parts interact?

These seem broadly to me as analogous to respectively maintenance programming and systems development.

Edit: I think the part I don’t understand is why you think the abstract system (the code) is qualitatively different from the concrete system (the engine).

Yes, abstract thinking is different than concrete thinking. Thinking about an engine part is concrete thinking. Manipulating that part in imaginary 3d space in your mind is abstract thinking. 100% of programming is abstract thinking. Abstract and concrete thinking are done in different parts of the brain. Here's the first google result I found on the topic. I think this stuff is really interesting.

https://www.goodtherapy.org/blog/psychpedia/abstract-thinkin...

At the end of the day, we all implement systems - large and small. The more moving parts a system has, the more complex its implementation tends to be. Decomposing large systems with moving parts takes many tries to master.
The rest of the 5%... well, that obviously is reserved for naming things.
It's as hard as you want it to be, right at the limit of what you're currently capable of dealing with if done right.

I find that with increased experience, writing (any kind of) code is more intense. Because I'm pattern matching 35 years of experience at every turn.

As a result, I tend to write code in short (2-4h) but very intense bursts with plenty of rest in between lately.

> I tend to write code in short (2-4h) but very intense bursts with plenty of rest in between

At any level it’s a great idea to ruminate and plan what you’re going to do. Alas, current management trends are about keeping developers busy typing — which is something of a vicious cycle.

I feel like there are two articles here one about politics and the other one about programming. I don't see how they are related in the way the author presented?
The hardest part of programming is finding workarounds for undocumented bugs. And I don't mean the kinds of details people have to be told on stackoverflow after refusing to read the manual. I mean genuinely showstopping bugs with nonobvious workarounds like HTML5 audio didn't fire the ended event in Safari until iOS 13.4. There's a guy who complained about it on his blog for eight years until Apple allegedly fixed it. And I say allegedly because I consider the fix to be broken and I still use a workaround which I had to discover for myself through original research.

The only part of programming which is easy is copying example code from the documentation. Once you build a nontrivial project that solves a real problem you're already working beyond the trivial. Real work gets real challenging real fast. If you're staying within the percentage of programming that's still easy then you're just doing copypasta and not really programming.

Almost everything I've ever attempted to do on my own, once I did enough research into how to do it, turned out to be extremely easy. The only time it's been difficult is when some other person put arbitrary and/or unexpected rules or limitations in place. Examples:

"Oh you want to make a website? You knew how to make a website in 2004? Well too bad, the tools you used in 2004 are now hopelessly broken. New tools have arisen to take their place, but you have to learn everything from scratch."

"Oh you want to make a video game? You learned OpenGL? Too bad that won't work on iOS. Looks like it's back to the studying phase for you..."

"Oh you want to compile Hadoop from scratch? Too bad it's a 9,000-step process, and anything could go wrong along the way."

"Oh you want to add a feature at the company you work for? Too bad someone wrote that module in an unnecessarily confusing way, so have fun tracing through that mess to figure out how it works before you can add the feature."

I've found these cases account for 95% of programming. So I guess I'm saying that 95% of programming is hard.