Ask HN: What change in your programming technique has been most transformative?

236 points by dmux ↗ HN
As an example, the team I work on has been adding more precondition checking to all of our applications. The simple act of stepping back from the perceived data-flow and explicitly declaring what we believe should hold true has uncovered several bugs in our understanding of our applications.

We've likened it to having someone review a paper you've written: you often read what you think you wrote, not what's actually written.

This got me to questioning what others have found to be transformative in their development practices.

213 comments

[ 3.1 ms ] story [ 212 ms ] thread
By far, starting to write automated testing as I code. The sheer number of bugs I find immediately and the number of bugs I find from changes that I didn't expect would be enough to justify it, but what is even better is that it forces me to keep my code testable, which also happens to correspond fairly closely to well-architected code. Not perfectly, but for something that is automated and always running, it's really helpful.

When I'm able to greenfield something myself, and use this from day one, I tend to very naturally end up with the "well-separated monolith" that some people are starting to talk about. I have on multiple occasions had the experience where I realize I need to pull some fairly significant chunk of my project out so it can run on its own server for some reason, and it's been a less-than-one-day project each time on these projects. It's not because I'm uniquely awesome, it's because keeping everything testable means it has to be code where I've already clearly broken out what it needs to function, and how to have it function in isolation, so when it actually has to function in real isolation, it's very clear what needs to be done and how.

Of all the changes I've made to my coding practice over my 20+ years, I think that's the biggest one. It crosses languages. At times I've written my own unit test framework when I'm in some obscure place that doesn't already have one. It crosses environments. It crosses frontend, backend, command line, batch, and real-time pipeline processing. You need to practice writing testable code, and the best way to do it is to just start doing it on anything you greenfield.

My standard "start a new greenfield project" plan now is "create git repo, set up a 'hello world' executable, install test suite and add pre-commit hook to ensure it passes". Usually I add what static analysis may be available, too, and also put it into the pre-commit hook right away. If you do it as you go along, it's cheap, and more than pays for itself. If you try to retrofit it later.... yeowch.

Ah man. Reading these comments make me so happy. Automated testing from the start. Code as you go. Love every bit of it.
To piggy back on here. Don’t use mocking frameworks in tests. I’ve found it to be a great way to expose design flaws in the code.
I throw away rough drafts all the time for non-trivial work that I don't fully understand yet. There are different approaches to it, but I usually just create a throwaway branch to hack out a naive solution until I run into the non-obvious roadblocks and then try again. I used to try to _really_ understand something before coding a solution, but not being afraid of throwing away a rough draft has helped my mindset a bit in terms of working out difficult problems. It's not a novel or huge concept but it works for me.
This is an interesting idea, I think I often do this subconciously but I might take your mindset next time and tell myself I can chuck the first draft away.
We call this phase the PoC effort (proff of concept). Rough draft, I like that too.
Recognizing that there's a special power in resilient technologies. Those that keep chugging along decade after decade, and getting stronger, too. Not from inertia or monopolist effects or other kinds of random lock-in. But because they tackle a certain set of fundamental problems very, very well. Despite endless complaints about their fundamental limitations, conceptual flaws or alleged lack of scalability.

SQL, Python, PHP, JavaScript, and many aspects of Unix and the C/Make toolchain all come to mind.

Mind you, I don't like all of the above. At least 2 in particular I definitely wouldn't mind seeing "just go away."

But I do recognize that they have special staying power. And they didn't get this power by accident.

This is certainly not true. Evolution of programming languages is circumstantial. There are many flaws in all of the ones you had listed.

The marginal cost of deprecating something or breaking something is higher than the incremental cost of workarounds. This happens all the time from the laryngeal nerve in Giraffe to the evolution of a programming language, say JavaScript.

Please don’t mix good design with popularity. It’s a correlation/causation fallacy.

Look close enough and everything is flawed.

Evolution doesn’t ‘design’ anything, least of all anatomy.

The ‘special power’ is that they have stuck around, for whatever reason or circumstance.

I never claimed that evolution “designs” something. In fact, I’m talking about the opposite - Evolution has no hindsight. Therefore, the marginal cost of undoing something is larger than incrementally adding or bolting on fixes.
> The marginal cost of deprecating something or breaking something is higher than the incremental cost of workarounds.

So why are we in this python 2 to 3 mess?

(comment deleted)
1. Using REPLs: The tool or the principle.

REPL means Read-Eval-Print-Loop and its a place where you can run code immediately and get immediate feedback. For example F12 in your browser and use the console to do 1+1. But you can also use that console to interact with your code in the browser, the DOM and make http requests.

But I also see REPL as a principle - to get as quick feedback from the code you write as possible, and things that help this are:

* Quick compile times

* Unit tests

* Continuous integration

So that each stage is as quick as possible. I write code and within as second or two know if it failed a test. Within a few seconds maybe I can play with the product locally to test it manually. Once committed, I can see it in production or a staging environment at least pretty quickly.

You can then have bigger REPL loops where a product manager can see your initial code fairly quickly, give you feedback and you can get stated on that right away and get it out again for review quickly.

I don't think there is any excuse not to work like this given the explosion of tooling in the last 20 years to help.

2. YAGNI

Writing over elaborate code because it is fun! That's fun at first but you soon learn it's better to write what is needed now. There is a balance and writing absolutely shit code is not an excuse, but adding too generic code because of stuff that might happen is also a problem.

I would say: Automated testing and a functional programming mindset.

Even for legacy projects, starting with adding regression tests for every bug you find is a great way to introduce testing. And when you add new features, you can write tests for that as well.

I find that a FP mindset is helpful mostly because it tends to reduce the amount of global or spread-around state. This in turn makes testing and quickly iterating in a REPL a lot easier. Also when later the time comes to debug, it's much simpler if you don't have a huge amount of state to set up first.

And even if a lot of state is required, having it be an explicit input to a procedure is helpful because it makes it much clearer what you need to set up when doing manual testing.

Probably learning to use a debugger and decreasing my reliance on print() statements. It's a nonzero initial investment but saves time in the long run.

At the team level, probably CI/CD. It forces us to break the monolith into digestible chunks and makes regression testing easier.

Honestly at a team level - code reviews. At least one other developer reviewing every line of code that is written is transformative in the quality of code that is produced. A lot fewer problems, a lot more maintainable.
I've recently joined a team that focuses massively on test-driven development, and it's just such a great way to shake out dumb bugs and focus on the requirements and the problem you're trying to solve.

Also recently did Thorsten Ball's Interpreter/Compiler books which focus heavily on unit testing the functionality (I can't recommend these books enough).

I now can't imagine going back to a pre-TDD world

I did Learn Go With Tests which was my first intro to TDD and I found it enjoyable even if the author is little overly pedantic about his preferred testing approach.

However at work, I often find that I feel like I can't write unit tests before starting code because I just don't know how it's going to get built until I start poking at the code. I'm not sure how to break out of this

What does your pre-code design process look like? Might beef that up; its what has helped me quite a bit.
> However at work, I often find that I feel like I can't write unit tests before starting code because I just don't know how it's going to get built until I start poking at the code. I'm not sure how to break out of this

You'd start with shallow functions and quality asserts. Initially, you'd have broken tests, you have to write code to fix them, and that's your TDD.

This is definitely the best way. Start with the how you want everything to look and make your code match your expectation.
Hello, I wrote learn go with tests!

Your comment is exactly what I try to get across in the book but it's not always easy.

If you cant decide how you want something to look (as that's not always easy) just take a punt on something. Make sure it's a small decision and make something useful. Sure you might have to change it, but at least you'll be basing that on some real feedback.

Hi, thank you for writing the book! Great introduction to Go.

One thing I was uncomfortable with about TDD is the step to "just write enough to make the test pass". The danger seems to be when you have to walk away from some code and you or whoever picks up your code doesn't know what you were up to.

In a simple app and a few tests you could probably tell, but the larger an application gets, the less you can put effort into finding out "this test works because the application works how it should" or "this test works because someone wrote just enough code and hardcoded some value somewhere in order to make the test pass".

It seems "safer" to write tests that are going to stay broken until every little corner of everything works properly, even though this is less iterative.

As mentioned above I have little TDD experience, so maybe I'm missing something.

The point of "just writing enough to make the test pass" is to get you to the point where you have working software (proven by the test, even if the code is "bad")

This is the _only_ point where you can safely refactor, if your tests are failing how do you know you haven't broken something? So long as you keep things "green" you know you're ok

> The danger seems to be when you have to walk away from some code and you or whoever picks up your code doesn't know what you were up to.

Just don't do this. You're not "done" with the TDD cycle until you make it pass and have refactored.

I reflect this in my git usage too, roughly it goes

- Write a test -> Make it pass -> git commit -am "made it do something" -> refactor -> run tests (if i get in a mess, revert back to safety and try again) -> git add . -> git commit --amend --no-edit

> It seems "safer" to write tests that are going to stay broken until every little corner of everything works properly, even though this is less iterative.

The problem with this is how do you safely refactor when you have potentially dozens of tests failing?

A big point of this approach is it makes refactoring a continuous process and makes it easier because you have tests proving you haven't accidentally changed behaviour.

Thank you for the thorough reply! I'll keep hacking away at it :)
I'm not even thinking about "how it's going to get built" when I write test-first. I'm just writing (test) code as a client that's calling an API. It just so happens that the API (method, or method + object) in question doesn't actually exist yet, so the first thing I do is let the IDE generate them so it compiles.

Once the code compiles I go into the implementation and make it work.

Now and then I realize during implementation that I need some additional parameter, but it's easy to add that.

I guess to me:

1) Writing a test that calls an API and asserts that everything resulted correctly from it is more of an integration test in my opinion - in this case I can agree this can be written beforehand.

2. If I'm changing my tests as I write my code then it doesn't matter if I go test-code-test-code vs code-test-code-test. I have still changed my definition of "correct" on the fly based on something I found out as I implemented.

How have you achieved a balance between too little and too much testing? I worked on a project that had a ton of tests, but they seemed to be "over fitted."
The trick is to get something covered to the point where you could comfortable refactor the bejeezus out of it, and know that it's still working from a higher level. The test should be how you know the code works, and under any situation.

That is, you don't need to test that inbuilt standard library stuff is working as expected (e.g does a string reverse) but rather, does the combination of possible inputs match your expectations?

Hot code reloading. It was one of the best decisions I made when I added it to a hobby project I've been working for the last six years. Most of my development time for this project is spent adjusting code while the app is running. The feedback loop is incredibly tight and the most engaging of all of the projects I've ever worked on.

I'm working on extending auto reloading to all of the assets in the project because I know that tight feedback loops are that important.

What language? Is it a game?

I've been wanting to try that ever since I saw a video of someone developing an FPS in Common Lisp while playing it at the same time. They would modify the bullets and the way they made collision, then fire after each change to see the effect.

It's an open world roguelike in Clojure. It was counter intuitive to me at first, but modifying real-time behavior like rendering is even easier because the feedback loop doesn't involve player interaction.
Sounds like you're remembering John Carmack developing VR using racket:

https://www.youtube.com/watch?v=ydyztGZnbNs

edit:

Also check Arcadia for Clojure:

https://www.youtube.com/watch?v=_p0co13WYPI&feature=emb_titl...

And developing flappy bird in clojurescript (canonical figwheel example):

https://www.youtube.com/watch?v=KZjFVdU8VLI

No, the one I saw was different. They were walking around the world and shooting stone building walls. They also weren't part of a conference. The video was just their screen split with their code on one side and the game window on the other. I'm pretty positive it was Common Lisp too because I think I was looking for SLIME videos at the time.

Thank you for the links.

Programming in a team and accepting that it is not 'my' code but 'our' code. Not feeling slightly pissed when someone else changes the (not my) code. For me, this was a total game changer and a complete change in programming style. The focus changed to using the most common idioms, the clearest and cleanest and most concise way of writing stuff down, and avoiding smart hacks or, if necessary, encapsulating and documenting smart hacks. Not being shy of writing stuff multiple times until it really is clean and inviting to be read and understood. Using a very strict style (including when/where to put white space and line breaks) that everyone else also uses so that code is easy to read for everyone. It improved my programming enormously and the bugs/line ratio went down. It also gave a much better intuition to smell fishy code. It also took quite a while to internalise, but I now always code like that, also in private projects and in quick hacks scripts in, say, Perl.

My second most important change was to learn how to use contract based programming, or, if the language has poor support, at least to use a ton of asserts. This, for me, feels like stabilising the code very quickly and, again, improved my bug/line ratio. It forces me to encode what I expect and the code will relentlessly and instantly tell me when my assumptions are wrong so I can go back and fix that before continuing to base the next steps on broken assumptions.

I agree with the first bit; however it's hard to find a balance between accepting that your code can be improved, and standing by your solution. What I see in my team is that someone else has another solution, and whatever that is, should be implemented, if you argue against it, then it's often "I'm just trying to improve your code". However his solution might not necessary be better, it's just a different approach.

We also follow very strict style guides, but this is all defined in formatters and linters, so there's no arguing against it.

Even for my personal projects, the first thing I do is usually set up .editorconfig and TSLint (if I use Typescript).

I stopped doing small commits. In the past as soon as I was "done" with something I would commit, even if one line.

What would end up happening sometimes later in the day I would decide revert or modify the change, so my commit history ends up flooded with bunch of small commits.

I ended up writing a script that checks my current work, and if enough changes or time has past, then a commit is recommended:

https://github.com/stvpn/sunday/blob/master/git-train/git-bo...

I have the opposite problem, my commits are way too large and I'm trying to get in the habit of making more frequent commits as you can always squash them.
You can have your cake and eat it too by initially generating a series of small commits and then going back later to reorder them and squash them into a much shorter series of logically coherent commits.

This makes it way easier for someone (possibly you) looking later to understand what was done and why.

(I once almost took a job where the repo was just a series of huge commits taken at more or less regular intervals, but with no logical coherence and no attempt to document what was happening. Noped right out of there.)

(comment deleted)
This is exactly what I do, with one addition: Sometimes I don't check everything in as I go because I'm doing a lot of experiements at high speed. However, I will locally do "junk" commits for trivia such as comment typos and short, obvious bugs that are noticed.

At this stage I don't worry much about the commit message, as it's purely local, and many of them will be squashed (discarding the message). So the trivial ones get one-liner messages like "WIP: Adder: Comment fix" or "WIP: Parser fix '...', needs testcase'. "WIP" commits are a useful tag for me: They are never allowed to be pushed to a shared repo.

Then the "git add -p" stage.

When a task, feature or bug has been dealt with, I'll start using "git add -p" to separate out parts of files, and commit them as logically independent changes. At this stage, I don't mind separately committing even very scrappy little changes, such as comment typos, as separate commits because I will merge them later. The key is to separate out logically separate kinds of things in the delta from worktree to HEAD. During this I will usually find a few things that are untidy or comments that could be worded better, and add tiny commits for those changes.

The "git add -p" stage is a great time to get some perspective on what logical units were actually needed for the main task, and what else was refactored or fixed in passing, and this "pick up the pieces later" method frees me up to fix things and do small refactors without having to switch context while doing it.

Then the "git rebase -i" stage.

When the adding is done, "git rebase -i" to reorder into a sequence of logical, coherent and explainable changes. Ideally in an order where things still work if they are partially checked out in that order (bisectability), and squash together separate "git add -p" chunks that really must be one logical unit. Also, squashing trivial fixes such as typos in comments, whitespace etc into logical commits.

I may then go back and clean up and flesh out some of the commit messages before pushing the lot upstream for review. Or, in practice, there's usually no review of my work other than testing it, but so that others can at least read through the commit messages and patches to see what was done and how; hopefully learn from it.

The above cycle is usually done about every 1-2 workdays, but it can be longer if there's a tough problem being debugged or a complex new feature (but for big new features a branch is more useful). If something turns out to be too big, though, it doesn't matter because I can always commit any work in progress to a new local branch or stash, and rewind back to a stable worktree.

The final, logically coherent and documented set of commits is very satisfying to push, and rarely contains "junk" commits or unexplained changes, even though what I commit locally at first is often like that. I guess it helps to know Git quite well.

If I don't commit often enough, then I would forget which part was stably written or not by seeing the lines changed in the editor when I come back to a certain part of the code.
Leaning on the type system for as many guarantees as possible. A few of my teams use Kotlin's sealed classes and extension methods to prevent a lot of goofy invalid state and improve composability (using a Result<T, E> sum type very similar to what Rust has built-in).

I liked this article I recently came across discussing the topic: https://lexi-lambda.github.io/blog/2019/11/05/parse-don-t-va...

Many incremental changes (some already mentioned here) but none comes close to grokking macros in Lisp and lisp-like programming languages such as Common Lisp, Scheme, Racket, Emacs Lisp and Clojure. That was truly, truly transformative.
(comment deleted)
Moving from my college years of emacs on the server to a proper IDE on the desktop.
It wasn't college for me, but the first IDE I used that seemed like a step up from vi and makefiles was Visual Age for Java. And man, that was a huge step up.
Do you use any emacs key bindings in that IDE?
Yes, but compared to an IDE (Pycharm in this case) there is no comparison.
Avoiding cleverness at all costs. Simple code that is easy to both read and write without strong efforts to follow senseless DYI or take on abstraction for the sake of abstraction.
+1 from me. Every time I revisit code that I wrote cleverly I roll my eyes at myself. What was I trying to prove, now I have to put more effort in understanding my "cleverness" instead of quickly going through the change I have to implement.
"Everyone knows that debugging is twice as hard as writing a program in the first place. So if you're as clever as you can be when you write it, how will you ever debug it?"

http://www.linusakesson.net/programming/kernighans-lever/

Your link goes on to say that this doesn't mean the obvious. They say that cleverness isn't innate, and that writing code that you have trouble understanding later will force you to grow as a programmer

> If we deliberately stay away from clever techniques when writing code, in order to avoid the need for skill when debugging, we dodge the lever and miss out on the improvement. We would then need other sources of motivation in order to grow as programmers, and if no such motivation appears, our abilities stagnate (or even deteriorate).

I'm not experienced enough to guess if this is accurate, but I found it very interesting

Amen! Developers spend their first decade learning how to write complex code, and be clever, and compress everything into the fewest and hardest-to-read smallest amount of code possible.

Their second decade they learn that simplicity and readability is always better, even if it does require more code.

It's better to have 300 lines of simple, straight-forward, easy-to-read code than to have a 30 line version of the same code that's difficult to understand, and difficult to debug.

Always code for your future-self and future-others who really don't want to spend an hour figuring out what the hell you did in that awesome looking 30 lines of entropy.

Sometimes you're lucky enough to have this expedited; my first CL at my first job was beautiful and elegant and concise and needlessly complex, and I was told to unroll it to make it more readable. The lesson sunk in the minute I had to start reading other people's code.
"Expedited" haha. yep. I mean there are times when something is going to cause just totally complex code to be required, but in those cases there should almost always be a full paragraph of "explanation" (documentation) for about every 10 or so lines of code.

Ends up with actually more documentation text than code itself, which is a good thing. But for the majority of code it an be kept simple and self-documenting.

IME it's about time spent maintaining code rather than an age/experience thing, devs that flock like locusts from green field to green field will never learn, devs that maintain "clever" code will learn pretty quickly.
Sure, but the converse is that when you have a 10k+ LOC project, you can't read that program linearly anymore. I find that sometimes proponents of "straight-forward code" strive for code to be immediately understandable line-by-line, but in big projects, this is not how you navigate a project; you have to create the right modules and abstractions, otherwise you will get lost.
I've worked mostly on massive apps with 400K to 1 or 2 million lines of code, in many projects, so the concept of linearly reading thru the code is something that just never happens.

But with a "keep it simple" philosophy the number of lines of actual code never really matters that much, while code where the developers were clever and complex at every opportunity (often just showing off) the code is quicksand and a quagmire every every single line of the way, and is a nightmare.

Can anyone give a specific example of code they've worked with that was "too clever"?

Based on my experience, if a coworker wanted to "avoid cleverness" I imagine we'd end up arguing over something like a for-loop versus a map, but such small code decision matter very very little in comparison to overall architecture, and where you place the "seams" in your system.

So I ask, when you have felt that code is "too clever", was it because someone used a map, but you're more comfortable with for loops, or was it bigger than that?

My first job was maintaining some PHP. The author didn't know what a function was. The code was a 5,000 line script, top to bottom, with basic control and looping logic nested up to 17 deep (I counted). It was horrible. Yet surely, the author had avoided cleverness at all costs; he had built a working system with only the most basic tools, those being all he knew.

Generally bigger than that, though long clever statements using functional idioms like you suggest could be hard to read in some languages.

Cleverness also refers to architecture, designing meta-types to encapsulate all sorts of things that just don't need the flexibility, over-designing a system in anticipation of future needs that may never come. Sometimes it's a delicate balance and often only experience can dictate how "clever" one should be.

Many developers just out of university, or still in it, in certain languages, like to run with clever things they can do with the type system to abstract away all sorts of stuff, which becomes painful later.

A couple of examples from Java/Kotlin code:

- Too much business logic hidden behind dependency injection (using Dagger in this instance). I think DI should be used sparingly, to decouple large subsystems like the database or the network. It can feel clever to inject everything, so your system is super modular and decoupled, but that just makes it much harder to understand, with little or no real benefit.

- Somewhat related, excessive use of annotations to accomplish tasks that could be done in a more straightforward way with normal code. For example, in Android, you might have an annotation that adds some fragment to your activity as a mix-in; but is that really easier than just calling a function to do the same thing?

This stuff starts to cause real trouble when there are 1000 occurrences sprinkled through your code, and suddenly you need to step through it to debug a tricky problem. Straight-line code is vastly easier to deal with.

So I’d say “clever” is more of a problem at an architectural level, rather than inside individual functions.

At the low level, shorter is almost always better. If somebody comes up with a clever way to reduce a function from 10 lines to 4, say, that’s great -- as long as its purpose is clear and it’s testable.

Interesting examples. I am still conflicted about DI. I have seen it used too little which means that classes become basically untestable. OTOH, things like Spring (don't know about Dagger) seem so overcomplex and hard to figure out, I can certainly understand your point. I've had some luck with just hand-rolled DI in certain cases (e.g. a class that I know is mostly coordinating other classes; in that case, I might just inject all the dependencies manually for easy testing but provide a default constructor for actual production use), but I think it's hard to find the right balance.

Agreed about the annotations; it's one of the annoying things about Java, IMHO. The language itself is so inexpressive that people have to resort to annotations to do basic stuff. I think some level of "metaprogramming" can be useful (regardless of the language), either for boilerplate reduction or for removing aspects like logging from the main code, but it's too easy to become "clever" about it (not just in Java; Rubyists abuse metaprogramming way too often too, for example). I generally favour "explicit over implicit", which also means that I generally dislike inheritance because of all the non-local reasoning.

I've had some luck with just hand-rolled DI in certain cases (e.g. a class that I know is mostly coordinating other classes; in that case, I might just inject all the dependencies manually for easy testing but provide a default constructor for actual production use)

Same here! I think that approach works really well when you’re able to use it.

I’d love to try removing the DI framework and see what you get just rolling it all by hand, but that’s a tough sell in a large pre-existing project.

you could do it as a personal side project without merging it into the mainline and you might learn one of three things:

- handrolled is better and the framework stinks

- there are some disadvantages to the framework, but after doing it all by hand you also understand how it makes a lot of things easier

- it's a tradeoff that largely amounts to which kinds of problems you're personally more willing to put up with (quite likely outcome)

in any case you would learn something

> Can anyone give a specific example of code they've worked with that was "too clever"?

Code that relies on implementation details of a library that a good portion of developers wouldn't necessarily know. In C# if your code relies on LINQ being lazily evaluated to produce the correct answer it's probably too clever.

Using reflection in static languages like Java or C# when it's not necessary.

Writing code that passes functions around when you don't need to.

Chaining together a series of map/filter/reduces in order to avoid writing a simple for loop.

> Based on my experience, if a coworker wanted to "avoid cleverness" I imagine we'd end up arguing over something like a for-loop versus a map, but such small code decision matter very very little in comparison to overall architecture, and where you place the "seams" in your system.

> So I ask, when you have felt that code is "too clever", was it because someone used a map, but you're more comfortable with for loops, or was it bigger than that?

This rule basically comes down to all things being equal try to write straightforward code. If you can write two functions in 5 lines of code but one function will be more easily understood by a more novice programmer, go with the more straightforward approach.

>My first job was maintaining some PHP. The author didn't know what a function was. The code was a 5,000 line script, top to bottom, with basic control and looping logic nested up to 17 deep (I counted). It was horrible. Yet surely, the author had avoided cleverness at all costs; he had built a working system with only the most basic tools, those being all he knew.

No one is arguing that writing straightforward code is the most important code quality to strive for, just that before you implement that currying solution to reduce the number of lines of code from 40 to 35. Think twice.

> Can anyone give a specific example of code they've worked with that was "too clever"?

-I remember being called to help rewrite a few lines of Perl (the other developer didn't manage to do it): it took me two or three hours with constantly looking at a manual to rewrite these few lines in a Perl that a beginner could understand.

The end result had the same line number than the previous version..

-Configuration files made of C++ template for dubious reasons..

I once decided that the SNMP agent for an embedded device should not implement the leafs (responses) by simple hard coded C functions, one for each implemented leaf on the great standardized ASN.1 tree.

Instead I implemented some kind of ASN.1 tree parser, which parsed to lisp, then I tied that through C bindings to some kind of callback to the lisp functions, and clever reuse of datatypes and whatever. I don't even remember how it worked!

I don't remember if it did all the processing on the device or if I had (the same flavour) lisp running in the build system too. I just remember that in the end, even I didn't know exactly how it worked or how to hunt down the inevitable bugs. It looked very elegant though, with lots of autogenerated types from the ASN.1 spec.

The C code which eventually replaced my unholy mess had to handle the datatypes manually in each function, but it was easy to understand and easy to update. Fewer bugs too, I'm sure. And one less dependancy. (The lisp engine.) It didn't have the automatic integration with our vendor MIB file, you'd have to manually add or update functions in the C code whenever someone in another part of the company decided to update the MIB file.

Small price to pay. I'm not saying the "elegant" idea could have been made workable and easy for other developers to understand, but definitely not by me in that point in time. Lisp at the time was my hammer and I saw a lot of nails. (I wasn't even any good with the hammer for actual nail like things, I think.)

Clever:

    for(let i=0;i<100;)console.log((++i%3?'':'fizz')+(i%5?'':'buzz')||i)
Readable:

  for (let i = 1; i <= 100; i++) {
      if (i % 3 == 0 && i % 5 == 0) {
        console.log("FizzBuzz");
    } else if (i % 3 == 0) {
        console.log("Fizz");
    } else if (i % 5 == 0) {
        console.log("Buzz");
    } else {
        console.log(i);
    }
}
I like that example. Since I'm preaching to prefer functional style programing and to separate the side effect from the rest of the algorithm, I'd write the code this way:

  Array
  .from({length: 100}, (v, k) => k+1)
  .map(i => {
    if (i % 3 == 0 && i % 5 == 0) return "FizzBuzz"
    if (i % 3 == 0) return "Fizz"
    if (i % 5 == 0) return "Buzz"
    return i
  })
  .forEach(console.log)
I was working on a pipeline that took data in, did a bunch of work concurrently, then decorated the original data and shuttled it off elsewhere.

I spent a whole sprint building out "the RAD" - a rooted acyclic digraph - that ensured all sorts of correct behavior and powerful options for working with the graph. It felt awesome building what I thought was this super elegant thing that, to me, made total sense since I had been working with it for 3 weeks.

The other devs were confused by it and we're afraid to touch the code. I admitted failure and worked with them to rewrite it. It ended up being an adjacency list (i.e. a graph) with breadth-first traversal, but as long as I didn't call it that, they understood it and liked it.

And in hindsight, we didn't need all the guarantees that it provided, so they were right.

One example would be anything called a "validation framework", hand rolled or as a library. They add so much complication for something easily handled by a validate function and a bunch of linear if statements in the "not clever" version.

One I deal with at work is someone trying to be "clever" and coming up with some annotation based tool to automatically serialize and deserialize objects to a third party system. It's got complex class heirarchies, interception points, etc and I regularly have to crawl through this code. The "not clever" way would be a function that translates the objects to the very simple csv format. In fact I've done that to test interacting with it and when the bash version is simpler to read than the c# version you know it's over complicated.

Another frequent one is "this code is repeated, better move it to a function", problem is you then need to make a change and you have to go through every code path to check it's relevency or add optional parameters, then next time it needs to change you have to check all code paths and inspect which ones are using which paramters. The "not clever" way is to leave the repetitive code, this can have it's own downsides like fixing one place and missing another, but those are much easier to deal with.

Basically use the least level of abstraction that can reasonably get the job done. Your php example probably could have used a little more abstraction like functions and structures but most "enterprisy" code I see could use a lot less.

>Basically use the least level of abstraction that can reasonably get the job done.

There is a great section of John Ousterhout's "A Philosophy of Software Design" that goes into this. He mentions that moving code to functions / methods doesn't eliminate complexity, it nuat kind of shifts it: it adds complexity to the "interface" of the module.

Sometimes leaving code inline with its original context is better.

Not mentioned yet

1) Start with consistent naming conventions, notation and structures

2) Rely on autocomplete to simplify typing/readability/programming effort. It’s a tremendous force multiplier.

Ninja’d

3) TDD

4) Functional programming

Consistent naming does not mean “use the same word for all similar things”. It means using one word every time you have the same thing.

Words stop meaning anything when you use it all the time.

Use a thesaurus. Naming everything “handler” or “node” is just naming them “function” and “object”.

I ask myself what's the smallest, dumbest, least-coupled program I could possibly write to get the job done. Turns out that the answer is often "very", and short, simple programs are a lot easier to produce bug-free and work with over time.
Functional Programming fundamentally changed the way that I solve problems. It also changed my expectations for how code should be designed.

The concepts that influenced me the most were immutability and the power of mapping+filtering. Whenever I read a for/while loop now, I’ll attempt to convert it to the FP equivalent in my head so that I can understand it better.

map and filter are not concepts of FP - they are just syntactic sugar around for loops. fully evident if you check any native source code. many languages (such as go which I currently use) do not even have these functions but can apply FP. this is by design of simplicity and visibility: it does not get much easier to read than a simple for loop and know the exact iterations count at runtime.
One of the key principles of functional programming is, well, functions. A for loop isn't a function and isn't composable, map and filter are.
They are not equivalent - for loop prescribes the ordering of processing, but map and filter can work in any order. They are often associated with FP because to define these operators, you need to be able to pass functions as parameter to other functions, which is a thing that surprisingly many languages struggle with.
That's like saying SQL is just syntactic sugar around loops for databases. Might be true but it misses the whole point of declarative programming.
I’d do the complete opposite. Instead of using mapping and filters I’d prefer for loops.
I can think of 4 things:

1. Not being afraid to look at the code of the libraries that my main project depends on. It's a slow, deliberate process to develop this habit and skill. But more importantly, as you keep doing this, you will develop your own tactics of understanding a library's code and design, in a short amount of time.

2. Not worrying about deadlines all the time. Not a programming technique as such, but in a world of standups and agile, sometimes, you tend to work for the standup status. Avoiding that has been a big win.

3. (Something new I've been trying) Practicing fundamentals. I know the popular opinion is to find a project that you can learn a lot from, but that may not always happen. Good athletes work on their fundamentals all the time - Steph Curry shoots like > 100 3 point shots everyday. I'm trying to use that as an inspiration to find some time every week to work on fundamentals.

4. Writing: essays, notes. In general, I've noticed I gain more clarity and confidence when I spend some time writing about a subject. Over time, I've noticed, I've become more efficient in the process.

What would be some fundamentals to practice for a software developer?
There are quite a few, and you can create a list of categories you consider as fundamentals for the nature of your work. As an example, I would think Algorithms and Data Structures is a fundamental subject. These are the easiest to practice.

You could for example pick something as simple as a HashTable, and implement it from scratch. Then, you could add more complexity to it, like HTs that won't fit in memory, expanding and shrinking HTs efficiently etc.

Or, you could use one of the several practice websites like LeetCode to practice Algorithms/DS problems.

Once you start building a habit, you will also become better at organizing your practice routine and finding out more about what to work on, and where to look for study/practice materials. But mind you, this is a slow process, which you want to build as a habit. There is no end goal here (like cracking Google interview or such), this is a process to get better at the fundamental skills in your field.

Some game changers for me off the top of my head:

* Automated regression testing and build/deploy pipeline. The machine will do things quickly and repeatedly exactly the same way, given the same input conditions/data.

* TDD. Create tests based on requirements, get one thing working at a time, and refactor with a safety net.

* Mixing FP style and OO style programming to get the best of both worlds.

* Understanding type systems, and how to use types to catch/prevent errors and create meaningful abstractions.

* Good code organization, both in physical on-disk structure and in terms of coupling & cohesion.

* Validate all incoming data and fail fast with good error messaging.

Learning generics and the streaming system in Java. There's a clear improvement in expressiveness and they minimize a lot of duplication and visual clutter.
Autoformatting (gofmt, prettier, etc). I can't believe how much time I was wasting having to manually indent, split lines etc.
+1... black for Python. I didn't agree with some of the formatting decisions, but it has gotten better with passing versions, and the gains of not thinking about it at all are huge.
Writing simple functions which take an A and return B in any context. Then, composing those simple functions together to build up more complex functionality.
I would mention generics also. It has forced me to a clean architecture from the start without throwing away code.

Although sometimes the domain is not clear enough before you start, that's shitty.

Honestly, I used to be such a computer nerd, to a fault. So over the last quarter century of programming for money, I'd have to say the most transformative has been writing comments like my livelihood depends on it (since it does).

Other development practices that boosted my output significantly: Regular cardio exercise like running, strength training by lifting weights, and regularly reading source code for pleasure.