Ask HN: What habits make a programmer great?
A quote by Kent Beck:
I'm not a great programmer; I'm just a good programmer with great habits.
So what are the habits I should cultivate to be a great programmer or even a good one? What are the bad habits that I should drop?
190 comments
[ 6.4 ms ] story [ 188 ms ] threadI often encounter situations where a day or two were spent creating a mess of classes and abstractions, yet the actual logic to solve the problem still hasn't materialised. A defense about "code quality" or "not wanting to write spaghetti code" is often made.
My recommendations for a problem you don't have a clear solution in mind for:
1) Hack some pseudo-code spaghetti together in a blank file until you think "you got it"
2) Write some theoretical test scenarios in another blank file (given state, steps, outcome) to verify that it's actually "solved". Discuss with a team member if possible.
3) then write out those functions from 1) inside the existing codebase
4) verify stuff is working
5) only then do abstractions / splitting up into multiple files
For a problem where the solution is already existing, but the implementation lacking (e.g. a refactoring)
1) design the new code (data structures, interfaces)
2) always propose to the team - no ninja-architecture/refactors
3) ensure the solution can be tested, document what should be tested
4) execute on the implementation
YMMV
To convert from a prototype to an end-product, you need to do the other steps.
But:
If you know multiple programming languages. Sometimes adjust for the toolset. Eg. Python or NodeJS for crawling websites. I always abstract it with an api. For eg. Try take a screenshot of a website in c# ( my most proficient language).
SOME tools do get you much faster to the end result with more options. Just make sure you tackle that problem with that programming language and try to abstract it away.
Edit: My solution for website as a screenshot, a c# wrapper for a nodejs api
The API in Nuget ( dot nets package manager): https://www.nuget.org/packages/WebsiteAsImageWebService.Api/
Proper understanding of the problem is extremely important. Great advice.
There is a lot to be gained by going along with the dominant groupthink on a topic, and there is a lot of risk involved in declining to do so. This is especially true as the groupthought picks up steam and gains powerful backers, who will be offended that little-old-you would dare to contradict someone of their high status.
The likelihood that your successes over the conventional/trendy method will be undeniably obvious is very low, especially when you consider that as larger interests and personalities invest in the groupthought, their PR machines will be working to discredit/counteract those who stand to imperil their investment. You will probably not get the punchy ending that you'd need to justify the burn you initiated by going your own way.
You could try to argue that the rest of the team is just trying to follow the zeitgeist instead of thinking about how to generate objective value, but this will hurt much more than just smiling and using the "Tech Flavor of the Week" on the project.
Team cohesion is more important for large-scale productivity than technical implementation details. It's easier to work around a buggy framework than it is to get people to cooperate and release their grudges.
If you want to be a "great programmer" in the sense that you have maximal self-satisfaction, then yeah, you should use whatever you think is best.
If you want to be a "great programmer" in the sense that you're a hot commodity on the market with a strong network full of people who want to give you money, you must recognize that your field of operation is not really computer code, but the delicate egos of the humans around you.
---
Along a less social track: the bigger the technology gets, the more likely that its kinks will get worked out and that it or something based on it will become a driving force in something that is objectively useful/valuable down the road, at which time having the investment in the platform makes it easier to adopt the better tech from both a technical (increased familiarity with tech-specific syntax and concepts) and personal perspective (don't have years of resistance that you've brainwashed into yourself, + accompanying reputation).
JavaScript is a good example of a technology that has been difficult to justify at a technical level in any case where it wasn't mandatory up until very recently. However, its widespread faddiness is now starting to cause actually useful new things to be JavaScript-first or JavaScript-based, even if they're doing that just so they can ride the hype wave themselves.
My experience has taught me the exact opposite: no amount of Google Docs or whiteboarding meetings will tell you what the real problems are, you're only going to find them when you start implementing.
I got burned on my last project: we spent several weeks mostly talking and writing (English) about problems that turned out to be trivial, and got totally blindsided by problems that were not detectable until we actually connected to the firehose of incoming data (it's a streaming/analytical system) and then verified our work. There were lots of subtleties in the way a small portion of messages could be related over days and weeks. Nothing we would have been able to forsee with unit tests or manual inspection. We had to actually write the program to find out what would be hard about it.
When you hear them say: "I need a modern site", don't start until you're clear why they want it. For example "My customers told me my site doesn't look very trustworthy" will lead to a different solution than "My site is too slow" or "My site doesn't work on mobile devices".
Once you know the customer's why, you are also better able to make decisions like "It's okay to lose 1% of the data in case of failure in order to have a simpler system", or "We only look at the results once a day anyway, so using batching is acceptable".
http://threevirtues.com/
To give "urge" more meaning: These days I get a noticeable weird feeling in my gut if I'm working in a environment where there are too many unknowns.
That's fear for me. Always a chance I'll knock something important over...
further, try to create an environment where others (particularly others who are less powerful) feel comfortable voicing their concerns and are secure enough to state when they don't know or don't understand something.
e.g. if people are worried they are going to be fired or ridiculed in a group setting for not knowing a thing then it would be unreasonable to expect them to say "i don't know".
Take error handling really seriously. A program that can gracefully reject input with a detailed error message is far better than one that crashes mysteriously
That attitude hurts everyone who has to support what you write.
edit: Reformatted as a habit "Make sure whatever you write can be supported without unnecessary headaches."
Assertions, however, are great. If you can force the program to throw that NPE closer to the point where it actually illegitimately became null, the debugging session becomes much shorter. Null-safety in the type system (like with most modern languages: Swift, Haskell, Kotlin, Scala, Rust, etc.) is even better.
And: Using a different language is always a great idea, but often not possible.
Code should tolerate all sorts of bad inputs. Code should produce pristine outputs.
Just like being a model member of society
Googled and found it:
https://en.wikipedia.org/wiki/Robustness_principle
https://en.wikipedia.org/wiki/Jon_Postel
"In his lifetime he was known as the "god[2] of the Internet" for his comprehensive influence on the medium."
1. Re-structure your logic so the error cannot happen
2. If you can't do 1, recover from the error silently and continue on with the program
3. If you can't do 2, handle the error gracefully and notify the user of what state is changed, letting them choose whether or not to continue
4. If you can't do 3, inform the user of what happened and ideally provide the user corrective action they can take, then exit cleanly with an error code
5. If you can't do 4, exit cleanly with as descriptive of an error code as you can
6. If you can't do 5, you're probably faced with crashing or exiting with some unhandled exception. This is a bug that should never go into production.
Always be looking for opportunities to move your code up that hierarchy. Moving every 6 to 5 is my personal bar for barely acceptable quality. 4->3 is usually a big win (basically anything that prevents the program from exiting).
I think that defensive programming is the worst way of dealing with errors. You are now silently diverging from your business requirements by making up data.
1. Start small, then extend.
2. Change one thing at a time.
3. Add logging and error handling early.
4. All new lines must be executed at least once.
5. Test the parts before the whole.
6. Fix the known errors, then see what’s left.
Taken from here: https://henrikwarne.com/2015/04/16/lessons-learned-in-softwa...
1. If at all possible start from something existing. It is entirely possible by the time you are done nothing remains of the existing. That's fine. To me, it's easier to tweak existing code than fill in an empty editor window. To some, an empty editor window is all opportunity, to me, it's analysis paralysis.
2. Always be ready to throw everything away. You perhaps will save a few choice lines but be ready for your first, sometimes even the second solution to suck badly. This is fine. Putting in more time to have an easier to maintain solution is always useful.
3. To continue from there, easier to maintain code is king. Unless you are writing something extremely time critical do not try to be clever. A little slower is okay (and yes, I am in the performance consultancy business) if it significantly decreases the maintenance burden. Clever hacks belong to toy projects and blog posts. The next person who maintains it will be stupid to the code -- even if it's yourself. That clever hack is now a nightmare to untangle. In short: always code under the assumption that you will need to understand this when the emergency phone kicks you out of bed after two hours of sleep in the middle of the night. The CTO of Cloudflare was woken to the news of Cloudbleed at 1:26am.
What if you write a function in Go that's littered with the usual
and the error that's being propagated is hard to reproduce, e.g. a filesystem error? I never cover those "return err" in actual programs.Creating tests for some of these cases would involve mocking things out, and then you have to make assumptions that may not hold anyway. That's why I like to see it execute in the real environment at least once.
Yes, a lot of things like that are possible. I remember in one client-server project I worked on, we unplugged the network cable from one PC (while the program was communicating between client and server), to see if we got the correct error message that we had written for that case. And did other stuff like that.
This was done in a project I handled, for the World Bank. I had a team of 9 motivated though young developers (first project for most of them). Near the end, before we sent the software to the client, a colleague was assigned to test the software - testing by a member external to the team, a slightly senior guy, ex-CITIL (a Citi IT subsidiary which had more mature software processes than we did then), though of course we did test the software ourselves a lot, first. I remember what happened:
We prepared the machines and software for the test. Called him over.
He said: I am going to sit down and destroy your software.
We said: Go ahead, please try.
And he tried to. But could not. He only found a few minor cosmetic defects.
Getting up after a few hours of rigorous testing, he said something like - I am happy with your work.
1. Return something meaningful, not the bare error you received above.
2. If you do nothing but decorate the error, ignore coverage. Having decorated the error helps debug the full system when the error occurs.
3. If your error handling is more complex (remove temporary files, close handler, etc.), you might want to factor the cleanup into its own function and test it there.
Finally, your code is not really "littered." If you had exceptions, there would be no trace of the possible errors that might occur on the production system. Your coverage will be higher, but when the software will blow up in production, it will not matter then least.
If you know how these things work internally you'll be 3x better than most programmers.
Fail as much as possible in an environment that is okay with failure. My environments have been side projects at home and professional work as well. Make sure you know why it failed and what you can do better next time.
Some that occur frequently are:
* Take care of yourself. Get enough sleep, exercise and healthy food. Have hobbies.
* Constantly learn new stuff
* Practice communication skills. Building stuff fast and well doesn't help if you're building the wrong thing, and you need communication skills to prevent that
* Think about the context that your program will run in (related to the reason above)
* Practice empathy
Any tips on how to do this? I don't often feel things for myself and it's even rarer to experience empathy at a level I can detect. I meditate to try to better understand and learn to detect my feelings, but haven't made much progress yet (I use Headspace).
Its hard to empathise if your thoughts are elsewhere. When a situation requires/deserves empathy then be aware (mindful, if you like) of the locus of your thoughts and, when they wander, direct them back to the subject. Your meditation practice will definitely help there.
Consciously recognising when your empathy is needed is the hard part. All I can suggest is to try to recognise and avoid habitual distraction.
Everybody struggles with this. I find it hugely embarrassing. Don't bother calling people out for making mistakes, if they're any good, they know they've made a mistake. Instead, give advice about how you avoid those kinds of errors. We've all been there. It sucks. Instead offer up some tricks or techniques for avoiding that kind of problem in the future. If it's really bad, a war story about how you really messed up bad can be calming.
The gist is, your coworker is probably feeling a lot of emotions. We've all felt those emotions, and it's going to be ok. Later, we'll have a post mortem and find a way so nobody can ever have that problem again (or make it harder). It sucks coworker had to be the one to break things in that way, but coworker is helping ever other person to come after them. Somebody was going to do that eventually, coworker just got unlucky.
Backed by science! https://www.scientificamerican.com/article/novel-finding-rea... https://www.theguardian.com/books/booksblog/2013/oct/08/lite... https://www.washingtonpost.com/news/speaking-of-science/wp/2... https://www.psychologytoday.com/blog/the-athletes-way/201412...
I've been confronted with this question a couple of times even before I read the articles linked above and I always answered the same. Always felt like it's such an "obvious" answer since reading fiction puts you in the perspective of another person/character, so it's good empathy exercise. My personal experience confirms this as well, that when I want to shift my perspective to that of another, reading fiction helps.
(Emphasis on "personal" as, in this use case, YMMV.)
Two very different books but, Between the World and Me, and How to Get Filthy Rich in Rising Asia both made me think in ways I cant quite describe.
They can't speak. You have to empathize with them to have any relationship with them. In social situations you'll end up reading body language and worrying about other's feelings without even thinking about it due to pet practice.
Note: I don't have empathy for myself very well, but my own behavior can tread bizarrely enough that I am forced to examine my own behavior in a way that can be explained to others reasonably. If you can explain others' behavior in this way that isn't condescending towards them, that takes all aspects into their behavior into account, you will come off as compassionate and empathetic regardless of how you feel in the situation. At least this has been my experience.
Raise kids. No joke. They can do some pretty stupid things, and keeping them out of danger and helping them grow has actually helped me understand and empathize with my team quite a bit.
Be generous with your life (from small things like hosting guests in your home, to traveling to help people survive and thrive in other countries). When you see what others are going through, you'll probably have a gut reaction. You should probably trust your gut.
Read books on leadership. It helps to learn how other good leaders coped with similar circumstances. -- and yeah, I just called you a good leader without meeting you. Just asking this question and being introspective indicates you're on the path to being a good leader.
https://www.amazon.com/Emotions-Revealed-Recognizing-Communi...
https://www.amazon.com/Unmasking-Social-Engineer-Element-Sec...
Doing all the other self-care that OP mentioned is a good way to start moving in that direction. Be patient and kind to yourself. Once you've learned that, you actually start developing and being able to feel that way about others. It sounds a little hippie dippie, but once you have patience and you're at peace with yourself, you're much better at being able to accept the faults of others.
Especially seems that a lot of devs forget about the first point.
In my experience, when I've started doing gym & swimming I became to feel less "burned out" as well as my self-esteem improved :-)
If you try to work on a complex algorithm with little sleep, you tend to throw a wrench into it.
Taking the time to properly understand the concepts, protocols, formats, apis and tools that you use when working on something.
This I never seem to have the time or mental energy to do.
See videos like Gary Bernhardt - The Unix Chainsaw on how he gathers very nice data on just about anything in his system with a few pipes. It's easy to try to be awesome in OOP, or in javascript but waste time and energy clicking around or digging manually in repos for information.
I find that is the nicest and most efficient advice I ever took.
The other habit is: improving your maths skills (sophisticated combinatorics, probabilities, statistics) ..
http://bookofhook.blogspot.in/2013/03/smart-guy-productivity...
They are very concerned about compile times and having a short edit/compile/run cycle.
2. Check your ego at the door. A good friend begins a class he teaches having coders say "I'm a programmer and I make mistakes"
3. I've seen a lot of simple systems that did useful things that grew into complex systems. I've never seen a system that started off complex being anything but a huge PITA for everybody involved. Keep It Simple, Stupid
4. Learn all the major programming paradigms and be able to solve problems in all of them. In my opinion, this is the first step to being any kind of programmer at all, much less a great one
5. We write code to help people. We write code alongside other people. People are a critical component, yet we hardly talk about them. Programming is about 95% social, 5% technical. But reading the latest industry news, you'd never think that. Keep your focus on the people, not the tech. Knowing and manipulating the tech is how you get in the game. How your work impacts real people is how you master it. You should be a professional and write clean code, but nobody cares about your standards if you're not getting along with others or making something somebody actually wants
6. Study mistakes and disasters. In every true professional field, folks spend a lot of time picking over and thinking about things when they go wrong. You should do the same. A big part of success is simple failure avoidance
7. Pair/mob program and use TDD if you're deep in OO or complex/mutable code. Nothing shows you how much you still have to learn like making what looks like a simple change and watching dozens of tests go red
8. Keep learning more about text processing and command line utilities. There are a ton of people right now writing complex code for things that can be done in a few lines of shell scripting. Don't try to solve everything in the shell, but know that a little bit of code can replace a ton of work if you know what you're doing.
9. Pick 2 or 3 IDE shortcuts and practice them each week. The less you touch your mouse and the more you can do work without moving your hands the faster you'll be. (This is one I need to work on)
10. Hang out with people that are better than you and pay attention. Greatness tends to rub off over time.
Seems to be Gall's law:
https://en.wikipedia.org/wiki/John_Gall_(author)#Gall.27s_la...
Every programmer I admire has worked on their chosen field for years. You can keep flitting between languages and frameworks, but mastery ultimately comes from hankering down on one thing and advancing our collective knowledge a bit further. You can achieve mastery in any field - it can be a business domain - as mundane as a Time Tracking or a Todo list app, or it can be a technical domain - like compilers or databases or browsers. You however can't achieve mastery by consuming what others produce. Becoming an expert in CSS and its quirks is a good vocational investment, but you're no closer to a fundamental understanding of the field by just consuming an API.
You should spend years working on a problem domain that interests you. Look at prettydiff (https://github.com/prettydiff/prettydiff). It is a diff tool, a parser, a pretty printer, a minifier, all rolled into one. It is the most comprehensive publicly available tool in terms of language and dialect support in the world right now. Its code is not the most modular and new contributors might have trouble getting in, but its author has been at this problem for years, and if you called a conference for pretty-printing, he would be one among the few thousand people in the world to meaningfully participate.
Consider blueimp's jQuery File Upload plugin (https://github.com/blueimp/jQuery-File-Upload). While this isn't as foundational a work as prettydiff, the project spans seven years and a thousand commits. It solves a mundane problem, but the sheer effort that has went into it makes it a formidable solution.
Consider Martin Odersky, watch Compilers are Databases (https://www.youtube.com/watch?v=WxyyJyB_Ssc). He has been writing compilers for like forever. He's worked on the field for decades, and has written compilers for everything from Fortran to Java to Scala.
You don't have to be a genius to be a master. You just need to go as deep into a field as you can. Here are a few interesting links you might want to read:
The Joys of Having a Forever Project: http://web.archive.org/web/20130125011224/http://www.dev.gd/...
Rich Hickey on becoming a better developer: https://gist.github.com/prakhar1989/1b0a2c9849b2e1e912fb
You and Your Research by Richard Hamming: http://www.cs.virginia.edu/~robins/YouAndYourResearch.html
(...Don't polarize on one end of the Specialist vs Generalist debate.)
Since improvement of experience and quality of output is roughly logarithmic, it is (depending on the specialisation of the task at hand) better to have a person that has worked ten years to be 7/10 in 5 displines than 10/10 in one and 2/10 in the remaining 4.
Of course, if you only care about the one discipline, get the 10/10 guy. But in real use cases, things are only occasionally atomic, and lack of understanding of adjacent processes and disciplines can prove fatal.
There are varying definitions of "understand", but I am confident that I understand something when I can give a lecture on it with no preparation... or when I can code it without checking any references.
+
There is no one true way.
=
Anybody that claims that there is only one way to do something is selling dogma.
that and, listen to many voices, especially those that aren't the loudest.
"Researchey" green-field development for data-science-like problems:
1. If it can be done manually first, do it manually. You'll gain an intuition for how you might approach it.
2. Collect examples. Start with a spreadsheet of data that highlights the data you have available.
3. Make it work for one case before you make it work for all cases.
4. Build debugging output into your algorithm itself. You should be able to dump the intermediate results of each step and inspect them manually with a text editor or web browser.
5. Don't bother with unit tests - they're useless until you can define what correct behavior is, and when you're doing this sort of programming, by definition you can't.
Maintenance programming for a large, unfamiliar codebase:
1. Take a look at filesizes. The biggest files usually contain the meat of the program, or at least a dispatcher that points to the meat of the program. main.cc is usually tiny and useless for finding your way around.
2. Single-step through the program with a debugger, starting at the main dispatch loop. You'll learn a lot about control flow.
3. Look for data structures, particularly ones that are passed into many functions as parameters. Most programs have a small set of key data structures; find them and orienting yourself to the rest becomes much easier.
4. Write unit tests. They're the best way to confirm that your understanding of the code is actually how the code works.
5. Remove code and see what breaks. (Don't check it in though!)
Performance work:
0. Don't, unless you've built it and it's too slow for users. Have performance targets for how much you need to improve, and stop when you hit them.
1. Before all else (even profiling!), build a set of benchmarks representing typical real-world use. Don't let your performance regress unless you're very certain you're stuck at a local maxima and there's a better global solution just around the corner. (And if that's the case, tag your branch in the VCS so you can back out your changes if you're wrong.)
2. Many performance bottlenecks are at the intersection between systems. Collect timing stats in any RPC framework, and have some way of propagating & visualizing the time spent for a request to make its way through each server, as well as which parts of the request happen in parallel and where the critical path is.
3. Profile.
4. Oftentimes you can get big initial wins by avoiding unnecessary work. Cache your biggest computations, and lazily evaluate things that are usually not needed.
5. Don't ignore constant factors. Sometimes an algorithm with asymptotically worse performance will perform better in practice because it has much better cache locality. You can identify opportunities for this in the functions that are called a lot.
6. When you've got a flat profile, there are often still very significant gains that can be obtained through changing your data structures. Pay attention to memory use; often shrinking memory requirements speeds up the system significantly through less cache pressure. Pay attention to locality, and put commonly-used data together. If your language allows it (shame on you, Java), eliminate pointer-chasing in favor of value containment.
General code hygiene:
1. Don't build speculatively. Make sure there's a customer for every feature you put in.
2. Control your dependencies carefully. That library you pulled in for one utility function may have helped you save an hour implementing the utility function, but it adds many more places where things can break - deployment, versioning, security, logging, unexpected process deaths.
3. When developing for yourself or a small team, let problems accumulate and fix them all at once (or throw out the codebase and start anew). ...
It's helped that it's really been 12 (well, 13+change) years of experience, rather than one year repeated 12 times. Each year has brought something new and different that's just out of my comfort zone.
> Don't ignore constant factors. Sometimes an algorithm with asymptotically worse performance will perform better in practice because it has much better cache locality.
Forget the cache, sometimes they're just plain faster (edit in response to comment: I mean faster for your use case). I've e.g. found that convolutions can be much faster with the naive algorithm than with an FFT in a pretty decent set of cases. (Edit: To be specific, these cases necessarily only occur for "sufficiently small" vectors, but it turned out that was a larger size than I expected.) Caching doesn't necessarily explain it I think, it can just simply be extra computation that doesn't end up paying off.
> sometimes they're just plain faster
Not faster for sufficiently large N (by definition).
But your general point is correct.
I've best seen this expressed in Rob Pike's 5 Rules of Programming [0], Rule 3:
Rule 3. Fancy algorithms are slow when n is small, and n is usually small. Fancy algorithms have big constants. Until you know that n is frequently going to be big, don't get fancy.
[0] http://users.ece.utexas.edu/~adnan/pike.html
True, but supposedly researchers keep publishing algorithms with lower complexity that will be faster only if N is, like 10^30 or so.
Or so Sedgwick keeps telling us.
This is my #1 pet peeve with GitHub. When I first look at an unfamiliar repo, I want to get a sense of what the code is about and what it looks like. The way I do that with a local project is by looking at the largest files first. But GitHub loves their clean uncluttered interface so much, they won't show me the file sizes!
Check out Octotree if you're using Chrome. No file sizes still, but I've found that when you just want to quickly explore some potential new source this beats having to clone the repo first.
I think this Chrome extension called "GitHub Repository Size" might be exactly what you are looking for
This is true not just for data science but when trying to solve any numerical problem. Using a spreadsheet (or a R / Python notebook) to implement the algorithm and getting some results has helped me in the past to really understand the problem and avoid dead ends.
For example, when building a FX pricing system, I was able to use a spreadsheet to describe how the pricing algorithm would work and explain it to the traders (the end users). We could tweak the calculations and make sure things were clear to all before implementing and deploying the algorithm.
Great advice!
I work with a lot of traders. One antipattern I noticed is that when there's a problem with the data, they'll do all sorts of permutations and aggregations and then scratch their chins and ponder about it for hours.
Go to the fucking source and find an example of the problem! Read it line by line, usually it will be obvious what happened.
Corollary: Don't assume your data is correct, most outliers ina large data set are problems with the data itself. Build a few columns that serve as sanity checks. One good example is a column that shows the distance between this sequence number and the last, anything >1 is a dropped message.
6. Use assertions for defining your expectations at each stage of the algorithm - they will make the debugging much more easier
On HN, I often check specific commenters activity, or new comments in a thread that can be days-old, because I often find hidden gems. A link to more elaboration would definitely count as such. Perhaps an audience of even just a dozen or so people like me might be worth it.
EDIT: Coincidentally (I swear), there's exactly a dozen of comments that positively engage with your comment!
I reformatted this in a Google Doc, if anybody wants: https://docs.google.com/document/d/1Pix3-l3Qz1aLOuxoiiP1PTV6...