Ask HN: Thoughts on Rich Hickey's advice on becoming a better developer?
Very recently I stumbled onto a comment by Rich Hickey which I've reformatted in a gist[0] (unfortunately, the source is no longer available). Honestly, Rich Hickey is one of my programming heros and I'm a big fan of his ideas about programming. However, reading the response below, I couldn't help but feel disheartened.
Rich Hickey says - You don't level up by switching games all the time, but by sticking with one long enough to gain advanced skills. And, you need to be careful to recognize the actual game involved. Programming mastery has little to do with languages, paradigms, platforms, building blocks, open source, conferences etc.
Quite seriously, this is more or less opposite of what I've lived by in my career till now. As a backend engineer I can't help not getting excited by React or not spending a month learning OCaml or building side-projects tangential to my 9-5 line of work. The worst part is, that till now I always felt that it is a sure fire way to level up as a developer.
The confusing part is that Peter Norvig, another great programmer has quite the opposite to say[1] (although one can argue that overall they mean the same) - Learn at least a half dozen programming languages. Include one language that emphasizes class abstractions (like Java or C++), one that emphasizes functional abstraction (like Lisp or ML or Haskell), one that supports declarative specifications (like Prolog or C++ templates), and one that emphasizes parallelism (like Clojure or Go).
At this point, I'm really confused and I spent better part of the day mulling over whether I should go through the lectures of a recent Coursera course that I'm doing or should I dive deeper in my current stack e.g. Python / Django.
What are your thoughts? Is there one school of thought that you agree to?
[0] - https://gist.github.com/prakhar1989/1b0a2c9849b2e1e912fb [1] - http://norvig.com/21-days.html
113 comments
[ 3.0 ms ] story [ 219 ms ] threadhttps://gist.github.com/prakhar1989/1b0a2c9849b2e1e912fb
http://norvig.com/21-days.html
Those are the two career paths illustrated. Different people have difference preferences.
To me it sounds like he's saying "understand the underlying patterns and techniques that make good programs in any language" not "stick with one, and only one, language until you know it inside and out".
Choose one or two language and get mastery from these, but also look for other languages to learn from.
That way, you'll have a balanced portfolio.
Think how all programming languages can be reduced to lambda calculus or Turing machines. And think how design patterns and algorithms can exist across all programming paradigms. If you can master design patterns and algorithms in one general-purpose language, then to pick up other languages you just need to learn the syntax and idioms of that language.
On the other hand, a quick way to expose yourself to different programming concepts is to take a crash course in very different languages, as Norvig suggests. The risk with this method is that you never attain mastery in any one thing ("jack of all trades, master of none", s/trades/programming languages/)
I think Hickey is right in this one, but fundamentally they are talking about two different things. Hickey is talking about mastery, while Norvig is talking about "programming success" ("success" is word with a slippery definition, but to be "successful" definitely does not require one to be a "master")
You get a better idea of what 'advanced' is both by diving deep in to one language but also by digging in to other languages - seeing how they do stuff, what's possible, what's elegant, what's a hack, etc.
Re: parallelism, as a useful and simple exercise, try writing a load tester in Python and then try it in Go (simple program that hits yoursite.com as hard as your network connection will allow).
For example use Ruby/Rails on the backend or a web app, learn Objective-c to create an app that gets data from that RoR app, and maybe use go to create an API for a section that is performance heavy, etc... So that everything builds up. You need to establish goals.
But here he is saying that you can't only explore the breadth of programming, you also have to dive deep at some point and get the experience of mastery in something. I can't see either one disagreeing with the statement that you want to have some breadth of experience first but that mastery only comes afterwards from focus.
If you are thinking of learning a new language, ask yourself why. Is it because it's going to make you a better programmer? Is it because all the cool kids down at the Club House are talking about it? Is it because you're stuck with a problem in a language, and instead of overcoming that problem, you're just jumping to a new language hoping that language won't have any problems?
When you've really mastered one language, mastering a second one is much easier. Whereas if you learn the basics of one language, and then the basics of another, ad infinitum, you never really master anything.
Compare to learning to play musical instruments. Who is the better musician - somebody that can play Chopsticks on 20 instruments, or somebody that can play Brahams but only on the piano? The first has a lot more to learn than the second.
Finally, it's better to master a domain than a programming language. In the long run, it's more rewarding (personally and financially) to be the world's leading expert in, say, security in C++ than it is to be the world's 10,000th best developer in Ruby, Clojure, Go, etc.
"We value “T-shaped” people.
That is, people who are both generalists (highly skilled at a broad set of valuable things—the top of the T) and also experts (among the best in their field within a narrow discipline—the vertical leg of the T). This recipe is important for success at Valve. We often have to pass on people who are very strong generalists without expertise, or vice versa. An expert who is too narrow has difficulty collaborating. A generalist who doesn’t go deep enough in a single area ends up on the margins, not really contributing as an individual."
[0] - http://www.valvesoftware.com/company/Valve_Handbook_LowRes.p... page 46
It also makes me wonder about their ability to manage people with different skill levels and areas of expertise. And of course, having used Steam, I'm not at all surprised.
> It also makes me wonder about their ability to manage people with different skill levels and areas of expertise. And of course, having used Steam, I'm not at all surprised.
Weird that that's the conclusion you come to. Valve is an incredibly successful company with over a billion dollars in the bank. They've produced some of the most influential games of the past two decades (Half-life, Counter Strike, TF2, Left 4 Dead, Portal, DoTA2... it's actually hard to name them all).
The management failures at Valve have been pretty spectacular, and expensive, of late, it's just that because they are a private company, and because Steam makes all of the money, they can continue to have large expensive failures like the VR or the Steambox until doomsday.
Valve is not other companies, and their position is such that it's not advisable to take any too serious advice from how they run things into the real world.
Besides shelving the VR project that she was reportedly working on, there's also the Steambox which still doesn't exist (the only hardware to even pass prototype stage eventually had to abandon SteamOS and release on its own), the utter crapshoot that is their attempts at virtual economies (TF2 hats: prints money, trading cards: useless boondoggle, everything involving Counter-Strike: possibly lucrative now, but PR disaster after PR disaster with their fans).
Valve spins the utopian image, and generally keeps the veil down enough that goodwill and fandom manage to sustain the myth, but I suspect the cart's been off the rails for sometime, and if it weren't for Steam we'd have long seen a very public failure by now.
Sure; every employer does. But it's dodging the more interesting question, which is: faced with one candidate who has broad but shallow knowledge and another who has deep but narrow knowledge, which do you pick?
Actually, it's not that difficult. Of what you listed, only HL was original IP.
Valve put a lot of work into digital distribution, and have pretty much owned the PC digital distribution market.
With console and mobile gaming dwarfing the PC market, and big publishers like Ubisoft and EA getting into their own digital distribution, I'm curious to see how Valve continues to grow.
That said, I admire them as a company.
PC is stronger this generation than it was last generation, I'm not sure about the numbers but I suspect that it is ahead of both consoles, and it might be ahead of both of them combined, and I'd expect this to only be get more true. Porting to PC is pretty much just porting the parts that interact with the OS or GPU for this generation, due to both consoles using x86-64.
And mobile is a joke. market might be huge, there's a reason very few established game companies care about it at all. It's extremely unpredictable, and the expected price of games is far below what is realistic. Even if you make a hit, the amount of money you'll make is far from the amount of money you make from a hit on any other platform (hell, it doesn't even come close to the budget of most games). Honestly, I think the only way a company can hope to turn a profit on mobile is with IAP or shovelware (or both), and I don't see that displacing console or PC gaming any time soon.
I wouldn't mind seeing steam displaced, but I don't think it will be by EA or Ubisoft, largely due to the fact that they're both about as popular as Comcast, which seems to be due to them being massively out of touch.
When I see my friends 3 year old daughter playing Geometry Dash on his phone, I know that the PC can't compete with that.
PC ports are still and after thought. Games use a lobby system instead of dedicated servers. Only a few companies (Valve being one of them) target the PC market first. Pretty much every other AAA publisher looks at the PC market as an after thought.
Don't get me wrong, I'm a die hard PC gamer.
PC vs Console vs Mobile would be a more reasonable competition.
From some reports PC gaming market seems very healthy indeed from monetary perspective.
http://usfinancepost.com/pc-still-gaming-market-is-double-co... http://www.gamesindustry.biz/articles/2014-01-28-pc-gaming-m...
Mobile has been predicted to reach the size of PC gaming, but not to necessary dwarf it anytime soon.
http://www.mobilemarketer.com/cms/news/research/3892.html
So certainly if you combine mobile and console it's larger than PC. Says a lot that it takes to distinct markets to overtake PC.
In my opinion all the platforms have their pro's and con's, and none of them are going to disappear anytime soon.
I wasn't suggesting that PC gaming will disappear. However, it's clear that PC gaming isn't a growth sector. Given that is where Valve dominates, where does that leave them?
Valve may not consider themselves a growth oriented company, but they do look for opportunities to innovate.
That said, actually using Steam is an awful experience. Literally 50% of the time, when I launch Steam, it immediately stops and gives me a dialog saying I'm not connected to the Internet, despite that I have other Internet-using apps open and working fine. Quitting and immediately restarting works just fine for no apparent reason. I have several examples of things like this that show a lack of quality in their products. In my opinion, that is usually an indication of poor management as they aren't concerned with the quality of their product.
Another meme that's been thrown around is: "You are not the complete developer if you've not written a compiler." I don't necessarily buy that, but there is some merit: there is a lot of different skills required in a compiler, from domain modelling and DSL design, to theoretical CS like grammars and parsing, to very practical software engineering such as IO handling, string manipulation, encoding, documentation, etc.
[1]: https://yourlogicalfallacyis.com/no-true-scotsman
Hm, funny thing you mentioned startups. Apart programming related startups, I don't think that startup-ers are "great programmers". I get the feeling that most startup programmers (not all of them, but most of them) are somewhere between average and novice level.
I get the feeling that most "great programmers" are not that interested in startup (making money, living the thrill or whatever).
Most famous programmers, work in big corps for medium-to-huge salaries and (most of the times) even getting paid to work on their side-projects (Dropbox pays G.V. Rossum for Python-dev, Heroku pays Matz for Ruby-dev, etc.).
Beyond the scope of HR, in my experience it's the mix of both breadth of knowledge (context) and the depth of understanding (skill + experience) that is most useful.
Having a high-level understanding allows you to step back outside of the 'work' to instead focus on the objective, and someone with significant background knowledge and experience can utilize that to first conceptualize the optimal approach before hammering the nails.
Knowing what to apply, where, and when is equally as important as how. And there are fantastic programmers & hackers on each end of the spectrum, but in my experience developing that contrast is more intrinsically valuable.
There isn't one right answer though. The answer depends a lot on what you want to do. Do you want to lead a team, become an architect, go contracting, become a consultant, go into academia, join a Google, join a start up, write financial software, help cure cancer, invent new ways to do things, become an expert? There are lots of ways you can develop a career in software and whatever happens will most likely be part chance.
Don't worry about it too much, except to make sure you don't get too comfortable in a particular area. If you keep trying to challenge yourself and doing things that are interesting, you will keep getting better. Doing things that are interesting to you is important because it'll help you avoid getting too hung up on what everybody else knows and will help fuel you through the parts that are painful or tedious.
[0] - http://www.valvesoftware.com/company/Valve_Handbook_LowRes.p...
While it is true that learning different programming paradigms can help you evolve as a programmer, I strongly believe (I have been programming professionally for about 20 years) that that is far from the best way to evolve as a developer, and learning algorithms and data structures will help you "level up" faster and better.
The reason is simple. Programming languages are a medium of transforming programmer intent into something the computer can execute (perhaps indirectly). This is not an easy task by any means, but all an ideal (nonexistent) general purpose programming language can hope to achieve is the reduction of what is known as accidental complexity. Programming languages vary only in how much burden, and -- just as important -- what kind of burden they place on you when you commit your chosen algorithm from your mind to code on the screen.
Knowledge of algorithms, on the other hand, will help you tackle the far more important essential complexity of the problem at hand.
So, in a way, focusing on programming languages is indeed a distraction, at least as long as you haven't yet familiarized yourself with the more fundamental -- and more useful -- tools of computer science.
I recently spent a few hours reading about and implementing A* for the NPCs in a video game I'm working on.
Are you talking about being able to write A* and other algorithms from scratch with no reference material?
Its useful to have a steel beam and know how to apply it to construction. Knowing how its made and the metallurgy and how that impacts the application levels you up. Then there's a whole sea of lower level construction vocational laborers who don't know steel beams exist and they'll just bodge something together with stuff from the junk yard and when it collapses, well, there's no guarantees in life and it was cheap and the academics never know anything anyway so stop complaining, and last but not least the social butterflies who heard the Jones's next door, who are billionaires, built their wall out of recycled plastic so we'll use that to hold up the roof instead of proletarian wood or even this steel beam stuff.
At it's simplest you have things like checking for existence using linear scans of arrays. Anyone with experience sees that as a bad thing to do because they understand that it's inefficient for any sort of sizeable array. But those who don't understand algorithms or data structures won't even know there's an issue. And that's the simplest case really.
When you have a list and you need to efficiently add and remove items from both ends - then you want some sort of double-ended queue. I know where to find the one in my std lib and I know when I need to use it - which it turns out is almost never - I thought I did the other day, then I found a simpler way, which was actually a bit disappointing :)
Then, as you say, you have things like A* or R-trees. The more of these you have an handle on - even just to know they exist, the better position you'll be in when you need them.
But I don't even think that's the most important part of knowing data structures and algorithms (and really, I'm sure none of that's news to you if you're implementing A* searching). For me it's more about understanding efficient patterns for data access. So much of the work I do is about efficiently pushing data around in different forms. Often having a core chunk of data and then a couple of different indexing structures to allow me to interrogate it depending on the info I need.
No, I mean knowing what algorithms/data-structures are available, what their characteristics are, and how to tweak them. This way you can say stuff like: with those data access patterns, we'll need a B+-tree with links for concurrency, modified to not delete depleted nodes until such-and-such. Or, something like: doing that efficiently requires an optimization that's NP-hard because the problem reduces to graph partitioning, but we can use an approximate solution that should work well-enough in this case.
Second, you should know when to use A* or something else, the trade-offs etc.
Third, you should know how to implement A* , when needed, with reference material (you'd be surprised how many programmers can't read/understand the reference material even when its given to them, or don't know enough programming to put it in code).
Of course you could get by with never using A* or tries or whatever advanced algorithm, not even in some ready-made API form. E.g. if you just do some CRUD or some simple web programming. But for the kind of programmers we're talking here, those three are paramount.
The thing you mention, implementing it from scratch with no reference material might be impressive, but it's more of a circus act.
In my experience, knowing which prebuilt algorithms and data structures to reach for matters more than being able to implement them myself. Not saying there isn't value in learning or that it doesn't help flex the problem solving muscles, but in most software domains, I think that practically speaking there are more important skills to focus on after you've mastered the basics of time and space complexity.
Because for people working in those kinds of problems we're talking about (we started from Valve and games IIRC, but it could also be Google, Facebook engineers etc) you don't just download some off the shelf API.
You'll need to create something of your own to:
- have the whole IP
- mold it specifically to the domain logic you need
- take control of it's memory and performance characteristics based on your constraints
- implement it in the language your company uses, for which no ready made A* (or some other algorithm) is available
- you'll need to make it talk to different infrastructure, libs etc
- you might be the one writing the library API yourself
As I said, this is not about what a CRUD programmer will need. But as you can see everyday in HN, people write these and other algorithms all the time in their jobs doing more serious engineering.
I've seen entire projects which do not require specific knowledge or application of any algorithm. Even when they do, one can google and find "best algorithm for XYZ", then find a library which has the algorithm implemented and off you go.
I understand the use of algorithms in low level languages and specific domains, but when you need to sort a ruby array you just "array.sort" and a default (IIRC the default in ruby is a version of quick-sort) sorting algorithm is applied, which 9 out of 10 is the faster solution you'll get given the language constrains.
Can you offer an example of specific situation, in any OO-language with reasonable amount of available libraries, where deep knowledge of design and analysis of algorithms is actually required?
Interviews :)
You won't get very many good replies because you are asking the wrong questions. The truth is you don't even need a decent understanding of OO for most day to day dev. You will just produce suboptimal work.
A better question is: "...where reasonable knowledge of design and analysis of algorithms allows you to do a better job?"
And the answer to this more reasonable question is: Every single project I've worked on as an enterprise developer over the last 10+ years. Being able to say "hey actually this problem we are working on can be formulated as a graph problem" can turn a 3 month problem into a 3 day problem.
For example we recently had a rules engine rule dependency issue that turned out to be expressable as graph problem and was then easily solved. If nobody in the team had a good knowledge of algorithms we still could have solved the issue but it just would have taken a lot longer.
And unfortunately until you can phrase the problem in the right way you can't just google "find best algorithm for XYZ".
Also, I believe such knowledge will make you a better programmer, even if you don't have to build everything from scratch. I have seen experienced developers using a list to store unique elements, when the only read operation they would do was to check if an element was in the list. Their solution to speed up the checks was to sort the list every time it was modified and then run a binary search. In this case, better knowledge of algorithm complexity and data structures could lead to a much faster solution, using a set for example.
[1] http://www.academia.edu/6128585/A_Heuristic_Approach_to_Trai...
I wrote a book specifically on how to go from good to great or at least how to become the best you can be... targeted to developers in the first 5 or maybe even 10 years of their career.
Level Up! How to Become a Great Professional Software Developer
https://leanpub.com/level_up
None of it has to do with any specific language or platform. The techniques I describe are not the only way to do it but I guarantee if you take at least some of them to heart, you will improve a great deal. I do espouse going deep on at least one platform or stack for at least several years. You might even get so formal as to build a competency map to track your progress. Much of what you learn is transferrable. 10 years or 10,000 hours will get you there no matter what but following the techniques I describe will get you there faster.
Steven Talcott Smith, Master Developer, Chief Happiness Officer, ÆLOGICA
http://aelogica.com | Great Rails teams for hire
http://appexpress.io | Rapid Application Development
It doesn't matter what or how you program, as long as you have: The User.
If you have to spend your time competitively learning some new-fangled slippery, slidey, pretty little ball of tangled chains of bits and pieces of - whatever - it doesn't matter for a fig if you don't have: The User.
With The User as the principle focus of your goals and career as a developer, programmer, coder, hacker, binder-of-pretty-gthings, you can escape all misery and just learn to enjoy the ride completely. Because its The Users' bus, and if you don't have The User driving it, it ain't going no-Where.
So the whole perspective about Industry standards and real reasons for just using one thing, and one thing only, and sticking to it and becoming good at it: this is only ever acknowledged, worth a fig, if you have: The User. I've still got users of 20+ year old software stacks that I'd love to replace with some newfangled jangle. Have you?
That is how you prepare yourself for a career, in service of The User.
Your advice is correct about how to get better serving users, but that was very much not the question being asked.
And even if Developers can themselves be Users, in the sense that they gain some sort of joy out of consuming the developer tools of the trade, its still just navel-gazing until there is actually some sort of use out of the activity.
Real developers build things for users. It doesn't matter how good or bad they are, as developers, if there is a User: win.
Instead, set aside time to become a better marketer. If you're a solo developer you will need to get the word out to investors and customers about yourself and what you can do. Working for someone else, you'll additionally have to sell your ideas to your co-workers and management.
Even in just asking this question, you are demonstrating motivation to do your best as you create. However, your creations will typically be far better served by your ability to promote them.
Being able to convince others of the value of one's contributions tends to create an expectation for even greater contributions - this driving force can easily be directed toward improvement as a programmer.
But mastering a platform - be it a language, a framework, a library - will help make you a much better developer, it is true. If you don't know enough to exploit the relevant features of the platforms you're using, you'll produce a poorer solution than otherwise. You'll use more code, or fail to encapsulate. Your solution will be harder to maintain and adapt.
But then again, if you're busting your ass writing parallel applications in C++, managing locking and threads, when you could be using actors in Akka, your solution will have the same shortcomings compared to the one you could have developed had you been aware of the qualities of that platform.
So yes and no :-)
In the beginning, everything is new and it can be overwhelming. Humbleness and discipline or dedication is required in the beginning. Small successes lead to pride and enthusiasm, and gradually, over years, your skills grow. In playing guitar and glassblowing, i recall various times where I was satisfied and or pleased with my skills, and then looked back at those times years later and was amazed by how much more sophisticated my skills have become.
With that experience, I never felt frustrated learning Python, JavaScript or SQL because the process reminded me of my previous learning experiences and I knew then all I needed to do was put in time and seek information.
As an aside, I might note that there are mathematical aspects to creating, playing it and even appreciating music. Every system of music has rhythm and a set of tones that is based in someway on numbers – the 12 notes of the western octave, time signatures, 8th and 32nd notes and so forth. Learning the complex relationships of systems like jazz chords on the guitar or piano is very much a mental exercise as well as a physical one.
Creating a new project unlike any you've done in the past will do more to contribute to your skills as a developer, whether that project is in a new language or one you already know, than just picking up a new language and making your 10th simple blog.
http://jasonrudolph.com/blog/2011/08/09/programming-achievem...
At some point I was no longer learning anything from perl and I tackled something different. Erlang was my next deep dive.
What Rich is saying has less to do with how many languages you've absorbed whether a lot or a few. What he is saying is that leveling up requires taking the time to truly absorb and understand what you are working with.
I'm fairly polyglot these days. But the number of languages that I know deeply is less probably about 3. With the time spent learning them measured in years not months.
Peter Norvig is actually saying something similar. He's advocating a slightly different route to the same goal. Learn the foundations of programming deeply. The 4 broad categories he lists are foundations of programming that any developer needs to learn deeply.
Both Rich and Peter advise investing the time to learn something thoroughly. They just came at it from slightly different directions. I imagine they would agree with each other though.