Ask HN: How best to learn software design principles?
Recently I had several rounds of interviews with one of the big tech companies, and while they said my problem solving and coding skills were good, they said they eventually decided to reject me due to my inadequate solutions to design problems, particularly software design.
In the long term, I'm sure the usual advice of reading through great open source code is the way to go. In the short term, where can I find write-ups of software design, going from 'How would you build a spreadsheet program?' to actual UML designs, discussion of design patterns, etc.
99 comments
[ 0.26 ms ] story [ 260 ms ] threadWhat videos would prioritize over the others?
Then I'd move on to Domain Driven Design and probably Refactoring and Growing Object Oriented Software, Guided By Tests.
Those four will give a good start to what enterprise/OO people are expecting in design.
You'll find as you watch them that the signal-to-noise ratio drops with each episode. In the last few, there's about ten minutes of useful information mixed in with about 50 minutes of cosmology lessons, tours of his house and neighbourhood, and babble amongst his various alter egos (Data, Sherlock, Spock, Minecraft guy, Ruby guy, Microsoft fanboy, and so on).
Everyone has their own threshold for how much of that they can tolerate. For me, it crossed that about five episodes ago.
https://twitter.com/garybernhardt/status/485978334434586625
You could do worse than learning from his book when it comes out!
It gets so weird you'll soon forget that first thumbnail.
Design patterns are the greatest tool there is for learning software design. If you know them, you can apply them, but much more importantly they show you how you can apply the basic principles of OO development to various problems. By learning them, you're learning the thought process of how they came to be, and you'll be able to apply those thought processes to your own problems. Consider it like learning algebra by doing exercises. You can read theory all you like, but unless you apply it it just won't click.
http://philip.greenspun.com/seia/
http://www.amazon.com/Software-Architecture-Primer-John-Reek...
http://martinfowler.com/design.html is also good.
"Architects look at thousands of buildings during their training, and study critiques of those buildings written by masters. In contrast, most software developers only ever get to know a handful of large programs well—usually programs they wrote themselves—and never study the great programs of history. As a result, they repeat one another's mistakes rather than building on one another's successes.
"Our goal is to change that. In these two books, the authors of four dozen open source applications explain how their software is structured, and why. What are each program's major components? How do they interact? And what did their builders learn during their development? In answering these questions, the contributors to these books provide unique insights into how they think.
"If you are a junior developer, and want to learn how your more experienced colleagues think, these books are the place to start. If you are an intermediate or senior developer, and want to see how your peers have solved hard design problems, these books can help you too."
edit: a major advantage to aosa is that all the source is available.
I was going to reply about experience (and code review) being the best teacher, but this is better. Critics from your peers are good, but critics from the experts are even better.
But I'd say in my oh 10-15 years of programming I've learned that the way to really learn the importance of a pattern is by deriving why it is really needed. You can't really make shortcuts there.
The difference with software is that it's soft. We're not actually creating buildings. Yes we ship products so it's related. But it's more on the spectrum towards disciplined writing or math than architecture in the traditional sense.
It's actually more to your advantage to learn via practicing with the minimum tools for the job.
Anyway, still important to learn others' patterns. But it's less meaningful in terms of how it fits in your mind -- in particular, knowing when to apply them in the future -- than deriving them on your own.
And there's less risk in learning via mistakes unlike with building physical things. Software just has faster iterations.
I used to work for a top5 consulting engineer - architects some times just do the very high level design the consulting engineers actually turn this into a practicable design which is then built by the contractors and the navvy's.
And I am sure that my dept boss Dr Shair (one of the pioneers of the cable stay bridge design) would consider him self an engineer and not an architect
- Designing software in minute detail beforehand is generally neither necessary nor possible.
- It is impossible for someone to be a capable software "architect" unless they are an experienced software "builder".
- It is impossible for someone to be a capable software "builder" unless they are also capable of designing it.
- Software can be reused once built, and this matters while building it.
- Software is rarely "finished", but continues to grow over time.
- Software can be used by an unbounded number of users at the same time.
- Software can be reproduced infinitely.
- Software can be forked and incrementally changed.
- Anyone can design and build their own software, given some talent and a computer.
- The business motivations for software development are fluid, and often change rapidly over the course of a single project.
I could go on and on. As with many (most?) metaphors, it offers some superficial familiarity, but is ultimately harmful to your understanding.
Software is software. Understand it by reading about software and writing software.
edit: to be more specific about the top three; the first rests on the word "minute" which can be as large or small as you want, depending on what you're trying to prove. The second may be true now, but that's largely because we lack a specific language of high-level software abstraction, so the only way to learn it is to build a lot of things (the general point of the original passage, btw.) The third is just wrong - plenty of people are useful for building parts of software who would have no ability to design a large application. I suspect that those people are a majority of the industry.
In context here, "minute detail" is obviously a relative term comparing the requirements of software and architecture design.
You just made up the "specific language" thing. The reason we don't get unicorns to write software for us is they don't exist either.
There are many incompetent software devs out there, but I don't see how anyone can possibly build any amount of software _well_ without having an appreciation of how to design it. This is why I used the word "capable".
The thing is, even if some of these things were similar to architecture, it would be by accident. They are, on the surface, totally different fields. On a deeper level, they're still totally different. The onus is on you to show the linkage, if you believe it to be applicable.
+ Free Book "The Architecture of Open Source Applications" : http://hackershelf.com/book/54/the-architecture-of-open-sour...
* Large-Scale C++ Software Design
* The Practice of Programming
* Code Complete
These are a good start but in the end it's experience. It takes a lot of practice to build a good intuition when making software design decisions.
http://gameprogrammingpatterns.com/
The C2 wiki has interesting entries on software patterns and related ideas: http://c2.com/cgi/wiki?WikiPagesAboutWhatArePatterns
Stanford CS 240 has a great list of papers on system design: http://web.stanford.edu/class/cs240/
One paper that's no longer on that list, but I remember from when I took the class: End-to-end arguments in system design http://web.mit.edu/Saltzer/www/publications/endtoend/endtoen...
Read the AOSA book, read Design Patterns, and read something like Domain-Driven Design. Realize that no-architecture is frequently chosen due to ignorance ('pragmatism'). Know SOLID back and forth. Understand why people strive to isolate things when doing TDD. Recognize and know how to decrease coupling, and what the costs of that are. You can practice these concepts every day, and you should, if only to develop the necessary element of taste.
You'll go through a phase where you adhere to these ideas religiously, then eventually assimilate the knowledge so that it becomes almost unconscious.
Best of luck.
I think that's a surprising characterization; it doesn't jive with my own perception (as subjective as that may be). I think the past decade has (rightly or wrongly) seen a reaction in our industry against enterprise-style "architecture astronauts" but that doesn't equate to being "anti-design". (I'm thinking of the emergence of Rails-like frameworks vs. the JEE stack, and then the emergence of micro-frameworks like Flask/Express/Sinatra after that.) Simple/reduced structure is a valid design choice (of course there are trade-offs that come with that, and of course they aren't always appropriate, but I don't think that means it's popular to be "anti-design").
I agree fully. I feel that the shift has come about because there has been an unconscious recognition that restricting your tools to one particular problem domain is a valid constraint to make on a design. Enterprise scale, "architecture astronauts" and the like were trying to build tools with which you could build anything, for anyone. And it's clear from the results that this is just a vector for complexity to creep in. Focused tools, from libs to frameworks unashamedly limit themselves to one particular problem domain, or at least used to.
I've certainly noticed a shift in that these hard learnt lessons are being ignored in order to add features to frameworks etc; which increase their applicable scope, with the increase in complexity that arises. The primary driver of which appears to be the younger generations desire to use the same tools for different purposes rather than learn a new set of applicable tools for a different problem domain. A good example is the shift from v1 Zend Framework to Zend Framework 2 which just appears to be so heavily "Java/Enterprise" influenced I simply refuse to use it; its far far too complex a framework for what is essentially a framework for building web sites. Building web sites is simple. These modern frameworks make it massively more complex than it needs to be.
Software architecture is focused on giving advice about things that are peripheral to programming, so by nature they are not lively reading if you not steeped in it. And it is opinions. It's all arbitrary, and it changes with the latest fad. Not least, it is patronizing. It's like getting a lecture on proper handwriting or the correct pronunciation of words, and being grimly informed that half of your letters are written incorrectly, and this is unacceptable and must change.
Frankly, HN would have to change culture entirely to cover design advice, and I would stop reading.
The kicker about software architecture is that it gives bad advice about writing software. Now, it's true, the individual bits of advice are all over the map. They don't all agree. But the premise was to prevent bad code from getting written by syntax changes and style conventions, hopefully stopping bad practices in their tracks and encouraging good practices in their place. I think the premise failed -- bad code hasn't stopped -- but the experiment continues, and the net sum of all the advice and conventions has been to encourage more complexity and more code, while simplicity is what yields good software.
And no more.
Abstractions are also useful to separate the arrangement of work from the actual work itself. These boundaries are useful for dividing up work among colleagues.
Even books and articles on complex subjects are fairly light on structure.
In a piece of software, there are many schemes by which you might organize the source code:
- Organize by author of code
- Organize by chronological order of appearance (e.g. what version or patch # this appeared in)
- ... by order of execution (of course, this is only a partial order)
- ... by library
- ... by importance
- ... by some arbitrary property of the code (by algorithm, by what data it touches, etc.)
Each one makes sense in a different context, and you can't have all at once. But you don't want to double-down on one choice at the expense of other, equally-valid choices with excess structure.
In particular, the papers covered topics of software design from failures (like Therac-25) to examples of successfully designed large scale systems (like Map Reduce). I'd highly recommend reading through the papers, most of which can be found via Google search.
As for FP languages for a starter, I'd recommend any of Erlang/Elixir, Clojure or Scala (only for its FP features and great learning resources)
1. Its easier to define your own abstractions in libraries. Even advanced stuff like async programming can often be put on a library.
2. Languages like Haskell and OCaml have really cool type systems. Its a killer feature, IMO.
I wouldn't put that much emphasis on the purity aspect. In Haskell the reason for the purity to allow for lazyness and similar languages (Ocaml, F#, etc) are perfectly OK with strict evaluation and mutation.
That said, for mobile apps having a language that operates well with the rest of the ecosystem is probably a bigger concern. Its easier to choose your own language when you have more control over the running environment.
You can also work through the "Learn you some Erlang" exercises online.
The thing about Erlang/OTP isn't just that it's a functional language, but that--for a beginner--it's very small and practical. I describe it as "a functional language written by people whose jobs depended on it".
OTP (effectively, production runtime library and development patterns) is built on really sound engineering principles, and you can learn a lot from the explanation of why you build things a particular way while reading the pirate book.
EDIT:
The main reason that the functional aspects of Erlang are so cool when contrasted with Haskell or Scala or whatever is that they are so deeply tied into how the language functions in a production setting. It's not a matter of "my, how clever", but more of "oh, that makes implementing this reilably a shit-ton simpler".
As an example: actors in Erlang (processes there) are implemented as side-effect free functions which accept messages and manually pass state around. As a consequence, we get code hot-loading for free--you can simply swap out the update functions and run a helper to update the actor state before resuming it's run cycle using the new update function.
That's wicked cool, and is something that just falls out from the pragmatic language decisions made earlier in its history.
The difference is subtle but important. You need to master imperative programming for the exposure to functional to produce a evolution. You can probably get the same effect in the reverse path (functional first, imperative next) but it's quite uncommon as a learning path.
The same reasoning leads me to advise exposure to logic languages (prolog and the like), to compiler design (a complex state machine) and to a complete algebraic abstraction (relational algebra is probably the best candidate). Each will bring into your toolbox an important top level tool, no matter which paradigm you use at any moment in your life.
http://www.amazon.com/Beginning-Java-Objects-Concepts-Editio...
http://www.amazon.com/Program-Development-Java-Specification...
A wiki with some nice explanations of design principles.
Books can help to a certain degree, but you have to make a conscious effort to practice what you're reading. The best thing to do, assuming you can't get the experience on the job, would be to find a big project and understand and contribute.
Note that a lot (most?) comp sci programs don't necessarily address design principles. If you're in a software engineering track they should, but my curriculum was mostly algorithms, data structures, operating systems, and programming languages. My senior year I had one class where design and project management was really a topic.
So read books, build things and read code.
I like to dig into open source libraries/frameworks/tools that I use in my projects and try and understand their design. It would be great if the documentation people write focused a little on that, instead of the more common "this is how you install" and "this is how you use" (and sometimes "this is how you build it") - but writing docs is hard/tiring and so that knowledge gets lost.
You won't be able to infer this stuff from studying software that works well. There are specific words they want to hear. Functional programming is probably not going to help at those interviews. FP isn't used at big corporations.
Software is the same way. Studying SOLID principles, application boundaries, encapsulation, etc. only makes sense within the context of a working application. Not only that, but making sense of design decisions is not as easy as "seeing the result" like you can in a drawing. With software design, you have to take a step further and continuously ask questions:
I'm an instructor at a coding school[1], and one of our biggest goals is to inculcate software design principles into our students. Frameworks are useful, but if you never learn proper software design, you'll always be stuck as a framework developer (e.g. Rails/Angular developer), and never become a software developer (i.e. frameworks and languages are just details).[1] http://www.makersquare.com/
[1] http://www.saasbook.info/