Ask HN: Suggested software /design/ books?

I am not sure if a book can teach you how to design large complex programs. I certainly haven't run into one that does it effectively. Perhaps a good "Design Patterns" type book would be useful on the micro-scale, but on the macro scale I can't really think of a good book.

That said it doesn't mean you can't learn this skill! I think the best place to start is finishing a decently sized program. The first time I did this I was extremely unhappy with my design. However, each time it got easier and my designs started getting better.

Second, start reading some large code bases. These are often difficult to read, but they can teach some tricks and some methodology. This summer I read Tomcats source code with the aid of "How Tomcat Works: A Guide to Developing Your Own Java Servlet Container."[1] While not enjoyable (it was for my job), the experience was worth doing.

In the end, all programmers constantly struggle to make scalable, reusable, and maintainable systems. Few programmers achieve a soundly designed system but we can certainly strive to achieve perfection in our designs.

[1] by Budi Kurniawan; Paul Deck, ISBN: 978-0-9752128-0-6

I might as well get all of the big names out of the way.

* Design Patterns: Elements of Reusable Object-Oriented Software

* Refactoring: Improving the Design of Existing Code

* Clean Code: A Handbook of Agile Software Craftsmanship

* Code Complete: A Practical Handbook of Software Construction

And some others:

* Working Effectively with Legacy Code

* Domain-Driven Design: Tackling Complexity in the Heart of Software

There is a great book called, experience, and that you cannot buy from amazon.

You feel your code is clunky because of lack of design. And lack of design comes from the fact that we usually don't know what we want to design till we design it. This is where prototypes come in handy because prototypes shed the light on problems that we didn't foresee. Rule of thumb for prototype code is to throw it away once you are done.

What I am trying to say here is that you don't need a book to learn what you are asking for, you need experience.

I've read a lot of books and I've never found any that were good beyond the basics of a language. It's been much, much more instructive to do work on code bases that are already managing the complexity.

Find an open source project on the web that has the kind of complexity you want and start digging through the code. Better yet, start fixing bugs. It'll force you to wrap your head about how they did things. With enough of that, you'll even figure out why they did it, and that's as much as anyone else can teach you.

Try reading a non-programming design book too. I'm reading 'Notes on the Synthesis of Form' by Christopher Alexander at the moment, and it has some interesting stuff on tackling problems elegantly.

I find myself taking the first solution that appears to me, which is not great. I'm trying to adopt more of a sketching approach, writing a few bits of code to see how things would interact, and working out where the complexity would bite me in a design.

In his book "ANSI Common Lisp" PG gives an account of designing reusable software from the bottom up i.e. "changing the language to suit the problem. In Lisp, you don't just write your program down toward the language, you also build the language up toward your program." (as opposed to exclusively top down design) http://www.paulgraham.com/progbot.html

I'm a big fan of IEEE Software magazine. http://www.computer.org/portal/web/software/home

They have a lot of good authors who focus on different aspects of design and architecture.

I also enjoyed Implementation Patterns by Kent Beck. It is not so much on the architecture of the system, but focuses on the implementation of classes and routines.

* David Hanson: C Interfaces and Implementations

* David Hanson: A Retargetable C Compiler

* John Lakos: Large-Scale C++ Software Design

* VA: Beautiful {Code, Data, Testing}

* Design pattern books maybe

* Code Complete

* Fred Brooks: The Mythical Man Month

* Fred Brooks: Design of Design

* "Best of" books like C++ Pearls or More C++ Pearls

* Michael Abrash: Grapics Programming Black Book

* Looking at John Carmack code, eg. in my github repo at http://github.com/mtrencseni

* In the old days they had trade magazines, which unfortunately gave way to blogs. If you're interested in C/C++, there was The C/C++ User's Journal, if you ask them they'll just send you a free DVD with all the issues, and there was the C++ Report

There are many aspects to maintaining a good project:

* knowing your tools; if you use Makefiles/svn/make/gcc/Visual Studio, buy a book on it, write some helper scripts, etc.

* knowing a comfortable subset of your language; pick 2 (at most 3) languages and be good at those, really good at one

* know the idioms of your language and framework(s)

* physical layout: putting code and resources into folders, files, including, naming, etc. in a consistent manner; eg. this is covered in Large Scale C++

* consistent naming and commenting: your code should be pleasant to look at in terms of naming, declarations, length of blocks, tabulation, etc., eg. look at John Carmack code

* DRY, KISS, etc.

* logical design, distributing code and functionality across your program: that's the tough one. Personally I don't find DPs to be generally useful; instead:

* constant refactoring: your motto wrt. refactoring must be "no fear", and you have to set aside time for this. eg. I spend weeks refactoring

* when writing a program, don't start hacking away to get something out the door, think it over, get to know your subject, flesh out its details (eg. read up on it, how have others solved it), so you can appreciate its beauties and difficulties, and if possible

* copy your predecessor's successful ideas and designs

* build yourself a nice 'lib' that you reuse and grow across your projects, but be careful not to turn it into a monster =)

* trick: you can distribute some of these among team members, eg. see the chapter on the Surgical Team in Mythical Man Month, which I subscribe to

Having to assume a timeframe, I do not think you have actually learned the languages. It usually takes years to be really good with one. That is not to say you have learned nothing (exposure to many paradigms is good), but that it takes a lot of effort, may be confusing and that you would probably be better-served by concentrating on one or two and working on actually programming first. The dual nature of languages is that they are nontrivial (at the "mastery" level) and they can profoundly affect how you program but that at the same the specific one you end up using is kind of immaterial to programming which is to say composing logic into a functioning program.

All of that is a long-winded way of introducing the suggestion that you learn to program by programming. The second-best is understanding others' programs (select well-respected, progressively larger codebases). Books, suggestions of which I hope others will provide, may give you some good ideas and thought patterns. "Software design" is a vast field with tons of completely obsolete advice, and terrible amounts of overengineering. Perhaps something like an agile workflow book, "The Pragmatic Programmer" once you can construct programs.

Try to read good code like TeX, Plan9, Arc, Lua or Tcl.

Others have already listed good books. That's great, except I'd suggest spending time looking at data structures and algorithms books as well as software engineering and design books: a right data structure or algorithm can mean less code. However, I'll suggest something others only touched upon.

You already know more programming paradigms than most working programmers including those that have contributed to software you use daily. That's great and invaluable. Each of these paradigms told you something new about managing complexity (message passing in Squeak, reflection in Java, metalinguistic abstraction in Lisp). You can now apply them outside of those languages (that's what Design Patterns are about): multiple dispatch via Visitor Pattern in Java, message passing OO in Lisp[1], interfaces and implementations in C and so on.

[1] http://okmij.org/ftp/Scheme/oop-in-fp.txt

What you don't have is practice applying these lessons iteratively: put in effort, ship something (even with inadvertent mistakes), receive constructive criticism, find bugs and missing features, put in effort to improve design as needed to add features and fix bugs, ship an improved version and so on.

You're afraid of shipping something you're embarrassed of, or of failing to ship something that delivers on its promises. Every programmer feels that, it's perfectly natural and fine: its what motivates us to improve. Acknowledge this fear, but don't let it paralyze you.

There's a Russian proverb: eyes fear, hands do. Apply it.

I would agree with some of the commenters: you need experience and write more code using a single programming language of your choice.

But before you do so, here are some of my suggestions, take it with a grain of salt because it may not work for you.

1) Quit HNews.

You won't learn how to build software here. Most people rehash the same "technical" speak again and again about consumer application (web scaling, javascript, functional, anti-enterprise, etc).

2) Clear your mind

Go out. Walk in the park more. Don't think too much about programming languages. I suspect you're drawn to a distortion field of "learning programming language X will make you better at Y". Once you got that cloud away, start exploring what you wanted to do.

3) Pick a language and be good with it

Again, don't worry too much about the other programming languages unless you want to learn them for the fun of it. If your goal is to build something, stick with one.

Over two decades of programming, I've read most of the books listed in this thread. Some of them contain real pearls, while others are undeservedly popular. In my opinion, each language is suited to (and perhaps designed for) certain kinds of problems. There is no 'general' software design approach. Each language has developed its own idioms and patterns, which are best learned by tracing through production code of competent programmers who are solving problems that are somewhat similar to yours. Most books actually need to be taken with a grain of salt, as they often sound more authoritative and general than they really are.

Programming languages are useful tools, but their purpose is to allow designs to be expressed in an executable form and not to aid your decision process. The design pattern books (and a few others) abstract away the details of programs so that you can see the chassis and principles upon which it is built. They are sometimes helpful as you learn about systems structures. (For example, the Model-View-Control pattern is very common and useful, but it's in not always intuitive.)

Perhaps the most educational thing you can do is to choose a real project and write the user level documentation for it, and the use cases, the APIs and the unit tests: all before you begin to code.

Get other people to read your documentation and comment. If anyone can identify elements that are not needed, remove them. Don't worry much about implementation or optimization at this stage, worry about clarity and elegance of design.