Ask HN: Is Knuth's TAOCP worth the time and effort?
I'm starting the first volume.
Right at the start, Knuth introduces Induction Mathematical Proof, and more specifically tries to show that we can describe it as an `algorithm mathematical proof`.
I showed it to a friend who is quite good at math, and he told me the book may be trying too hard especially in the examples variety, and how it might not be needed for comprehension's sake.
Would you still recommend this book, and if yes, in what circumstances?
191 comments
[ 4.7 ms ] story [ 206 ms ] thread[1] https://wiki.c2.com/?MappersVsPackers
They do make a good doorstop though.
Edit: for instance, there is now a SIMD article on HN #1; how many people actually are really interested in that? Those should overlap the people who would enjoy these books.
Does TAOCP help me with that? Well, maybe, mostly no though. I still treasure the parts I've read (I freely admit not having read all of it, but some parts and done some of the exercises) even if from some strict capitalist time = money point of view they probably wouldn't have been worth it.
Generally people that continue reading through TAOCP and invest the time and effort in doing so do get value from that or are foolish to keep going.
I read the early volumes thoroughly in the 80s and 90s and got a lot of insight into, for one example, methods for constraining error when doing chained numerical calculations.
His approach alone was inspiring at the time even if specific chapters weren't of any immediate interest.
They're not especially useful for general journeyman programmers and they're but one star in the firmament for people aiming higher .. but they are worth the effort if the material is of interest.
When I read TAOCP it was right after university at the culmination of an 8 year process where I had self-studied a literal heap of graduate level texts and papers. I had been wrongly advised that TAOCP would be better saved as a capstone, where I could understand better its depths.
It had no depth, and literally everything in it was better and more thoroughly explained by other works I had already engaged with.
Oh, I did learn one very valuable thing: not to blindly trust the advice of others. I wish I had learned that earlier, in which case I could have thrown it on the DNF pile and gotten that month of my life back.
I just learned what I needed for that from sources other than Knuth, and those sources did a better and more thorough job explaining it.
There was a lot of useless knowledge that wasted time to cover, like how to optimize for his totally made up machine language.
But it's also a weirdly esoteric, crusty machine language. If it was a Turing tape machine, or an idealized stack machine, that I'd understand. Those are invented architectures that are amenable to analysis and in which the analysis (e.g. maximum stack size, maximum tape use) reflects fundamental properties of the algorithm.
But MIX is just this weird thing that neither resembles CPUs you are likely to use, nor is it particularly useful for analysis.
Of course decimal computers and self-modifying code, and a lot of other things besides, went out of fashion, so he designed MMIX to replace MIX, during 1999–2011. The MMIX update to TAOCP started being put online in 1999 and was published in 2005 (Fascicle 1) and all the individual programs were finally put in book form by Martin Ruckert in 2015 (MMIX Supplement). The design of MMIX is close enough to (some/many) actual CPUs of the present/future — it's basically a RISC architecture like MIPS or RISC-V; in fact Knuth closely worked with Hennessy and Dick Sites in designing it. Yes it is “nicer” in some ways to write programs in than real CPUs (e.g. it has a whopping 256 registers, handles alignment automatically), but that's a fine choice for pedagogy IMO (avoid dealing with register spilling etc, while still being able to look into machine-level concerns like pipelining etc).
Maybe you read TAOCP at some point before 2005 (or missed the “don't bother learning MIX” part of the preface), in which case your complaint is valid (and of course it's not very convenient to read some parts from a different book), but IMO the machine language being a weird one has not been a problem for over two decades.
(1) TAOCP shouldn't have used an assembly language at all,
or
(2) the specific assembly language used in TAOCP (MIX in the 1960s, MMIX since the 1990s) has a bad design.
In the comment I replied to, it seemed the argument was (2) rather than (1), so that's what I replied to.
As for (1), it has been discussed many times before, but apart from the reasons he himself gives (https://cs.stanford.edu/~knuth/mmix.html#:~:text=Why%20have%... and the preface to TAOCP and to the MMIX supplement — “Indeed, Tony Hoare once told me that I should never even think of condensing these books by removing the machine-language parts, because of their educational value”), in short:
• [quantitative] Most of the book is in English pseudocode, dropping down to assembly language only on occasions when the concreteness is relevant. (See also https://news.ycombinator.com/item?id=38444482 above.)
• [historical] The job he was hired by Addison-Wesley to do (you've written a few compilers in machine language — e.g. https://ed-thelen.org/comp-hist/B5000-AlgolRWaychoff.html#7 — write a book for us showing others how to do the same), meant a book for working programmers writing real programs (most of whom would be writing in languages for various different machines): no one in the 1960s was writing production compilers in FORTRAN or ALGOL (or COBOL or BASIC!); I may be wrong but I imagine very few compilers were written in “high-level” languages (PL/I notwithstanding) until C took off in the latter half of the 1980s, and Turbo Pascal was written in assembly even in the 1990s. A language that is similar to what serious readers would use, but abstracts away some of their finicky details and is a bit nicer for teaching in, is exactly what MIX and MMIX are. (Yes MIX was getting seriously out of date by the 1990s; that's valid criticism and why he spent so many years replacing it with MMIX.)
• [cultural] Among people seriously interested in computing/programming/algorithms, on the one hand there are the theorists, most of mainstream academia, who are happy thinking abstractly and asymptotically, not bothering too much with constant factors or machine-level considerations. Knuth, despite being part of academia and having to a great extent spawned this field, is at heart very much on the other side (here not counting the even large number of programmers who simply do it as a job): of hackers with “mechanical sympathy” who simply care about accomplishing a bit more with computers, getting their programs to run faster, and so on. In fact, this subculture is somewhat underground (occasionally surfacing like HAKMEM) and Knuth may be one of its few “respected” representatives in academia. For example: his presentation of circular lists of course includes the XOR trick to save one field (https://en.wikipedia.org/wiki/XOR_linked_list) from the very first edition (Exercise 2.2.4–18). “Packing” tries to put the children of one nodes among the “holes” of others? Of course (Exercise 6.3–4, and how his student Liang and him did hyphenation in TeX, and what he used for the word count program) (unlike CLRS which barely even mentions tries https://news.ycombinator.com/item?id=21924265). He has written (still being published) multiple volumes devoted to “backtracking” programs that many theorists are not interested in be...
There was an interesting time in the late 70's when many vendors had their own high-level systems software development programming languages: CDC's Cybil, Sperry-UNIVAC's PLUS, &c., and these were often used for compiler development as a replacement for assembly language. At Cray, the CFT compiler was in assembly language, and CFT77 was in Pascal.
The main problem is that algorithm examples are specified in a fake machine code. It’s not that useful when things can just be Googled now.
People just seem to elevate Knuth to divine status and treat his books like the Bible. Why, idk.
https://norvig.com/21-days.html
1) Time is important, there's ageism in tech and you need to plan your career wisely - no one is going to do it for you. The best way to learn is to get hired ASAP and get professional experience.
2) College degree is very important. In this market, people are struggling to get jobs with experience & degrees. You just have much less likelier chance of success without a degree. This appears all the time, even actual prodigies like George Hotz are overcompensating and always appear eager to prove that they have the fundamentals down like a CS grad.
3) Moving fast into building things which generate hype is the best course of action for young people
I know a friend that got to interview stage with a FAANG they told him the position was for university degree holders. This is an old essay and I'd always take with a grain of salt what people say versus how the reality is. It's very easy to get wrong advice that's no longer applicable.
I know someone who did something very similar (but in math) and jumped into a grad math program at age 17.
Reading TAOCP is the least efficient way to learn about algorithms. Just because something is harder doesn't mean it's better. Can't even put TAOCP in resume without appearing cringe.
>I know someone who did something very similar (but in math) and jumped into a grad math program at age 17.
Can you share more about this, even IMO medalists go to undergrad first.
"Learning the algorithms" isn't a binary checkbox. It's a gradient of thinking and math skills, ranging up to a researcher in the field. TAOCP is the only book I know if that will give you that depth.
Learning those skills is not for everyone, but can be extremely valuable, and open a lot more doors than graduating with a class of 10,000 other CS students.
> Can you share more about this, even IMO medalists go to undergrad first.
He was an "undergrad" whose first math class was graduate real analysis.
Which would you think they'd choose?
>He was an "undergrad" whose first math class was graduate real analysis.
This is unlikely, he'd still have to pass undergrad Math exams. I'd wager there are plenty of those for a 4 year degree.
> but can be extremely valuable, and open a lot more doors than graduating with a class of 10,000 other CS students.
You need to convince the hiring manager that the skills are extremely valuable. No one is going to take the word for it. Deep theoretical CS doesn't always translate to industry success. The flaw in your reasoning is that you'd try to impress some hardcore CS guy from super-duper company with TAOCP. But they already get 10s of thousands of applications. It won't even get the resume read without a BsC. All the companies which talk like broken records that they don't care about degrees, actually do care about them a *lot* and in your first intro call they'd ask about it if you even get that far.
Keep in mind during interview you have to implement the algorithm fast in a common language Java, C++ or Python. Just being theoretical about it isn't enough.
I read through volume 1 several times. I then skimmed through the arithmetics part of volume 2 and read only the section about random numbers, which is very fascinating but you will be able to live without unless you are doing simulations or cryptography/cryptoanalysis.
Whether you will need volumes 4a, b, c depends on whether your work has much to do with combinatorics. I'm a professor and I did not need most of it, but had a good laugh about the hidden gems and jokes like the exercise about Knuths personal pipe organ.
As a software engineer, you may occasionally consult volumes 3 and 1 (searching, sorting, data structures).
[Side node: If I was Knuth, I don't think I would have created three subvolumes instead of the planned single volume 4. While it is true that the material exploded, that will always be the case, and is also true for the other chapters. In his shoes, I would have worked on succession planning and trying to map out the remaining volumes so that the book can be completed one day. Under Knuths approach (esp. going back to update the existing volumes repeatedly on the way) TAOCP will unlikely ever be completed, which would be a shame.]
It does feel that sometimes he goes deeper than what you'd expect from a 7 volume book.
As with any complicated topic, you need to give yourself some sleep to be able to truly intuit it.
One day I was working on a problem and I realized that I was storing a lot of 0's in my matrix and I remembered that I read about sparse matrices in TAOCP (this was before you could just go to numpy and make a sparse matrix). So I went back to that chapter and used that to solve my problem.
That's happened to me many times in my career now, I remember that I read the general idea in TAOCP and then when I realize I need it, I go back and learn it for real.
If you want to be someone who understands computation (using computers to solve real problem), then it's probably good to have "read" TAOCP just to see what kinds of problems other people have encountered and know that there is at least one good solution to them. Then if you ever need to solve that problem, you know where to look for a start.
A little more philosophical: a culture needs a common core set of ideas that everyone can reference. When working together as a group, having a set of common ideas is a great shortcut in communicating. I'm not religious, but I see the bible as a historical way that this worked for the western cultures - there was a set of "givens" that everyone could hang their ideas off of. Every field of knowledge needs this common understanding. I view TAOCP as this for the field of computing. There are better ways to solve many of the problems now, or common libraries that provide the solutions, but having the problems named and discussed gives everyone a good starting place to talk.
tl;dr - yes, TAOCP is good, read it to learn the concepts and ideas that people smarter than you solved years ago.
https://en.wikipedia.org/wiki/Introduction_to_Algorithms
> he told me the book may be trying too hard
Knuth isn't trying to write a novel or something to learn from, rather a concise, comprehensive attempt to cover the field of Computer Science. It's a repository of knowledge. He marks some exercises at 50 which, from memory, means that if you can solve it fully you are in the running for a Turing award. One of the Python core developers was once asked what the developers did in their sparetime: We read Knuth so you don't have to [0].
[0] https://www.norvig.com/quotations.html
You'll either be interested, or return it and not feel a desire to get it again. And maybe years later you'll want to dive in.
What books if you liked will you like TAOCP
If you hated skip TAOCP.
Me:
Loved SICP
totally wore out Godel, Escher, Bach
Somewhat angry at A New Kind of Science
For example i never thought i would use the basic compiler knowledge i gained during my bachelor. But at one of my first consulting gigs it did helped knowing about abstract syntax tree, where you can add and remove nodes. This knowledge helped me create a highly automated refactoring tool for an old but massive enterprise code base.
This was the only situation in my whole career for now where i needed this knowledge, the rest has all been about creating webpages and rest apis :p
Knuths Organ Works https://www.youtube.com/watch?v=OBO613Q8hAw
His primary interest lies in the patterns, structures, and the mathematical beauty of the scriptures, rather than in theological or moral interpretations, which are still there and deeply influence and corrupt society, despite his ignoring them.
And that shows that expertise and rigor in one field doesn't necessarily translate into expertise and rigor in another field.
Knuth hasn't written much about theology, but he has written some. How much of it have you read? Reading it myself, I don't get the impression he's at all uninterested in the theological or moral implications of scripture. (But I do get the impression he's used the tools of his trade in his public presentations on the subject.)
TAOCP is extremely rigorous and detailed, the discussion of any topic there will cover every detail. For practical software engineering, it's not useful, you don't deal need that level of theoretical understanding as a software engineer (if you're one of the few who need it, you'd know you are). And you need a very solid mathematical background to actually understand it.
If you want to understand the theory of CS algorithms, I think the CLRS "Introduction to Algorithms" is an excellent textbook. It covers about everything most people are likely to encounter, has comprehensive mathematical analysis but is much easier to follow than TAOCP. For an understanding of the mathematics involved, Concrete Mathematics is a good book, it teaches the mathematical concepts and tools that are the foundation of other, more rigorous books.
https://theory.stanford.edu/~tim/won1samplefinal.pdf
In 1956, Knuth graduated high school and entered college, where he encountered a computer for the first time (the IBM 650, to which the series of books is dedicated). He took to programming like a fish to water, and by the time he finished college in 1960, he was a legendary programmer, single-handedly writing several compilers on par with or better than professionals (and making good money too). In 1962 when he was a graduate student (and also, on the side, a consultant to Burroughs Corporation), the publisher Addison Wesley approached him with a proposal to write a book about writing compilers (given his reputation), as these techniques were not well-known. He thought about it and decided that the scope ought to be broader: programming techniques were themselves not well-known, so he would write about everything: “the art of computer programming”.
This was a time when programming a computer meant writing in that computer's machine code (or in an assembly language for that machine) — and some of those computers were little more than simple calculators with branches and load/store instructions. The techniques he would have to explain were things like functions/subroutines (a reusable block of assembly code, with some calling conventions), data structures like lists and tries, how to do arithmetic (multiplying integers and floating-point numbers and polynomials), etc. He wrote up a 12-chapter outline (culminating in "compiler techniques" in the final chapter), and wrote a draft against it. When it was realized the draft was too long, the plan became to publish it in 7 volumes.
He had started the work with the idea that he would just be a “journalist” documenting the tricks and techniques of other programmers without any special angle of his own, but unavoidably he came up with his own angle (the analysis of algorithms) — he suggested to the publishers to rename the book to “the analysis of algorithms”, but they said it wouldn't sell so ACP (now abbreviated TAOCP) it remained.
He polished up and published the first three volumes in 1968, 1969, and 1973, and his work was so exhaustive and thorough that he basically created the (sub)field. For example, he won a Turing Award in 1974 (for writing a textbook, in his free time, separate from his research job!). He has been continually polishing these books (e.g. Vols 1 and 2 are in their third edition that came out in 1997, and already nearly the 50th different printing of each), offering rewards for errors and suggestions, and Volume 4A came out in 2011 and Volume 4B in 2023 (late 2022 actually).
Now: what is in these books? You can look at the chapter outlines here: https://en.wikipedia.org/w/index.php?title=The_Art_of_Comput... — the topics are low-level (he is interested in practical algorithms that one could conceivably want to write in machine code and actually run, to get answers) but covered in amazing detail. For example, you may think that there's nothing more to say about the idea of “sequential search” than “look through an array till you find the element”, but he has 10 pages of careful study of it, followed by 6 pages of exercises and solutions in small print. Then follow even more pages devoted to binary search. And so on.
(The new volumes on combinatorial algorithms are also like that: I thought I'd written lots of backtracking programs for programming contests and whatnot, and “knew” backtracking, but Knuth exhausted everything I knew in under a page, and followed it with dozens and dozens of pages.)
If you are a certain sort of person, you will enjoy this a lot. Every page is full of lots of clever and deep ideas: Knuth has basically taken the entire published literature in computer science on each topic he...
The elementary vernacular foundations of modern mathematics is (more or less) naive set theory; the starting tools of serious foundational work (as arcane as it is even within maths as a whole) are logic and more rigorous set theory, perhaps with some computability mixed in; the main tool of a mathematician who wants to reason in great generality is category theory (with some handwaving in the direction of the previous point about universe hierarchies and whatnot). There are some signs of convergence between these (and of mainstream mathematicians starting to once again take foundations seriously), but at the basic level those are what you’ll be dealing with.
None of them existed in their current form at the time PM was written, even logic (no Kripke semantics! no forcing! and no Gödel of course). Some did not yet exist at all. Some changed quite drastically in direct response to PM. And of course PM is the origin of (embryonic) type theory, which is the inspiration of the unified approach I referenced above. So as a historically important text, sure, if that’s what you want, but as a gateway to understanding more interesting maths it’d be terribly inefficient.
In that respect TAoCP was uniquely lucky. It was also a self-obsoleting book: it ceased to be comprehensive months after it was published, exactly because it told you everything there was to know about algorithms to date. Yet none of the stuff that’s in it is itself obsolete, there’s just immesurably more stuff now. PM, on the other hand, was attempt at “rationalization” in the 19th-century sense, and mostly a failed one except for serving as fertilizer for all of the later ones.
If you are interested in the underlying goal of Principia Mathematica, I urge you to check out the Metamath Proof Explorer (MPE): https://us.metamath.org/mpeuni/mmset.html
By itself, Metamath doesn't have built-in axioms. MPE uses Metamath to first state a small set of widely-accepted axioms, namely classical logic and ZFC set theory, and then proves a tremendous amount of things, building up to a lot of mathematics from careful formal proofs that rigorously prove every step.
Some things cannot be proven, but that doesn't mean that proof is dead.
https://us.metamath.org/mpeuni/mmset.html
Don’t get me wrong, PM is marvelous and there’s no gainsaying its enormous historical impact.
If your characterization of Metamath is correct, I don’t think that’s in the spirit of PM at all. One of the major problems PM had was the rejection of (what later became) the Axiom of Choice in favor of Russell’s convoluted theory of types. Indeed, that set theory (ZFC) is needed to get the rest of the way R&W were trying to go is one of the signal failures of the logicist program behind PM.
However, if you think the main point of Principia Mathematica was the very specific set of axioms that they chose, then that's different. The PM authors chose to use a "ramified" theory of types, which is complex. It does have sets, it's just not ZFC sets. Few like its complexities. Later on Quine found a simplification of their approach and explained it in "New Foundations for Mathematical Logic". There's a Metamath database for that "New Foundations" axiom set as well (it's not as popular, but it certainly exists): https://us.metamath.org/index.html
More Metamath databases are listed here, along with some other info: https://us.metamath.org/index.html
There are pros and cons for both systems.
The biggest problem NF has is, as usual, a social one: there just ain’t a lot of people working on, or interested in working on, NF compared to other set theories.
https://www.principiarewrite.com/
But then again, as a result of all this work from him, pulling together and categorizing and cataloguing and analyzing all the disparate ad-hoc “tricks” that programmers had come up with, Knuth basically put (this area of) computer science on good foundations and made it a respectable academic discipline. So in that sense he successfully executed a Principia-like project.
To put it another way, the computer programming/algorithms world is generally divided into hackers who derive satisfaction from elegant ways of bending computers to their will, and theorists who are happy to talk asymptotics, treat their problems abstractly as a branch of mathematics, and often never write a single program. What sets Knuth apart is that he straddles both words: is firmly a hacker at heart, but is able to bring prodigious theoretical skill to bear on the area.
In 2015, I decided I was no longer poor and should buy the box-set as a treat. I set aside time to read a chapter in the first week after it arrived, but realised that it would take years to read and actually understand the material in that one book, let alone the other two. I decided I'd just dig in from time to time, but found that I never actually did, because in all honestly I'm less interested in abstract algorithms now, and prefer to spend free time doing other things in life that don't involve computers.
So, my copy is still essentially untouched after almost a decade.
Also, it's really heavy (I'd say at least over 1kg), so by the time I sold it at less than the current price because it's no longer new (also mine is 1-4A, the current edition is 1-4B) and paid postage to package and ship it somewhere, it's probably not really worth the effort.
You should imagine that you are forced to sell it every day. Eventually you'll either sell it (or donate it (your local library might appreciate it)) or start reading it.
Semi-serious question: Which of the following do you mean:
A. Every day, you should imagine that you are forced to sell it; or
B. You should imagine that you will, every day, be forced to sell it.
(I teach law students to spot and fix ambiguities in draft contracts; this would be a nice practice example :-) )
OP imagines that there is a point in the future where they read the book without commiting to it. Selling/donating it would put an end to that possibility and force OP to imagine a future where they cannot read it, and hopefully helps them assessing one va the other, and prioritizing accordingly.
https://about.usps.com/notices/not121/not121_tech.htm
I've used it to buy/sell large tomes like this and it is really cheap ship things this way. It's not the speediest of delivery methods.
For example if you are interested in large integer arithmetic the 2nd edition of Volume 2 has some exercises that develop some methods that, if I recall correctly, I found more suited to my needs than the methods in the text when I was working on a large integer library.
The 3rd edition dropped those. I've long forgotten most of what I learned when working on my large integer library, but if I had to learn that subject again I'd probably actually start with the 2nd edition, and then check the 3rd edition to add to that.
I'd love to own them and be the kind of person to read them, but I know I would also struggle to give them the proper attention (not with the hundred other books to read on my shelf).
There is also a contrast between the two: they lie on opposite ends of a spectrum from shallow to deep, of things that make you really think / put in honest effort, and also give a deeper enjoyment.
As for reading other (good) books: like anything else, it's ultimately a question of what gives you most joy, I guess. Learning (say) real analysis from Rudin would also be deep and make you work, but Knuth has more jokes.
I don't get what you find strange about comparing reading TAOCP vs reading social media. Your time is limited, and each activity that you choose bears an opportunity cost. Of course it is an interesting question to ask whether you should read another book instead, but I bet that most people waste a lot more time on social media than reading a book that is not TAOCP, so svat's comparison does make a lot of sense in my opinion.
I read social media when I'm too tired to really engage with anything in a serious way.
I (try to) read TAOCP sometimes when I have a surplus of energy and ambition.
Admittedly, sometimes ambition comes along a little bit after I force myself to do something challenging, but TAOCP is so difficult for me that I really have to be feeling it to make any headway.
Any recommendations/suggestions?
1) Foundations of Multithreaded, Parallel, and Distributed Programming by Gregory R. Andrews - This is one of my favourites even though old. You get to learn/compare the different paradigms in one single book.
2) Parallel Programming: Concepts and Practice by Bertil Schmidt et al. - This is a more recent book with good explanations/coverage including CUDA.
3) The Art of Multiprocessor Programming by Maurice Herlihy et al. - Well known classic though with a main focus on shared-memory architectures.
Along with the above you also need more detailed language/library specific implementation books;
a) C++ Concurrency in Action by Anthony Williams.
b) Programming with POSIX Threads by David Butenhof.
c) UNIX Systems Programming: Communication, Concurrency and Threads by Kay Robbins and Steve Robbins.
Finally; for studying concurrency at the OS level where software meets hardware, i have found nothing better than;
i) Unix Systems for Modern Architectures: Symmetric Multiprocessing and Caching for Kernel Programmers by Curt Schimmel.
https://silezukuk.tumblr.com/image/616657913
---
PS- not to be "that guy" but I do think that a future edition should port the MMIX to RISC-V because it'll be actually runnable and I'd bet money that RISC-V is the educational standard going forward for the next 50 years.
1. Most of the time you are not implementing foundational algorithms like sorting or SAT solving. You use mature implementation in libraries.
2. If you are in fact implementing foundational algorithms, then the existing volumes of Knuth cover only a very limited set of problem areas.
3. If you are implementing something in an area covered by Knuth, the books are worth looking into as a reference, but often writing performant and robust code requires considerations not in Knuth. This is because Knuth works with an idealised machine ignoring things like parallelism, memory hierarchy etc. and understandably does not get into software design issues.
They do also go much wider than just combinatorial math.
I read TAOCP out of curiosity long time ago. Has it been useful for me? Very much. But the usefulness came from abstract connections that clicked later in my life almost randomly. I am not a common individual.
If you expect a direct practical application of what you read, I believe you better choose other material that suits your needs better.
But in any case the algorithms are taught in English not MIX; the MIX programs are only a very tiny part of the books (https://news.ycombinator.com/item?id=14520230 — the 3000+ pages of Vols 1–4A have only 90 assembly programs), used only when the low-level details are actually relevant. So they can be skipped if one is not interested in them.
Your comments put TAOCP in a historical context. Yes, for the time, TAOCP was a huge achievement and it is a magnificient work. History is interesting, but the question here is about today, and I don't think it's the best way to learn the stuff today. (Again, just my opinion, there's no right and no wrong here.)
> source code is the primary source of understanding a concept, and English text is just an accompanying explanation, not the other way round
note that most algorithms in TAOCP are only described in English (pseudocode), no (MIX) source code. (So if you want source code as primary you simply won't find it… but then again, a typical algorithm textbook like CLRS also would have only pseudocode).
The TAOCP pseudocode is also in a unique style that's not similar to modern programming languages — e.g. here's a note that Knuth wrote just earlier this year: https://cs.stanford.edu/~knuth/papers/cvm-note.pdf explaining (and generalizing/correcting) in his own style, the algorithm from a recent paper (https://arxiv.org/pdf/2301.10191v1.pdf): compare how the algorithm is described in Knuth (first page) vs the paper (second page): Knuth is simultaneously more high-level and more low-level (and also uses “goto” rather than “structured programming” for-loops, about which heretical preference of his I can write a long essay some other time :D).
It is possible to have different opinions about which one is the best way to learn the stuff today, but clearly Knuth rewrote it in his style because he thought it was better. ;)
I think you can read at various levels though.
When I say I've read most of Volumes 1-3, I mean literally that, I read the text through. At that level it isn't too difficult to read, and you'll learn a lot like that.
However some bits I've studied in much greater detail, like for instance the multi-precision arithmetic algorithms which I had to implement. For that I read it multiple times, implemented the algorithms, did some of the exercises. That takes much much longer than just reading it through.
It is rather like reading a research paper which you do roughly like this, stopping after any step where you think it is no longer providing value to you.
To get the most out of it, you'll need to go to level 5, but you can get a great deal out of it at the earlier levels.So I've read most of TAOCP at level 3, half a volume at level 4 and only 50 pages at level 5.
If nothing else I've loaded the index in my head!
PS the examples are notorious in TAOCP. The easy ones will take a few minutes and the hard ones are unsolved research problems!
I haven't read the book, so he poses research questions as exercises to the reader...?
He's like a depth-first system guy... I suspect he only planned the 7 volumes because it was a requirement to start out wide.
"Develop computer programs for simplifying sums that involve binomial coefficients"
Which was solved in the book titled "A = B":
https://www2.math.upenn.edu/~wilf/AeqB.html
or
https://www.amazon.com/B-Marko-Petkovsek/dp/1568810636
For instance, I just opened randomly to the binary tree traversal material in Vol. I. It's very mathematic or formal, e.g. "T2. [P = Λ?] If P = Λ, go to step T4." and so on. To fish out some of the more interesting details, you have to look through the questions and then flip to the answers.
Meanwhile, https://en.wikipedia.org/wiki/Tree_traversal has the same information but somehow more clearly presented.
If I flip to other random pages and scan for 30 seconds, I can't find much that is actually interesting---particularly in the early volumes.
-----
The Art of Electronics is more my speed, as it's a collection of real-world applications, deep dives into each component type, quick comparison tables, and clear overview of a specific design's pros and cons. It's as much interesting to read through as it is useful as a reference.
I can flip to basically any random page of The Art of Electronics and find something eliciting, "Oh cool." I can't say the same for TAOCP.