I hear recommendations for this book all the time; I've cover-to-covered Strang several times and have digested maybe 20% of Axler (I just dip into random sections of LADR when something annoys me in a problem set or whatever), and I'm super curious why the aura on Lay's book is so positive.
Lay explains things very clearly and it's very good for going back to when I need to review something. Lay covers all the foundational linear algebra topics you would need for say, ML or graphics programming. I found that roughly the first half of LADR felt like a review of what Lay covered.
You won't get anything new from Lay's book if you've done Strang. It is a basic textbook and doesn't cover anything advanced. It's just a good textbook.
As someone who's watched all Strang's lectures and read most of Axler's book, I found Linear Algebra and Its Applications to give the most interesting treatment on the topic. What made it so special was its clear explanations and the use cases it introduced.
The applications not only provided the context to get a deeper understanding of the concepts, it also made the math relevant to the real world. By showing a range of problems, from the chemical stoichiometry to the algorithmic SVD, it inspires the reader to look for more problems that can be solved with the toolset being introduced.
if self teaching you need a book that holds your hand and doesnt try to put you through the usual trials of "lets see if you can figure this out yourself.. oh you cant? maybe repeat the subject next term", or major proofs left as exercises for the reader. get a friendly hand-holdy book. im reading linear algebra, theory, intuition, code by mike cohen atm (up to ch4) and so far between that and a chatbot for clarification/links, im re-learning the L.A. i should have learn't at uni but for whatever reason just did not understand back then. those courses move fast and if you get stuck somewhere its game over. i think L.A. is way harder than calculus as a beginner, but once things start to click and fall into place it starts to feel easier.
If you are SELF-teaching, you need to learn how to figure things out.
When you can’t, there’s plenty of sources. It’s not the 1990s. Your favorite LLM probably gives good explanations of linear algebra ideas even on fast mode. Gemini was surprisingly good for me.
The thing about linear algebra is that it isn't really a cohesive subject in the same way that calc is. This is why you get as many perspectives as you get commenters.
There are some central concepts, and which ones matter to you depends on what you're planning to do with your future. The key points are:
* Matrices/vectors as grids of numbers/computational tools.
* Matrices/vectors as positions and transformations of those positions.
* Matrices/vectors as more abstract geometric objects
* Matrices/vectors as algebraic objects.
There is no 'done right' imo. My sense is LADW is probably the best option for a motivated honors math student, because unlike Axler the author doesn't hate determinants for whatever reason. Strang would work well for engineers. Axler is mostly concerned with the last two, but IMO this makes him kinda niche.
honestly students should just study ode's to motivate and learn linear algebra which otherwise is lacking substance, and peek into pde's to see why vector space necessary and insightful especially for nonlinear pde.
For math majors, LADR is far better than LADW, almost most of LA books use the determinant to prove the advanced stuff (diagonlaization etc), what sets LADR apart is the intentional delay of introducing determinants in order to prove (and hence understand) what's the goal behind concepts such as diagonalization, I love this method.
Axler is usually recommended as a great second book for linear algebra (unless you're extremely comfortable with proofs).
For a first exposure, these days it's hard to do better than Math Academy [0]. It'll diagnose all your weak points (including what prereqs you might not have) and handle all the scheduling using spaced repetition and knowledge about how the topics relate to each other.
All you have to do is keep showing up and do the work for 30+ minutes per day.
For those who find Linear Algebra Done Right too much to start with, and those who don't get why Strang starts with matrices, I can't recommend more "The dark art of linear algebra" read this first. With this you can then tackle every other book on the topic more easily
I found it really insightful (and always overlooked) to distinguish between vector and co-vector spaces. It doesn't necessarily produce new knowledge, but makes things more clear.
Overrated and tendentious book. There are many better linear algebra texts. His polemic against determinants is poorly motivated, misguided, and distracting. The writing is quite formal and not terribly inspiring. The coverage is adequate but nothing more.
Lately been deep diving into linear algebra. And a way which i engage with it is that I tell AI to generate interactive examples + questions on Lean or Haskell. Its so fun, just deriving the intuition in these languages.
My bag of tricks is better than your bag of tricks. Alright.
As with most textbooks, it fails to motivate why reading it is worth the investment.
Perhaps it is a millennial old tradition of the Greek mystery schools, that the rite of passage came by proving your commitment to material knowledge without anything but fate in the school itself as motivation.
I wanted to learn the underlying principles of LLM/AI and got myself Shilov's book. Wow that was so thick, each paragraph took a while to figure. This could be a nice option..
This is supposedly based on Sheldon Axler's earlier and shorter paper "Down With Determinants!" [0]. I lectured mathematics for a while at a "former polytechnic" and used to enjoy leaving print-outs of this sort of paper in the faculty communal areas.
Note that "done right" means done with Axler's completely subjective and unusual hatred of determinants, chronicled here [0]. It is in no way "done right" in some definitive, rigorous way; most math professors I have spoken to either strongly disagree with the presentation or have no particular preference.
Determinants are easy to use but very hardly to grasp intuitively, this is not a minority point of view. See countless of StackOverflow questions begging for a conceptual exposition of determinants.
The easiest conceptual handle is geometric: volume expansion, but seeing how this is related to the combinatorial sum over all permutations, or how those two point of views are related to the algebraic one (that a set of equations having a solution or not), is not easy to see even in the 2D case.
I won't be surprised if math professors don't have this issue like you said (especially if someone is comfortable with wedge products), but the vast majority of newcomers who are interested in understanding why something works rather than just how to use it struggle all the time with determinants.
I think the volume explanation is one of the most intuitive pieces of math in existence, personally! Uninvertibility of a tranformation corresponds to a volume of zero because the transformation must squish two dimensions together, leaving them impossible to differentiate, det(AB) = det(A)det(B) because applying two transformations applies their scaling successively, det(A^-1) = 1/det(A) because you have to undo the scaling to invert a transformation etc. I don't think the permutation definition is even strictly necessary; if I recall correctly Linear Algebra Done Wrong defines the determinant in terms of its geometric definition and develops its formula from the properties it must have. I think that the concept is well worth the investment of initial confusion. Axler disagrees, however.
But if you see the geometric view then the permutation sum, for certain obsessive learners they want to see why they are equivalent and thats the hard part.
It seems to me more like determinants are often badly motivated. The way I remember it from Apostol's Calculus was like "a volume multiplier would be very useful; it needs to be a signed volume for linearity, which implies antisymmetry. Here are axioms collecting these requirements. They're uniquely satisfied by the determinant. Proof: ..."
Agreed that it should help if you got to learn wedge products first (I didn't).
This is indeed a good concise description of how to connect the two, but even making peace with that there is something still magical in how the permutations in the sum cancel neatly (in an inclusion-exclusion kind of a way) to get the volume.
I found that “properties of the determinant uniquely determine this formula that I guessed” approach to determinants to be extremely unconvincing when I was learning linear algebra.
This might come down to details of how you explain it: iirc Apostol took those basic moves (axioms) and calculated what the formula would have to be, rather than starting with a formula and checking that it has the properties of a signed volume.
But I don't know, it's been a very long time for me. It's good to have a variety of approaches to the subject.
Yes, but that doesn't prove that the determinant is actually the real signed volume multiplier.
I think this is what led me to feel unhappy about determinants when I was a first-year university student. You need to actually prove that the determinant is the volume of the N-dimensional parallelepiped, and the axiomatic proof doesn't do that.
So you need basically two extra lines after proving those things so that people can say "okay, the determinant eats ignores all input vector non-orthogonality so that it gives volume".
I got halfway though the exercises with the help of a reading group. They were very hard, bit thought provoking, so I would definitely recommend. Don't feel discouraged if you get stuck and try not to look at the solutions right away.
In the 1980s, being not entirely adept at mathematics I recall scouring every library and bookstore I could find for any snippet that would explain a proof, or even a concept, so that I could understand it. Videotaped lectures by other professors were sometimes available on campus too.
59 comments
[ 28.6 ms ] story [ 1226 ms ] threadThe contenders seems to be:
- Linear Algebra Done Right - Sheldon Axler
- Liner Algebra Done Wrong - Sergei Treil
- Introduction to Linea Algebra - Gilbert Strang
- Introduction to Applied Linear Algebra: Vectors, Matrices, and Least Squares by Stephen Boyd and Lieven Vandenberghe
[1] https://www.youtube.com/playlist?list=PLZHQObOWTQDPD3MizzM2x...
(I don't doubt it at all! I'm just curious.)
The applications not only provided the context to get a deeper understanding of the concepts, it also made the math relevant to the real world. By showing a range of problems, from the chemical stoichiometry to the algorithmic SVD, it inspires the reader to look for more problems that can be solved with the toolset being introduced.
If you are SELF-teaching, you need to learn how to figure things out.
When you can’t, there’s plenty of sources. It’s not the 1990s. Your favorite LLM probably gives good explanations of linear algebra ideas even on fast mode. Gemini was surprisingly good for me.
There are some central concepts, and which ones matter to you depends on what you're planning to do with your future. The key points are:
* Matrices/vectors as grids of numbers/computational tools.
* Matrices/vectors as positions and transformations of those positions.
* Matrices/vectors as more abstract geometric objects
* Matrices/vectors as algebraic objects.
There is no 'done right' imo. My sense is LADW is probably the best option for a motivated honors math student, because unlike Axler the author doesn't hate determinants for whatever reason. Strang would work well for engineers. Axler is mostly concerned with the last two, but IMO this makes him kinda niche.
For a first exposure, these days it's hard to do better than Math Academy [0]. It'll diagnose all your weak points (including what prereqs you might not have) and handle all the scheduling using spaced repetition and knowledge about how the topics relate to each other.
All you have to do is keep showing up and do the work for 30+ minutes per day.
[0] https://www.mathacademy.com/courses/linear-algebra
Linear Algebra Done Right 58 points, July 2023, 4 comments https://news.ycombinator.com/item?id=36576114
Linear Algebra Done Right – 4th Edition, 631 points, Oct 2023, 294 comments https://news.ycombinator.com/item?id=38060159
Linear Algebra Done Right [pdf], 85 points, Sept 2024, 39 comments https://news.ycombinator.com/item?id=41416799
Possibly paired with some numerical algebra free text (many on the Internet)
As with most textbooks, it fails to motivate why reading it is worth the investment. Perhaps it is a millennial old tradition of the Greek mystery schools, that the rite of passage came by proving your commitment to material knowledge without anything but fate in the school itself as motivation.
Rigor before Worth.
(Yes this is a pet peeve of mine :)
Thanks!
I am assuming game developers are interested in the applications of Linear Algebra and there are many other books that are far suited for that.
Anyone from game dev community care to elaborate ?
[0] https://www.axler.net/DwD.html
[0] https://www.axler.net/DwD.html
The easiest conceptual handle is geometric: volume expansion, but seeing how this is related to the combinatorial sum over all permutations, or how those two point of views are related to the algebraic one (that a set of equations having a solution or not), is not easy to see even in the 2D case.
I won't be surprised if math professors don't have this issue like you said (especially if someone is comfortable with wedge products), but the vast majority of newcomers who are interested in understanding why something works rather than just how to use it struggle all the time with determinants.
Agreed that it should help if you got to learn wedge products first (I didn't).
But I don't know, it's been a very long time for me. It's good to have a variety of approaches to the subject.
I think this is what led me to feel unhappy about determinants when I was a first-year university student. You need to actually prove that the determinant is the volume of the N-dimensional parallelepiped, and the axiomatic proof doesn't do that.
So you need basically two extra lines after proving those things so that people can say "okay, the determinant eats ignores all input vector non-orthogonality so that it gives volume".
On a daytime episode of David Letterman, Isaac Asimov predicted fiber optics would one day bring about television studios in people's homes: https://youtu.be/cIB1b_8hqB0?si=212sGzZ71VIZORML&t=696
All sources of understanding are so very much appreciated.
i think lot of ppl are under the impression that long courses are a prerequiste to even start exploring ai/ml.
This is the best I know: https://www.youtube.com/watch?v=Fnfh8jNqBlg&list=PLlXfTHzgMR....
[0] I meant natural language