If you actually want to learn linear algebra, don't use this blogpost. It's real weaksauce compared to the wealth of free information and resources available online.
Firstly, the real illustrated guide to linear algebra is the youtube series "The Essence of linear algebra" by 3blue1brown[1]. It has fantastic visualisations for building intuition and in general is wildly superior to this, which seems fine but extremely superficial.
If you're done with 3b1b and want to take things further, then the go-to is the excellent 18.06SC course by the late and legendary Gilbert Strang. It's amazing, it's free. [2]
Still want more? OK now you're talking my language. If you are serious about linear algebra (Up to graduate level, after that you need something else) then you want the book "Linear Algebra Done Right" by Sheldon Axler. It's available for free from the author's website[3] and he has made a bunch of videos to supplement the book. There's also an RTD Math full lecture series[4] that follows the book and he explains each thing in a lot of detail (because Axler goes fast, so it's beneficial to unpack the concepts a bit sometimes).
For most people going into science and engineering as opposed to pure mathematics, Poole's "Linear Algebra: A Modern Introduction" is probably more suitable as it's heavy on applications, such as Markov chains, error-correcting codes, spatiel orientation in robotics, GPS calculations, etc.
Comparing this blog post to a 500-page book or a multi-hour course and calling it “weaksauce” misses the point. This post is meant as an introduction to the dot product, and it does that really well. The formal definition (6.1) and explanation in Axler’s book wouldn’t make a good starting point for most people, it isn't even a good next step in my opinion. It’s great that you’re passionate about the topic, really, but helping more people discover math means meeting them where they are and appreciating content like this for what it’s trying to do.
Find a copy of Yousef Saad’s books (Iterative Methods, and the Eigenvalue Problems). Find a problem you want to solve, and implement one of the solutions he describes. If you don’t understand something, that’s what the first chapter is for.
mathacademy has a course on linear algebra. currently working my way back up from nothign to get to it. easily the best resource for learning math on the internet.
When I see the word "illustrated," I expect to see graphs or something that would help me visualize how linear algebra works. The only thing "illustrated" about this post is that he hand drew some table which could have been easily with some basic HTML+CSS.
That's your preference. However "To illustrate is to make something more clear or visible. Children's books are illustrated with pictures. An example can illustrate an abstract idea. "illustrate" comes from the Latin illustrare 'to light up or enlighten.'"
What graphical illustration do you think this is missing? How would that make things better? Have you ever seen http://matrixmultiplication.xyz/? Great graphical illustration. Also: a really unhelpful way to understand matrix multiplication.
This is part 2 of a series, all under the same name; the first part is extensively illustrated (and I'm not sure the part 1 illustrations are all that helpful).
I think this is a reasonable expectation. I'm writing about the dot product and there is a geometric visualization for dot products, so it's fair to expect that. As I said in this comment (https://news.ycombinator.com/item?id=45802969) I focused on teaching the reader the bare minimum they would need to know to get to matrix multiplication.
I use illustrations for a lot of other things though. I use them to pace my posts, to call out things that are especially important, to reinforce written ideas with images, and to give the reader's eye a place to rest.
Some hand-written (not AI-generated) prompts to consider:
"An expert in university-level linear algebra, including solving systems of equations, matrices, determinants, eigenvalues and eigenvectors, symmetry calculations, etc. - is asked the following question by a student: "This is all great, professor, and linearity is also at the heart of calculus, eg the derivative as a linear transformation, but I would now like you to explain what distinguishes linear from non-linear algebra."
"What kind of trouble can the student of physics and engineering and computation get into if they start assuming that their linear models are exact representations of reality?"
"A student new to the machine learning field states confidently, 'machine learning is based on linear models' - but is that statement correct in general? Where do these models fail?"
The point is that even though it takes a lot of time and effort to grasp the inner workings of linear models and the tools and techniques of linear algebra used to build such models, understanding their failure modes and limits is even more important. Many historical engineering disasters (and economic collapses, ahem) were due to over-extrapolation of and excessive faith in linear models.
A great resource that isn't mentioned often is the linear algebra chapters in Birkhoff and Mac Lane's Survey of Modern Algebra. Chapters 7,8,9, and 10 (in the 4th and 5th editions anyway) are a self-contained book-within-a-book of about 200 pages on both the computational and theoretical aspects of vector spaces, matrices, linear transformations, and determinants.
Many times I've been puzzled by a concept just to go there and find it made simple and obvious. It's a real golden nuggett... Plus if you then want to go further into groups, rings, fields, and Galois theory, that's also there.
> Summary: A dot product is a weighted sum of two vectors.
Nope. This is incorrect. The dot product is a weighted sum of a vector's elements, where the weights are the elements of the other vector. Weighted sum of two vectors would require a third entity to provide the weights.
Hey everyone, I'm the author. I'm seeing a lot of the same comments here, so I want to address them.
I teach math by leading with examples. I try to show the intuition behind an idea, and why it is interesting. For this series, my reader is someone who knows algebra, and likes learning new things, especially when a teacher shows what is interesting about a topic.
## You didn't cover x about the dot product.
I try to only teach as much as is necessary to get the student to the next point, which is matrix multiplication. I usually end up cutting a lot of material out of my chapters to keep them simple. In this case, I cut out a whole section on the properties of a dot product, as well as a discussion about inner and outer products, because those weren't necessary to get to matrix multiplication. I think this context was lost while posting to HN.
## 3B1B already has a series on this.
I love 3B1B, but his style of teaching and mine are quite different. Even though we both teach visually, his videos are densely packed with information and his expectation is that you will watch the video a few times till you understand the topic. He also leads with math more than I do. My posts are written more like stories. My goal is they should be easy to get into, and by the time you have finished reading, you should understand more about the topic. I don't expect readers to read through multiple times. I personally learned linear algebra through Strang's videos and textbook, and those videos are awesome, but can be confusing. If you found the Strang or 3b1b videos confusing, hopefully my posts will make it easier for you to follow them. I think comment is spot on: https://news.ycombinator.com/item?id=45800657
If these ideas resonate with you, I think you'll like this post, and if not, there are plenty of guides that go the more traditional route. You can also read the first post in the series and see if you like it: https://www.ducktyped.org/p/an-illustrated-introduction-to-l...
The comments here are a case study in conflict drives engagement. And not in a good way.
Try not to drive more conflict if you respond.
Facts:
The target audience of this post is someone who does not know anything about the dot product. It provides a mechanical/computational introduction. It does not include any geometric interpretation of the dot product.
Critique:
After you read the post, you should have seen something to connect to a matrix-vector product (imo), but the example matrices need to be transposed to look like a traditional matrix-vector product. Maybe the author is intending to multiply on the left by the transpose of the weight vector. The weights don’t seem to consistently appear on either side.
17 comments
[ 2.9 ms ] story [ 30.3 ms ] threadFirstly, the real illustrated guide to linear algebra is the youtube series "The Essence of linear algebra" by 3blue1brown[1]. It has fantastic visualisations for building intuition and in general is wildly superior to this, which seems fine but extremely superficial.
If you're done with 3b1b and want to take things further, then the go-to is the excellent 18.06SC course by the late and legendary Gilbert Strang. It's amazing, it's free. [2]
Still want more? OK now you're talking my language. If you are serious about linear algebra (Up to graduate level, after that you need something else) then you want the book "Linear Algebra Done Right" by Sheldon Axler. It's available for free from the author's website[3] and he has made a bunch of videos to supplement the book. There's also an RTD Math full lecture series[4] that follows the book and he explains each thing in a lot of detail (because Axler goes fast, so it's beneficial to unpack the concepts a bit sometimes).
[1] https://www.youtube.com/watch?v=fNk_zzaMoSs&list=PLZHQObOWTQ...
[2] https://ocw.mit.edu/courses/18-06sc-linear-algebra-fall-2011...
[3] https://linear.axler.net/ and https://www.youtube.com/watch?v=lkx2BJcnyxk&list=PLGAnmvB9m7...
[4] https://www.youtube.com/watch?v=7eggsIan2Y4&list=PLd-yyEHYtI...
https://news.ycombinator.com/item?id=38060159
For most people going into science and engineering as opposed to pure mathematics, Poole's "Linear Algebra: A Modern Introduction" is probably more suitable as it's heavy on applications, such as Markov chains, error-correcting codes, spatiel orientation in robotics, GPS calculations, etc.
https://www.physicsforums.com/threads/linear-algebra-a-moder...
Quote from https://www.vocabulary.com/dictionary/illustrate
This is part 2 of a series, all under the same name; the first part is extensively illustrated (and I'm not sure the part 1 illustrations are all that helpful).
I use illustrations for a lot of other things though. I use them to pace my posts, to call out things that are especially important, to reinforce written ideas with images, and to give the reader's eye a place to rest.
If you are looking for graphs, this video about the dot product from 3B1B may be more your speed: https://www.youtube.com/watch?v=LyGKycYT2v0
"An expert in university-level linear algebra, including solving systems of equations, matrices, determinants, eigenvalues and eigenvectors, symmetry calculations, etc. - is asked the following question by a student: "This is all great, professor, and linearity is also at the heart of calculus, eg the derivative as a linear transformation, but I would now like you to explain what distinguishes linear from non-linear algebra."
"What kind of trouble can the student of physics and engineering and computation get into if they start assuming that their linear models are exact representations of reality?"
"A student new to the machine learning field states confidently, 'machine learning is based on linear models' - but is that statement correct in general? Where do these models fail?"
The point is that even though it takes a lot of time and effort to grasp the inner workings of linear models and the tools and techniques of linear algebra used to build such models, understanding their failure modes and limits is even more important. Many historical engineering disasters (and economic collapses, ahem) were due to over-extrapolation of and excessive faith in linear models.
Many times I've been puzzled by a concept just to go there and find it made simple and obvious. It's a real golden nuggett... Plus if you then want to go further into groups, rings, fields, and Galois theory, that's also there.
Nope. This is incorrect. The dot product is a weighted sum of a vector's elements, where the weights are the elements of the other vector. Weighted sum of two vectors would require a third entity to provide the weights.
I teach math by leading with examples. I try to show the intuition behind an idea, and why it is interesting. For this series, my reader is someone who knows algebra, and likes learning new things, especially when a teacher shows what is interesting about a topic.
## You didn't cover x about the dot product.
I try to only teach as much as is necessary to get the student to the next point, which is matrix multiplication. I usually end up cutting a lot of material out of my chapters to keep them simple. In this case, I cut out a whole section on the properties of a dot product, as well as a discussion about inner and outer products, because those weren't necessary to get to matrix multiplication. I think this context was lost while posting to HN.
## 3B1B already has a series on this.
I love 3B1B, but his style of teaching and mine are quite different. Even though we both teach visually, his videos are densely packed with information and his expectation is that you will watch the video a few times till you understand the topic. He also leads with math more than I do. My posts are written more like stories. My goal is they should be easy to get into, and by the time you have finished reading, you should understand more about the topic. I don't expect readers to read through multiple times. I personally learned linear algebra through Strang's videos and textbook, and those videos are awesome, but can be confusing. If you found the Strang or 3b1b videos confusing, hopefully my posts will make it easier for you to follow them. I think comment is spot on: https://news.ycombinator.com/item?id=45800657
If these ideas resonate with you, I think you'll like this post, and if not, there are plenty of guides that go the more traditional route. You can also read the first post in the series and see if you like it: https://www.ducktyped.org/p/an-illustrated-introduction-to-l...
For another example of my writing, see my series on AWS: https://www.ducktyped.org/p/a-mini-book-on-aws-networking-in...
Try not to drive more conflict if you respond.
Facts:
The target audience of this post is someone who does not know anything about the dot product. It provides a mechanical/computational introduction. It does not include any geometric interpretation of the dot product.
Critique:
After you read the post, you should have seen something to connect to a matrix-vector product (imo), but the example matrices need to be transposed to look like a traditional matrix-vector product. Maybe the author is intending to multiply on the left by the transpose of the weight vector. The weights don’t seem to consistently appear on either side.