Wow very nice. Lots of content in here, with no lengthy explanations but useful point-form intuition.
The .epub has very clean math done in HTML (no images), which is a cool way to do things. I've never seen this before. I wonder what the author used to produce the .epub from the .tex?
The organization and formatting of the single .tex file is such that one could almost read the source alone. Really nice. Also, I had no idea that GitHub did such a good job rendering the LaTeX math in markdown, it's imperfect but definitely good.
Tried to pick a book to get into linear algebra recently, the experience was fairly hellish. First course this, second course that, done right, done wrong... I'd to the LADR4e route, but I don't have the proof-it chops yet...
You might want to checkout the book Practical Linear Algebra: A Geometry Toolbox by Dianne Hansford and Gerald Farin (its 1st edition was simply named The Geometry Toolbox: For Graphics and Modeling) to get an intuitive and visual introduction to Linear Algebra.
Pair it with Edgar Goodaire's Linear Algebra: Pure & Applied and you can transition nicely from intuitive geometric to pure mathematical approach. The author's writing style is quite accessible.
Add in Introduction to Applied Linear Algebra: Vectors, Matrices, and Least Squares by Stephen Boyd et al. and you are golden. Free book available at https://web.stanford.edu/~boyd/vmls/
I had the same experience when I first learned linear algebra. I don't have any book recommendations, but I did want to say that for some topics, it is better to learn it by applying it than by using a book. Linear algebra was like that for me, but oddly enough I was able to learn tensor calculus from a book later (after doing a lot of problems).
It's crazy that Linear Algebra is one of the deepest and most interesting areas of mathematics, with applications in almost every field of mathematics itself plus having practical applications in almost every quantitative field that uses math.
But it is SOOO boring to learn the basic mechanics. There's almost no way to sugar coat it either; you have to learn the basics of vectors and scalars and dot products and matrices and Gaussian elimination, all the while bored out of your skull, until you have the tools to really start to approach the interesting areas.
Even the "why does matrix multiplication look that way" is incredibly deep but practically impossible to motivate from other considerations. You just start with "well that's the way it is" and grind away until one day when you're looking at a chain of linear transformations you realize that everything clicks.
This "little book" seems to take a fairly standard approach, defining all the boring stuff and leading to Gaussian elimination. The other approach I've seen is to try to lead into it by talking about multi-linear functions and then deriving the notion of bases and matrices at the end. Or trying to start from an application like rotation or Markov chains.
It's funny because it's just a pedagogical nightmare to get students to care about any of this until one day two years later it all just makes sense.
I went through Khan academy on linear algebra a long time ago because I wanted to learn how to write rendering logic, for me I was implementing things as I learned them with immediate feedback for a lot of the material. Was probably the single most useful thing I learned then.
I found the university level presentation of "Vector spaces -> Linear functions -> Matrices are isomorphic to linear functions" much more motivating than the rote mechanics I was taught in highschool, but it's hard to see if I would've had that appreciation without being taught the shitty way first.
I had the opposite experience when learning linear algebra as I was also doing 3D computer graphics at the time. It was super interesting and fun. I guess you just have to find an application for it.
Linear algebra was such a tedious thing to learn I skipped over it to abstract algebra and doubled back once I had some kind of minimally interesting framework to work with it against. Normally I think this is a foolish way to do things, but sometimes things are just so dull you have to take the hard route to power through at all.
Actually, the calculation of matrix multiplication is extremely easy to understand if you show it done with a row and column vector first. Once your students understand dot products, they will recognize matrix multiplication as the mere pairwise calculation of dot products.
>Even the "why does matrix multiplication look that way" is incredibly deep but practically impossible to motivate from other considerations. You just start with "well that's the way it is" and grind away until one day when you're looking at a chain of linear transformations you realize that everything clicks.
Can you expand on your experience with this? I do some graphics programming so I understand that applying matrix transformations works, and I've seen the 3blue1brown 'matrices are spreadsheets' explanation (luv me sum spreadsheets), but the intuition still isn't really there. The 'incredibly deep "why matrix multiplication looks that way"' is totally lost on me.
Always nice to see CC-licensed textbooks. This one looks fairly minimal, not including much of explanation, illustrations, or proofs; I think those are generally useful for the initial study, but this should still work as a cheat sheet, at least.
A linear algebra course without graphics? When I learnt it at school almost 25 years ago, the teacher made schematics all the time to explain the visual intuition behind each concept. I was totally confused when he introduced the abstract definition of a vector space with the addition and scalar multiplication. Then he drew some arrows. Then it all suddenly made sense.
Highly recommend to anyone struggling with linear algebra to check out Linear Algebra Done Right, by Sheldon Axler. Do always keep in mind that some concepts are very verbose, but truly out of necessity. If you're talking about an N by N matrix, you're naturally going to have to distinguish N^2 different elements.
You can go very far without touching matrices, and actually find motivation on this abstract base before learning how it interops with matrices.
A little surprising to me this doesn’t come up the most! Excellent text, and look, you can get the 4th edition (2024) for free as a pdf at http://axler.net.
Someone should convert all the examples into C code so it's more intelligible to programmers who are, let's admit, the main audience for something like this.
To the best of my knowledge: Scalars are variables. Vectors are arrays. Matrices are multi dimensional arrays. Addition and multiplication is iteration with operators. Combinations are concatenation. The rest like dot products or norms are just specialized functions.
But it'd be nice to see it all coded up. It wouldn't be as concise, but it'd be readable.
This was one of the reasons for me wanting to study the J programming language. I have a notion AI/ML programming might be better done using array languages.
As someone who took a standard undergrad linear algebra course but never really used it in my work, what are some good ways to get acquainted with practical applications of linear algebra?
In terms of understanding why something is like that, Linear Algebra belongs to Geometry more than Algebra. Every formula in Linear Algebra which ultimately is justified by "it's just that way" can be better justified by geometry. The algebraic formulas are like animals who lost their natural habitat and were put in a zoo, making people think that these animals evolved in the zoo itself.
To give an example: A simple multiplication of two numbers is better seen as rotating one of the numbers to be perpendicular to the other and then quantifying the area/volume spanned by them. This gives vector dot product.
While geometry might better address "why", algebra gets into the work of "how to do it". Mathematics in old times, like other branches of science, did not encourage "why". Instead, most stuff would say "This is how to do it, Now just do it". Algebra probably evolved to answer "how to do it" - the need to equip the field workers with techniques of calculating numbers, instead of answering their "why" questions. In this sense, Geometry is more fundamental providing the roots of concepts and connecting all equations to the real world of spatial dimensions. Physics adds time to this, addressing the change, involving human memory of the past, perceiving the change.
Not just Linear Algebra but every branch of Science/Mathematics should be taught as much as possible using Geometry (and other visualizations) before being mapped to abstract algebraic symbols. Basic Geometry (along with simple Arithmetic) is the oldest subfield of Mathematics for the reason that most of its concepts are intuitive and feels "natural" and mappable to the "Real World" by us Humans. While abstraction via symbol manipulation is necessary to generalize and extend mathematics it should come at a later stage after we have developed some intuition of the concepts being represented by the symbols, however limited/restricted they might be. All abstraction requires some mathematical maturity which can only happen over time.
It starts with the axioms of being able to draw one line parallel to another, and a line through point, and builds up everything from there. No labeled Cartesian axes. Just primitive Euclidean objects in an affine space.
I love this. It's a great companion to all the resources online like Kahn Academy, 3blue1brown etc. and mathisfun, Wolfram Mathworld and Google's Gemini.
A little bit too concise IMO. E.g. it doesn't really explain where the normal equations come from and it mentions eigenspaces without ever defining them.
42 comments
[ 4.6 ms ] story [ 64.2 ms ] threadThe .epub has very clean math done in HTML (no images), which is a cool way to do things. I've never seen this before. I wonder what the author used to produce the .epub from the .tex?
Instead, it is replaced with a red error box saying: [ Unable to render expression. ]
I wonder if there is an artificial limit for the amount of latex expression that can rendered per page.
Pair it with Edgar Goodaire's Linear Algebra: Pure & Applied and you can transition nicely from intuitive geometric to pure mathematical approach. The author's writing style is quite accessible.
Add in Introduction to Applied Linear Algebra: Vectors, Matrices, and Least Squares by Stephen Boyd et al. and you are golden. Free book available at https://web.stanford.edu/~boyd/vmls/
Thanks to everyone recommending books too!
But it is SOOO boring to learn the basic mechanics. There's almost no way to sugar coat it either; you have to learn the basics of vectors and scalars and dot products and matrices and Gaussian elimination, all the while bored out of your skull, until you have the tools to really start to approach the interesting areas.
Even the "why does matrix multiplication look that way" is incredibly deep but practically impossible to motivate from other considerations. You just start with "well that's the way it is" and grind away until one day when you're looking at a chain of linear transformations you realize that everything clicks.
This "little book" seems to take a fairly standard approach, defining all the boring stuff and leading to Gaussian elimination. The other approach I've seen is to try to lead into it by talking about multi-linear functions and then deriving the notion of bases and matrices at the end. Or trying to start from an application like rotation or Markov chains.
It's funny because it's just a pedagogical nightmare to get students to care about any of this until one day two years later it all just makes sense.
Short, simple answer to that question by Michael Penn: https://www.youtube.com/watch?v=cc1ivDlZ71U
Another interesting treatment by Math the World: https://www.youtube.com/watch?v=1_2WXH4ar5Q&t=4s
There's no impenetrable mystery here. Probably just bad teaching you experienced.
I wonder if these days that would be a better starting point.
Can you expand on your experience with this? I do some graphics programming so I understand that applying matrix transformations works, and I've seen the 3blue1brown 'matrices are spreadsheets' explanation (luv me sum spreadsheets), but the intuition still isn't really there. The 'incredibly deep "why matrix multiplication looks that way"' is totally lost on me.
You can go very far without touching matrices, and actually find motivation on this abstract base before learning how it interops with matrices.
To the best of my knowledge: Scalars are variables. Vectors are arrays. Matrices are multi dimensional arrays. Addition and multiplication is iteration with operators. Combinations are concatenation. The rest like dot products or norms are just specialized functions.
But it'd be nice to see it all coded up. It wouldn't be as concise, but it'd be readable.
http://t3x.org/klong/index.html
Some books for studying Mathematics using J are listed here - https://code.jsoftware.com/wiki/Books
To give an example: A simple multiplication of two numbers is better seen as rotating one of the numbers to be perpendicular to the other and then quantifying the area/volume spanned by them. This gives vector dot product.
While geometry might better address "why", algebra gets into the work of "how to do it". Mathematics in old times, like other branches of science, did not encourage "why". Instead, most stuff would say "This is how to do it, Now just do it". Algebra probably evolved to answer "how to do it" - the need to equip the field workers with techniques of calculating numbers, instead of answering their "why" questions. In this sense, Geometry is more fundamental providing the roots of concepts and connecting all equations to the real world of spatial dimensions. Physics adds time to this, addressing the change, involving human memory of the past, perceiving the change.
https://www.youtube.com/watch?v=yAb12PWrhV0&list=PLBQcPIGljH...
It starts with the axioms of being able to draw one line parallel to another, and a line through point, and builds up everything from there. No labeled Cartesian axes. Just primitive Euclidean objects in an affine space.
Starts from linear transformations and builds from there.
Edit: Also the mathematics stackexchange.