Ask HN: the road to learning useful math
I've recently come across quite a few interesting stats-heavy compsci papers where the mathematical notations and concepts expressed escape me. I'd like to correct this -- and I've seen a few comments floating around here that this stuff isn't actually that hard, and this has got me thinking.
Could HN recommend any 'math for programmers' resources or suggest structured approaches (ie. 'look into X, then Y, then try understanding Z) to go about learning this sort of stuff? I think I'm probably not alone here in wanting to learn more about this sort of thing.
A quick google provides some interesting links to get started:
Math for programmers: http://steve-yegge.blogspot.com/2006/03/math-for-programmers.html
'Concrete Mathematics' by Graham, Knuth & Patashnik: http://en.wikipedia.org/wiki/Concrete_Mathematics
32 comments
[ 2.8 ms ] story [ 57.3 ms ] threadAlso, you might want to check out the wikibooks High School Mathematics Extensions textbook: http://en.wikibooks.org/wiki/High_School_Mathematics_Extensi... - there's a whole lot of rigarous but accessible introductions to some pertinent mathematical concepts. Marcus du Sautoy's 'The Music of the Primes' is also great as a bit of light reading to give you a sense of the history (and some very basic mathematical concepts) of the subject.
Cheers,
Tim
I suggested Numerical recipes coz it contains the implementation of Various Liner Algebra algorithms which form the basics of Math for Programmers..
An approach that might work for some is to take a good undergrad textbook of something you're interested in, but that is a bit too hard (preferably not too dense, with enough explanatory text). Try to plough through it, and look up all the stuff you don't understand! Don't expect to get very far very quickly, but at least you'll get led to useful reading material for the fields you care about, and it might also motivate you more than starting from axioms.
(For me it was Information Theory, Inference, and Learning Algorithms that sparked off a lot of study: free online at http://www.inference.phy.cam.ac.uk/mackay/itila/ ).
Mathematics for Computer Science is perhaps tailor made for your needs - http://ocw.mit.edu/OcwWeb/Electrical-Engineering-and-Compute...
Wikipedia often helps with the notations.
I think learning useful maths is best done by reading up on the stuff you presently need. I have several maths books that I never read because while it seemed like a nice idea, in the end there just wasn't enough motivation for it.
Are you reading these stats-heavy comp-sci papers for a particular reason beyond mere curiosity, like for a project or for work? If so, then you will have what I consider the most crucial element needed for learning a difficult subject: motivation, real lasting motivation.
My advice: identify a concrete need to learn about a particular area of math/stats, like solving a real-world problem with software, then don't start by getting bogged down in a particular equation, but instead, find out the area of math/stats which that equation is a part of, and seek out the most interesting essay or video lecture about it. It's important that this video or essay be composed by someone with a reputation for making the complicated accessible to mere mortals.
Good luck!
_Mathematics: From the Birth of Numbers_ by Gullberg and Hilton (http://www.librarything.com/work/23014) is also a good overview. It starts with counting (Roman numerals, numbers in ancient Egyptian writing, etc.) and builds up from there. The writing seems quite clear, and it covers both the mathematical concepts and their history. (I like this approach quite a bit, though my background is in history. YMMV.) I haven't gotten to the chapters for math I'm completely new to yet, but it seems like it would introduce you to enough basic concepts that you could have a clearer idea where to go next.
_The Cartoon Guide to Statistics_ by Larry Gonick (http://www.librarything.com/work/12735) seems like a good statistics refresher / overview. It's not that deep, and it's rather hokey, but it's a quick read. An important part of reading math papers is not getting thrown by the dense notation, so starting with something less formal may be helpful.
http://github.com/ionfish/notes/blob/master/discrete-maths.m...
The formatting is a little unreadable—I need to fix the Markdown line breaks—but it does have a fair few links.
Practical Foundations of Mathematics http://www.cs.man.ac.uk/~pt/Practical_Foundations/index.html
Linear Algebra - Gilbert Strang (Book & Videos)
Linear Algebra and Applications http://www.math.unl.edu/~tshores1/linalgtext.html
P.S. There was another Introduction to Statistics book that I am not able to find the link for.
The way to get around this initially is by brute force. For example, say you have a matrix function (from matrices to matrices). The formula uses the exponential of a matrix, and you're not sure what that actually looks like. So find the definition, and write it out. Think about what that means, maybe do a couple of examples to get a sense of it. Write out your matrix function with this definition expanded (which probably means you'll write it element-wise), so that you know what each element of the output matrix is as a function of the elements of the input matrix. Work a couple of examples, think about it.
Eventually, you get familiar with the exponential of a matrix, and you do the mapping in your head - so your brain has just acquired a new abstraction. The process above applies to basically every mathematical structure you'll run into. Sometimes you'll have to recurse a few levels.
It's not about books, it's about process. You can know how to do lots of math problems and still not grok papers - and you can learn to quickly grok papers and be able to implement them / reason about them without being very good at math problems (I'm in the 2nd bunch). In terms of books, you only really need Wikipedia / Wolfram MathWorld, plus maybe one of the standard math references like Kreyszig or Boas. Learning to use Mathematica can make your life much easier as well.
One last comment:
A book that totally changed my relationship to math is Peter Szekeres' Modern Mathematical Physics: Groups, Hilbert Space, and Differential Geometry. The explanations are uncompromisingly lucid, written in plain english, with minimal symbol walls. The book takes you essentially from zero, and introduces nearly all the math that underpins modern physics. This book really changed my conception of myself - I've lost my fear, when reading papers, that maybe this is the one I'm not smart enough to understand. When that fear goes, the stress goes - and you know it's just a matter of learning the definitions. Then it becomes pleasurable, like a puzzle.
From reading it I get most of what it's about, and how they work, but the embedded probability stuff is mostly squiggles to me.
I guess my ultimate aim with this is would be able to quickly grok papers on various compsci subjects, and then go around and code implementations to test things out.
Comp sci is notorious for having a huge gap between published work and working implementations. It often takes heroic efforts to put together a working version of the algorithms described, sometimes so much work that nobody has ever done it (Chazelle has some algorithms that have never been implemented in code). Even if it's plausible to do, don't be surprised if it takes weeks to get it working right. The more "on the edge" the algorithms are, the more difficult it tends to be, and it's not really likely that you'll be able to quickly experiment with new methods unless the authors have published code.
This happens far less often than it should, because simple possession of a working algorithm is often enough to squeeze several more papers out of a topic, so authors have no particular incentive to share theirs, or even make it easy to create one from scratch. Computer graphics papers are particularly bad on that front, often omitting so much detail that even with significant study and effort you could never reproduce their results without researching the problem on your own anyways.
Other people have offered good advice: go through it in excruciating detail, think about every equation as its own sub-problem, and make sure you understand what's happening at each step. These papers are meant for detailed study, not quick reading, even if the reader already knows a lot about the field. Google should help you through most of the math stuff, and try going to book search or the papers search if you're not finding anything online, which will give you good keywords to look for books on.
Yeah, so, read ITILA, and do all problems up to 3, and attempt as many 4 as possible. You can skip the info. theory half and go straight to Bayesian stats and then neural nets - but you'll want to go back and learn the info theory too, eventually.
BTW, you're jumping into an enormous kettle of fish called "graphical models," which are very useful but endlessly varied upon and often needlessly overcomplicated. I think what you really need is Zoubin: http://videolectures.net/mlss07_ghahramani_grafm/
If that's a bit heavy, browse around the site for an easier introduction.
I get ... how they work, but the embedded probability stuff is mostly squiggles to me
Pardon me for saying so, but no, if you don't understand why each of those equations is what it is, then you don't understand how it works. That's kind of like saying "I get how the car works, but that bit in the front is just a giant hunk of metal to me."
If it looks like squiggles, you need to untangle it. It's like untying a giant knot. You need to find some end, some piece of thread you can tug on, and then slowly tease each equation apart until you get how the various pieces move. (What happens if Z is bigger/smaller than what it is? Why is it there at all? What is a "clique"? Why would I add up potential functions in a clique?)
Thanks for the video lecture links, I'll definitely look into those.
There are certainly problems with learning on your own, not the least of which is maintaining the drive to push through the icky parts. But in the absence of external motivational structures, you can give your self-discipline a workout. I'm not a particularly disciplined person, but I can still commit to doing 1 hour of math a day. I guess I think of it like my daily mental exercise. It's hard to develop this habit though, and if you can't, school's not a bad option.
One of the other problems with learning math on your own is not having anyone to ask questions. To help with this, I've actively tried to make friends (of sorts) with math professors at a nearby school. I get lunch with one every once in a while, and we talk about sports and life and math. This can kinda substitute for the community of like-minded people that school provides.
All this goes to say is that you have to be aware of how you learn and how you get motivated. If you are externally driven, and don't mind learning at a pace you did not establish, school will be useful to you. If you don't grok extrinsic motivation, school's rough and you should learn on your own.
Actually the instrument analogy is apt--the world class concert musicians you see on Youtube are practicing 5 hours a day. The guy down the hall banging out a Chopin prelude practices one. Both are "good" musicians, but only one is great. It's a question of what you're looking for.
In the interim, you could start with David Freedman's "Statistics" (written along with Pisani and Purves) to at least get a feel for the basic ideas of statistics from one of the great statistical writers of the 20th century.
If there are other things you want to learn about, I'd be happy to make suggestions, as I went in the opposite direction as you're trying to go now, having been a math major as an undergrad who picked up CS along the way. For cryptography, for instance, you'd need to learn number theory and then abstract algebra, but those things will be of little use to you in other contexts.
"Advice to Mathematics Teachers on Evaluating Introductory Statistics Textbooks" by Robert W. Hayden
http://statland.org/MyPapers/MAAFIXED.PDF
"The Introductory Statistics Course: A Ptolemaic Curriculum?" by George W. Cobb
http://repositories.cdlib.org/cgi/viewcontent.cgi?article=10...
Both are excellent introductions to what statistics is as a discipline and how it is related to, but distinct from, mathematics.
A very good list of statistics textbooks appears here:
http://web.mac.com/mrmathman/MrMathMan/New_Teacher_Resources...
- Practical Mathematics by C. I. Palmer - Mathematics for the Nonmathematician by Morris Kline
Practical Mathematics is an old book, first published in the early nineteen hundreds. It covers everything beginning with multiplication to advanced algebra and trig. It uses practical trade problems in gear cutting, cam ratios in engines, &c... The book has only been re-published by the company Lindsay's Technical Books. You can also find it in Google's book repository.
Mathematics for the Nonmathematician is a bit more recent and written by a strong mathematician with a lot of history and background. This book got me hooked on mathematics, no other book has done that; his in-depth scholarly work plus his skill in mathematics makes it both interesting and educating to read.
A large majority of programming problems can be tackled with solid arithmetic or algebra.