HN Explorer

I suck at math, please divulge resources

47 points by cookiecaper ↗ HN
This post is inspired by the "Wikipedia list of algorithms" submission at http://news.ycombinator.com/item?id=457579 . Props to soundsop.

On several different occasions I've brought up the Wikipedia page on discrete cosine transforms and hoped to understand it. This selection was made because DCTs are apparently fundamental algorithms for lossy data compression like that found in JPEGs or MP3s or whatever. I want to be able to write decoders for these things -- I've poked around in mplayer's source and want to change stuff, want to create a JPEG decoder as proof-of-concept to myself, etc.

But, here's the hitch: I have no mentionable background in mathematics. I got up to geometry in high school (i.e., I know cosines in triangles but have no idea what a cosine wave is, how it works, and/or how they're related if at all). Where can I learn all of the things I need to learn for these things to make sense to me, and to become blessed with the skills and understanding necessary to create and/or decode such algorithms? I need something somewhat simple and plain, something I don't have to chain through five hundred articles to understand -- that's the problem with WP; I don't understand one thing, so middle-click it, do that about ten times per math article, including on the articles I opened to understand the first article, and there's a never-ending lot of WP articles for me to read, and that's not really tenable. I need something focused on teaching, not necessarily technical precision and exhaustiveness.

I know what a logarithm is, I looked it up. I've also tried #math on Freenode but those dudes just don't know how to talk to someone without any mathematical training. Ask for a simple explanation there, it's usually kind of funny.

Please help me. Thanks. : )

38 comments

[ 0.27 ms ] story [ 64.1 ms ] thread
tjic
I've run into the same problem that you have, in the context of Wikipedia articles on physics.

Fundamentally, Wikipedia is very good at helping you move from step n to step n+1, but it's a pretty poor textbook (which is to say, it's not great at moving you from step 3 to step 303).

I'd say that you want to reframe your question as "what textbook should I get?"

lliiffee
The answer to that is "What is Mathematics?" by Courant and Robbins.
[dead] zack
I believe that book requires more background than the OP suggests he has.
RiderOfGiraffes
You can't learn all of everything, and you certainly can't do it quickly. What you can do is learn something that you're motivated to learn, but you can't do it by simply reading. You need to work on it.

Math requires work. Euclid said, "In geometry there is no royal road."

The DCT is a complicated thing to "understand", and I'm not sure I do myself. I would guess that the people who wrote the code you're seeking to modify didn't necessarily understand it either - they may simply have coded the formulae they were given.

So don't despair. You can learn things like this, but you can't do it all at once, and you can't do it quickly. You need to start somewhere.

What do you really want to know?

paulgb
http://planetmath.org and http://mathworld.com are other encyclopedia-like sources that are useful for things Wikipedia doesn't explain well.

Other people will be better than I am at suggesting good books, so I won't try to do that. If you have a software development background, consider getting involved in an open-source project that deals with the kind of math you want to learn.

harpastum
If you're really interested in learning a lot about mathematics, I would definitely recomment the Princeton Companion to Mathematics (see below for URL). It came out very recently, and while i'm only about 100 pages in (of well over 1000), it's down to earth writing and low prerequisites are very appealing to me.

http://www.amazon.com/gp/product/0691118809?ie=UTF8&tag=...

or if you'd rather not use my affiliate link:

http://www.amazon.com/Princeton-Companion-Mathematics-Timoth...

comatose_kid
You are trying to run a marathon without training.

Here's what I would do:

1) Look at a signals and system course online.

2) Look at the course's prerequisites.

3) Do you understand concepts in the pre-req? If not, goto 2)

4) Once you get to a course you understand reasonably, move on to the next course you don't understand. Do this (unwind the stack) until you reach the signals and systems course.

A reasonable chunk of an electrical engineer's education is required to get to the point you wish to reach, so there is a lot of effort involved.

edit: that said, this link might also be helpful: http://www.redcedar.com/learndsp.htm

aminuit
This is great advice. I did EE and I don't think it was until about the 2nd or 3rd year that we got a treatment of systems and signals that was sufficient to understand DCT. You seem to be focusing on the cosine part but it turns out that the cosine part of the DCT is probably the easiest part to understand. It probably won't make much sense until 1) you understand something about discrete signals and 2) you understand why transforms from one domain to another are particularly useful for solving certain problems.

I recommend 'Linear Systems and Signals' by B.P. Lathi. It was the book I used. It doesn't cover DCT explicitly, but it does have a friendly, Knuth-like treatment of the history of the mathematics. It makes for a nice break between sections of heavily technical material. Here is an excerpt from the beginning of the book on the topic of complex numbers and why they are useful. It should give you an idea what the rest (and the general subject) is like:

"If we want to travel from City a to City b (both in Country X), the shortest route is through Country Y, although the journey begins and ends in Country X. We may, if we desire, perform this journey by an alternate route that lies exclusively in X, but this alternate route is longer. In mathematics, we have a simliar situation with real numbers (Country X) and complex numbers (Country Y). All real-world problems must start with real numbers, and all the final results must also be in real numbers. But the derivation of results is considerable simplified by using complex numbers as an intermediary. It is also possible to solve all real-world problems by an alternate method, using real numbers exclusively, but this would increase the work needlessly."

larryfreeman
There are so many great books written that I would suggest buying a book and enjoying.

Among my favorite authors are:

Keith Devlin (check out his latest: The Unfinished Game about the invention of probability.)

William Dunham (check out: Journey through Genius: a survey of historically important math proofs that does not require too much background)

The book that got me to return to mathematics after college was the Mathematical Experience by Reuben Hersh and Phillip J. Davis.

There's also been some fantastic NOVA documentaries (my own favorites are "The Proof" about Andrew Wiles and the "Trillion Dollar Bet" about the mathematics of finance.)

I find Wikipedia is fantastic for reference material but not so great for study material. Better for self-study is MIT online. (Here's a link to a series of excellent video lectures on linear algebra: http://ocw.mit.edu/OcwWeb/Mathematics/18-06Spring-2005/Video...)

I have my own blog about mathematics (http://fermatslasttheorem.blogspot.com) which attempts to provide complete and clear proofs for famous results in mathematics. Some months, I get as many as 20K unique visitors so you might find it helpful.

There are also fantastic links on Hacker News on mathematics. I've found some great ones about Bayes' Theorem, for example. (http://www.google.com/search?hl=en&q=Bayes+Theorem+site%...)

brianto2010
> I got up to geometry in high school (i.e., I know cosines in triangles but have no idea what a cosine wave is, how it works, and/or how they're related if at all).

Hmm... I guess these could work...

http://www.coolmath.com/algebra/index.html

http://www.purplemath.com

If you want video tutorials, this might work:

http://www.youtube.com/user/khanacademy

For a calculus resource based on analogies and stories, try this:

http://www.karlscalculus.org/

Trying to create an implementation of a lossy compression is a bit over-the-top, especially when you are trying to learn it through Wikipedia. Try a Google search and filter through those results. Maybe there will be a less technical, but still informative explanation. Also try Youtube!

http://www.youtube.com/watch?v=sckLJpjH5p8

(Haven't watched it all the way through)

Going from basic Trigonometry all the way to Discrete Fourier Transforms isn't exactly a 'overnight-hack' type of thing. Grokking everything will take a lot of time, if you do attempt it. Please don't hurt yourself.

kalid
Shameless self-promotion, but I write about math in an intuitive way here: http://betterexplained.com

I think the key is realizing that math is about manipulating a mental model of an idea. The wrong model can make certain tasks impossible -- for example, thinking that numbers must be 1-dimensional makes complex numbers a paradox. But when you see that they could represent items in 2 dimensions, you see how complex numbers represent a rotation. And from that, numbers can have arbitrary dimensions (vectors), and so on.

Whenever I study a new concept, I try to find the mental model behind it. For a book recommendation, I'm going through Visual Complex Analysis (http://www.usfca.edu/vca/) and find it extremely well written, with an intuitive approach.

drhowarddrfine
And betterexplained is one of the best sites on the web, too.
neovive
++ for betterexplained ... great website!!
kalid
@drhowarddrfine, @neovive: Thanks, it's nice hearing when people find it helpful :).
harpastum
I had never heard of this site before. All of these articles look really interesting! There goes my weekend.
kalid
Thanks -- ironically, I was planning on spending this weekend writing some more :)
kaens
I have pointed many of my friends at your site; it's a wonderful resource.

Thanks for taking the time to make betterexplained - the world is a better place with it existing.

kalid
Thanks for the kind words! It really means a lot knowing it's coming in useful.
mizhi
This site is awesome. It helped me unscrew my understanding of complex numbers.
kalid
Thanks :). Complex numbers befuddled me for a long time too.
[deleted]
(comment deleted)
earl
Actually, it's easy: time. Just put in the hours.

If you want specific books, you need to say what your current level is, which you kind of explained, but much more importantly, what exactly you want to do. While most math is useful, different fields have different bodies of math I'd learn first: machine learning vs stats vs algorithmic optimization vs 3d graphics vs fluid dynamics vs financial mathematics, etc. I'll try to watch this post and put up book recommendations if you are specific.

Also, the last thing to note is unlike most other subjects, math is cumulative. Calculus is really easy, and almost thing new, if you understand trigonometry and algebra. Seriously, only like 10% new material. The trick is math rewards really deeply understanding subjects and penalizes you for not doing so. Learning half the material but deeply understanding it is, IMO, more valuable than the reverse.

hotpockets
Google books could be your friend here. Sounds to me like you might benefit from a physics 101 type text, such as Halliday and Resnick. All these types of books will have a section on waves and their applications in the real world. Some, on fourier series too. Some of these books will have a preview mode on google books (though not Halliday & Resnick). Try to find the ones that do.
vivekkhurana
Dont worry, all good programmers I know have sucked at maths at one point or the other. Try sage http://www.sagemath.org/ , its a free software written in python. The software has a web interface which lets you program the equations and see visual output in form of graphs or other suitable figures.

Have a look at sage demo at http://wiki.sagemath.org/interact and try it after downloading. If you still have issues, write back, we would love to help you...

danbmil99
I learned DCT's without a good math background, so it can be done. I looked at sample code, and compared it to the equations. Eventually it started to make sense. I used Excel to work it all out graphically. Maybe I should post those spreadsheets...

BTW if you just want to write decoders, you're probably better off using a library and just think of it as magic. Why do you need to really understand it anyway? Is that some sort of fetish of yours?

kaens
Assuming you've been coding for a while, I'd recommend taking a look at Concrete Mathematics ( http://www.amazon.com/Concrete-Mathematics-Foundation-Comput... ). It's a challenging book, but it's extremely well-written, and you should be able to get through it with a minimal amount of other resources. I was able to work my way through it, and I have no background in formal education past the high-school level, let alone a background in maths.

This is, in my opinion, required reading on the same level as SICP for people who are serious programmers - at least those with a weak math background.

chriskelley
Depending where you live, check with a local college about extension courses.

I'd been trying for ages to advance my math skills through solo-study, but I always ended up with too many unanswered questions. Very recently I finally bit the bullet and enrolled in some extension courses at UCLA, and I'm loving (almost) every second of it and learning leaps and bounds more than I was on my own, at a minimal cost. If you've got the time and the money, I give it my vote.

Everyone has a different style for learning, but like others have mentioned, study study study.

krepsj
I would quote Stevey's Blog here. It says: "The right way to learn math is breadth-first, not depth-first. You need to survey the space, learn the names of things, figure out what's what."

It comes from his own experience. He decided to renew his math skills and declared plan "Math every day". After 15 months he posted another article about results of this experiment. You can read both articles as they are inspirating in many respects.

About his motivation: http://steve.yegge.googlepages.com/math-every-day

and about his results: http://steve-yegge.blogspot.com/2006/03/math-for-programmers...

I would add that you should really enjoy math. If you only want to quickly understand one particular thing and seek for some miraculous explanation, this approach will surely not work for you.

markessien
You're doing it wrong. It's like trying to clean up a messy warehouse - if you walk in and start picking up things, you'll despair very quickly, because the task seems to massive and complex. If however, you simply pick up a tiny corner, and clean it up only, then move to the next corner and so on, the task becomes manageable.

Address this problem the same way - start implementing, and deal with each problem individually. For example, you may want to start with loading a bitmap and breaking it up into a matrix. Make sure that works first, test it also. Then do the quant comparisons or whatever else.

Break the problem into chunks and then only face one chunk while ignoring the rest. Do this in the context of actually working on something with a result, and you'll find that things become way easier and more interesting.

Trying to start by learning several years of mathematical theory is an exercise in frustration.

davo11
Try a maths for engineers course at your local Uni. If you don't want to or can't enrol, you could probably just sneak in up the back and take notes, I've done this in the past for courses I was just curious about, lecturers usually don't mind. often they enjoy it - someone who actually wants to learn!

There may be something of interest here too, this is the link to Itunes U in ITunes,

http://deimos3.apple.com/indigo/main/main.xml

jlouis
Another point worth mentioning: Math in general is heavily interconnected. The more math you do, the better you become at it. A large amount of math is knowledge. That mess of symbols is something you will have to understand. As soon as you understand what it means, everything is just trivial.

My advice would be to start with some books you can understand and then build up there towards your goal. It may take a couple of years, but the time used are going to be paid back in masses when you hit other math problems: They suddenly got easier too!