HN Explorer

Ask HN: What to study to become a machine learning/ai master?

24 points by supremus_58 ↗ HN
I dropped out of school a few years ago and have been working as a developer. I plan to start attending school again in the coming fall. My question: what is the best route of study to become a master of all things machine learning? My plan is to go in as a double major for math and computer science as I was before. Which topics in mathematics are most prevalent for machine learning and more specifically neural networks?

11 comments

[ 2.7 ms ] story [ 64.8 ms ] thread
cr0sh
Probability/Stats, Calculus, Linear Algebra
supremus_58
this is the math necessary for most comp-sci majors. specifically to machine learning, which topics in mathematics are most relative to be pursued? what pathway should a math major take to have the most relative knowledge to pursue ML
[deleted]
(comment deleted)
LeanderK
i am a bit confused. What do you mean with "school"? If you really want to become an "ML/AI master" this is more complicated then you seem to think.

If you mean university it will take some time (and rightly so) until you can choose your courses. ML is rapidly changing, so you can't just learn "one trick", you need to be able to adapt quickly and learn new techniques (this can be really fundamental compared to other branches like programming). Also most of the concepts are hard and you can't (or it's difficult to) learn them without some theoretical background (for example a bachelor in math/physics/engineering). Of course you can assemble something in TF based on a quick MOOC, but this doesn't make you an "ML/AI master", just somebody who can use a library and who will meet his boundaries quickly.

The usual requirements are Probability/Stats, Calculus, Linear Algebra. To really scratch the surface you need to take about 2 or 3 semester of courses. When i was in school i didn't imagine it to be this challenging. I also didn't imagine it to be so rewarding, but i know that i would be unable to pass them if i were studying alone.

There are always exception to the rule and i only started diving into ML recently, so take this advice with i grain of salt. But just because you taught yourself programming it doesn't mean you can do the same with ML/AI that quickly and also alone. Take it seriously and don't expect to get results too quickly. But each step is really rewarding, so just start to study and see where it takes you.

supremus_58
sorry that the question came out so ambiguous. yes, by school i mean uni. the question may have better been asked as "what is the best route to take for a mathematics student to learn the most relevant ideas in ML?" i've take all the classes which you've listed, i dropped school somewhere halfway through. I know how to write code for ML, this isn't totally new to me.(by this i mean i know how to use the python ml library) i ask topically, what math is most important in the further development of ai?(ie complex analysis, combinatorics, graph theory, stochastic methods are all things important in ML, what other math is worth noting?) the math you listed are simply things required for most comp-sci majors.
LeanderK
ah ok. I am also starting with ML, so take this with a grain of salt.

While the courses i listed are required for most CS majors, i think they are more important for ML than many other fields of CS (of course there are exceptions). I think a deeper understanding of LA, Calculus and Numeric compared to normal CS-majors is beneficial (but if you majored in math you probably had to take these in more depth than most CS-majors). Also i would double down on Probability (so a course on Probability-Theory, Markov-Chains etc.). Optimisation Theory is also important, but often already somewhat covered in the other courses. I don't think one can point to a specific course that will be really important for NN in the future, nobody really knows whats coming. Just learn as much math as possible, so that you are prepared to learn and understand all the future theory and techniques that are not yet discovered. Maybe we will all be solving stochastic differential equations in the future, because they accurately predict the performance of various NNs?

Edit: I forgot tensor-mathematics, i haven't taken a course yet, but it seems pretty straightforward.

supremus_58
awesome, thank you! I took linear an analytic linear algebra class some years ago, but admittedly don't remember too much. I'm beginning to study more in depth matrix analysis, this and combinatorics are useful for understanding fundamentals of many algorithms. I've learned a little of Markov Decision Process, but i've been told by people working in the field that MDP isn't used much for neural net anymore, though this doesn't mean it shouldn't be learned. I'll definitely look into tensors.

here's a link i found looking into the subject: http://datascience.ibm.com/blog/the-mathematics-of-machine-l...

and here's some recommendations for math/ml texts i got in another thread that may be of use: https://news.ycombinator.com/item?id=13275031

LeanderK
i was also told that Markov-chains are not used for NN anymore, but i learn them because often they get combined with NNs and used for other aspects of the problem (if i remember right AlphaGo used them beside NNs).

the blog article was useful, thank you! A problem with most of the recommendations is that they are targeted at software engineers wanting to learn some ML. They mostly don't need to understand back propagation fully and don't have that complex of a problem. So the requirements are the absolut basics (what are vectors/matrices and how to multiply them). Also their math-background is either non-existent or a long time ago. I am a student and constantly learning math, so i am in a different position. I am also interested in really understanding NN, including recent papers. They vary, some being not very math-heavy (the dropout paper) and some being the opposite (bayes neural network, only skimmed some papers). Many paper introduce a new idea borrowed from other branches of math, so you can't specialise too much. A broad understanding is important. Also we still lack a unifying theory, so we definitely don't know what math will become really important in the future.

oumua_don17
For a rigorous learning plan: http://web.archive.org/web/20101102120728/http://measuringme..., don't forget to read the section "Straight into the Deep End".

See related discussions here: https://news.ycombinator.com/item?id=1187237 and https://news.ycombinator.com/item?id=1055042.

I am talking a middle of the path approach. Working through the following in order: 1. Linear Algebra 2. Calculus 3. Probability & Stats 4. Optimization Theory. For 1 and 2, using Manga Guides. Working through Manga guide for LA now. Note that I have a 3D graphics background and learning ML just to add to my skill set, so my plan is very specific to my use case. YMMV. Hope this helps a bit. Thanks.

hackermailman
Obv you would want to talk to whoever your advisor is or a professor in the ML department at your school. Otherwise, find existing books/courses and see what you'll need for background such as this free book: http://neuralnetworksanddeeplearning.com/

http://www.cs.cmu.edu/~mgormley/courses/10701-f16/schedule.h... (general grad level intro)

http://www.cs.cmu.edu/~bapoczos/Classes/ML10715_2015Fall/ (in depth grad level intro, both include a youtube lecture playlist).

The requirements listed here http://www.cs.cmu.edu/~bapoczos/Classes/ML10715_2015Fall/faq...

  -Basic probability and statistics are a plus.

  -Basic linear algebra (matrices, vectors, eigenvalues) is a plus. Knowing functional analysis would be great but not required.

  -Ability to write code that exceeds 'Hello World’. Preferably beyond Matlab or R.

  -Basic knowledge of optimization. Having attended a convex optimization class would be great but the recitations will cover this.