Ask HN: What math should I know well to be able to learn ML
I studied math and computer science 20++ years ago on university of mathematics where I had 20 different math subjects but in my last 15 years of work I forgot most of it since I don't use any more advanced math on my current job as software architect where I deal with automation, development, system design, operating systems, integrations etc.
I am very experienced in many different IT domains but I would like to attack this problem in more deep way then just starting immediately with online courses or tutorials (but not so deep that it consumes all my free time, nor would I want it, I have many different hobbies and interest). My end goal is to know to apply ML methods and understand enough of it to improve/troubleshot anything that doesn't fit immediately the typical use case.
13 comments
[ 2.7 ms ] story [ 24.7 ms ] threadAlso, have a look here: https://github.com/off99555/machine-learning-curriculum
I don't know what level you're at right now, I was pretty far behind and had to spend a good deal of time with the basics to make sure I really understood them i.e. algebra, trigonometry and calculus and proofs, etc.
After that though I'd say the following are must know areas:
1. Linear Algebra
2. Probability Theory
3. Statistics
4. Optimization Theory
5. Signal Processing (Information Theory) Fourier Transforms, etc.
But the good thing is once you understand those fields pretty well, applying the methods and solving problems using them is really not that difficult.
This sounds awesome and fast. I'm curious to know:
1) What background did you have at the start?
2) What resources did you find most valuable, and how did you use them? (e.g. did you just get a book per topic, and work through chapter-by-chapter, doing the exercises?)
3) Did you get help from friends/colleages, or were you able to work through all the materials independently?
Jumping in deep can really suck your time as you need to brush up on lots of fields. Also it will be sometime before you have anything meaningful to show, which can be quite frustrating.
May I suggest a case by case approach wherein you take a problem and try to apply ml kit like scikit-learn to see which algorithm produces the results you desire and trying to learn why it does and how?
You can find all the gaps in your knowledge as you learn and fill them on a need by need basis. This is from someone who tried real hard to do it the bottom to top way and found it frustrating to learn with a full time job and family. Now I am going to try the top to bottom way and focus on one niche. Bottom to top approach works better when you are at school where you have specific assessments to measure progress and the end of semester to signal the completion of each course/subject. Since I have just started this alternative approach, I do not have an exact map to propose.
If someone has walked this path, any inputs would be most welcome.
However, I would also like to renew math because of its general effects on the brain and because its hard. Without stressor, there is no improvement. I wanted a long time to devote few hours a week to study general math but if I can learn something that has usage in ML along the way, I consider it a double win.
For example, Fourier transformation and wavelates were very handy for me in understanding how certain DSP processes work while I had personal audio workstation and was doing digital composition most of my time.
-Multivariate Calculus
-Statistics & Probability (with Calculus)
-Linear Algebra
-Optimizations
as a baseline. You'll need more as you progress.
You want to be comfortable with the math review in: http://www.deeplearningbook.org/
I'd wager to say most people on HN would drop out of an actual graduate level course on ML. These courses (from my experience) are more research oriented and less about "hacking".
---
1.1 Who Should Read This Book?
This book can be useful for a variety of readers, but we wrote it with two main target audiences in mind. One of these target audiences is university students(undergraduate or graduate) learning about machine learning, including those who are beginning a career in deep learning and artificial intelligence research. The other target audience is software engineers who do not have a machine learning or statistics background, but want to rapidly acquire one and begin using deep learning in their product or platform.
--
For a rigorous learning plan: http://web.archive.org/web/20101102120728/http://measuringme..., don't forget to read the section "Straight into the Deep End". See related discussions here: https://news.ycombinator.com/item?id=1187237 and https://news.ycombinator.com/item?id=1055042.
I am taking a middle of the path approach. Working through the following in order: 1. Linear Algebra 2. Calculus 3. Probability & Stats 4. Optimization Theory. For 1 and 2, using Manga Guides. Working through Manga guide for LA now. Note that I have a 3D graphics background and learning ML just to add to my skill set, so my plan is very specific to my use case. YMMV. Hope this helps a bit. Thanks.
The math for things that work (these days) ^(TM) is quite boring and would take a quarter of lectures (see any of a number of MOOCs). Some of it is quite tricky like convex optimization and graphical models, but you'll likely never have to know these details unless you're in academia. The latter have also fallen out of popularity, so you know... less things to know. Convex optimization these days is all about stochastic-distributed things, which is probably not the ML you (or most "data scientists") have in mind.
Best thing to do would be to work through the homeworks of some course offering at CMU/Stanford/UW. The principle is generally simple, but you'll have to wade through the symbols, and implement something to make sure you understand the symbols. This is probably the best exercise if you want to understand things.
You could also just look at and modify the code in scikits, since this will save you a lot of work with regards to the engineering bit. Kaggle might be help, but I found them mind-numbingly boring (usually with leaky datasets), which have to be hacked for performance with endless tweaking on XGboost.
To be honest, I really think ML is overhyped (it is also quite boring as a job). If you want to do math, you should be doing math.