HN Explorer

Ask HN: Is it important to have sharp math knowledge for deep learning?

3 points by thiago_fm ↗ HN
So I've been learning deep learning from Andrew Ng courses and it's great. I actually remember a lot of algebra and calculus, as I used to be a TA in uni, but almost 10 years have already passed and I'm not that sharp anymore. For instance, I have to frequently review how to transpose a matrix, review derivate tables etc.

Is it worth to really review everything and do people that do deep learning professionally have always amazing knowledge about math, or is it normal to dig it up when you need it?

I could buy some books and go through them, but I'm not sure if it's worth the time given that I can invest on learning all the ML/DL things I don't know, which are many.

3 comments

[ 2.7 ms ] story [ 17.9 ms ] thread
codingslave
Depends, itll probably help to get hired. If you are in a highly applied position, probably better to know different network architectures, and why they work in specific ways. For instance, Restnet versus VGG16, what was the innovation? Which optimizer might be better for the two? When to slow learning rate, when to speed it up? Theres an art to tuning and getting neural networks to work that goes beyond mathematical understanding. But some positions may require their employees to understand the archaic math behind machine learning, down PCA and its relationship to eigenvectors, methods of non differentiable optimization, etc
sgillen
My honest advice. If you don’t already have a really good understanding of the math I recommend you focus on the more applied stuff (like everything coding sleeve mentioned). I fear many of the roles that actually would require you to review the math might be looking for PhDs or similar.

You probably should at least know how to do the basic stuff in numpy or whatever though.

thiago_fm
Alright, that I can do. I meant more, that there is a big difference between knowing everything by heart and comprehend everything easily and having to look it up and stare at some formula a bit. Math is a rabbit hole, like, you can do integrals, then line integrals, triple integrals and it goes on.

For me it isn't also very clear until where it is useful to learn math.

I'm also more on the practical/software engineering side, so I don't even have intentions of doing research. I just want to follow the most up to date methods.