HN Explorer

Ask HN: As a professional, does it worth it to learn the math behind NN?

6 points by IWantToRelocate ↗ HN
In 2021, as a (junior) professional working directly with ML/AI, does it worth it to learn what's going on in terms of mathematics behind Neural Networks? Or should we just focus on the pratical side of things, like daily applications using frameworks/libraries and metrics/performance?

17 comments

[ 3.0 ms ] story [ 53.1 ms ] thread
PaulHoule
How much do you know already?
IWantToRelocate
Well, not much. Data isn't the main product where I work for, so I end up just using python libraries for some mundane applications and small datasets (image classification and recommender systems). But I'm the "need to know from scratch" kinda person and I wonder if I should waste my time learning the math behind it or if its useless
PaulHoule
What's the first derivative of

   x * cos(x)

?
IWantToRelocate
wolfram alpha told me its: cos(x) - xsin(x) lol kidding, i had calculus 15 years ago. so it sounds I should recap derivatives.
PaulHoule
This book is a little old but you should learn whatever math it takes to understand it

https://www.amazon.com/Networks-Recognition-Advanced-Econome...

Another oldie-but-goodie is

https://www.amazon.com/Neural-Networks-Lecture-Computer-Scie...

which tells the secret of when to stop when you're doing "early stopping", something I've seen many modern deep learners fail to get right.

Off the top of my head I would say the fundamental math about deep networks is not really new. Most of the work in that field is pretty ad-hoc and not a lot is proven; probably people that are proving things are using difficult graduate-level math but you don't need to go there.

IWantToRelocate
thank you Paul for the references, will def take a look mainly in the early stopping problem.
andyxor
there is not much math, don't let the academic papers razzle dazzle you, they make things look complicated to get their grants

in practice, as long as you've studied basic calculus and understand how to find a minimum of a function via derivative you're good, there is your "gradient descent" in a nutshell: https://www.mathsisfun.com/calculus/maxima-minima.html

everything else is plug-and-play from existing libraries

you can ask any "data scientist" or "ML engineer" what they do all day, it's a whole lot of copy paste, and tweaking the data and parameters through trial and error until it fits

Edit: Ok , it would also help to understand dimensionality reduction via PCA/SVD at least once, it's available in any linear algebra book: https://en.wikipedia.org/wiki/Singular_value_decomposition , https://en.wikipedia.org/wiki/Principal_component_analysis that's probably the best and most "scientific" part of ML

IWantToRelocate
Thank you! appreciate your reply, honestly. i have an engineering degree but it has nothing to do with computers, so i kinda struggle sometimes. I basically want to get fluent with the metrics/performance when applying ML/NN, when to tweak what, how to improve some stuff, what algorithm works best for some of the problems, etc.
aborsy
This, in a nutshell.
mindcrime
My personal feeling is that it's always "worth it" to learn more maths. Not saying it's absolutely required, and I'm a big fan of (generally speaking) prioritizing the pragmatic / practical / hands-on over the purely theoretical. But, even within that belief system, I would say that - all things being equal - any opportunity to learn more maths is worthwhile, at least up to a certain point. Figuring out the details of the "certain point" is left as an exercise for the reader though.
aborsy
Frankly, I rarely see this complicated math in academic papers to be useful in practice.

Most of the stuff that is valuable in practice does not require much math.

Many people believe math actually hindered the development of AI.

wnkrshm
The papers are interesting to people making tools - if you can see what mathematical assumptions are made you can see what limitations a method has and whether it can apply to cases you're interested in.
jstx1
Since you're working in the field - yes, obviously. It's not even that much math - the building blocks are fairly straightforward linear algebra and calculus.

Here's a nice, fairly detailed, summary - https://explained.ai/matrix-calculus/. But if you haven't taken undergraduate level single/multivariable calculus and linear algebra, I would take them from MIT Opencourseware even if it's just to fight off the impostor syndrome a little bit.

ozzythecat
I recommend learning multi variable calculus and linear algebra. You can take courses online for free. Personally, I didn’t really understand what was going on until I refreshed my math.

It won’t make you an ML expert, but it’s like the difference between knowing how to write a sort function vs. only knowing how to call a sort function. Ideally, you understand both. Not saying you need to understand how every sort function works internally and write a proof for it… but a basic understanding is helpful.