Ask HN: What does it really mean to do ML/DL at the hobbyist/research levels?
I liked the courses where math was a huge focus because it allowed me to understand from first principles.
When i took the Udacity course it was just learning the tensorflow library for the most of it. Conceptually i understood CNNs and RNNs but i couldn't implement it from scratch because i didn't truly understand it. I could maybe implement the forward propogation for RNNs but didn't understand the nuance of the backprop.
I ended up basically just understanding the tensorflow library, which is still useful and powerful. But that left me wondering why do people who do this for research and hobby really do?
I highly doubt researchers are spending their time re-inventing the wheel to implement their own basic CNN or RNN model, but they obviously are doing more than just function calls to a set library right?
What should i be trying to learn if i am a hobbyist wanting to really understand ML/DL? Is implementing theses advanced DL models from scratch even useful? What steps do i take to become "better" at ML/DL rather than just following the tutorials and courses that taught me how to use the tensorflow library to create basic RNNs/GANs?
When people say they are "better" at ML/DL than they were before does that just mean they have a better intuition for what model, how many layers, how much dropout, batch sizes, etc to use for a certain problem?
I feel lame just piecing together models with high level code like `keras.add_fully_connected(layer_size)` or `tf.MultiCellRNN`. It feels like cheating because i'm just throwing things together and seeing what works. The science aspect feels lost.
2 comments
[ 2.9 ms ] story [ 21.5 ms ] threadFirst thing you need to understand is that data is king. To use DL models you need lots of data. If you don't have lots of data, you will probably have more success with older ML models.
Hardest part of any DL/ML project is getting/cleaning/augmenting data. In the real world, dataset preparation will consume >75% of your time/effort. Size and quality of your dataset will affect the resulting accuracy more than a choice of a particular model.
Once you have data, you can just try several models which have been used for your task, and see if size of model makes a difference. Language or frameworks is not that important, but most people use Python and Tensorflow or Pytorch. Again, does not matter. Of course you need to understand the basics. For example backprop in RNN is tedious to trace, but you should be able to do it, if necessary: it's no more complicated than linear circuit analysis, or designing a FSM. There are no "nuances", just apply the chain rule backwards, and think in terms of unrolled computation graph. You rarely need to think about low level details like that, but occasionally you might need to use an operation which is not differentiable, or does not have gradient implemented in the framework you're using.
If you want to get "better" at ML/DL, go to Kaggle, and try to solve problems. Read about winning solutions. Read relevant papers. Read code. Try things. Eventually you'll get it.
1. Go very technical, try to win kaggle competitions, try to create the very best solutions possible. This probably means learning all the cutting edge techniques, being very up-to-date with what's going on in the algo space, and lots of practice/experience, and you probably have to have a knack/passion for this kind of competition as well.
2. Go very domain/business driven - force yourself to optimize thinking about high impact business questions that you'll be answering with data. If you can craft a strong answer that leads to high level business strategy, it doesn't matter what techniques you used, people value that. This kind of person is not ML centered, they are more of a stats person with some tensorflow mixed in.
Generally speaking, Google and AI/tech companies prefer #1, non tech companies prefers #2.
Most people who want to get into data science do it for the exact same reasons you do - it's math-y and new. Other types of programming are old, not so math-y, boring. However, data science at a business level is completely different from data science at a hobby level because doing data science is very expensive. It requires much more capital and infrastructure than regular programming. Definitely more expensive than developing hardware in my opinion, especially if you look at data collection costs, data scientist salaries, and how much work is needed to extract meaningful business value out of data.
A hobby level data scientist is probably similar to a hardware or robotics hobbyist in my opinion - they both play with fun tools and spend money/time but have difficulty translating it into real businesses/profession due to the high capital cost needed to even access the really good tooling, let alone do something at a competitive level with those tools.