I wish HN had a parallel community for these kind of silly posts. I use it as a worksafe site to take a break at work and sometimes jokes like this, while not being in the spirit of HN, are things I very much appreciate.
Levity does seem inconsistently received, not least because it so often relies on shared cultural understanding. I was somewhat confounded by this until realising that timezone matters. Nevertheless I don’t think puns and wry observation are entirely out of place, and even surreal irony or absurdist tales can go over quite well; sometimes very well if they convey a parable.
Satire and parody are probably the hardest to pull off. They’re often at someone’s expense, and that is frowned upon; even when they’re not intentionally so, someone with different points of reference may read it as such.
Slightly crestfallen that the “ML” here is machine learning and not the programming language. I still refer to my vintage paperback of L. C. Paulson’s
ML for the Working Programmer from time to time.
Neuro-linguistic programming definitely had its day. It seems to have fallen out of fashion in the last 3 decades (it was really just a phenomenon in the 80s -- I remember those days) so it's quite likely that modern references to NLP don't refer to it.
I usually read NLP as "Nonlinear Programming" (nonlinear optimization) which is the community I come from. This acronym is not widely used outside the community so if I'm not reading the optimization literature, I'm pretty sure NLP doesn't refer to it.
Natural language processing is more in vogue these days, so that tends to be my default reading. The term itself seems to have existed for decades, so it's not like it came after the others but this is the most likely reading today.
Oh, for a second I thought the GP was referring to nonlinear programming. Then I remembered this is an ML thread.
There are some nonzero number of papers referring to NLP (as in nonlinear programming) in ML, mostly for the purposes of constrained optimization, but I agree with your current breakdown.
For people w/ a physics background, I would still recommend https://www.inference.org.uk/itprnn/book.pdf. Some of it is a bit obsolete, but then DL made a lot of stuff around generalization/overfitting somehow obsolete. It makes a lot of connection between different kind of approaches in ML, information theory, (Bayesian) statistics, and physics.
It is not a very good book if you only care about applications (in which case the Keras book, for beginner, or fastai/etc. are much more appropriate).
David MacKay was an absolute rockstar and this book is grossly underrated among beginners in machine learning. This should be THE complementary reference for anyone who uses the Bishop book. Both these books follow a philosophy which some people may not completely agree with, that from the "church of Bayes".
My guess is that information theory went through its hype phase and much of the ideas are so pervasive across real systems that people forget how important those connections are.
To tell you its importance, skim this work on information-theoretic probing [1]. I find this so satisfying. Its most famous alternative, linear probing, always felt inelegant. This paper experimentally shows how terribly linear probing fails.
> I see modelling is almost always only open to phds/masters.
I think this varies pretty widely based on employer, e.g. if you are at a smaller company (and you show interest and have the necessary skills) then you're much more likely to be able to contribute on the modeling side. It's easier to get there if you have an advanced degree, but definitely not necessary.
That being said, IMHO book lists like this aren't very useful because there's no incentive to keep them short and realistic. Reading seminal papers and implementing them is a different learning philosophy, maybe, but lists like this are probably more feasible to complete: https://dennybritz.com/blog/deep-learning-most-important-ide...
the field is getting pretty crowded. there are probably better things to do at this point for pure career advancement and money, for the amount of effort you would have to spend to get viable professional-level skills. i haven't touched the job market recently though.
however, it remains intrinsically very interesting and can be fun to learn about. more so than most resume items.
20 comments
[ 4.8 ms ] story [ 17.5 ms ] thread[0] https://marxistleninist.wordpress.com/2005/01/20/study-guide...
I wish HN had a parallel community for these kind of silly posts. I use it as a worksafe site to take a break at work and sometimes jokes like this, while not being in the spirit of HN, are things I very much appreciate.
Satire and parody are probably the hardest to pull off. They’re often at someone’s expense, and that is frowned upon; even when they’re not intentionally so, someone with different points of reference may read it as such.
Neuro-linguistic programming definitely had its day. It seems to have fallen out of fashion in the last 3 decades (it was really just a phenomenon in the 80s -- I remember those days) so it's quite likely that modern references to NLP don't refer to it.
I usually read NLP as "Nonlinear Programming" (nonlinear optimization) which is the community I come from. This acronym is not widely used outside the community so if I'm not reading the optimization literature, I'm pretty sure NLP doesn't refer to it.
Natural language processing is more in vogue these days, so that tends to be my default reading. The term itself seems to have existed for decades, so it's not like it came after the others but this is the most likely reading today.
There are some nonzero number of papers referring to NLP (as in nonlinear programming) in ML, mostly for the purposes of constrained optimization, but I agree with your current breakdown.
Speech and Language Processing From Dan Jurafsky,
Available at: https://web.stanford.edu/~jurafsky/slp3/
For people w/ a physics background, I would still recommend https://www.inference.org.uk/itprnn/book.pdf. Some of it is a bit obsolete, but then DL made a lot of stuff around generalization/overfitting somehow obsolete. It makes a lot of connection between different kind of approaches in ML, information theory, (Bayesian) statistics, and physics.
It is not a very good book if you only care about applications (in which case the Keras book, for beginner, or fastai/etc. are much more appropriate).
My guess is that information theory went through its hype phase and much of the ideas are so pervasive across real systems that people forget how important those connections are.
To tell you its importance, skim this work on information-theoretic probing [1]. I find this so satisfying. Its most famous alternative, linear probing, always felt inelegant. This paper experimentally shows how terribly linear probing fails.
[1] https://arxiv.org/abs/2003.12298
So is studying all that stuff just for being a MLE/ data engineer worth it, if you are already a software developer (full stack)?
I think this varies pretty widely based on employer, e.g. if you are at a smaller company (and you show interest and have the necessary skills) then you're much more likely to be able to contribute on the modeling side. It's easier to get there if you have an advanced degree, but definitely not necessary.
That being said, IMHO book lists like this aren't very useful because there's no incentive to keep them short and realistic. Reading seminal papers and implementing them is a different learning philosophy, maybe, but lists like this are probably more feasible to complete: https://dennybritz.com/blog/deep-learning-most-important-ide...
however, it remains intrinsically very interesting and can be fun to learn about. more so than most resume items.