I would really like the opportunity to get into RL. At work my team uses GANs and RNN/LSTMs almost exclusively. Any suggestions for specific RL online classes at Coursera, eDX, or Stanford?
I especially like that this framework is set up for experimenting with four popular models and the means to measure how well your models are doing.
I came across this a few days ago:
For Reinforcement Learning specifically, the standard text is Reinforcement Learning: An Introduction[1]. Dave's UCL Course on RL[2] is great too (playlist of all lectures)[3].
RL is really really brittle, i.e. one simple bug that wouldn't matter in GAN/LSTM (such as e.g. adding one more or less number than needed) would cause your RL algorithm not to converge and you'd spend days-weeks debugging it - it requires extreme discipline verifying every single step of your algorithm. Often you don't know if your model didn't converge because it is not strong enough or if there was some tiny little error/wrong assumption leading to divergence.
Yes, some types of NNs are resilient to errors in implementation.
I hired David Rumelhart as a consultant in the 1980s (when he was still at UCSD) when I was writing SAIC’s neural network product for Windows. He warned me that he was used to seeing projects with big implementation errors that still worked very well.
I do find GANs brittle. I use them for generating numeric/categorical spreadsheet data, not for images.
> He warned me that he was used to seeing projects with big implementation errors that still worked very well.
For any sufficiently complex function, if you tune the constants enough, the wrong function will sometimes approximate the results the correct one would give if it were set up with the correct constants.
15 comments
[ 3.2 ms ] story [ 45.5 ms ] threadI especially like that this framework is set up for experimenting with four popular models and the means to measure how well your models are doing.
Source: Julian Schrittwieser works on Deepmind at Google http://www.furidamu.org/
[1]http://incompleteideas.net/book/the-book-2nd.html
[2]http://www0.cs.ucl.ac.uk/staff/d.silver/web/Teaching.html
[3]https://www.youtube.com/watch?v=2pWv7GOvuf0&list=PL7-jPKtc4r...
- "An Introduction to Deep Reinforcement Learning" by Vincent François-Lavet et al (https://arxiv.org/pdf/1811.12560.pdf)
- "A (Long) Peek into Reinforcement Learning" by Lilian Weng (https://lilianweng.github.io/lil-log/2018/02/19/a-long-peek-...)
- "Deep Reinforcement Learning: Pong from Pixels" from Andrej Karpathy (https://karpathy.github.io/2016/05/31/rl/)
Those are the basics. Some more resources listed on this post: https://news.ycombinator.com/item?id=18219620
This goes over the theory behind RL mostly, so if you are looking for practical implementations you might want to find other resources.
I hired David Rumelhart as a consultant in the 1980s (when he was still at UCSD) when I was writing SAIC’s neural network product for Windows. He warned me that he was used to seeing projects with big implementation errors that still worked very well.
I do find GANs brittle. I use them for generating numeric/categorical spreadsheet data, not for images.
For any sufficiently complex function, if you tune the constants enough, the wrong function will sometimes approximate the results the correct one would give if it were set up with the correct constants.
I’m interested to hear more about this. Care to elaborate?
[1] http://rail.eecs.berkeley.edu/deeprlcourse/