> In US regions, each K80 GPU attached to a VM is priced at $0.45 per hour while each P100 costs $1.46 per hour.
The $300 free tier gets you ~600 hours of K80. The spinning up guide suggests iterating models in <5 min, so that's 7200 iterations.
> start with vanilla policy gradient (also called REINFORCE), DQN, A2C (the synchronous version of A3C), PPO (the variant with the clipped objective), and DDPG, ... VPG...
that's 6 algorithms, combined with a half a dozen tasks to try, whittles it down to a few hundred iterations per task/algo combo.
that, combined with a lot of paper-reading, and perhaps clever blogging is probably enough to get started.
Still, it seems beneficial to democratize DL by making these 5 minute iterations free, doesn't it?
The guide doesn't assume access to specialized hardware. Experiments and iterations with all of those algorithms can be done on a normal CPU that anyone has. :)
Quote:
"Iterate fast in simple environments. To debug your implementations, try them with simple environments where learning should happen quickly, like CartPole-v0, InvertedPendulum-v0, FrozenLake-v0, and HalfCheetah-v2 (with a short time horizon—only 100 or 250 steps instead of the full 1000) from the OpenAI Gym. Don’t try to run an algorithm in Atari or a complex Humanoid environment if you haven’t first verified that it works on the simplest possible toy task. Your ideal experiment turnaround-time at the debug stage is <5 minutes (on your local machine) or slightly longer but not much. These small-scale experiments don’t require any special hardware, and can be run without too much trouble on CPUs."
I appreciate a lot they are thinking of beyond-human runs these days, for example just put a few agents into big, number-crunching climate change models and see if they are able to “win” (keeping planet Earth ecosystem on this side of divergence), how (for example, beating the one or two or more most critical opponents) and at what cost for implementable actions for mankind (how many resources are needed?). Basically, treat these scenarios as active strategy games with rewards and a final goal instead of passive, parametric Montecarlo runs a la think tanks or supranational agencies?
DRL has definitely shown some excellent results but can someone in the field comment on this paper, Simple random search provides a competitive approach to reinforcement learning [1].
Not a comment on that paper, but for those interested I would highly recommend Ben Recht's (ont of the authors of that paper) blog series at http://www.argmin.net/2018/06/25/outsider-rl/.
12 comments
[ 4.4 ms ] story [ 42.8 ms ] threadhttps://cloud.google.com/blog/products/gcp/new-lower-prices-...
> In US regions, each K80 GPU attached to a VM is priced at $0.45 per hour while each P100 costs $1.46 per hour.
The $300 free tier gets you ~600 hours of K80. The spinning up guide suggests iterating models in <5 min, so that's 7200 iterations.
> start with vanilla policy gradient (also called REINFORCE), DQN, A2C (the synchronous version of A3C), PPO (the variant with the clipped objective), and DDPG, ... VPG...
that's 6 algorithms, combined with a half a dozen tasks to try, whittles it down to a few hundred iterations per task/algo combo.
that, combined with a lot of paper-reading, and perhaps clever blogging is probably enough to get started.
Still, it seems beneficial to democratize DL by making these 5 minute iterations free, doesn't it?
You can attach a GPU for free, and, if I recall, even a TPU. See https://colab.research.google.com/notebooks/gpu.ipynb
Quote:
"Iterate fast in simple environments. To debug your implementations, try them with simple environments where learning should happen quickly, like CartPole-v0, InvertedPendulum-v0, FrozenLake-v0, and HalfCheetah-v2 (with a short time horizon—only 100 or 250 steps instead of the full 1000) from the OpenAI Gym. Don’t try to run an algorithm in Atari or a complex Humanoid environment if you haven’t first verified that it works on the simplest possible toy task. Your ideal experiment turnaround-time at the debug stage is <5 minutes (on your local machine) or slightly longer but not much. These small-scale experiments don’t require any special hardware, and can be run without too much trouble on CPUs."
Neural Episodic Control https://news.ycombinator.com/item?id=13843282
Exploration by Random Network Distillation https://news.ycombinator.com/item?id=18346943
Evolution Strategies as a Scalable Alternative to Reinforcement Learning https://news.ycombinator.com/item?id=13953980
Recurrent World Models Facilitate Policy Evolution https://news.ycombinator.com/item?id=16860247
Playing Atari with Deep Reinforcement Learning https://news.ycombinator.com/item?id=8484313
[1] https://arxiv.org/abs/1803.07055
I maintain one for the Ruby programming language https://rubybib.org/.