HN Explorer

Show HN: A batched, concurrent wrapper for OpenAI gym library

12 points by MutatedFlood ↗ HN
Hello HN, new user here, so please let me know if I break some rules.

Currently I've been working on training reinforcement learning agents, and OpenAI gym, while is great, runs only one agent at a time. Hence I decided to extend it.

I built a wrapper around OpenAI gym, such that it now runs several environments concurrently. All while (mostly) having the same call signature as OpenAI gym. And it is published to PyPI for anyone interested.

For more details, please visit: https://github.com/Chimpan-Z/agymc

Feedback really appreciated! Have a good day everyone!

5 comments

[ 3.6 ms ] story [ 20.5 ms ] thread
mooneater
Is this different than VecEnv in baselines?

https://github.com/openai/baselines/blob/master/baselines/co...

MutatedFlood
Thanks for the feedback.

OpenAI said they used multiple processes (it seems that they used multiprocessing module from your link). Processes is handled by OS and, it depends, sometimes will only switch to another process on long CPU stalls. I used an event loop to manage that, which supposedly should be fixed those long stalls.

Also, this version runs on a single CPU while OpenAI's probably won't, given a multicore CPU. My initial thought was that extending using multiprocessing is much easier than wrapping it using an event loop, hence I only implemented the event loop part.

wookayin
When simulation is CPU-heavy, it will never scale. Though I admit that event-based concurrency might work better in many cases (like where communication overhead is greater than computation), but in some environments you would need real parallelism using many CPU cores. This library also lacks benchmark.
MutatedFlood
You're right. I plan to add multiprocessing support such that there would be numCPU processes, each running one event loop. Which should scale properly.

What kinds of benchmark do you suggest? I'll do my best to provide one.

Thanks for the feedback.