The main highlight is a new version of AllReduce native to TensorFlow. This, like Horovod, builds a ring and has both native and NCCL implementations. Early results (albeit on a single host) show that it outperforms Horovod:
https://groups.google.com/a/tensorflow.org/forum/#!topic/dis...
Somewhat off topic: I'm rediscovering the dance required to set up a new Windows 10 laptop with Python, CUDA, and TensorFlow. If I understand correctly, pyenv does not work on Windows. Is there something similar?
Also, if I installed TensorFlow in WSL, it wouldn't be able to use the GPU, would it?
Python 3.7 is supported if you build master from source. But that can be a lot of work and though it may solve your TF problem, it may create other problems with other Python packages.
In my experience conda is the way to go for setting up your environment. I did not have great luck with pyenv, YMMV.
I don't know about WSL. Maybe setup a dual boot with Ubuntu on your system?
10 comments
[ 4.3 ms ] story [ 33.8 ms ] threadhttps://github.com/tensorflow/tensorflow/issues/20517
Somewhat off topic: I'm rediscovering the dance required to set up a new Windows 10 laptop with Python, CUDA, and TensorFlow. If I understand correctly, pyenv does not work on Windows. Is there something similar?
Also, if I installed TensorFlow in WSL, it wouldn't be able to use the GPU, would it?
In my experience conda is the way to go for setting up your environment. I did not have great luck with pyenv, YMMV.
I don't know about WSL. Maybe setup a dual boot with Ubuntu on your system?
Is this different than the existing multi_gpu_model? (https://keras.io/utils/#multi_gpu_model)