From the Troubleshooting section on the website linked by the parent:
“CPU performance is faster than GPU on your network. Find out if your workload is sufficient to take advantage of the GPU. On small networks running with small batch sizes, the CPU may perform faster overall due to the overhead related to dispatching computations to the GPU. This will get amortized when the batch or model sizes grow, since the GPU can then take better advantage of the parallelism in performing the computations.”
The Mac chips only have a couple more GPU cores than CPU cores, while discrete graphics cards have hundreds or even thousands of cores. If the benefits of training on a GPU come from massive parallelization, why would a GPU with relatively few cores be expected to provide a benefit vs. CPU? Am I misunderstanding something about what a Mac GPU "core" is?
The M1 integrates an Apple designed[17] eight-core (seven in some base models) graphics processing unit (GPU). Each GPU core is split into 16 Execution Units, which each contain eight Arithmetic Logic Units (ALUs). In total, the M1 GPU contains up to 128 Execution units or 1024 ALUs
I'm guessing when you see NVidia/AMD "cores"/"shaders", they're counting ALUs, whereas an Apple GPU "core" is just a "block of GPU" that can be copy/pasted more times to increase performance. I think NVidia has "shader modules"/SMs which are blocks of X number of "cores"/ALUs. I'm being very imprecise but I think that's what's going on.
Yes, I think Apple’s definition of a core is quite different from, say, NVidia’s.
From Wikipedia:
“The M1 integrates an Apple designed eight-core (seven in some base models) graphics processing unit (GPU). Each GPU core is split into 16 Execution Units, which each contain eight Arithmetic Logic Units (ALUs). In total, the M1 GPU contains up to 128 Execution units or 1024 ALUs, which Apple says can execute up to 24,576 threads simultaneously and which have a maximum floating point (FP32) performance of 2.6 TFLOPs.”
NVidia has its own terminology. For example a 1080 GTX GPU has 20 stream multiprocessors (SM), each containing 128 cores, each of which supports 16 threads. That makes 40,960 threads which is more than the M1 GPU at 24,576 threads, but not massively so.
9 comments
[ 262 ms ] story [ 1080 ms ] threadhttps://developer.apple.com/metal/tensorflow-plugin/
And also try this to get a significant boost:
This is literally NVIDIAs moat: The perpetual immaturity of alternative platforms.
“CPU performance is faster than GPU on your network. Find out if your workload is sufficient to take advantage of the GPU. On small networks running with small batch sizes, the CPU may perform faster overall due to the overhead related to dispatching computations to the GPU. This will get amortized when the batch or model sizes grow, since the GPU can then take better advantage of the parallelism in performing the computations.”
https://en.wikipedia.org/wiki/Apple_M1#:~:text=The%20M1%20in....
I'm guessing when you see NVidia/AMD "cores"/"shaders", they're counting ALUs, whereas an Apple GPU "core" is just a "block of GPU" that can be copy/pasted more times to increase performance. I think NVidia has "shader modules"/SMs which are blocks of X number of "cores"/ALUs. I'm being very imprecise but I think that's what's going on.
From Wikipedia:
“The M1 integrates an Apple designed eight-core (seven in some base models) graphics processing unit (GPU). Each GPU core is split into 16 Execution Units, which each contain eight Arithmetic Logic Units (ALUs). In total, the M1 GPU contains up to 128 Execution units or 1024 ALUs, which Apple says can execute up to 24,576 threads simultaneously and which have a maximum floating point (FP32) performance of 2.6 TFLOPs.”
NVidia has its own terminology. For example a 1080 GTX GPU has 20 stream multiprocessors (SM), each containing 128 cores, each of which supports 16 threads. That makes 40,960 threads which is more than the M1 GPU at 24,576 threads, but not massively so.