CPUs outperform GPUs on deep learning task (software.intel.com)

That's interesting. Intel has implemented efficient CPU kernels.

> Intel Math Kernel Library for Deep Neural Networks (Intel MKL-DNN) is an open source performance library for Deep Learning

But the catch is that it’s currently lacking support for GoogleNet and ResNet. So it is limited.