1 comment

[ 2.8 ms ] story [ 13.3 ms ] thread
TLDR; Make 1x1 convolutions sparse, write fast Sparse Matrix Multiplication kernels, get a nearly 2x speedup with smaller models.