TensorFlow and PyTorch under use linear algebra optimisation (arxiv.org)

The paper claims TF and PyT fail to exploit linear algebra optimisation tricks: " for instance, reducing the number of scalar operations by applying the distributive law, and automatically identifying the optimal parenthesization of a matrix chain"