> Inspired by Google® JAX, functorch is a library that offers composable vmap (vectorization) and autodiff transforms. It enables advanced autodiff use cases that would otherwise be tricky to express in PyTorch
It's very nice that they're open about borrowing ideas that they like from other projects.
I've used it for Stable Diffusion on my M1 mbp, works with no problem. I am not a AI/ML person so my experience with M1 PyTorch is limited to just using SD.
Yeah works great, Tensorflow works great as well, but if using pip need to install tensorflow-metal and tensorflow-macos. Plus I always need to downgrade protobuf to 3.20. And couldn't get the C bindings for Tensorflow to work through metal, only through CPU.
It doesn’t support a few things, and has bugs in a few of the functions but besides that it works OK. Need to make sure your batch size is high enough to fully utilise the GPU cores. I think it was maybe 2-4 times faster than the CPU in my case.
Most things require workarounds, some things aren't possible (or we haven't found workaround yet) and it's not as fast as CUDA. But stable-diffusion inference works, and so does textual inversion training. I was also able to run training of a T5 model with just a couple of tweaks.
I'd stick with PyTorch 1.12.1 for now. 1.13 has problems with backpropagation (I get NaN gradients now when I attempt CLIP-guided diffusion -- I think this applies to training too), and some einsum formulations are 50% slower (there is a patch to fix this; I expect it'll be merged soon), making big self-attention matmuls slow and consequently making stable-diffusion inference ~6% slower.
19 comments
[ 5.5 ms ] story [ 46.2 ms ] threadIt's very nice that they're open about borrowing ideas that they like from other projects.
- 'pip install torch' doesn't work inside of a plain Python env (not miniforge)
- 'conda install torch' doesn't work inside miniforge
- 'pip install torch' works inside miniforge
In my very limited experience it's not a ton faster than using a CPU backend though.
Pretty good machine learning experience on M1.
I'd stick with PyTorch 1.12.1 for now. 1.13 has problems with backpropagation (I get NaN gradients now when I attempt CLIP-guided diffusion -- I think this applies to training too), and some einsum formulations are 50% slower (there is a patch to fix this; I expect it'll be merged soon), making big self-attention matmuls slow and consequently making stable-diffusion inference ~6% slower.