DiffusionBlocks – Block-Wise NN Training via Diffusion Interpretation (github.com)

"DiffusionBlocks, a principled framework that partitions transformers into independently trainable blocks, reducing memory requirements proportionally while maintaining competitive performance across diverse architectures and tasks."