PyTorch 2.2: FlashAttention-v2 integration, AOTInductor (pytorch.org) 15 points by egnehots 2y ago ↗ HN
[–] two_in_one 2y ago ↗ > now supports FlashAttention-2, yielding around 2x speedups> torch.compile improvementsso far 2.1 didn't work well with MoE GPT, at least in my implementation, due to dynamism in data flow. will check how 2.2 does
1 comment
[ 3.0 ms ] story [ 15.5 ms ] thread> torch.compile improvements
so far 2.1 didn't work well with MoE GPT, at least in my implementation, due to dynamism in data flow. will check how 2.2 does