- Up to 1350+ FP8 TFLOPS on Hopper GPUs
- No heavy dependency, as clean as a tutorial
- Fully Just-In-Time compiled
- Core logic at ~300 lines - yet outperforms expert-tuned kernels across most matrix sizes
- Supports dense layout and two MoE layouts
1 comment
[ 3.3 ms ] story [ 15.3 ms ] thread- Up to 1350+ FP8 TFLOPS on Hopper GPUs - No heavy dependency, as clean as a tutorial - Fully Just-In-Time compiled - Core logic at ~300 lines - yet outperforms expert-tuned kernels across most matrix sizes - Supports dense layout and two MoE layouts