The CPU to GPU dispatch overhead this aims to eliminate is a real bottleneck I've measured: my multi-pass Winograd kernel on MI300X (github.com/Jayluci4/nova-wino-amd) launches 3 HIP kernels + 1 rocBLAS GEMM per forward…
The CPU to GPU dispatch overhead this aims to eliminate is a real bottleneck I've measured: my multi-pass Winograd kernel on MI300X (github.com/Jayluci4/nova-wino-amd) launches 3 HIP kernels + 1 rocBLAS GEMM per forward…