Ggerganov made a statement on this "redundancy" which I really like:
> In addition to what @SlyEcho answered, my view is that since we now have a relatively good way to implement decoupled GPU backends (CUDA, Metal, OpenCL, etc.), there is no good reason not to keep doing it. As long as the new implementations do not touch the core ggml API and don't break the general workflow of other backends, we can keep adding support for new backends. Developers can join and help maintain the respective implementations. We can even have multiple implementations for a given single backend since we are still learning and finding new ways to make things more optimal and we don't know yet what is "the best way".
> In the long run, we can eventually obsolete and deprecate certain implementations, but for now the main goal is to experiment and explore the most efficient ways for hardware acceleration. Supporting more backends also helps to find certain good patterns in the implementations and even though currently there is a lot of "copy-paste", at some point we can think about the best way to consolidate the code and reuse the best techniques that we have found across all architectures.
1 comment
[ 0.25 ms ] story [ 12.7 ms ] thread- https://github.com/ggerganov/llama.cpp/pull/2059
- https://github.com/ggerganov/llama.cpp/pull/2039
Ggerganov made a statement on this "redundancy" which I really like:
> In addition to what @SlyEcho answered, my view is that since we now have a relatively good way to implement decoupled GPU backends (CUDA, Metal, OpenCL, etc.), there is no good reason not to keep doing it. As long as the new implementations do not touch the core ggml API and don't break the general workflow of other backends, we can keep adding support for new backends. Developers can join and help maintain the respective implementations. We can even have multiple implementations for a given single backend since we are still learning and finding new ways to make things more optimal and we don't know yet what is "the best way".
> In the long run, we can eventually obsolete and deprecate certain implementations, but for now the main goal is to experiment and explore the most efficient ways for hardware acceleration. Supporting more backends also helps to find certain good patterns in the implementations and even though currently there is a lot of "copy-paste", at some point we can think about the best way to consolidate the code and reuse the best techniques that we have found across all architectures.