Handrolled Assembly will be better, but i preferred code i can read and understand.
MDMA is faster than llama.cpp in some models, because i did read the c code and rewrote it better. Rust is not the cause, i could the C code and open a PR and then hope it get merged.
The point of MDMA is you can have multiple nodes providing models while your main mode see all of them.
Handrolled assembly is always better (on the long run).
You can read this assembly code the same than C until your data structures are properly exposed and documented.
There are some tricks though: avoid abuse of macro pre-processor (ffmpeg was bitten by jus that), reasonable name mangling/hierarchical namespaces (usually done with a pre-processor that to avoid too long identifiers on deep projects).
And still, if your microsoft rust code does work with the microsoft rust to plain and simple C transpiler it will decouple it from c++ llvm which is a good thing.
3 comments
[ 0.27 ms ] story [ 12.5 ms ] threadDoes the C code generated by the rust->plain and simple C transpiler work?
Handrolled Assembly will be better, but i preferred code i can read and understand.
MDMA is faster than llama.cpp in some models, because i did read the c code and rewrote it better. Rust is not the cause, i could the C code and open a PR and then hope it get merged.
The point of MDMA is you can have multiple nodes providing models while your main mode see all of them.
You can read this assembly code the same than C until your data structures are properly exposed and documented.
There are some tricks though: avoid abuse of macro pre-processor (ffmpeg was bitten by jus that), reasonable name mangling/hierarchical namespaces (usually done with a pre-processor that to avoid too long identifiers on deep projects).
And still, if your microsoft rust code does work with the microsoft rust to plain and simple C transpiler it will decouple it from c++ llvm which is a good thing.