[–] martynassubo 1y ago ↗ Hello,I recently completed a small project where I explored the potential improvements in model serving by switching from naive PyTorch setups to the ones utilizing ONNX Runtime, as well as ONNX Runtime with Rust's Actix web server.Here are the findings:https://github.com/martynas-subonis/model-servingAdditionally, I wrote a broader article on Substack for anyone interested:https://github.com/martynas-subonis/model-servingHope somebody finds it interesting/useful as well.
1 comment
[ 3.4 ms ] story [ 12.3 ms ] threadI recently completed a small project where I explored the potential improvements in model serving by switching from naive PyTorch setups to the ones utilizing ONNX Runtime, as well as ONNX Runtime with Rust's Actix web server.
Here are the findings:
https://github.com/martynas-subonis/model-serving
Additionally, I wrote a broader article on Substack for anyone interested:
https://github.com/martynas-subonis/model-serving
Hope somebody finds it interesting/useful as well.