octoml-profile is a python library and cloud service designed to provide the simplest experience for assessing and optimizing the performance of PyTorch models on cloud hardware. With octoml-profile, you can easily run performance/cost measurements on a wide variety of different hardware and apply state-of-the-art ML acceleration techniques, all from your development machine, using the same data and workflow used for training and experiment tracking, without tracing or exporting the model.
Apply just a few code changes, run your code locally, and you instantly get performance feedback on your model's compute-intensive tensor operations.
No more:
- exporting the models and stitching them back with pre/post processing code
- provisioning the hardware
- preparing the hardware specific dependencies, i.e. the version of PyTorch, Cuda, TensorRT etc.
- sending the model and data to each hardware and running the benchmarking script
1 comment
[ 3.4 ms ] story [ 13.8 ms ] threadApply just a few code changes, run your code locally, and you instantly get performance feedback on your model's compute-intensive tensor operations.
No more: - exporting the models and stitching them back with pre/post processing code - provisioning the hardware - preparing the hardware specific dependencies, i.e. the version of PyTorch, Cuda, TensorRT etc. - sending the model and data to each hardware and running the benchmarking script
More info: https://octoml.ai/blog/octoml-profiler-provides-deep-intelli...
Live demo April 6: https://octoml.ai/cp/octoml-profiler-demo/