Petals runs Llama 2 (70B) from Colab at 5 tokens/sec (github.com)

5 points by borzunov 3y ago ↗ HN

3 comments

[ 5.7 ms ] story [ 21.4 ms ] thread

borzunov 3y ago ↗

Chat web app: http://chat.petals.ml

Colab: https://colab.research.google.com/drive/1uCphNY7gfAUkdDrTx21...

Project description: https://github.com/bigscience-workshop/petals

borzunov 3y ago ↗

We've moved to a new domain, the chat is now at https://chat.petals.dev

amrb 3y ago ↗

Great project and I'm happy to see it expand to more models!