Show HN: Run Llama3.1 405B on a 8GB VRAM challenge [video] (youtube.com) 19 points by lyogavin 1y ago ↗ HN How to run Llama3.1 405B on a 8GB VRAM
[–] langcss 1y ago ↗ https://github.com/lyogavin/airllmWhat sort of speed do you get? [–] lyogavin 1y ago ↗ Not fast, not a good fit for realtime interaction applications, but for offline data processing cases, it works perfectly. I use it for my offline tasks.
[–] lyogavin 1y ago ↗ Not fast, not a good fit for realtime interaction applications, but for offline data processing cases, it works perfectly. I use it for my offline tasks.
2 comments
[ 2.5 ms ] story [ 17.3 ms ] threadWhat sort of speed do you get?