Show HN: Run Llama3.1 405B on a 8GB VRAM challenge [video] (youtube.com) 19 points by lyogavin 2y ago ↗ HN How to run Llama3.1 405B on a 8GB VRAM
[–] langcss 2y ago ↗ https://github.com/lyogavin/airllmWhat sort of speed do you get? [–] lyogavin 2y ago ↗ Not fast, not a good fit for realtime interaction applications, but for offline data processing cases, it works perfectly. I use it for my offline tasks.
[–] lyogavin 2y ago ↗ Not fast, not a good fit for realtime interaction applications, but for offline data processing cases, it works perfectly. I use it for my offline tasks.
2 comments
[ 3.3 ms ] story [ 11.8 ms ] threadWhat sort of speed do you get?