https://wow.groq.com/hey-elon-its-time-to-cease-de-grok/
See this reply: https://news.ycombinator.com/item?id=39437239
yes
I think this post is output from the chatbot itself
GPUs can't scale single user performance beyond a certain limit. You can throw 100s of GPUs at it but the latency will never be as good.
Its using vanilla llama-2 from Meta with no fine tuning. The point here is the speed and responsiveness of the underlying HW and SW.
Its not using GPUs. Its using Groq's own HW - LPUs
https://wow.groq.com/hey-elon-its-time-to-cease-de-grok/
See this reply: https://news.ycombinator.com/item?id=39437239
yes
I think this post is output from the chatbot itself
GPUs can't scale single user performance beyond a certain limit. You can throw 100s of GPUs at it but the latency will never be as good.
Its using vanilla llama-2 from Meta with no fine tuning. The point here is the speed and responsiveness of the underlying HW and SW.
Its not using GPUs. Its using Groq's own HW - LPUs