There is a lot out here. I gave a seminar about the overall approach recently, abstract: https://shorturl.at/E7TcA, recording: https://shorturl.at/zBcoL. This two-part AMA has a lot more detail if you're already…
You can chat with these new models at ultra-low latency at groq.com. 8B and 70B API access is available at console.groq.com. 405B API access for select customers only – GA and 3rd party speed benchmarks soon. If you…
If you're interested in this sort of stuff, you might like this diff-based CLI tool I wrote: https://github.com/freuk/iter It runs on Groq (the company I work for), so it's super snappy.
You've got good ideas. What I like to personally say is that Groq makes the "Copilot" metaphor real. A copilot is supposed to be fast enough to keep up with reality and react live :)
(Groq Employee) As I'm sure you're aware, XTX takes its name from a particular linear algebra operation that happens to be used a lot in Finance. Groq happens to be excellent at doing huge linear algebra operations…
Sweet, thanks! It seems like this research ecosystem was incredibly rich, but Moore's law was in full swing, and statically known workloads weren't useful at the compute scale of back then. So these specialized approach…
(Groq Employee) Thanks for the feedback :) We're always improving that demo.
(Groq Employee) Yes! Determinism + Simplicity are superpowers for ALU and interconnect utilization rates. This system is powered by 14nm chips, and even the interconnects aren't best in class. We're just that much…
(Groq Employee) Agreed, one should care, and especially since this particular service is very differentiated by its speed and has no competitors. That being said, until there's another option at anywhere that speed..…
(Groq Employee) It's hard to discuss Tok/sec/$ outside of the context of a hardware sales engagement. This is because the relationship between Tok/s/u, Tok/s/system, Batching, and Pipelining is a complex one that…
(Groq Employee) You're right - we are comparing to independently-clocked logic. I wonder whether async logic would be feasible for reconfigurable "Spatial Processor" type architectures [1]. As far as LPU architectures…
[dead]
There is a lot out here. I gave a seminar about the overall approach recently, abstract: https://shorturl.at/E7TcA, recording: https://shorturl.at/zBcoL. This two-part AMA has a lot more detail if you're already…
You can chat with these new models at ultra-low latency at groq.com. 8B and 70B API access is available at console.groq.com. 405B API access for select customers only – GA and 3rd party speed benchmarks soon. If you…
If you're interested in this sort of stuff, you might like this diff-based CLI tool I wrote: https://github.com/freuk/iter It runs on Groq (the company I work for), so it's super snappy.
You've got good ideas. What I like to personally say is that Groq makes the "Copilot" metaphor real. A copilot is supposed to be fast enough to keep up with reality and react live :)
(Groq Employee) As I'm sure you're aware, XTX takes its name from a particular linear algebra operation that happens to be used a lot in Finance. Groq happens to be excellent at doing huge linear algebra operations…
Sweet, thanks! It seems like this research ecosystem was incredibly rich, but Moore's law was in full swing, and statically known workloads weren't useful at the compute scale of back then. So these specialized approach…
(Groq Employee) Thanks for the feedback :) We're always improving that demo.
(Groq Employee) Yes! Determinism + Simplicity are superpowers for ALU and interconnect utilization rates. This system is powered by 14nm chips, and even the interconnects aren't best in class. We're just that much…
(Groq Employee) Agreed, one should care, and especially since this particular service is very differentiated by its speed and has no competitors. That being said, until there's another option at anywhere that speed..…
(Groq Employee) It's hard to discuss Tok/sec/$ outside of the context of a hardware sales engagement. This is because the relationship between Tok/s/u, Tok/s/system, Batching, and Pipelining is a complex one that…
(Groq Employee) You're right - we are comparing to independently-clocked logic. I wonder whether async logic would be feasible for reconfigurable "Spatial Processor" type architectures [1]. As far as LPU architectures…
[dead]