Could Endpoint SLMs Replace Cloud LLMs? Would Datacenter Race Shudder to a Halt?

2 points by aniijbod ↗ HN
Yeah, an SLM on an endpoint like a phone will have fresh latency issues as it goes online to fill gaps in its inference engine's knowledge base that a cloud LLM might not have, but cloud LLMs aren't exactly latency-free either, so the latency/performance issue isn't necessarily LLM's winning card.

0 comments

[ 3.1 ms ] story [ 10.1 ms ] thread

No comments yet.