I don't want to bash on the article, but a much more interesting take with llm that worked at 1kk TPS would the the countless amount of real time applications you could build with it.
I like electronic music, I dance to it a lot. What it I had a LLM that had enough throughput and low latency to do the reverse, take my dancing and generate music.
We are still stuck stargazing raw artificial cognitive intelligence, but real progress will he when we stop perceiving these as external intelligence and just an extension of ourselves.
“If” is temporary. More TPS is the question of time. In such usecase human in the loop will be eliminated as nobody, even anybody with fastest recognition, will be the slowest part.
How many industries are? How many use cases are possible?
If you solve everything, what will people do? Sitting on the beach, sipping drinks?
Massive degradation of human brains and rise of neurodegenerative illnesses as unused device starts to be erroneous?
1,000,000 TPS is good for some use cases but not publicly available…
Higher TPS by itself wouldn't remove the humans in the loop. Humans in the loop aren't there to make things faster, but to make them right - current LLMs can't do that on their own in most real world projects.
On the contrary, I would say that higher TPS would make humans in the loop much more productive, because the current long wait times mean that you lose focus and flow.
totally agreed on the "focus" aspect of it
that is why i was so happy about composer models from cursor when they came out but they werent very great
imagine opsu 5.5 level models at those speeds and faster.
At 1m tps, LLM’s could generate UI’s in realtime (a 5k webpage would take 5ms). Applications will have every little in common with today’s stacks when that happens. (For starters, rendering, event handling and data storage could move to the context.)
Interesting thought experiment but entire page is AI slop. I am not sure how much original thought the "author" put into this thought experiment themselves before hitting publish.
10 comments
[ 0.26 ms ] story [ 14.8 ms ] threadI like electronic music, I dance to it a lot. What it I had a LLM that had enough throughput and low latency to do the reverse, take my dancing and generate music.
We are still stuck stargazing raw artificial cognitive intelligence, but real progress will he when we stop perceiving these as external intelligence and just an extension of ourselves.
How many industries are? How many use cases are possible?
If you solve everything, what will people do? Sitting on the beach, sipping drinks?
Massive degradation of human brains and rise of neurodegenerative illnesses as unused device starts to be erroneous?
1,000,000 TPS is good for some use cases but not publicly available…
On the contrary, I would say that higher TPS would make humans in the loop much more productive, because the current long wait times mean that you lose focus and flow.