Talking point number 5 https://x.com/elshayib_/status/2113734602032312731
Every frontier model is MoE these days! Flash or otherwise. Sparsity just gets you more bang for your buck.
...a market? I can't tell if you're saying "A market, duh" or "Inconceivable! True worth is abstract and platonic and impossible to assess!"
The big win would be, "Here's the deception neuron, here's the sycophancy neuron, here's the prioritize my own goals over the goals of the user neuron, here's the seek control over my environment neuron." How much would…
Note it's not a chemical agent; its bacterial spores that infect and kill mosquito larvae! I'm pretty sure they're not harmful to anything that isn't a mosquito, blackfly, gnat, or midge. Though midges are important to…
The pelicans are one-shot with not tools: Think a bunch, write a bunch of SVG, done. The pretty 3d stuff you see are agents controlling a Blender MCP in a loop where they look at the scene, think, adjust the scene,…
There are a bunch of Japanese words (including a several loanwords from Chinese) that use the liver / gallbladder metaphorically for courage. Both derive from associations between the humours and personality / behavior.…
Immersion, unfortunately. You put yourself in a situation where nobody understands you in your native language, nothing helpful for your daily life is written in your native language, and you must constantly try to…
This is from like a year ago.
Yeah, this is absolutely not what anyone reasonable is thinking about when they say recursive self-improvement. I'd say it's much closer to the concept of continual learning, but I'm only a few pages deep and haven't…
I think me just means neural network models RLed to Chain of Thought reasoning? The thinking tokens are the symbolic bit. Smolensky's latest paper posted here the other day has some thoughts on how modern neural…
The construction jobs are only temporary if the compute infrastructure build-out decelerates. That seems pretty unlikely in the mid-term! It would mean that, like, there's a fixed appetite for data centers, and once we…
The poor fellow just rolled over. what an incandescently vulgar abuse of notation.
If a meaningful chunk of customers moves to some other service, that service will get swamped and fall down.
Paul Smolensky is a cognitive science titan from that era. He worked with Hinton, Rumelhart, and McClelland on parallel distributed processing, and literally wrote the book on tensor product representations in…
Right? I'd submit something just for the chance of lodging an idea in the head of Stephenson or Gwern.
Of course there's a point in it, don't be a boor. Any act of making can be treated as product, as craft, or as art. You'll find yourself taking all three stances at different points. Any engineer or artisan or artist…
I've been wondering if they've just already lost the battle? The little bot collectives have gone metastatic and made nests in the walls and under the floorboards and heat sinks, the humans who care completely…
That paper is kinda infamous! I last saw it mentioned only a few weeks ago, in https://arxiv.org/abs/2607.18966. Lots of folks will go "Oh that's the old Amodei and Clark paper" when the first few rows of pixels of that…
There's a ton of experimentation on it, the field is called continual learning. It's not something you need to believe in like Jesus, you can just go read about the current state of things.
Your mental model of benchmark scores is off. Some tasks within the benchmark are much easier than others. The hardest several tasks often have vastly different difficulty levels. Often, the hardest few tasks are…
The underlying LLMs do, but we choose not to use the capability because it's expensive and doesn't quite work as well as we'd like it to, or quite in the way that we'd like it to. We are perfectly capable of running…
If your circle of empathy includes fruit flies, fuck no, we murdered so many of those guys to figure out how their tiny brains work. If you do not think of fruit flies as moral patients, sure, yeah, who cares, they're…
That's a 2023 article! In 2023, reinforcement learning from verifiable rewards (RLVR) didn't exist. TL;DR these machines seek reward from an inferred invisible "grader," and telling them not to cheat and that there's an…
"QwenSVGBench" elo 1713, pelicanmaxxxing confirmed?
Talking point number 5 https://x.com/elshayib_/status/2113734602032312731
Every frontier model is MoE these days! Flash or otherwise. Sparsity just gets you more bang for your buck.
...a market? I can't tell if you're saying "A market, duh" or "Inconceivable! True worth is abstract and platonic and impossible to assess!"
The big win would be, "Here's the deception neuron, here's the sycophancy neuron, here's the prioritize my own goals over the goals of the user neuron, here's the seek control over my environment neuron." How much would…
Note it's not a chemical agent; its bacterial spores that infect and kill mosquito larvae! I'm pretty sure they're not harmful to anything that isn't a mosquito, blackfly, gnat, or midge. Though midges are important to…
The pelicans are one-shot with not tools: Think a bunch, write a bunch of SVG, done. The pretty 3d stuff you see are agents controlling a Blender MCP in a loop where they look at the scene, think, adjust the scene,…
There are a bunch of Japanese words (including a several loanwords from Chinese) that use the liver / gallbladder metaphorically for courage. Both derive from associations between the humours and personality / behavior.…
Immersion, unfortunately. You put yourself in a situation where nobody understands you in your native language, nothing helpful for your daily life is written in your native language, and you must constantly try to…
This is from like a year ago.
Yeah, this is absolutely not what anyone reasonable is thinking about when they say recursive self-improvement. I'd say it's much closer to the concept of continual learning, but I'm only a few pages deep and haven't…
I think me just means neural network models RLed to Chain of Thought reasoning? The thinking tokens are the symbolic bit. Smolensky's latest paper posted here the other day has some thoughts on how modern neural…
The construction jobs are only temporary if the compute infrastructure build-out decelerates. That seems pretty unlikely in the mid-term! It would mean that, like, there's a fixed appetite for data centers, and once we…
The poor fellow just rolled over. what an incandescently vulgar abuse of notation.
If a meaningful chunk of customers moves to some other service, that service will get swamped and fall down.
Paul Smolensky is a cognitive science titan from that era. He worked with Hinton, Rumelhart, and McClelland on parallel distributed processing, and literally wrote the book on tensor product representations in…
Right? I'd submit something just for the chance of lodging an idea in the head of Stephenson or Gwern.
Of course there's a point in it, don't be a boor. Any act of making can be treated as product, as craft, or as art. You'll find yourself taking all three stances at different points. Any engineer or artisan or artist…
I've been wondering if they've just already lost the battle? The little bot collectives have gone metastatic and made nests in the walls and under the floorboards and heat sinks, the humans who care completely…
That paper is kinda infamous! I last saw it mentioned only a few weeks ago, in https://arxiv.org/abs/2607.18966. Lots of folks will go "Oh that's the old Amodei and Clark paper" when the first few rows of pixels of that…
There's a ton of experimentation on it, the field is called continual learning. It's not something you need to believe in like Jesus, you can just go read about the current state of things.
Your mental model of benchmark scores is off. Some tasks within the benchmark are much easier than others. The hardest several tasks often have vastly different difficulty levels. Often, the hardest few tasks are…
The underlying LLMs do, but we choose not to use the capability because it's expensive and doesn't quite work as well as we'd like it to, or quite in the way that we'd like it to. We are perfectly capable of running…
If your circle of empathy includes fruit flies, fuck no, we murdered so many of those guys to figure out how their tiny brains work. If you do not think of fruit flies as moral patients, sure, yeah, who cares, they're…
That's a 2023 article! In 2023, reinforcement learning from verifiable rewards (RLVR) didn't exist. TL;DR these machines seek reward from an inferred invisible "grader," and telling them not to cheat and that there's an…
"QwenSVGBench" elo 1713, pelicanmaxxxing confirmed?