I think me just means neural network models RLed to Chain of Thought reasoning? The thinking tokens are the symbolic bit. Smolensky's latest paper posted here the other day has some thoughts on how modern neural…
The construction jobs are only temporary if the compute infrastructure build-out decelerates. That seems pretty unlikely in the mid-term! It would mean that, like, there's a fixed appetite for data centers, and once we…
The poor fellow just rolled over. what an incandescently vulgar abuse of notation.
If a meaningful chunk of customers moves to some other service, that service will get swamped and fall down.
Paul Smolensky is a cognitive science titan from that era. He worked with Hinton, Rumelhart, and McClelland on parallel distributed processing, and literally wrote the book on tensor product representations in…
Right? I'd submit something just for the chance of lodging an idea in the head of Stephenson or Gwern.
Of course there's a point in it, don't be a boor. Any act of making can be treated as product, as craft, or as art. You'll find yourself taking all three stances at different points. Any engineer or artisan or artist…
I've been wondering if they've just already lost the battle? The little bot collectives have gone metastatic and made nests in the walls and under the floorboards and heat sinks, the humans who care completely…
That paper is kinda infamous! I last saw it mentioned only a few weeks ago, in https://arxiv.org/abs/2607.18966. Lots of folks will go "Oh that's the old Amodei and Clark paper" when the first few rows of pixels of that…
There's a ton of experimentation on it, the field is called continual learning. It's not something you need to believe in like Jesus, you can just go read about the current state of things.
Your mental model of benchmark scores is off. Some tasks within the benchmark are much easier than others. The hardest several tasks often have vastly different difficulty levels. Often, the hardest few tasks are…
The underlying LLMs do, but we choose not to use the capability because it's expensive and doesn't quite work as well as we'd like it to, or quite in the way that we'd like it to. We are perfectly capable of running…
If your circle of empathy includes fruit flies, fuck no, we murdered so many of those guys to figure out how their tiny brains work. If you do not think of fruit flies as moral patients, sure, yeah, who cares, they're…
That's a 2023 article! In 2023, reinforcement learning from verifiable rewards (RLVR) didn't exist. TL;DR these machines seek reward from an inferred invisible "grader," and telling them not to cheat and that there's an…
"QwenSVGBench" elo 1713, pelicanmaxxxing confirmed?
Only, if he had instead fallen in love with the version of this idea in which a community acts as the principal, rather than an individual. (I would like to be known to the agent serving my family, that serving my…
OpenAI hiring him as a consultant, obviously, which may well be the case.
Decades ago, long enough that NDAs are long expired, I analyzed ad network traffic at Google, handling big entities with involved contracts like IAC, Mozilla, AOL, Yahoo, and so on. We looked for weird traffic that…
What on earth is the point of limiting membership to such random and specific groups?
The name was given to the project when it was supposed to be a demo for nerds, not a product. They accidentally a product, and woe! Too late, the name was stuck and wouldn't come off, even if you scraped at it with your…
Scaling curves don't need to be drawn at particularly enormous parameter counts to be useful! If you can do a 300M and 1.2B run (like the authors do here), then you can do 150M, 300M, 600M, and 1.2B runs with only 50%…
I'm terribly sorry, but scaling curves or GTFO. Any random pile of linear algebra works fine-ish at small scales. Very few random piles of linear algebra push the Pareto envelope at large scales.
I've got 5 & 6 year old kids. They have a a VHS player / tiny CRT monitor with a few dozen tapes, a tiny janky mp3 player with all my ripped post-y2k era albums, and lots of books and art supplies. VHS tapes are so…
I think me just means neural network models RLed to Chain of Thought reasoning? The thinking tokens are the symbolic bit. Smolensky's latest paper posted here the other day has some thoughts on how modern neural…
The construction jobs are only temporary if the compute infrastructure build-out decelerates. That seems pretty unlikely in the mid-term! It would mean that, like, there's a fixed appetite for data centers, and once we…
The poor fellow just rolled over. what an incandescently vulgar abuse of notation.
If a meaningful chunk of customers moves to some other service, that service will get swamped and fall down.
Paul Smolensky is a cognitive science titan from that era. He worked with Hinton, Rumelhart, and McClelland on parallel distributed processing, and literally wrote the book on tensor product representations in…
Right? I'd submit something just for the chance of lodging an idea in the head of Stephenson or Gwern.
Of course there's a point in it, don't be a boor. Any act of making can be treated as product, as craft, or as art. You'll find yourself taking all three stances at different points. Any engineer or artisan or artist…
I've been wondering if they've just already lost the battle? The little bot collectives have gone metastatic and made nests in the walls and under the floorboards and heat sinks, the humans who care completely…
That paper is kinda infamous! I last saw it mentioned only a few weeks ago, in https://arxiv.org/abs/2607.18966. Lots of folks will go "Oh that's the old Amodei and Clark paper" when the first few rows of pixels of that…
There's a ton of experimentation on it, the field is called continual learning. It's not something you need to believe in like Jesus, you can just go read about the current state of things.
Your mental model of benchmark scores is off. Some tasks within the benchmark are much easier than others. The hardest several tasks often have vastly different difficulty levels. Often, the hardest few tasks are…
The underlying LLMs do, but we choose not to use the capability because it's expensive and doesn't quite work as well as we'd like it to, or quite in the way that we'd like it to. We are perfectly capable of running…
If your circle of empathy includes fruit flies, fuck no, we murdered so many of those guys to figure out how their tiny brains work. If you do not think of fruit flies as moral patients, sure, yeah, who cares, they're…
That's a 2023 article! In 2023, reinforcement learning from verifiable rewards (RLVR) didn't exist. TL;DR these machines seek reward from an inferred invisible "grader," and telling them not to cheat and that there's an…
"QwenSVGBench" elo 1713, pelicanmaxxxing confirmed?
Only, if he had instead fallen in love with the version of this idea in which a community acts as the principal, rather than an individual. (I would like to be known to the agent serving my family, that serving my…
OpenAI hiring him as a consultant, obviously, which may well be the case.
Decades ago, long enough that NDAs are long expired, I analyzed ad network traffic at Google, handling big entities with involved contracts like IAC, Mozilla, AOL, Yahoo, and so on. We looked for weird traffic that…
What on earth is the point of limiting membership to such random and specific groups?
The name was given to the project when it was supposed to be a demo for nerds, not a product. They accidentally a product, and woe! Too late, the name was stuck and wouldn't come off, even if you scraped at it with your…
Scaling curves don't need to be drawn at particularly enormous parameter counts to be useful! If you can do a 300M and 1.2B run (like the authors do here), then you can do 150M, 300M, 600M, and 1.2B runs with only 50%…
I'm terribly sorry, but scaling curves or GTFO. Any random pile of linear algebra works fine-ish at small scales. Very few random piles of linear algebra push the Pareto envelope at large scales.
I've got 5 & 6 year old kids. They have a a VHS player / tiny CRT monitor with a few dozen tapes, a tiny janky mp3 player with all my ripped post-y2k era albums, and lots of books and art supplies. VHS tapes are so…