Once you lose, you have lost. Ok, but how does that help us predict when something will lose?
[dead]
> with AI-generated content excluded from pre-training. > without distillation from third-party models sounds like zero unless they are lying.
My point is that if I made someone "smarter" they wouldn't suddenly know "What day, month, and year was Carrie Underwood’s album “CryPretty” certified Gold by the RIAA?" which is an example of a question in the SimpleQA…
Knowledge benchmarks can't really be improved upon via distillation or RL. It requires those facts be added to the training corpus and for the model to memorize them better. Neither distillation or RL really do that and…
"Shark attacks correlate strongly with ice cream sales" is an entirely true statement that some would argue is also misleading. Misleading should be removed as a category and replaced with a better hedge like "not sure"
My threshold for asking for help might be a little higher than the median, but, I like this operating style personally. Maybe it's just how I was raised, but the thought of not trying to figure something out for myself…
The null hypothesis isn't just the opposite of whatever your opposition believes. For LLMs the null hypothesis would be that there is no relationship between the input and output tokens. Something that is so obviously…
Sending an AI response to a question that someone asks you is insulting because it's a bit like sending them a link to letmegooglethat where it just animates typing the question you have into google. I think it's only…
If your AI alignment strategy is so fickle that it breaks if people simply discuss potential problems with the strategy then you didn't really have an alignment strategy to begin with.
It makes scams like that scalable. Once you discover one vector of scamming an AI bookkeeper, you can scam all of the users of that AI, using your own AI to scale it for you.
Well did they sell the website too? Or just the domain? Because the domain doesn't generate ad revenue, the original website did. Like just because I sell the domain name for my blog doesn't mean you also get the…
I think most of the people who pick blue would be empathic, loving people that are just kind of bad at game theory. I don't think I want to live in a world in which they all died out.
The downside of redding is that some portion of the world probably dies and you now have to live in that worse world that if you and 50% of the rest of the world has just blued, would not have happened.
can't wait for "our worst and dumbest model yet"
Instagram follows is not a good way to hire football players but it's probably a good way to hire instagram influencers. The football analogy is a little unfair because VCs are investing in more than just a company's…
Agreed. I think the starting comparison actually works here. It's a bit like the automobile. The advice of "just don't" doesn't work for cars. It takes a deliberate effort on every scale of society to accomplish, it's…
It's going for a rendition of the leaning tower of Lire.
Most breakthroughs that are published are for efficiency because most breakthroughs that are published are for open source.' All the foundation model breakthroughs are hoarded by the labs doing the pretraining. That…
Does the data not support a 2X increase in packages? Pre-ChatGPT, in ~2020, there were about 5,000 new packages per month. Starting in 2025 (the actual year agents took off), there is a clear uptick in packages that is…
It's less about performance and more about ecosystem lockin. It's a bit like imperial vs metric units. Why would you ever chose to learn imperial if you had the option to only ever use metric to begin with?
shhh just surf that deadly tsunami bro
That would be great if journals bothered publishing replication studies. But since they don't, researchers can't get adequate funding to perform them, and since they can't perform them, they don't exist. We can't look…
I am reminded by the perhaps revisionist history but still applicable belief that slavery was really ended by industrialization making abolition economically advantageous and not actually a socially driven movement. (In…
Once you lose, you have lost. Ok, but how does that help us predict when something will lose?
[dead]
> with AI-generated content excluded from pre-training. > without distillation from third-party models sounds like zero unless they are lying.
My point is that if I made someone "smarter" they wouldn't suddenly know "What day, month, and year was Carrie Underwood’s album “CryPretty” certified Gold by the RIAA?" which is an example of a question in the SimpleQA…
Knowledge benchmarks can't really be improved upon via distillation or RL. It requires those facts be added to the training corpus and for the model to memorize them better. Neither distillation or RL really do that and…
"Shark attacks correlate strongly with ice cream sales" is an entirely true statement that some would argue is also misleading. Misleading should be removed as a category and replaced with a better hedge like "not sure"
My threshold for asking for help might be a little higher than the median, but, I like this operating style personally. Maybe it's just how I was raised, but the thought of not trying to figure something out for myself…
The null hypothesis isn't just the opposite of whatever your opposition believes. For LLMs the null hypothesis would be that there is no relationship between the input and output tokens. Something that is so obviously…
[dead]
Sending an AI response to a question that someone asks you is insulting because it's a bit like sending them a link to letmegooglethat where it just animates typing the question you have into google. I think it's only…
If your AI alignment strategy is so fickle that it breaks if people simply discuss potential problems with the strategy then you didn't really have an alignment strategy to begin with.
It makes scams like that scalable. Once you discover one vector of scamming an AI bookkeeper, you can scam all of the users of that AI, using your own AI to scale it for you.
Well did they sell the website too? Or just the domain? Because the domain doesn't generate ad revenue, the original website did. Like just because I sell the domain name for my blog doesn't mean you also get the…
I think most of the people who pick blue would be empathic, loving people that are just kind of bad at game theory. I don't think I want to live in a world in which they all died out.
The downside of redding is that some portion of the world probably dies and you now have to live in that worse world that if you and 50% of the rest of the world has just blued, would not have happened.
can't wait for "our worst and dumbest model yet"
Instagram follows is not a good way to hire football players but it's probably a good way to hire instagram influencers. The football analogy is a little unfair because VCs are investing in more than just a company's…
Agreed. I think the starting comparison actually works here. It's a bit like the automobile. The advice of "just don't" doesn't work for cars. It takes a deliberate effort on every scale of society to accomplish, it's…
It's going for a rendition of the leaning tower of Lire.
Most breakthroughs that are published are for efficiency because most breakthroughs that are published are for open source.' All the foundation model breakthroughs are hoarded by the labs doing the pretraining. That…
Does the data not support a 2X increase in packages? Pre-ChatGPT, in ~2020, there were about 5,000 new packages per month. Starting in 2025 (the actual year agents took off), there is a clear uptick in packages that is…
It's less about performance and more about ecosystem lockin. It's a bit like imperial vs metric units. Why would you ever chose to learn imperial if you had the option to only ever use metric to begin with?
shhh just surf that deadly tsunami bro
That would be great if journals bothered publishing replication studies. But since they don't, researchers can't get adequate funding to perform them, and since they can't perform them, they don't exist. We can't look…
I am reminded by the perhaps revisionist history but still applicable belief that slavery was really ended by industrialization making abolition economically advantageous and not actually a socially driven movement. (In…