I mean, surely the main motivation for "use an LLM to rewrite a huge project in a new language" was excitement about the shiny new tech that made it possible.
TBF, they did it first with ada/babbage/curie/davinci. "Sol" is a much weaker branding, though.
Does anyone understand why LLMs have gotten so good at this? Their ability to generate accurate SVG shapes seems to greatly outshine what I would expect, given their mediocre spatial understanding in other contexts.
The Claude Plays Pokemon stream with a minimal harness is a far more significant test of model intelligence compared to the Gemini Plays Pokemon stream (which automatically maintains a map of everything that has been…
Notably 45 out of the 50 days of improvement were in two specific dungeons (Silph Co and Cinnabar Mansion) where 4.5 was entirely inadequate and was looping the same mistaken ideas with only minor variation, until…
In my experience with the models (watching Claude play Pokemon), the models are similar in intelligence, but are very different in how they approach problems: Opus 4.5 hyperfocuses on completing its original plan, far…
This writeup on the underground puzzle is worth reading, it's a pretty baffling "puzzle" design. https://pokemow.com/Gen2/ShutterPuzzle/ That said, it's definitely Gem's fault that it struggled so long, considering it…
There were no such writeups, 99% of the discussion about difficulties in Crystal were in twitch and discord chats where Google doesn't scrape. (It hadn't yet gotten the public attention that Claude and Gemini's runs of…
It's hard to say for sure because Gemini 3 was only tested with this prompt. But for Gemini 2.5, which is who the prompt was originally written for, yes this does cut down on bad assumptions (a specific example: the…
Exactly what I was going to post. Optimizations like loop unrolling slow down the N64 because keeping the code size small is the most important factor. I think even compilers of the time got this wrong, not just modern…
The busy beaver function is interesting precisely because you _can't_ come up with any computable function that grows faster.
Claude almost universally reacts to everything with a positive exclamation as its first sentence, regardless of whether it's good or bad. If you don't believe me, just watch https://www.twitch.tv/claudeplayspokemon for…
With 2, the real problem is that approximately 0% of the OpenAI employees actually believed in the mission. Pretty much every single one of them signed the letter to the board demanding that if the company's existence…
Are you thinking of Bismuth's "Speedrunning as a gateway to scientific endeavours", perhaps? https://www.youtube.com/watch?v=w8_1lQ2KH50
As an n=1 data point, that was my exact situation for a while. Also a lot of the people who put out high effort stuff are college students, which works for the same reason. More interestingly and more surprisingly, some…
I've wondered myself why there's so little overlap between these two closely related interests of mine. Some of it seems to be the "But I don't want to cure cancer. I want to turn people into dinosaurs." effect, where…
How long until we get to the point where models know that LLMs get this wrong, and that it is an LLM, and therefore answers wrong on purpose? Has this already happened? (I doubt it has, but there ARE already cases where…
In conversational language, "All my hats..." implies that the speaker has at least two hats, which theoretically means that the sentence could be a lie from them having exactly one hat (even a green one). However, in…
https://xcancel.com/legit_rumors/status/1861448164084978157#...
https://www.scottaaronson.com/writings/bignumbers.html
Two reasons why I don't particularly believe him: 1) Altman's companies have had similar clauses before: https://news.ycombinator.com/item?id=40396787 2) The entire OpenAI board debacle started because Sam wanted Helen…
If, like me, you are suddenly curious what would happen if you added a small fourth body: https://youtu.be/WrahPSY9pf0
I have just been informed that my above comment is false, the CLIP-L is in fact referring to OpenAI's, despite that also being the name of an OpenCLIP model.
One of the diagrams says they're using CLIP-G/14 and CLIP-L/14, which are the names of two OpenCLIP models - meaning they're not using OpenAI's CLIP.
You should never ask an LLM to answer questions about itself. The answer is guaranteed to be hallucinated unless Google specifically finetuned it on an answer of that question. The answer it gave you is meaningless.…
I mean, surely the main motivation for "use an LLM to rewrite a huge project in a new language" was excitement about the shiny new tech that made it possible.
TBF, they did it first with ada/babbage/curie/davinci. "Sol" is a much weaker branding, though.
Does anyone understand why LLMs have gotten so good at this? Their ability to generate accurate SVG shapes seems to greatly outshine what I would expect, given their mediocre spatial understanding in other contexts.
The Claude Plays Pokemon stream with a minimal harness is a far more significant test of model intelligence compared to the Gemini Plays Pokemon stream (which automatically maintains a map of everything that has been…
Notably 45 out of the 50 days of improvement were in two specific dungeons (Silph Co and Cinnabar Mansion) where 4.5 was entirely inadequate and was looping the same mistaken ideas with only minor variation, until…
In my experience with the models (watching Claude play Pokemon), the models are similar in intelligence, but are very different in how they approach problems: Opus 4.5 hyperfocuses on completing its original plan, far…
This writeup on the underground puzzle is worth reading, it's a pretty baffling "puzzle" design. https://pokemow.com/Gen2/ShutterPuzzle/ That said, it's definitely Gem's fault that it struggled so long, considering it…
There were no such writeups, 99% of the discussion about difficulties in Crystal were in twitch and discord chats where Google doesn't scrape. (It hadn't yet gotten the public attention that Claude and Gemini's runs of…
It's hard to say for sure because Gemini 3 was only tested with this prompt. But for Gemini 2.5, which is who the prompt was originally written for, yes this does cut down on bad assumptions (a specific example: the…
Exactly what I was going to post. Optimizations like loop unrolling slow down the N64 because keeping the code size small is the most important factor. I think even compilers of the time got this wrong, not just modern…
The busy beaver function is interesting precisely because you _can't_ come up with any computable function that grows faster.
Claude almost universally reacts to everything with a positive exclamation as its first sentence, regardless of whether it's good or bad. If you don't believe me, just watch https://www.twitch.tv/claudeplayspokemon for…
With 2, the real problem is that approximately 0% of the OpenAI employees actually believed in the mission. Pretty much every single one of them signed the letter to the board demanding that if the company's existence…
Are you thinking of Bismuth's "Speedrunning as a gateway to scientific endeavours", perhaps? https://www.youtube.com/watch?v=w8_1lQ2KH50
As an n=1 data point, that was my exact situation for a while. Also a lot of the people who put out high effort stuff are college students, which works for the same reason. More interestingly and more surprisingly, some…
I've wondered myself why there's so little overlap between these two closely related interests of mine. Some of it seems to be the "But I don't want to cure cancer. I want to turn people into dinosaurs." effect, where…
How long until we get to the point where models know that LLMs get this wrong, and that it is an LLM, and therefore answers wrong on purpose? Has this already happened? (I doubt it has, but there ARE already cases where…
In conversational language, "All my hats..." implies that the speaker has at least two hats, which theoretically means that the sentence could be a lie from them having exactly one hat (even a green one). However, in…
https://xcancel.com/legit_rumors/status/1861448164084978157#...
https://www.scottaaronson.com/writings/bignumbers.html
Two reasons why I don't particularly believe him: 1) Altman's companies have had similar clauses before: https://news.ycombinator.com/item?id=40396787 2) The entire OpenAI board debacle started because Sam wanted Helen…
If, like me, you are suddenly curious what would happen if you added a small fourth body: https://youtu.be/WrahPSY9pf0
I have just been informed that my above comment is false, the CLIP-L is in fact referring to OpenAI's, despite that also being the name of an OpenCLIP model.
One of the diagrams says they're using CLIP-G/14 and CLIP-L/14, which are the names of two OpenCLIP models - meaning they're not using OpenAI's CLIP.
You should never ask an LLM to answer questions about itself. The answer is guaranteed to be hallucinated unless Google specifically finetuned it on an answer of that question. The answer it gave you is meaningless.…