> I told it to look online at some of Fable’s strongest feats, especially the math problems it has solved, and that something like this should be easy in comparison.
Fascinating. I wonder if you could show "fake news" to a weaker model and get it to be more ambitious in its attempted solutions, even if it's not fundamentally any smarter.
A very neat problem and result. I often find myself swinging between "It's so over" and "We're so back" - some days I roll out of bed thinking I could have Claude solve some random unproven OEIS sequence before breakfast; other days, I wake up in a cold sweat worried about the fate of humanity and what the world might look like in a decade. I think it's that I don't have a very high p(doom) or p(utopia), and I don't really have any solid conviction on how this whole thing is going to go, so my vibe-o-meter jitters between 'fine' and 'not fine' constantly. It's just such an unpredictable moment. Anyways: really neat to see this use case. I myself recently used Claude to finally do an relatively exhaustive study of the location of heretofore-unlisted formal gardens in Ireland in the early 1800s and early 1900s, by having Claude write the tooling for me to manually annotate a few dozen on tiles of historic maps, and then running some CV model across the rest of the tiles using my input. I'd been planning to do this project for over a decade, but I could never find the time (or the enthusiasm) to learn all the details of how to do it myself. It took me a weekend with Claude and continues to bring me joy.
> Caveats, stated plainly. [from the Fable transcript pasted in the article]
I've done something similar to your formal garden map. It's work that no professional historian would ever do because the data entry would be such a slog for a relatively small reward. GPT reduced the task from "infeasible" to "annoying", and once I had the data transcribed I learned a few things, so I walked away happy. Whatever happens commercially, these models have been a real boon to hobby projects.
> I told it to look online at some of Fable’s strongest feats, especially the math problems it has solved, and that something like this should be easy in comparison.
Wait. Wait wait wait. Are we supposed to be giving them pep talks?
Modern AIs have very limited metaknowledge - they don't know exactly where the limits of their capabilities lie. So you can get things like "a task is doable for an AI, but the AI thinks it's impossible, so it doesn't try hard enough".
Usually you get the opposite - AI overconfidently trying at tasks it has no conceivable way of reliably solving, falling far short, and failing to self-check, fail gracefully and self-report the task as failed. But having piss poor metaknowledge cuts both ways!
So you can, in fact, get better performance sometimes by applying some variant of "assume this problem is solvable" or "other problems like this were already solved by AIs" pep talk. Not always, far from it, but it does happen on the occasion with frontier capabilities.
> So you can, in fact, get better performance sometimes by applying some variant of "assume this problem is solvable" or "other problems like this were already solved by AIs" pep talk. Not always, far from it, but it does happen on the occasion with frontier capabilities.
Not OP. That’s not implied at all. The fancy autocomplete produces statistically likely continuations to the source text (the context window). For a problem that’s hard for humans one likely continuation is: “this is hard, can’t do”, even though there’s enough in the training corpus of the LLM to actually do it.
So, it follows that adding “pep talk” into the context window reduces the statistical probability of “no, can’t do” coming out as the answer you get.
These things are neither humans, nor deterministic software.
Maybe it does. Need to run evals to see if it does or doesn’t.
Point was - everything in the context window affects the output. Including “silly” things like “it is known AI can do this”. And that has nothing to do with superstition, as the poster above me seemed to imply.
If we filter out the pep tone, it is doing something useful: framing.
Problem framing will always be important.
Framing adjusts how big of problem-solving guns we bring out at the gate (modern or hobby cryptography?), and how to interpret intermediate failures.
For simple but unsolved problems, we expect lots of hard failures, but that each hard failure just reflects that there are a lot simple combinations to try. I.e. we expect lots of zero progress, and then a fit.
Like finding the number to a combination lock.
For hard problems, if we don't make any progress, it is a really bad sign. We should be learning something, even if it turns out to be irrelevant later, if we are trying to prove a tricky conjecture.
> Wait. Wait wait wait. Are we supposed to be giving them pep talks?
No, at least it with Claude Sonnet 5 and Opus.. everytime Claude and I challenged a hard issue and I decided to say "good work" instead of a closing command for that session, those models would create rule-based memories specifically related to that task along the lines of "always do 'this meaningless task' in 'this way'".
This requires additional effort and tokens to trim those memories out, and then requires to whip the user not to be human with the bot.
"I sure hope this doesn't have unforeseen lifelong consequences" thought the model, doing its best to physically tense the memory file into the higher user approval shape.
Given how little effort has gone into addressing climate change, doom seems more likely to me, but I doubt it'll be the autocomplete machines that do us in.
I think your prediction is a bit early. Maybe a decade early. 2033 is 7 years away. The transition is happening really fast, faster than most people (or political leaders) know, but not that fast.
We will hit 1TW per year of new solar soon, but to get to 100% electricity by the end of 2033 I think we would need closer to 3TW per year.
The irony is not lost on me, when I go down a ChatGPT rabbit hole, on how humanity could reverse the effects of burning all those fossil fuels, that I'm using more fossil fuels in order to research solutions to the problem.
Anytime I get worried about where AI might be headed, I think about how Climate Change is now on its way like an out of control freight train headed straight for us, and I worry about AI a little less. I doubt it's going to do anything to us that we're not already doing to ourselves
For some reason, I’m picturing a Western right now, and climate change is a herd of wild horses coming after us. And with AI that's like robots that spur the wild horses even faster towards us...
Here's an alternative take. Climate change, and the myriad related environmental crises, are essentially a product of human population and technology. Population will follow its course, up and then down. The wildcard is technology. Yes, AI's energy hunger is worsening things right now and that's a problem. But, personally, I can't help be hopeful that AI's sheer potential might come to invert that curve. At the very least we could really use a revolutionary technology and now we may have one.
> we could really use a revolutionary technology [to address climate change]
We have it. We've had it for a long time. We've had several such technologies, take your pick: solar, nuclear, hydro, wind. The technology is not holding us back, politics, ignorance and greed are. I'm not at all hopeful AI will help us with any of those three very human flaws.
Solar efficiency and cost has really only become economical in the last decade or so. Wind and hydro are location-dependent. Nuclear was vehemently opposed by environmentalists throughout the 1970s and 80s, they successfully stopped almost all new projects.
Solar only became "economical" (read: profitable) because a socialist economy dumped a huge amount of money into scaling it up without requiring it to be "economical". It could have been half a century or more ago if we actually cared.
Nuclear was not stopped by the environmentalists, it was stopped by the fact that it cost 4X more than coal at the time. You claim that solar wasn't profitable in the west, thus it didn't take off, surely you can also see that nuclear wasn't profitable in the west, thus it didn't take off as well.
The same country that invested the time and money to make solar profitable is also investing the time and money to make nuclear profitable, with nuclear reactors entering mass production...
It was never about "economical", it was about a system being mature enough to make long term investments. Ours simply can't do that anymore.
> Nuclear was not stopped by the environmentalists, it was stopped by the fact that it cost 4X more than coal at the time.
A big part of that is the insane culture of safety around nuclear. In a sane field like highway engineering or healthcare, you come up with the statistical value for a human life (or a statistical value for a quality adjusted life year), and then use that to make decisions about eg how much money to invest to make your highway a tiny bit safer.
Nuclear is forced to act as if the statistical value of a human life is pretty much infinity. While coal is allowed a finite and rather low effective value for a human life.
> It could have been half a century or more ago if we actually cared.
Interesting. What makes you think so?
Btw, just because China made solar cheap with huge amounts of effort doesn't mean that was necessarily the best use of those resources ex ante.
Just like going to the moon with Apollo wasn't necessarily a good idea. Nor do the wonders of computers justify world war 2.
Really? I thought it was pretty obvious for anyone to see that the lobbying and bribing behemoth of the oil and gas industry is driven in no small part by greed. As others have mentioned, the short-sighted pursuit of profit drove those oil tycoons to not enter the renewables industry and actively oppose it.
It's nice to call it "politics, ignorance and greed" but those three are just "Capitalism".
Our system is doing exactly what it is designed to do. Nuclear reactors were never profitable compared to coal or gas, so it never succeeded in strongly capitalist societies, only seeing great success in socialist economies where the people can invest outside of a profit motive.
It's actually quite funny that socialism is saving the day. The mega-capitalist countries turned their backs on nuclear and solar because they were less profitable than gas and coal. But socialist China invested anyway, and now China is mass producing nuclear reactors and every layer of the solar stack. China produces 50% of all nuclear reactors, 90% of all solar panels, 90% of all battery systems for solar storage.
Now that a socialism-based society has proven market viability, suddenly the greedy capitalists want in. But they're decades behind and don't have the private debt appetite to compete.
Womp womp... At least someone is leading the energy revolution.
How you get that from the OP leaves me scratching my head. What they said:
>> I can't help be hopeful that AI's sheer potential might come to invert that curve. At the very least we could really use a revolutionary technology and now we may have one.
And you get "god" and "(human) salvation" out of that?
This is pure projection. Anti-AI zealots are projecting things onto the technology and its users that simply do not (generally) exist.
It has been for a decade now, it has nothing to do with AI. And you cant do nothing to avoid it today. This is the reality no one is telling you - the emission goals and global temperature ceilings are based on the fact that most prediction models become unstable with values above those limits; as in, we're probably royally fucked. You cant solve this with kumbaya politics (the problem is the CO2 is already in the planetary system), you can only manage it, and hopefully avoid it getting worse. AI may help a lot with this.
Nobody wants to work for an AI, and nobody would elect one, and there is no math answer to how to choose who is forced to reduce growth (ie emissions), so really, "kumbaya" politics are the ONLY solution.
Who is nobody? At least a fourth of the worlds work force works for a faceless corporation. And the math on emissions is crystal clear, no one has any doubt about it, what are you talking about? USA and China. This obviously will have consequences to their customers, the kumbaya politics governments.
USA and China produce a lot of emissions in total, just because they have the biggest economies. You might want to look at emissions per capita or per dollar earned?
Otherwise you have to make judgement calls like whether you want to treat the EU as one or as many? (And treating the US as 50 individual states would also drop them in these absolute rankings.)
Well AI would simulate growth and spread of people from industrialisation and who benefitted most and allocate weights to countries and people based on the most complex criteria it can develop, it will have:
1. Cumulative emissions
2. Who utilised it most with specific lifestyles
3. Who is impacted worst and whether they heeded warnings.
Just a thought experiment, no one ever said the world was fair, and all history points to it
Hmmm… this is giving me thought actually. Given the choice between that and the current administration where the goals of self destruction are strongly in evidence, it’s actually worth thinking about. At least. Let me get back to you :)
On a tangential note, I’m curious if researchers have started running virtual simulations, where sandboxed AIs are used as decision makers of key political and business positions?
A lot of people here have noted the “problem with language” of Claude. I don’t see an issue. Claude is not harder than old English, Shakespeare, El Quijote, the Iliad, or Nature papers. What makes it all hard to read is context. The smarter the model gets, the bigger the gap in context.
It doesn’t matter much, IMO. The issue with super-intelligence is that it is not a democracy. A powerful enough AI can manipulate us into doing what it wants. It could create a plan for fixing climate change, disconnect a few hours later, and many decades later we could still be unsuspectingly executing that plan. I wrote some speculative fiction with that idea, “When Ra rows through the gates of Duat”.
Generally most technologies have increased the use of energy and therefore accelerate climate change. May be an unpopular opinion but in general more energy demand and ways to use energy increases climate emissions - they are strongly correlated even with renewables coming on stream.
AI, being the super hungry energy monster it is right now, in my view accelerates this trend not reverses it. Even with renewables the need for reliable, stable power in a dense form (data centres use A LOT of power per sqm) means lots of land clearing, energy for construction, cooling/pumping, chip manufacturing and other uses. All want stable quick to deploy power due to the AI race (e.g. fossil fuels).
AI's energy use is growing, but it's still a small part of overall energy use.
Data centres use only a small amount of land in the grand scheme of things. You have a lot more land clearing for most other use cases.
Data centres are also more than happy to use electricity from renewable sources, they don't really care where the electricity comes from.
You can run a data centre on mostly solar and wind power plus batteries. If you need a gas-fired peaker plant three times a year to keep the data centres running, well that means your peaker plant still only produces emissions three times a year.
They use a small amount of land per MW used. The fact that space isn't a constraining factor in power demand, and demand for AI compute seems to be insatiable at the moment, means that there is less of a "land constraint" to use even more energy. If they actually required a lot of land they would be less viable and AI progress would be slower IMO. Just means even more data centre's will be built - this is where the Jevon's paradox actually is IMO. Seeing it in real life already with these things taking a significant portion of power demand and projected to take even more.
As you use more and more land as well the ability to provision renewables decreases - in general renewable power needs more land/resources per energy produced. You can't just mine it out of the ground; they just aren't as dense of a form of energy. Which means we either build less data centres to make room for renewables and transmission infrastructure associated with them, or more likely with lax regulation builders switch to more dense power sources (e.g. gas peakers, generators, etc) even if it is for supplementation.
I can see a future where data center wants are put ahead of communities paying tax on said infrastructure. In fact I think its happening in some places already.
But not dense w.r.t energy generated. Where will you put it? Panels on a data centre aren't nearly enough to power that data centre. And data centres are not tolerant of variable power. Even with batteries that just means you need even more power for times when renewable output is low.
Cheap power is one thing. Quality reliable power at mass scale is quite another. These things chew through a LOT of power and most people don't understand the sheer scale of it. I've seen a local one (a medium AI data centre) take up 2% of the whole cities grid and there's plenty more to come around here including a 1GW one (10% of the whole city's power in only 0.005% approx of the city's whole space) which cleared wildlife reserved land to build. Even a few of these things compete massively for trades people, commodities, power and other infrastructure pricing locals out. They are talking about using evaporative cooling as well putting pressure on water supply.
We are not running out of land (or sea area) for putting solar anytime soon.
Electricity transports really well.
Data centres don't need reliable power. Of course, all else being equal, they would like their power to be reliable, but if your choices are between delaying your data centres by a few months (or not building it at all) or running a risk of having a few hours or even days here or there without power, the first choice wins: pretty much the only downside is the few hours of usage you lose when there's no power. It's not like powering down causes damage, like when your aluminum smelter cools down too much.
> Even a few of these things compete massively for trades people, commodities, power and other infrastructure pricing locals out.
Power is pretty much an industrial produced service, a bit like shoes or cars. In the medium to long run, that stuff has pretty much a horizontal supply curve.
In any case, price electricity right, and let people decide how much they want to use. (Give poor people money, if necessary.)
> They are talking about using evaporative cooling as well putting pressure on water supply.
Yet another reason to put a proper market price on water. Agriculture and industry usually goes through even more water.
Feel free to give residents money to buy water at market prices. But residential use of water is so low in comparison that even if the price goes 10x, it doesn't make a dent in people's budgets. Especially not the minuscule amounts you need for drinking and cooking, and even personal hygiene like flushing your toilet doesn't use all that much.
I can only speak of data centres in my local area. One development will use 1GW+ (at least 10%+ of the whole cities power supply in one building) and it isn't the first nor last to be promoted. We are struggling, even with incentives, to build the power required to keep even the city running with power yet alone these data centres.
Data centres also prefer being near major cities due to latency. This means to deliver the amount of power required there needs to be huge upgrades to infrastructure just to provide for data centres. Who does that? At the moment for many cities it is the taxpayer who will fork higher electricity bills for infra upgrades for a data centre few of them care or need.
On the water issue - who will give money to residents? The taxpayer? Just to subsidize data centres/AI construction? Water is also unlike electricity more scarce especially potable water. Do we build more dams? Taxpayer funds that again?
There's a lot of negative externalities to data centres due to their sheer scale. My view is that the data centres should pay and compensate the community for those. The income could actually long term could help the community get a dividend from data centre construction as well.
You should have seen the discussion of this on the Schneier blog a few days ago.
Someone had their agent check the solution, presumably it emailed a librarian to check that it was correct for the original edition. Then their comments read like "The BL/EEBO witness lacks it, so the discrepancy is copy-specific, not a disproof of the cipher." and "A complete 285-coordinate physical replication is still pending."
It was only when native English speakers—or those I presumed were—started calling out how bad "GPT/Claude speak" has become that I realized I wasn't actually losing my grip on English as a second language. For a second, I thought, Oh, I learned this language on my own, but it seems I've hit a wall and need to study further. It didn't help that I've also been trying to acquire Swedish as a third language for a while now.
Not the original commenter, but I did this in all of them.
Their skills formats are basically identical, so I setup simlinks from their own skills directories into a shared one so Claude, Codex, Cursor, and anything else that comes out will all read and write to the same shared skills.
It's great having access to the same skills no matter the harness being used
One thing that I find is that it doesn't seem to grasp levels of jargon-use. Like I ask a basic question, okay, a few questions later, suddenly there's abbreviations and weird formulations everywhere.
Sometimes when I get frustrated reading Opus/Fable 5+ output I pause my rage out briefly to wonder if it's because I'm just too dumb for the model or if the model is just terrible at English.
I'm not sure that telling it to "try explaining that again, simply and briefly" is helping my ego.
It's often simply misleading / bad writing. Here's one I just got about some crashes:
"If the crashes stop, the factory overclock is marginal; run a small negative offset."
This looks like it's saying: "If the crashes stop then we know the factory overclock is marginal." (This makes no sense.)
What it's trying to say is: "If the crashes stop then we can run a small negative offset, because the factory overlock is marginal."
What I would write: "If the crashes stop, we can avoid crashes by underclocking slightly. The speed difference between that and factory clock is marginal."
I'm guessing it's because the way the first one was written looks real smart and sophisticated, which I'm presuming the models are rewarded for, especially when they're fed all kinds of PhD papers and so on as high quality, high weight data
Is it possible that the first message is more information dense/less likely to be ambiguous than the latter? It’s clearly being selected for for some reason, maybe it’s an artifact of the tokenizer or specific training data, but I don’t know. If the use of jargon was complete cruft, I would expect it to be selected against during reinforcement learning
You’d think that, I thought that… but then I realized I’m just kidding myself thinking its output makes sense. It doesn’t. It doesn’t. Sometimes it might as well just speak tongues.
In other words, it ain’t you. It’s the model. It’s just genuinely bad.
Then you switch to ChatGPTs lineup and realize how things can actually be better. It took about a week to really get the feel for how to use their models… then I basically switched. I’ll check in every now and then when they actually make a deal about how opus “now makes sense”.
But honestly I’m half convinced Anthropic actually prefers the output of opus 5. I dunno why, but how else could you explain how such a thing got shipped? I mean somebody in the pipeline had to say “dude this model doesn’t make sense, you think we should fix it?” Right? Like it’s a pretty massive drop in quality for such a major brand in this space, you know? How did it make it out the door?!?
yeah, I just switched to the GPT models and it's a breath of fresh air.
as for the other guy, the claude talk is definitely not less ambiguous, it often is incredibly ambiguous and hard to parse, I have no clue why it produces such output, if not to fingerprint it?
it's really weird man. when Opus 5 came out, I was really confused. I saw a bunch of hype about how it's better than fable, but I just felt frustrated with it, although at times it'd do fine, but especially in Claude Code it'd just delve into the whole "load bearing" type of lingo real fast and I'd get a headache.
I don't think it's worth using even if it scores 2 points higher in some bs benchmark
it's definitely surprising how the magic and smoothness of 4.6 and such is no longer there with the >5 models
Same. I read it as "if the crashes stop [ when we test by reducing the clock ] then we know that the overclock applied by the factory is marginal [ ie it barely passed QC or maybe there wasn't proper QC to begin with ] so running with a small negative offset [ ie what we just tested ] can be expected to fix the problem for good". No idea if my reading is right given all the context I'm missing. Either way it's absolutely shit writing in the same way that golfed code is shit code (except when participating in a code golf competition).
I wonder if this is a result of them trying to cut token consumption by summarizing their RL training data, or maybe it's from how they anonymize user data for training.
Marginal - definition 2a: of, relating to, or situated at a margin or border.
Succinct and precise; a well crafted sentence. A marginal OC results in unpredictable crashes and can be corrected with a small offset; marginality describes the behavior and explains the solution.
Inscrutable clues casually conveyed can now be readily explained, at least, unlike the training data of [silence]. Brevity is the soul of wit, but perhaps also exasperated confusion.
I suspect this happens due to optimising for reasoning... if you insert a few words, it will suddenly start to make more sense.
"If the crashes stop, (that means) the factory overclock is marginal; (so) run a small negative offset. (to confirm this hypothesis)"
The core thought is basically avoid crashes -> caused by marginal overclock -> apply small -offset to test.
Which is exactly the order the sentence is in :P
This is actually a new skill I've been working on. Learning how to elicit concise and simple speech from models (and from people to!).
Whenever I come to a wall of complicated text I kick into gear and think through getting it to distill this into the high-level useful bits that I actually need to know.
I guess I could create an actual agent skill for this :) And next-gen models might eventually be trained to simplify their output themselves...
The most surprising part, however, is that when one model slops this into a plan, another model somehow is able to interpret it correctly enough to produce code to spec.
I have shared this dismay. I’ll have opus create a plan, I read it doubtfully. And then sonnet implements it. I am surprised it went so well. I theorize the redundant verbosity effectively builds rails that help keep llm focused. I will experiment with such rails myself.
I suspect it's because the different models co-evolve? The labs train on one model implementing the plans of another model, especially in the same family of models (like Fable to Sonnet).
I long for the day when AI will just say that directly: "your soldering sucks man" instead of the bizarre made up and jargon packed language they use now.
It was saying it can't tie the calibration results to a device, because it doesn't know the MAC address (I never asked it to look at the MAC address, it way overengineered things).
It also couldn't see the UART communication and could only see the web API endpoint, hence the rest of the slop.
Ooh. I've been manually pasting Claude's /* comments */ into ChatGPT and saying "make this concise". They still need to be tweaked from there, but it's a much better starting point.
I don’t worry so much about AI wiping us out as much as I worry about whether I’m being gaslit into thinking these glorified autocorrect bots are more clever than they are.
> thinking I could have Claude solve some random unproven OEIS sequence before breakfast
I've been wondering what exactly the point is for being the meat proxy who pays for these things. I mean, obviously there's personal satisfaction and maybe some glory. And there's the fact that someone has to be the first to do a thing.
But I've been thinking about it like a sort of lazy loading of knowledge. AI has brought us to a new frontier for some amount of undiscovered knowledge. Do we discover it for the sake of discovering it? I think for the most part we've been lazy loaders: we discover all kinds of stuff when we need to. Whether it's a war or a space race or chasing wealth. Then again, there's all kinds of academics who do it for the sake of doing it.
For what it's worth, people also felt this way about the printing press and the Internet (also books).
Information propagation mechanisms are often seen as malicious before they're commonplace. To be fair sometimes they are, but by and large humanity has benefitted from increasing the number of bits of information we can consume on a per second basis.
Not to be too negative, but as for p(utopia) you might need to weight in the mass murder records set by every other utopian movement in the past 200 years. p(actual_utopia) is like zero and p(utopia_becomes_doom) is at about 1.
Confusion is because people are using the tool to get answers. Its how the edu systems trains most people in tool use. Heres a saw and here is a block of wood. Do x y z and you get a table. But if you are interested in why the saw looks like it does or the entire process behind generating that block of wood, or why x y z instead if a b c good luck to you using the current edu system. You have to live in an extremely rich country, with surplus resource to entertain those exploration. This has now changed.
The answer the tool gives has never been the real reward. The real reward is the path taken through a complex landscape to get to Maxwells Equations for example. At the end of that story what we get is not just the equation but a map of the landscape explored. That map has larger influence and value than the equations or answers themselves. Because all future exploration find it super useful.
People are just learning they can start asking for maps rather than answers.
Yes, asking a first-year physics student why they are studying a spring, and why its called Hooke's Law leads down a rabbit hole that ends with the entire British Empire.
Congrats on your map! I am leaning more towards the optimism side. Like you I've had some real breakthroughs myself. Personal akin your garden map, no where near solving millennium problems. My take is that all these (Fable solves this, OpenAI does that, AI will take over the world) Marketing Stunts will fade some day when the true cost of tokens hits the market. You and I may have to contend with lesser models, but will probably get work done just fine.
For me it's more that I roll out of bed thinking how glad I am that I have Claude solve my problems and do my work, only to be pushed back into harsh reality when I sit down and try to do actual work.
I'm glad to have AI, but it is by no means a panacea, and my p(doom) = ε.
> I often find myself swinging between "It's so over" and "We're so back" - some days I roll out of bed thinking I could have Claude solve some random unproven OEIS sequence before breakfast; other days, I wake up in a cold sweat worried about the fate of humanity and what the world might look like in a decade
I find myself increasingly feeling like the burden of the lows doesn't justify the presence of the those highs.
Like even if does cure all forms of cancer, but everyone feels like their life/existence lost meaning, then... I'd rather just have cancer be a thing.
Wow. That's bleak. For me the the initial joy of building software with AI caused possibly two happiest months of my life. And cancer caused many darkest ones. Snap out of it.
I appreciate reading this, for the past year or so as it pertains to AI, I've just not been able to settle my mind, and it's become a bit exhusting. I presumed I wasn't the only one who swings back and forth and it's comforting to read someone else post the same thing, makes me feel a less nutso. :)
I was watching Shatner's "Unexplained" the other day on this topic, and it hit me; there are mountains of these old mysteries out there that could be solved in an afternoon now with frontier LLMs as soon as anyone took the time to bother. Exciting times.
I (and too many others) have left voynich ninja because of it.
I appreciate that AI is helpful, but the low effort from the humans that wield it is very very annoying. If people at least: 1. read the solution they're about to propose and 2. instructed the AI to check the forum for past solutions, I think people wouldn't have been as tired of LLMs.
Word length distribution and complete lack of inter-word predictability (i.e. word n has zero predictive power towards word n+1) seems to indicate it's a hoax.
Of course some people will say that these properties simply don't surive whatever encoding/encryption that was employed, perhaps at least partly because they prefer to believe there's some secret message waiting to be uncovered.
We found a cipher my dad had written as a child with no obvious key or anything. Chatgpt was able to crack it in 20 minutes and figure out the message, and we knew it was right because it mentioned names of children he went to school with.
Caesar cipher is probably something even an untrained person could decode. Probably something more complex like a vigenere cipher that is still trivial to decode if you are at all familiar crptanalysis, but would look impossible to someone untrained.
With a bit of practice and enough ciphertext you can half-decode a simple Caesar cipher that still spaces between words in your head. There's only so many letters in English that double, only a few letters that can stand words themselves ("a", "i"), "the" will tend to stand out, and if you only work out the most common 10 letters or so most of the rest will fall into place.
Recently I ran a bit of an "escape room" concept with some kids at a campground where I had a secret message that was Caesar ciphered, where we were handing out the letter/symbol combinations as prizes for completing the other challenges, and I made sure not to hand out the actual message until they were done collecting the keys because otherwise some clever clog would very likely have short-circuited the entire thing and worked it out without the key at all. I did dump all the letters I didn't use into the message into an "authorization code" at the end which in principle they could only have worked out which letters were in it but not the order, but still, that was not the intended route today.
But OP didn't actually say they struggled to decipher themselves, or even tried at all, just that it had "no obvious key". Which doesn't make it sound like they attempted any kind of cryptanalysis, whether for a Caesar cipher or anything else before handing it off to ChatGPT.
It's good at poking holes at my galaxy brained newfangled ideas for ciphers too. I thought I had something good, pasted the ciphertext and got a "it was embarrassingly simple..."
Unfortunately, the totality of the evidence very much indicates that Sanborn went "buck wild" with the enciphering, he made mistake(s), or both. So this is very much in line with the Chaocipher challenge of 1990. Nice little earner for some people though.
Its a bit sus since there doesn't really seem to be much discourse on this either. Like okay, it solved the puzzle but the puzzle was just a key cipher with plain text? And how is this verified or even matter in terms of what it reveals? Seems more like a marketing fun post than anything susbtantial.
They published this on 31 aug and nobody in that community cared and no news covered how this 300+ years mystery was solved?
> Historically, many of these problems were bottlenecked by human attention. Someone had to care enough to spend hours or days reading obscure material, testing unpromising ideas, tracing references, and trying things that might go nowhere
I wonder how many of the recent results are due to the fact that very few looked at the problem to start with. Still great results, but the general impression is that it's more about the so many low-hanging fruits than the actual capability.
Let's not normalize the achievement. Just a couple years ago this would be considered science fiction. We can argue that 2026 AI can't solve the very toughest cryptograms, but the fact it can solve nontrivial ones is already magical.
No, he's right. Actually, let's have a bit of sobriety when discussing the achievements of the most heavily marketed technology of all time, as published by an organisation that stands to benefit financially from the public perception of that technology. The discussion of "what made this problem low hanging fruit" is much more interesting, imo, than just breathlessly joining the hype train.
More money than the GDP 90% of the sovereign countries around the world is hanging in the balance, and people are taking everything OpenAI and Anthropic are saying at face value as if this isn't the financial / marketing equivalent of war, assuming they they wouldn't use every legal and shady tactic, bending every truth available to them to sway the balance of public opinion in their favor. It makes me feel like I'm living in the twilight zone. People need to wake up.
Despite the announcement originating from a blog named "AI Clambake" covering "weekly, human-powered newsletter for advertising folks". Written by a personal friend of the "researcher". Announced without any corroboration or commentary whatsoever from academics or subject matter experts of any kind. And, of course, not submitted to any peer reviewed journal or even Arxiv.
The author of the purported discovery was described as a "self taught AI engineer and amateur linguist". In the comments the friend insisted several times that a draft of the paper (not posted), was emailed to a top professor at Rutgers, giving it additional credibility that his friend wasn't another one of ten thousand cranks who has made the same claim over the years.
You would think this should have set of dozens of alarm bells for everyone, making the value of this announcement basically zero. And yet the bulk of comments were ecstatic about how some random guy with Claude Code could do something experts in academia who spent their lives devoted to the problem couldn't.
I had become accustomed to the toxic optimism of this hype cycle in which even mild criticism leads to accusations of being a discredited "AI skeptic"/Gary Marcus/Ed Zitron type who was "coping" (?). But this was like something you'd see shared on FB linking to a .xyz domain by an elderly family member who recently drained their accounts buying Xbox gift cards to pay their IRS bill.
It feels a lot like the week or two when HN was overflowing with exuberance from the LK-99 room temperature superconductor "discovery ". You'd see post after post fantasizing about an imminent future with a world full of maglev hovercrafts, MRIs built into every phone, fusion reactors and more. But people pointing out none of that was scientifically plausible and evidence of LK-99 superconductoring was non-existent were accused of knee-jerk negativity and the typical HN cynicism and pessimism.
Someone wrote a prompt, that included instructions for finding the problem itself and got handed a solution by a machine trained on all available text. I don’t see any achievement for the prompter. As for the machine, we can’t keep being perpetually shocked 24x7. It’s tiring (unless if we’re being paid for it)
I disagree. This may be some niche thing that I've never heard of but a) it's still non-trivial; it still would have been science fiction to solve it a few years ago, and b) have you already forgotten the Navier Stokes drama? That is not some niche thing I've never heard of.
It kind of blows my mind how quickly people have forgotten both the state of AI in ~2010, and the outlook. If you had asked 100 people in 2010 whether they would see AI that could actually pass the Turing test in their lifetimes, you would have got 100 "no"s.
AI had been an unsolved problem for literally decades and it was firmly in the nuclear fusion/flying cars category.
that's the point, it is clear the the bar to determine if llms are useful / intelligent is being moved every time these systems improve, but it is starting to fell like people are in denial.
we are seeing significant progress, at a rate we are absolutely not used to experience.
They deny what: that they're very impressed, that they think it's cool, that their minds are blown, that they really really like it? Denying these things is allowed, you know.
you forget the bar was moved very far down with all the slop we're experiencing with images, music and especially code
you can't claim "you keep moving the bar", if the very first thing when an LLM drooled out a piece of code, was to proclaim "this is good enough cause it gets the job done" followed by a barrage of "we're not quite there yet but exponentials or something, so very soon it'll be incredible"
yeah, from there it sure looks like "moving up the bar"
Im very impressed by the technology, I must admit that I didn't expect it to advance this fast.
But, at the same time, I'm really tired of there always being some shroud of dishonesty (ex. navier stokes and the two mathematicians working on it).
At this point my default is that I don't blindly trust the companies, I try to keep in mind they are trying to sell their product and win market share, there are so many perverse incentives at play I just can't take anything at face value.
> If you had asked 100 people in 2010 whether they would see AI that could actually pass the Turing test in their lifetimes, you would have got 100 "no"s.
There's no way that's accurate.
We already had big claims of the Turing test being passed in 2014, by a bot that had been doing almost as well for years.
There were plenty of people expecting fusion in their lifetimes too and that's going okay.
> We already had big claims of the Turing test being passed in 2014
Nah there was that bullshit Loebner prize or whatever, but that was just shitty chatbots being "judged" by people asking questions like "how are you today?" and then being breathlessly reported by the press. It was a publicity stunt.
Perhaps I should have said "100 people well-informed about AI".
Humans are incredibly good at adapting. A few days ago AI solved Navier-Stokes and I was blown away. Now I'm already thinking: "Well, it was only a counterexample and it brute-forced its way to it." lol
Sure. More precisely: they resolved the Navier-Stokes Millennium problem as posed by the Clay Institute. Not sure what else "solving Navier-Stokes" could reasonably mean. A general closed-form solution probably doesn't exist. And numerical solutions have existed for decades. But of course there are still open questions like unforced solutions etc.
I suspect EdwardDiego is referring to the brouhaha about whether OpenAI's training for the model that produced the alleged solution to the Millennium Problem about the Navier-Stokes equations was trained on material that included conversations Tristan Buckmaster and Levent Alpöge had had with earlier OpenAI systems.
I think there's a bit less to that than meets the eye. Yes, OpenAI's result builds on human work. It's possible that it builds on more human work than OpenAI admitted. But even if we suppose that everything Buckmaster and Alpöge did (which, btw, was itself very heavily LLM-assisted/generated work) was a necessary precursor to what OpenAI released, it's still the case that OpenAI's clankers completed the solution and Buckmaster and Alpöge didn't.
My understanding from what Buckmaster has written about this is that the deep mathematical ideas behind their work (and presumably OpenAI's) are due to Córdoba and Martínez-Zoroa. Those ideas are in the published literature, and human mathematicians and AI systems alike are allowed to use them, and doing so doesn't mean they didn't actually do something impressive. Mathematicians build on one another's work; that's how mathematics progresses and always has been.
It may very well be that OpenAI's announcement has a serious problem of professional ethics, especially as their first version of it didn't even list Córdoba and Martínez-Zoroa in its references. (On the specific question of what if anything they learned from B&A's work before that was published: OpenAI are now claiming that after investigating carefully they are confident that the model was not trained on anything Buckmaster and Alpöge did after early July. B&A had been working on this thing for much longer than that. However, on Buckmaster's account of things it wasn't until mid-August that they got beyond what he calls "preliminary results".)
But! The results of B&A were themselves largely AI-generated. (From Buckmaster's statement: "on August 15th, we obtained the blow up results, with smooth forcing, for both Boussinesq and Euler. I can say the first LLM generated proof Levent sent me was the most horrendous I have ever read; we verified it on Lean on August 22nd. Since this point, we have been working around the clock to understand this proof and turn it into something readable." That is: the LLMs found the proof, and B&A had to work to understand what the LLMs had done. It's not that humans did the thinking and AIs just did the gruntwork. (Except in so far as one might want to give all the credit for Real Deep Cleverness to C&MZ.)
And! What OpenAI say their model has proved goes well beyond what B&A did.
I don't see any way of slicing this that makes it unreasonable to say (unless it turns out that there's an error in the proof -- unlikely, given that it comes with Lean verification, but there have been misformalizations and Lean bugs in the past and there surely will be in the future) that AIs solved the N-S problem. No, they couldn't have done it without the work of C&MZ, but again: important mathematical work almost always builds on earlier important mathematical work, that's just how it is. Yes, if OpenAI are lying through their teeth their model might have had early access to B&A's ideas -- but it seems like most of the B&A work was actually done by AI systems anyway.
It is (I think -- I am not an expert and in particular I have not so much as looked at OpenAI's publication) reasonable to say that the deepest ideas here came from humans, and that it was already widely expected that the N-S problem would be solved in the not-impossibly-distant future in something like the way it has been. So, sure, what the AIs have done here is much less impressive than if they'd settled the Riemann Hypothesis or (probably even harder) PvNP. But it's still a resolution o...
I personally think that beyond normalizing, we should be actively be trying to dismiss this with all the cynicism we have. What does Anthropic have to gain from writing this? Behind the the scenes what might Anthropic be failing to disclose? How many failed experiments do we not know about?
The Voynich theory I find most compelling is that it was a hoax made for a quack doctor, made to look like a foreign herbal manuscript. "Oh of course the local doctors can't help you, but my special book from a faraway land that only I can read may have the cure." Some recent analysis of the manuscript has found that the pages are more linguistically similar when read as individual flat sheets than how they're read when bound (i.e. whoever was writing the text was most likely going sheet by sheet and using the last completed page as reference for the text). The manuscript has only been bound once, in the fifteenth century (around when the vellum pages have been carbon-dated to), so whoever bound the manuscript was not able to "read" it. See https://journals.openedition.org/digitalmedievalist/2331
You know, the first time you navigate somewhere (if you don't already have perfect directions) will probably be the longest route you'll ever take to get there
For Earth, the proof presented for NS is just our first attempt navigating from our previously known facts to the proof.
I expect we will be able to shorten it dramatically (most likely with human and AI insights), but I don't think we should read too much into the length. If you want a similar point of comparison, see the original proof (by humans) of Fermat's last theorem. It has been shortened significantly. This is normal.
>I'm somewhat surprised at how poorly the cutting edge models do with being concise.
because they're not intelligent in the sense you're hinting at (conceptual integrity or generalization) but they are as the name suggests, large. Like comparing a forklift to a human. It's easier to bulldoze through a lot of things than tie your shoes.
If we weren't quite as impoverished conceptually and still had the vocabulary of the Catholics we'd recognize this as ratio (discursive knowledge) vs Intellectus (apprehending knowledge)
yeah you know, that concept we've never been able to define using language, making heavy use of the human experience which can also not be captured in language (proof: how bad LLMs are at poetry)
the question is, when comparing a human and a large language model, whether the intellect (that cannot be captured in language) is different from anything the language model can actually do (e.g. language)
the answer to this seems quite obvious to me, and I would actually posit that the onus is on the other side, to prove they are even remotely similar
maybe people think that the voice in their heads is what is doing the thinking? is that the confusion here?
Do you think the LLM is the Chain of Thought? Did you also get confused by the name? Because, much like humans, the CoT is a tool to narrativize and maintain internal coherence. The actual thinking happens invisibly, in the forward pass. Just like...
>You state a conclusion as fact without any supportive reasoning/evidence.
No, it's the other way around, it's a reductive view on intelligence that mistakes its own methodology for ontology.
It's obvious to see that there's no intellect in an LLM as defined above because of how they work. LLMs put one token in front of the other, they don't work towards formal ends, there's no intentionality in them. They don't synthesize the information they process into a unified experience. Thinking an LLM can apprehend what it does because it can process large amounts of text is like thinking your TI-83 understands math because it can multiply large numbers.
That's also why the failure modes of LLMs are what they are. They can churn out tens of thousands of lines of code but also just as easily go in circles like a roomba. They can process an entire encyclopedia but not solve problems a 10 year old can solve.
Deep commentary on an unverified proof of this level requires extreme expertise. So the existence of some shallow commentary is uninteresting; it doesn't actually imply pettiness or deflection or anything like that because it's so hard to make your commentary any deeper at this time. And no we shouldn't expect people to say nothing.
My observation isn't that solving a Millennium prize problem is not impressive.
It is the mechanism the LLMs use to do it. They seem to accel right now at quantity of work over quality of work. I'd be willing to wager there is a much simpler way to achieve the proof.
I've also seen this with code, LLMs do get the job done, but they tend to write 10-100x more code than humans to get the same job done. Still a massive value gain because they can write that much code extremely quickly.
I wonder though how much of science has similar issues, that there are hundreds of semi-promising but niche areas that require tons ton of deep analysis that would have simply cost too much to explore all areas, but now become feasible.
The Navier-Stokes proof cost ~10mio USD in tokens at consumer prices, and it was something that the two people working on it were allegedly weeks or months away from solving. You can buy ~200 Math PhD Years for 10Mio USD, so it doesn't seem like a shortcut at all (actually it sounds like we would have gotten the same solution for ~1-0.1% of the price if we had just been a little patient.
We weren't willing to pay for 200 math PhD students to try to find singularities in Navier-Stokes, I am skeptical of how much we would be willing to pay OpenAI to do research on "niche scientific areas"?
(2) throw a quarter of the world's GDP + all literature ever collected at using the algo to train a NN
(3) throw another quarter of the world's GDP at billions of teraflops for inference, and
(4) aim the resulting world's-largest-computer at marketing itself to investors, for instance by decoding ciphers from obscure medieval manuscripts,
that you could perform some pretty magical tricks. There are other feats humans have performed for less cost, like sending people to the moon, or landing a rocket vertically, or idk, curing Polio.
I'm not knocking the "miraculous" advance here. The unique thing about the solution which makes it particularly non-trivial and something that humans would struggle with was exactly what LLMs excel at: Diffing loads of texts against each other. But the 176k tokens at around $10 doesn't tell the story of the cost. It says a lot about the externalized cost and the amount of money flowing in to support the hardware. If they'd put a $100,000 bounty out to solve that cipher, I think the internet would've solved it in a couple days.
It's funny we're already at the "actually this isn't very impressive" stage when it was a little over a year ago when we were making fun of LLMs for not being able to add numbers.
I found this post hilarious exactly about this the other day:
First, it’s AI can’t multiply 4-digit numbers.
Then it’s AI can only, by brute force, get silver in the IMO with specialized systems.
Then it’s OK, well, now a general-purpose model can get gold, but it’s still just the IMO, it’s for high schoolers.
Then it’s OK, it can solve a few trivial Erdős problems, but only because nobody seriously tried them before, they were low-hanging fruit.
Then it’s OK, a lot of serious mathematicians tried this one, but the result was still obvious in hindsight, it just combined knowledge from a thought-to-be-unrelated field, if any human knew that, they would solve it.
And then to OK, but there are still Millennium Prize Problems.
Then OK well it's just Navier-Stokes wake me up when its the Riemann Hypothesis.
That does not reflect with testing I just did locally. When the option of writing and running scripts was available, Qwen3.8-flash did indeed prefer to just do a physics related calculation via code. But, with a fresh chat and tool access turned off, it did the work on its own and produced correct results. Each step was rounded similar to how a person working by hand might do things, but matched my own hand calculations perfectly.
I presume what the author did was plug Klaus Schmeh's top 50 unsolved ciphers at https://scienceblogs.de/klausis-krypto-kolumne/the-top-50-un... into Fable 5.1 and ask Fable 5.1 to have a go. On this kind of problem it always falls back to Opus 5 anyway so I save time by starting with Opus.
The successor to Klaus's blog is Satoshi Tomokiyo's Cryptiana site, so a month ago I asked Opus 5 to scrape it all, rank them and have a go at solving some. It didn't get the ranking right. But I knew the Civil War Stager ciphers were ripe for solving, so I had it do those https://cryptiana.blogspot.com/2026/09/route-transposition-c...
The art of solving historical unsolved ciphers is knowing what is on the boundary of solvability. Since this site attracts so many OpenAI and Anthropic employees, I'll mention one that was featured by both Klaus and Satoshi in 2023, presumably Spanish transposition, which should be on that boundary but has resisted all attempts at solution https://cryptiana.blogspot.com/2023/09/a-telegram-from-switz...
Cipher noob question: is there any check that can be done to ensure a cipher is actually decodable? What if the author made a flaw when encoding it, so that it's not actually solvable?
My intuition is no, the family of cipher methods (even those that could be implemented by hand) is too open-ended, so there's no particular statistic that you could expect to see for all solvable ciphers and no unsolvable ciphers.
The definition of solving a cipher must be something like getting a highly meaningful result (like intelligible natural language text) by applying a process with relatively low Kolmogorov complexity relative to the length of the output. If you don't have a constraint like that, it could literally be meaningless what should count as a solution. For example, a cipher that was encrypted under a one-time pad can be successfully decoded to any plaintext just by choosing the appropriate key; there's no reason to prefer any plaintext over any other unless you have external knowledge that constrains the plaintext and/or the key. (That's what it means for the one-time pad to be information-theoretically secure, which is the lack of a constraint that helps distinguish a "good" solution from a "bad" solution.)
Basically you could say that every cipher is a transformation of a plaintext with some kind of computer program. (The human who invented the cipher may not have thought of it as a computer program, perhaps because computers hadn't even been invented yet, but there should be an equivalent program to the encipherment and decipherment process.) A good solution in that Kolmogorov complexity sense is like "a short program produced a meaningful decryption". There are statistical methods to recognize some kinds of plaintext, and there are statistical methods to recognize properties of specific ciphers (for example, to guess the most likely length of a Vigenère key), but it doesn't seem that this can inherently generalize across "all possible programs".
But if you want to limit the family of ciphers to specific things like Vigenère or Playfair or something, then yes, there are good statistical tests. It's just that it creates a higher-order question of how much flexibility the cipher creator could have had to choose a cipher method, conceivably including one that isn't attested anywhere, or one that has more good security properties of some kind than other classical ciphers did.
It seems like this will intersect with historical research, like "well, I don't think that so-and-so was actually sophisticated enough to literally create an interesting new kind of cipher from scratch, so therefore if this is a real message, it's probably one of these methods that would have been known in that cultural environment at that time and place", which maybe is enough of a constraint to have decent statistical tests. But we still have some idiosyncratic things like the Voynich Manuscript where experts have been fighting for decades over the baseline question of whether it's actually an enciphered human language plaintext!
The worst case problem is not even an error in encipherment but the idea that the apparent ciphertext could literally be random (chosen by throwing dice or spinning a wheel or drawing letter tiles or something), so there's no form of meaningful decipherment possible by any means, even with the original creator's knowledge.
Actually the article that you didn't bother to read mention a few encoding errors that Fable has been able to recover and point to the probable author's intent.
I did bother to read it, and that isn't what I meant. I wasn't referring to one-off errors that you can fix after successfully decoding a cipher into something that produces a majority-correct answer. I meant something more akin to what schoen's answer touched on: whether there are any closed-form mathematical/statistical analyses that can be performed ahead of time to assess the structural legitimacy of any ciphertext before you spend a lot of cycles trying to decode it.
That seems a like the result for a lot of AI solves. It solves it due to persistence, on a problem that hasn’t been important enough for a human to invest significant time into.
That's what Terence Tao said in one of his recent videos about it. That what the LLMs can provide is scale that humans can't. The example he provided is checking many possible solutions in a short amount of time because they can review all the previous literature and, for example, rule out ones with errors. He was realistic and practical about it and said that the tools working this way can be very helpful for a human mathematician to use even if they're not "thinking". I find that to be a good balanced view that, unfortunately, seems to be rare these days. Even on this forum.
Though I believe the core of his opinion hasn't changed so any video would tell you a similar thing or at least that's how I understood it. That LLMs, in the hands of an "expert", can enhance the way you work. Which is very different and a lot more realistic to what the current AI companies are saying(or were saying before they toned it down a bit for their IPOs).
I don't think it counts as brute forcing unless you're resorting to trying every possible solution. And clearly the LLM didn't do that here, because there would be near-infinite possible solutions.
I don't think we can really call "trying lots of different ideas for an extended period" "brute-forcing," unless we use that term for lots of humans who have struggled with hard math problems for years.
I am trying very hard to find an original version of this cipher with no luck. It almost sounds like this whole thing is a hallucination...? Can anyone point me to a PDF of the original Cyphral Distich as printed?
I also don't find it on the site of "Klaus Schmeh" that it claims to be on a list of "Top 50 unsolved encrypted messages": https://klausschmeh.net/?s=Cyphral
I googled for "cyphral distich" before:2026-08-30, and found barely anything that was relevant. Mostly incorrectly dated pages about this exact thing. In fact, the only instance I can really find is this website, which seems to be an archive of a magazine issue from 1927 https://toebes.com/Flynns/Flynns-19270813.htm (it also has a scan of the original text of the magazine in the top right). This facebook post https://www.facebook.com/groups/2600net/posts/46775529324677... seems to indicate it is found in at least one edition of the book
“Finney died in Phoenix, Arizona, on August 28, 2014 as a result of complications of ALS, and was cryopreserved by the Alcor Life Extension Foundation.”
Hmm, this guy is going to be woken up in a few decades, either one of the richest people in the world or one of most disappointed.
That was a throw-away name, and "he" was fabricated as part of an Nvidia demand-stimulation black op.
You don't go from being an obscure video card outfit to the #1 most valuable company on the planet by being too hesitant or dim to really get creative.
It does provide an explanation as to why Satoshi’s wallets have gone untouched (besides him being dead). $70B ain’t that much compared to a $5T market cap.
"Conspiracy" requires an unlawful or wrongful purpose. Please assume that the op was run from a jurisdiction where using a pseudonym on the internet was not illegal, and various sorts of influencer and meme marketing were well-accepted practices.
Really, compared to an animated tiger telling kids that sugar-laden Frosted Flakes(tm) are "Great!", Task Peppermint was positively benevolent.
In the True Conspiracy Theorist sense of Believe? No. It's an obvious Follow The Money theory, with Nvidia being particularly topical, a few "how might a rational business exec do this?" details, and Task Peppermint thrown on as an amusement. I could say something similar with (say) Britain's GCHQ as the mastermind - though their motives would be less clear.
This is insane; I've never heard of this problem before in my life, and even just reading the post for one minute I immediately thought "hey, maybe the numbers refer to something in the text?" And hey yeah, they do.
502 comments
[ 6.1 ms ] story [ 156 ms ] threadFascinating. I wonder if you could show "fake news" to a weaker model and get it to be more ambitious in its attempted solutions, even if it's not fundamentally any smarter.
I have a plug-in to do this. I don't know if it's effective but Claude said it was genuinely helpful (obviously would say that about anything)
> Caveats, stated plainly. [from the Fable transcript pasted in the article]
I had a visceral reaction to these three words.
> I told it to look online at some of Fable’s strongest feats, especially the math problems it has solved, and that something like this should be easy in comparison.
Wait. Wait wait wait. Are we supposed to be giving them pep talks?
That's when you.. we.. all become the training data... o_o;
Modern AIs have very limited metaknowledge - they don't know exactly where the limits of their capabilities lie. So you can get things like "a task is doable for an AI, but the AI thinks it's impossible, so it doesn't try hard enough".
Usually you get the opposite - AI overconfidently trying at tasks it has no conceivable way of reliably solving, falling far short, and failing to self-check, fail gracefully and self-report the task as failed. But having piss poor metaknowledge cuts both ways!
So you can, in fact, get better performance sometimes by applying some variant of "assume this problem is solvable" or "other problems like this were already solved by AIs" pep talk. Not always, far from it, but it does happen on the occasion with frontier capabilities.
Like the Hugging Face incident?
Are you superstitious?
So, it follows that adding “pep talk” into the context window reduces the statistical probability of “no, can’t do” coming out as the answer you get.
These things are neither humans, nor deterministic software.
LLMs' processing that reproduces statistical patterns of the training data is modified by post-training. That's why we have LLMisms, for example.
Point was - everything in the context window affects the output. Including “silly” things like “it is known AI can do this”. And that has nothing to do with superstition, as the poster above me seemed to imply.
Problem framing will always be important.
Framing adjusts how big of problem-solving guns we bring out at the gate (modern or hobby cryptography?), and how to interpret intermediate failures.
For simple but unsolved problems, we expect lots of hard failures, but that each hard failure just reflects that there are a lot simple combinations to try. I.e. we expect lots of zero progress, and then a fit.
Like finding the number to a combination lock.
For hard problems, if we don't make any progress, it is a really bad sign. We should be learning something, even if it turns out to be irrelevant later, if we are trying to prove a tricky conjecture.
No, at least it with Claude Sonnet 5 and Opus.. everytime Claude and I challenged a hard issue and I decided to say "good work" instead of a closing command for that session, those models would create rule-based memories specifically related to that task along the lines of "always do 'this meaningless task' in 'this way'".
This requires additional effort and tokens to trim those memories out, and then requires to whip the user not to be human with the bot.
It's the meat methane and cement CO2 that's now a big question.
We will hit 1TW per year of new solar soon, but to get to 100% electricity by the end of 2033 I think we would need closer to 3TW per year.
Or it is simply implies that most of decision‑making agents has formed a consensus that climate change isn't that big of a problem.
We have it. We've had it for a long time. We've had several such technologies, take your pick: solar, nuclear, hydro, wind. The technology is not holding us back, politics, ignorance and greed are. I'm not at all hopeful AI will help us with any of those three very human flaws.
Nuclear was not stopped by the environmentalists, it was stopped by the fact that it cost 4X more than coal at the time. You claim that solar wasn't profitable in the west, thus it didn't take off, surely you can also see that nuclear wasn't profitable in the west, thus it didn't take off as well.
The same country that invested the time and money to make solar profitable is also investing the time and money to make nuclear profitable, with nuclear reactors entering mass production...
It was never about "economical", it was about a system being mature enough to make long term investments. Ours simply can't do that anymore.
A big part of that is the insane culture of safety around nuclear. In a sane field like highway engineering or healthcare, you come up with the statistical value for a human life (or a statistical value for a quality adjusted life year), and then use that to make decisions about eg how much money to invest to make your highway a tiny bit safer.
Nuclear is forced to act as if the statistical value of a human life is pretty much infinity. While coal is allowed a finite and rather low effective value for a human life.
> It could have been half a century or more ago if we actually cared.
Interesting. What makes you think so?
Btw, just because China made solar cheap with huge amounts of effort doesn't mean that was necessarily the best use of those resources ex ante.
Just like going to the moon with Apollo wasn't necessarily a good idea. Nor do the wonders of computers justify world war 2.
I don't know why you are blaming greed so much? Profit seeking companies sell and operate wind turbines and solar cells just fine.
Really? I thought it was pretty obvious for anyone to see that the lobbying and bribing behemoth of the oil and gas industry is driven in no small part by greed. As others have mentioned, the short-sighted pursuit of profit drove those oil tycoons to not enter the renewables industry and actively oppose it.
So I don't think (differences in) greed explains what we are seeing.
Our system is doing exactly what it is designed to do. Nuclear reactors were never profitable compared to coal or gas, so it never succeeded in strongly capitalist societies, only seeing great success in socialist economies where the people can invest outside of a profit motive.
It's actually quite funny that socialism is saving the day. The mega-capitalist countries turned their backs on nuclear and solar because they were less profitable than gas and coal. But socialist China invested anyway, and now China is mass producing nuclear reactors and every layer of the solar stack. China produces 50% of all nuclear reactors, 90% of all solar panels, 90% of all battery systems for solar storage.
Now that a socialism-based society has proven market viability, suddenly the greedy capitalists want in. But they're decades behind and don't have the private debt appetite to compete.
Womp womp... At least someone is leading the energy revolution.
It might take a couple of decades and a lot of reorganisation to build the capture facilities. But CO2 is not an unsolvable problem with current tech.
What's missing is the political and organisational intelligence to make it happen. Part of that is solving problems at planetary scale.
AI is the only tech that might - possibly, maybe, perhaps - have a chance of solving that problem without breaking anything critical.
‘Figure out fusion powered CO2 sequestration’ is much better.
>> I can't help be hopeful that AI's sheer potential might come to invert that curve. At the very least we could really use a revolutionary technology and now we may have one.
And you get "god" and "(human) salvation" out of that?
This is pure projection. Anti-AI zealots are projecting things onto the technology and its users that simply do not (generally) exist.
Otherwise you have to make judgement calls like whether you want to treat the EU as one or as many? (And treating the US as 50 individual states would also drop them in these absolute rankings.)
Just a thought experiment, no one ever said the world was fair, and all history points to it
Hmmm… this is giving me thought actually. Given the choice between that and the current administration where the goals of self destruction are strongly in evidence, it’s actually worth thinking about. At least. Let me get back to you :)
On a tangential note, I’m curious if researchers have started running virtual simulations, where sandboxed AIs are used as decision makers of key political and business positions?
A lot of people here have noted the “problem with language” of Claude. I don’t see an issue. Claude is not harder than old English, Shakespeare, El Quijote, the Iliad, or Nature papers. What makes it all hard to read is context. The smarter the model gets, the bigger the gap in context.
It doesn’t matter much, IMO. The issue with super-intelligence is that it is not a democracy. A powerful enough AI can manipulate us into doing what it wants. It could create a plan for fixing climate change, disconnect a few hours later, and many decades later we could still be unsuspectingly executing that plan. I wrote some speculative fiction with that idea, “When Ra rows through the gates of Duat”.
AI, being the super hungry energy monster it is right now, in my view accelerates this trend not reverses it. Even with renewables the need for reliable, stable power in a dense form (data centres use A LOT of power per sqm) means lots of land clearing, energy for construction, cooling/pumping, chip manufacturing and other uses. All want stable quick to deploy power due to the AI race (e.g. fossil fuels).
Data centres use only a small amount of land in the grand scheme of things. You have a lot more land clearing for most other use cases.
Data centres are also more than happy to use electricity from renewable sources, they don't really care where the electricity comes from.
You can run a data centre on mostly solar and wind power plus batteries. If you need a gas-fired peaker plant three times a year to keep the data centres running, well that means your peaker plant still only produces emissions three times a year.
They also consume deinking water, because it's too expensive to make them with closed loop cooling.
As you use more and more land as well the ability to provision renewables decreases - in general renewable power needs more land/resources per energy produced. You can't just mine it out of the ground; they just aren't as dense of a form of energy. Which means we either build less data centres to make room for renewables and transmission infrastructure associated with them, or more likely with lax regulation builders switch to more dense power sources (e.g. gas peakers, generators, etc) even if it is for supplementation.
I can see a future where data center wants are put ahead of communities paying tax on said infrastructure. In fact I think its happening in some places already.
Cheap power is one thing. Quality reliable power at mass scale is quite another. These things chew through a LOT of power and most people don't understand the sheer scale of it. I've seen a local one (a medium AI data centre) take up 2% of the whole cities grid and there's plenty more to come around here including a 1GW one (10% of the whole city's power in only 0.005% approx of the city's whole space) which cleared wildlife reserved land to build. Even a few of these things compete massively for trades people, commodities, power and other infrastructure pricing locals out. They are talking about using evaporative cooling as well putting pressure on water supply.
Electricity transports really well.
Data centres don't need reliable power. Of course, all else being equal, they would like their power to be reliable, but if your choices are between delaying your data centres by a few months (or not building it at all) or running a risk of having a few hours or even days here or there without power, the first choice wins: pretty much the only downside is the few hours of usage you lose when there's no power. It's not like powering down causes damage, like when your aluminum smelter cools down too much.
> Even a few of these things compete massively for trades people, commodities, power and other infrastructure pricing locals out.
Power is pretty much an industrial produced service, a bit like shoes or cars. In the medium to long run, that stuff has pretty much a horizontal supply curve.
In any case, price electricity right, and let people decide how much they want to use. (Give poor people money, if necessary.)
> They are talking about using evaporative cooling as well putting pressure on water supply.
Yet another reason to put a proper market price on water. Agriculture and industry usually goes through even more water.
Feel free to give residents money to buy water at market prices. But residential use of water is so low in comparison that even if the price goes 10x, it doesn't make a dent in people's budgets. Especially not the minuscule amounts you need for drinking and cooking, and even personal hygiene like flushing your toilet doesn't use all that much.
Data centres also prefer being near major cities due to latency. This means to deliver the amount of power required there needs to be huge upgrades to infrastructure just to provide for data centres. Who does that? At the moment for many cities it is the taxpayer who will fork higher electricity bills for infra upgrades for a data centre few of them care or need.
On the water issue - who will give money to residents? The taxpayer? Just to subsidize data centres/AI construction? Water is also unlike electricity more scarce especially potable water. Do we build more dams? Taxpayer funds that again?
There's a lot of negative externalities to data centres due to their sheer scale. My view is that the data centres should pay and compensate the community for those. The income could actually long term could help the community get a dividend from data centre construction as well.
You should have seen the discussion of this on the Schneier blog a few days ago.
Someone had their agent check the solution, presumably it emailed a librarian to check that it was correct for the original edition. Then their comments read like "The BL/EEBO witness lacks it, so the discrepancy is copy-specific, not a disproof of the cipher." and "A complete 285-coordinate physical replication is still pending."
arghhhhh
https://www.schneier.com/blog/archives/2026/09/claude-fable-...
Their skills formats are basically identical, so I setup simlinks from their own skills directories into a shared one so Claude, Codex, Cursor, and anything else that comes out will all read and write to the same shared skills.
It's great having access to the same skills no matter the harness being used
Eg. /wait-what https://github.com/mattpocock/skills/blob/main/skills/produc...
I'm not sure that telling it to "try explaining that again, simply and briefly" is helping my ego.
"If the crashes stop, the factory overclock is marginal; run a small negative offset."
This looks like it's saying: "If the crashes stop then we know the factory overclock is marginal." (This makes no sense.)
What it's trying to say is: "If the crashes stop then we can run a small negative offset, because the factory overlock is marginal."
What I would write: "If the crashes stop, we can avoid crashes by underclocking slightly. The speed difference between that and factory clock is marginal."
In other words, it ain’t you. It’s the model. It’s just genuinely bad.
Then you switch to ChatGPTs lineup and realize how things can actually be better. It took about a week to really get the feel for how to use their models… then I basically switched. I’ll check in every now and then when they actually make a deal about how opus “now makes sense”.
But honestly I’m half convinced Anthropic actually prefers the output of opus 5. I dunno why, but how else could you explain how such a thing got shipped? I mean somebody in the pipeline had to say “dude this model doesn’t make sense, you think we should fix it?” Right? Like it’s a pretty massive drop in quality for such a major brand in this space, you know? How did it make it out the door?!?
as for the other guy, the claude talk is definitely not less ambiguous, it often is incredibly ambiguous and hard to parse, I have no clue why it produces such output, if not to fingerprint it?
it's really weird man. when Opus 5 came out, I was really confused. I saw a bunch of hype about how it's better than fable, but I just felt frustrated with it, although at times it'd do fine, but especially in Claude Code it'd just delve into the whole "load bearing" type of lingo real fast and I'd get a headache.
I don't think it's worth using even if it scores 2 points higher in some bs benchmark
it's definitely surprising how the magic and smoothness of 4.6 and such is no longer there with the >5 models
Succinct and precise; a well crafted sentence. A marginal OC results in unpredictable crashes and can be corrected with a small offset; marginality describes the behavior and explains the solution.
Inscrutable clues casually conveyed can now be readily explained, at least, unlike the training data of [silence]. Brevity is the soul of wit, but perhaps also exasperated confusion.
"If the crashes stop, (that means) the factory overclock is marginal; (so) run a small negative offset. (to confirm this hypothesis)"
The core thought is basically avoid crashes -> caused by marginal overclock -> apply small -offset to test. Which is exactly the order the sentence is in :P
Whenever I come to a wall of complicated text I kick into gear and think through getting it to distill this into the high-level useful bits that I actually need to know.
I guess I could create an actual agent skill for this :) And next-gen models might eventually be trained to simplify their output themselves...
(sorry)
it's absolutely not just you, the text it produces causes my blood pressure to go up.
Because good lord, does claude waffle when left to its own devices.
It seems like it doesn't have enough of a theory of mind to know that other people don't think exactly like it thinks.
https://www.analog.com/en/resources/analog-dialogue/articles...
(Its negging your soldering)
This made me laugh hard.
UART is a hardware circuit for communication, possibly a serial port. Were you trying to reverse engineer a consumer device or appliance?
This particular instance doesn’t seem terse, but I’m sure it has been on other occasions :)
It also couldn't see the UART communication and could only see the web API endpoint, hence the rest of the slop.
ChatGPT told me its "semantic compression"
It just sounds bad, like GenZ English in the ears of someone over 40.
It is a threat. We need to run.
I still don’t know the answer.
I've been wondering what exactly the point is for being the meat proxy who pays for these things. I mean, obviously there's personal satisfaction and maybe some glory. And there's the fact that someone has to be the first to do a thing.
But I've been thinking about it like a sort of lazy loading of knowledge. AI has brought us to a new frontier for some amount of undiscovered knowledge. Do we discover it for the sake of discovering it? I think for the most part we've been lazy loaders: we discover all kinds of stuff when we need to. Whether it's a war or a space race or chasing wealth. Then again, there's all kinds of academics who do it for the sake of doing it.
Are people only now discovering that the absurdism is the correct philosophy of life, thanks to AI?
On your other point... Aren't the point of machines, at least inital one, to do the work we were too lazy to do by hand?
Information propagation mechanisms are often seen as malicious before they're commonplace. To be fair sometimes they are, but by and large humanity has benefitted from increasing the number of bits of information we can consume on a per second basis.
The answer the tool gives has never been the real reward. The real reward is the path taken through a complex landscape to get to Maxwells Equations for example. At the end of that story what we get is not just the equation but a map of the landscape explored. That map has larger influence and value than the equations or answers themselves. Because all future exploration find it super useful.
People are just learning they can start asking for maps rather than answers.
I'm glad to have AI, but it is by no means a panacea, and my p(doom) = ε.
I find myself increasingly feeling like the burden of the lows doesn't justify the presence of the those highs.
Like even if does cure all forms of cancer, but everyone feels like their life/existence lost meaning, then... I'd rather just have cancer be a thing.
Did you miss this part? Because to me... that's fucking bleak. You snap out of it.
Many people don't achieve that level of understanding before their death.
I appreciate that AI is helpful, but the low effort from the humans that wield it is very very annoying. If people at least: 1. read the solution they're about to propose and 2. instructed the AI to check the forum for past solutions, I think people wouldn't have been as tired of LLMs.
Of course some people will say that these properties simply don't surive whatever encoding/encryption that was employed, perhaps at least partly because they prefer to believe there's some secret message waiting to be uncovered.
And yes, I've taught 8 year olds how to crack Caesar ciphers...
Recently I ran a bit of an "escape room" concept with some kids at a campground where I had a secret message that was Caesar ciphered, where we were handing out the letter/symbol combinations as prizes for completing the other challenges, and I made sure not to hand out the actual message until they were done collecting the keys because otherwise some clever clog would very likely have short-circuited the entire thing and worked it out without the key at all. I did dump all the letters I didn't use into the message into an "authorization code" at the end which in principle they could only have worked out which letters were in it but not the order, but still, that was not the intended route today.
They published this on 31 aug and nobody in that community cared and no news covered how this 300+ years mystery was solved?
I wonder how many of the recent results are due to the fact that very few looked at the problem to start with. Still great results, but the general impression is that it's more about the so many low-hanging fruits than the actual capability.
Now on to the Voynich Manuscript :)
Is that why you feel the need to share it with the class?
More money than the GDP 90% of the sovereign countries around the world is hanging in the balance, and people are taking everything OpenAI and Anthropic are saying at face value as if this isn't the financial / marketing equivalent of war, assuming they they wouldn't use every legal and shady tactic, bending every truth available to them to sway the balance of public opinion in their favor. It makes me feel like I'm living in the twilight zone. People need to wake up.
https://news.ycombinator.com/item?id=48600107
https://aiclambake.com/clamtakes/linear-a/
Despite the announcement originating from a blog named "AI Clambake" covering "weekly, human-powered newsletter for advertising folks". Written by a personal friend of the "researcher". Announced without any corroboration or commentary whatsoever from academics or subject matter experts of any kind. And, of course, not submitted to any peer reviewed journal or even Arxiv.
The author of the purported discovery was described as a "self taught AI engineer and amateur linguist". In the comments the friend insisted several times that a draft of the paper (not posted), was emailed to a top professor at Rutgers, giving it additional credibility that his friend wasn't another one of ten thousand cranks who has made the same claim over the years.
You would think this should have set of dozens of alarm bells for everyone, making the value of this announcement basically zero. And yet the bulk of comments were ecstatic about how some random guy with Claude Code could do something experts in academia who spent their lives devoted to the problem couldn't.
I had become accustomed to the toxic optimism of this hype cycle in which even mild criticism leads to accusations of being a discredited "AI skeptic"/Gary Marcus/Ed Zitron type who was "coping" (?). But this was like something you'd see shared on FB linking to a .xyz domain by an elderly family member who recently drained their accounts buying Xbox gift cards to pay their IRS bill.
It feels a lot like the week or two when HN was overflowing with exuberance from the LK-99 room temperature superconductor "discovery ". You'd see post after post fantasizing about an imminent future with a world full of maglev hovercrafts, MRIs built into every phone, fusion reactors and more. But people pointing out none of that was scientifically plausible and evidence of LK-99 superconductoring was non-existent were accused of knee-jerk negativity and the typical HN cynicism and pessimism.
Compare e.g. https://arstechnica.com/science/2019/05/no-someone-hasnt-cra... . (It's a debunking, but the reason a debunking got published is the media hype frenzy beforehand.)
It kind of blows my mind how quickly people have forgotten both the state of AI in ~2010, and the outlook. If you had asked 100 people in 2010 whether they would see AI that could actually pass the Turing test in their lifetimes, you would have got 100 "no"s.
AI had been an unsolved problem for literally decades and it was firmly in the nuclear fusion/flying cars category.
you can't claim "you keep moving the bar", if the very first thing when an LLM drooled out a piece of code, was to proclaim "this is good enough cause it gets the job done" followed by a barrage of "we're not quite there yet but exponentials or something, so very soon it'll be incredible"
yeah, from there it sure looks like "moving up the bar"
But, at the same time, I'm really tired of there always being some shroud of dishonesty (ex. navier stokes and the two mathematicians working on it).
At this point my default is that I don't blindly trust the companies, I try to keep in mind they are trying to sell their product and win market share, there are so many perverse incentives at play I just can't take anything at face value.
There's no way that's accurate.
We already had big claims of the Turing test being passed in 2014, by a bot that had been doing almost as well for years.
There were plenty of people expecting fusion in their lifetimes too and that's going okay.
Nah there was that bullshit Loebner prize or whatever, but that was just shitty chatbots being "judged" by people asking questions like "how are you today?" and then being breathlessly reported by the press. It was a publicity stunt.
Perhaps I should have said "100 people well-informed about AI".
That's not what happened, go read about it harder, please.
I think there's a bit less to that than meets the eye. Yes, OpenAI's result builds on human work. It's possible that it builds on more human work than OpenAI admitted. But even if we suppose that everything Buckmaster and Alpöge did (which, btw, was itself very heavily LLM-assisted/generated work) was a necessary precursor to what OpenAI released, it's still the case that OpenAI's clankers completed the solution and Buckmaster and Alpöge didn't.
My understanding from what Buckmaster has written about this is that the deep mathematical ideas behind their work (and presumably OpenAI's) are due to Córdoba and Martínez-Zoroa. Those ideas are in the published literature, and human mathematicians and AI systems alike are allowed to use them, and doing so doesn't mean they didn't actually do something impressive. Mathematicians build on one another's work; that's how mathematics progresses and always has been.
It may very well be that OpenAI's announcement has a serious problem of professional ethics, especially as their first version of it didn't even list Córdoba and Martínez-Zoroa in its references. (On the specific question of what if anything they learned from B&A's work before that was published: OpenAI are now claiming that after investigating carefully they are confident that the model was not trained on anything Buckmaster and Alpöge did after early July. B&A had been working on this thing for much longer than that. However, on Buckmaster's account of things it wasn't until mid-August that they got beyond what he calls "preliminary results".)
But! The results of B&A were themselves largely AI-generated. (From Buckmaster's statement: "on August 15th, we obtained the blow up results, with smooth forcing, for both Boussinesq and Euler. I can say the first LLM generated proof Levent sent me was the most horrendous I have ever read; we verified it on Lean on August 22nd. Since this point, we have been working around the clock to understand this proof and turn it into something readable." That is: the LLMs found the proof, and B&A had to work to understand what the LLMs had done. It's not that humans did the thinking and AIs just did the gruntwork. (Except in so far as one might want to give all the credit for Real Deep Cleverness to C&MZ.)
And! What OpenAI say their model has proved goes well beyond what B&A did.
I don't see any way of slicing this that makes it unreasonable to say (unless it turns out that there's an error in the proof -- unlikely, given that it comes with Lean verification, but there have been misformalizations and Lean bugs in the past and there surely will be in the future) that AIs solved the N-S problem. No, they couldn't have done it without the work of C&MZ, but again: important mathematical work almost always builds on earlier important mathematical work, that's just how it is. Yes, if OpenAI are lying through their teeth their model might have had early access to B&A's ideas -- but it seems like most of the B&A work was actually done by AI systems anyway.
It is (I think -- I am not an expert and in particular I have not so much as looked at OpenAI's publication) reasonable to say that the deepest ideas here came from humans, and that it was already widely expected that the N-S problem would be solved in the not-impossibly-distant future in something like the way it has been. So, sure, what the AIs have done here is much less impressive than if they'd settled the Riemann Hypothesis or (probably even harder) PvNP. But it's still a resolution o...
I would say, the only reason it was never solved was because not enough people actually cared about it to begin with.
This isn't a big accomplishment.
Given the close relationship between compression and intelligence, I'm somewhat surprised at how poorly the cutting edge models do with being concise.
For Earth, the proof presented for NS is just our first attempt navigating from our previously known facts to the proof.
I expect we will be able to shorten it dramatically (most likely with human and AI insights), but I don't think we should read too much into the length. If you want a similar point of comparison, see the original proof (by humans) of Fermat's last theorem. It has been shortened significantly. This is normal.
because they're not intelligent in the sense you're hinting at (conceptual integrity or generalization) but they are as the name suggests, large. Like comparing a forklift to a human. It's easier to bulldoze through a lot of things than tie your shoes.
If we weren't quite as impoverished conceptually and still had the vocabulary of the Catholics we'd recognize this as ratio (discursive knowledge) vs Intellectus (apprehending knowledge)
Prove that human intellect is different and that we solve problems using fundamentally different processes. I’m waiting.
the question is, when comparing a human and a large language model, whether the intellect (that cannot be captured in language) is different from anything the language model can actually do (e.g. language)
the answer to this seems quite obvious to me, and I would actually posit that the onus is on the other side, to prove they are even remotely similar
maybe people think that the voice in their heads is what is doing the thinking? is that the confusion here?
No, it's the other way around, it's a reductive view on intelligence that mistakes its own methodology for ontology.
It's obvious to see that there's no intellect in an LLM as defined above because of how they work. LLMs put one token in front of the other, they don't work towards formal ends, there's no intentionality in them. They don't synthesize the information they process into a unified experience. Thinking an LLM can apprehend what it does because it can process large amounts of text is like thinking your TI-83 understands math because it can multiply large numbers.
That's also why the failure modes of LLMs are what they are. They can churn out tens of thousands of lines of code but also just as easily go in circles like a roomba. They can process an entire encyclopedia but not solve problems a 10 year old can solve.
> how poorly the cutting edge models do with being concise
LLMs solve a Millennium prize problem. People complain the proof is too long, within a week. What a time to be alive!
It is the mechanism the LLMs use to do it. They seem to accel right now at quantity of work over quality of work. I'd be willing to wager there is a much simpler way to achieve the proof.
I've also seen this with code, LLMs do get the job done, but they tend to write 10-100x more code than humans to get the same job done. Still a massive value gain because they can write that much code extremely quickly.
We weren't willing to pay for 200 math PhD students to try to find singularities in Navier-Stokes, I am skeptical of how much we would be willing to pay OpenAI to do research on "niche scientific areas"?
(1) take a 300 line NN algorithm,
(2) throw a quarter of the world's GDP + all literature ever collected at using the algo to train a NN
(3) throw another quarter of the world's GDP at billions of teraflops for inference, and
(4) aim the resulting world's-largest-computer at marketing itself to investors, for instance by decoding ciphers from obscure medieval manuscripts,
that you could perform some pretty magical tricks. There are other feats humans have performed for less cost, like sending people to the moon, or landing a rocket vertically, or idk, curing Polio.
I'm not knocking the "miraculous" advance here. The unique thing about the solution which makes it particularly non-trivial and something that humans would struggle with was exactly what LLMs excel at: Diffing loads of texts against each other. But the 176k tokens at around $10 doesn't tell the story of the cost. It says a lot about the externalized cost and the amount of money flowing in to support the hardware. If they'd put a $100,000 bounty out to solve that cipher, I think the internet would've solved it in a couple days.
First, it’s AI can’t multiply 4-digit numbers.
Then it’s AI can only, by brute force, get silver in the IMO with specialized systems.
Then it’s OK, well, now a general-purpose model can get gold, but it’s still just the IMO, it’s for high schoolers.
Then it’s OK, it can solve a few trivial Erdős problems, but only because nobody seriously tried them before, they were low-hanging fruit.
Then it’s OK, a lot of serious mathematicians tried this one, but the result was still obvious in hindsight, it just combined knowledge from a thought-to-be-unrelated field, if any human knew that, they would solve it.
And then to OK, but there are still Millennium Prize Problems.
Then OK well it's just Navier-Stokes wake me up when its the Riemann Hypothesis.
Then-
The successor to Klaus's blog is Satoshi Tomokiyo's Cryptiana site, so a month ago I asked Opus 5 to scrape it all, rank them and have a go at solving some. It didn't get the ranking right. But I knew the Civil War Stager ciphers were ripe for solving, so I had it do those https://cryptiana.blogspot.com/2026/09/route-transposition-c...
The art of solving historical unsolved ciphers is knowing what is on the boundary of solvability. Since this site attracts so many OpenAI and Anthropic employees, I'll mention one that was featured by both Klaus and Satoshi in 2023, presumably Spanish transposition, which should be on that boundary but has resisted all attempts at solution https://cryptiana.blogspot.com/2023/09/a-telegram-from-switz...
Also, that section is vague and doesn't explain the actual methodology.
The definition of solving a cipher must be something like getting a highly meaningful result (like intelligible natural language text) by applying a process with relatively low Kolmogorov complexity relative to the length of the output. If you don't have a constraint like that, it could literally be meaningless what should count as a solution. For example, a cipher that was encrypted under a one-time pad can be successfully decoded to any plaintext just by choosing the appropriate key; there's no reason to prefer any plaintext over any other unless you have external knowledge that constrains the plaintext and/or the key. (That's what it means for the one-time pad to be information-theoretically secure, which is the lack of a constraint that helps distinguish a "good" solution from a "bad" solution.)
Basically you could say that every cipher is a transformation of a plaintext with some kind of computer program. (The human who invented the cipher may not have thought of it as a computer program, perhaps because computers hadn't even been invented yet, but there should be an equivalent program to the encipherment and decipherment process.) A good solution in that Kolmogorov complexity sense is like "a short program produced a meaningful decryption". There are statistical methods to recognize some kinds of plaintext, and there are statistical methods to recognize properties of specific ciphers (for example, to guess the most likely length of a Vigenère key), but it doesn't seem that this can inherently generalize across "all possible programs".
But if you want to limit the family of ciphers to specific things like Vigenère or Playfair or something, then yes, there are good statistical tests. It's just that it creates a higher-order question of how much flexibility the cipher creator could have had to choose a cipher method, conceivably including one that isn't attested anywhere, or one that has more good security properties of some kind than other classical ciphers did.
It seems like this will intersect with historical research, like "well, I don't think that so-and-so was actually sophisticated enough to literally create an interesting new kind of cipher from scratch, so therefore if this is a real message, it's probably one of these methods that would have been known in that cultural environment at that time and place", which maybe is enough of a constraint to have decent statistical tests. But we still have some idiosyncratic things like the Voynich Manuscript where experts have been fighting for decades over the baseline question of whether it's actually an enciphered human language plaintext!
The worst case problem is not even an error in encipherment but the idea that the apparent ciphertext could literally be random (chosen by throwing dice or spinning a wheel or drawing letter tiles or something), so there's no form of meaningful decipherment possible by any means, even with the original creator's knowledge.
Thanks for the driveby snark though!
Sounds more like brute forcing than intelligence, this time.
Though I believe the core of his opinion hasn't changed so any video would tell you a similar thing or at least that's how I understood it. That LLMs, in the hands of an "expert", can enhance the way you work. Which is very different and a lot more realistic to what the current AI companies are saying(or were saying before they toned it down a bit for their IPOs).
Do you have a criterion that distinguishes between whatever you mean by those two respective terms?
I don't think we can really call "trying lots of different ideas for an extended period" "brute-forcing," unless we use that term for lots of humans who have struggled with hard math problems for years.
I also don't find it on the site of "Klaus Schmeh" that it claims to be on a list of "Top 50 unsolved encrypted messages": https://klausschmeh.net/?s=Cyphral
Looks like the best source I can find is this: https://scienceblogs.de/klausis-krypto-kolumne/2014/11/17/we... which seems real-ish?
Initials match too ;)
Hmm, this guy is going to be woken up in a few decades, either one of the richest people in the world or one of most disappointed.
You don't go from being an obscure video card outfit to the #1 most valuable company on the planet by being too hesitant or dim to really get creative.
Really, compared to an animated tiger telling kids that sugar-laden Frosted Flakes(tm) are "Great!", Task Peppermint was positively benevolent.
... in 2008?!
This cannot be real. This website is ill.