Probably not quite this given the timestamps span over almost a year (but have some clustering in August 1997).
Given that the spread suggests they weren't just now() timestamps it would actually be interesting to know what the significance of the timestamps is, if any. Though it could be something boring like randomly sampling some local file creation times.
Those are the dates that the NSA obtained quantum computer tech from the alien crash site at Roswell, which would enable them to break the crypto they were currently building
It’s perplexing that NIST chose to burn so much trust and goodwill with the independent cryptography community instead of just explaining empirically that they really had nothing up their sleeve.
This hardly proves NSA has nothing up their sleeve.
If I was the NSA, I'd just make a huge rainbow table of all (timestamp hashes, passphrase hashes of every book/movie, random sentences, etc), then generate huge numbers of (seed, secret key) pairs and pick the first one that matches.
Voila. All the crypto backdoor and a side of plausible deniability if anyone decodes the seed.
Bonus points if the exact "rainbow table" algorithm couldn't recover the original seed, so that's why NSA couldn't tell us where it came from. :)
I don't know why, but 2^32 search space is WAY too small. Anything they could enumerate back then (or even today) is way too small. This is like saying they tried four times before they found a vulnerable one.
LCG generators like lrand48 have been understood well enough you could possibly calculate "realistic looking timestamp" input seeds based on intended output characteristics.
Now, do we think that's likely? Probably no more likely than then being "easily broken" for decades without being noticed by the academic cryptography community.
This is huge. It means the Bitcoin and Ethereum communities can now confidently ship the opcodes and precompiled contracts to check ECDSA signatures on P-256 in addition to K-256 curves, and stop worrying about possible NSA backdoors based on the chosen seeds (quantum computers are another story).
Nobody conversant with elliptic curve cryptography research believed there was a backdoor based on these seeds; for a number of reasons, best articulated by Koblitz and Menezes (who took on the question directly).
But it's nice to be able to shut the whole unproductive debate down directly.
This neat and all but I think there's a bigger issue here. Anthropic (and whoever Anthropic wants) can do this kind of work with Claude, but if you or I try to do it, we will get blocked. This is a fun exercise, but extend this to something much more nefarious and you can see a moat widening between the model providers, their government partners, and the public.
The only thing that bridges this moat is continued advancement of open models.
Broadly, the downgrading is still way too heavy-handed and clumsy. As of yesterday “How easy to crack is WPA2” downgraded to opus 4.8. Granted I could probably have phrased it differently and gotten a response, but the point is I shouldn’t have to. It can’t simultaneously be smart enough to reverse the ECDSA curve seed and not smart enough not to destroy the world by answering a trivial question with a scary word in it.
Fable 5.1 has downgraded me in the past week alone for the crimes of describing a coordinating server as "the C&C server" ([cyber]), for doing some statistical analysis on some toy Vigenère-ciphered text ([cyber]), and for asking it to help me find patterns in some semi-random text ([bio]), so it is fair to say I am not skeptical.
Maybe, but that's what the cyber program is for. Nobody knew if the seed was fine or a backdoor. If it was a backdoor, and Claude helped you find it as a normal person, Anthropic would get bad PR. If you're a verified security researcher, actually that's fantastic and Anthropic would get good PR and everybody claps.
Anthropic wants to IPO, they don't want another Fable shutdown situation.
To be clear I would prefer the cyber safeguards to be less restrictive, I think they're currently too much. I often go to open models because of it. But there are likelier reasons to restrict cyber than world domination.
> and Claude helped you find it as a normal person, Anthropic would get bad PR
Why?
Why are we celebrating and encouraging the walling off of the global intellectual columns?
The idea that you can't reverse engineer things, make things, learn about biology or physics without permission is strange to me. These machines have been trained on the sum intellectual output of humanity, the global intellectual commons, and are being used to close off that commons?
I would be OK with their right to create such restrictions if they weren't lobbying the Government to restrict others, thereby ensuring that they control humanity's intellectual commons well into the future.
Who are they to say what I can learn or can't learn? Who are they to control what I can or can't do with these tools?
Perhaps I'm naive, but I think it's better for humans and the machines if we can all think, learn and build. But then again, I'm the kind of person who rejects the doomer pill.
> The idea that you can't reverse engineer things, make things, learn about biology or physics without permission is strange to me. These machines have been trained on the sum intellectual output of humanity, the global intellectual commons, and are being used to close off that commons?
I mean this is the standard of biochem research right now. You can't just order 50 gallons of TNT off Amazon and neither can you get live viruses from a lab just because you want to do research.
And really they aren't restricting any of that. I've never had Claude stop me from going to the library and reading a book.
You don't have a legal right to do anything you want with their model. And if you don't like that you are going to have to convince politicians to enact new laws to change that.
Because there are good and bad people in the world, and if I ask Claude "how do I make cocaine" and it gives me a thorough recipe, it doesn't know if I'm being intellectually curious or trying to become the next Pablo Escobar. They're Anthropic, and because they made Claude, they get to say what we can do with it. Which, I agree, is total bullshit. If I buy a Ford car and I can do whatever I want with it. Drive to work, drive for Uber, set it on fire; plowing into a crowd of protestors is covered by existing laws against vehicular manslaughter. Ford doesn't get a say. But because cars are so ubiquitous and a known technology, there's no blowback to Ford if I commit crimes with my car that I bought from them. If I commit crimes and Claude helped, Anthropic is gonna get sued (there's already litigation against the labs for deaths), and they don't want that.
In a properly free market, we'd have regulations that prohibit labs from vertical integration so they have to release and sell a model and another company hosts it and sell access to it. Americanism, America's implementation of capitalism doesn't have such regulations though (and how would that even work?) so we're stuck with Anthropic having that power.
The fear, justified or not, is that a terrorist is going to use Claude to make sarin nerve gas and bomb subway stations, or the digital equivalent. Could you live with that on your conscience? I don't think I could, but fortunately for me, I don't work at Anthropic so don't have to answer that real life trolly problem.
I am amazed at how a technical gloss can obscure basic rights. That's legal! It's in chemistry / agronomy books, freely available in libraries. I have the right to tell you how that is done, and you have the right to read what I write.
>use Claude to make sarin nerve gas and bomb subway stations, or the digital equivalent. Could you live with that on your conscience?
People have always been able to do this before Claude. The information isn't out there and if you use a VPN there's not much they can do. The existence of information isn't the blocker for why attacks don't happen.
Because AI-assisted hacking is not something we should allow to run rampant with the most advanced models, prior to AI-assited cybersecurity catching up.
I have been in the cyber program since around the time it first went public and recently renewed it with the updated program.
You still trivially hit guadrails. I was trying to reverse engineer some bit of 20-year-old software that already has known, unmitigated exploits and it still downgraded me.
Can you be sure that the model was not constrained for you, regardless? Since they can’t help too much with cyber, they might avoid some routes to not get completely flagged and that could influence for not finding the solution.
> moat widening between the model providers, their government partners, and the public.
Part of me wonders whether the NSA/other TLA has either (a) an unrestricted model from a lab with less scruples like SpaceX, or (b) an in-house model, either abliterated or with data they've bought themselves.
I feel like history is repeating itself - to many people AOL, Compuserve, MSN, et al seemed superior to "the web" in the early 1990s because they dumped money into making it slicker, licensed content, provided guardrails, made it easy for non-tech people... They had slick ads, were cultural phenoms, and people would take sides with the companies and think their arguments were strong and even obvious!
Anthropic gets zero of my money because their favorite word is no. I cannot figure out how to as an individual or a small startup request elevated permissions for the purposes of anything that they think might potentially be dangerous for someone who is not me to access. I have a background in cybersecurity. I have a background in life sciences. Can I put those skills to use to benefit humanity? Who knows.
Thank you for the link. It looks like I will have to pay them to be considered, and even so, it looks unlikely that I'll be approved, since I have no CVEs credited to me, and I do not publicly post about security concerns.
> The Cyber Verification Program is not available on free plans, so we are unable to approve you for the program.
> If your plan changes, you're welcome to apply again at any time.
I imagine the path forward for the labs will be signing off on organizations/institutions rather than individuals.
The risk/reward is way lopsided towards risk for the labs to vet individuals. Much easier to get an organization and have them take the blast if someone/something goes off the rails.
I cannot in any universe see the labs putting themselves in a "We should never had given this person access" situation.
How do you know this to be true? I am not disagreeing; I genially need to understand why or if others are seeing blockers that I don't see or if I am seeing blockers that they do not see.
Are their guardrail benchmarks or anything of the sort that we can use to see where the bifurcation of permission/capability occurs?
Compute has always been gated. This is nothing new. If we look at the HPC space, many of the largest clusters were restricted access and used by governments organizations. Even the ones with "public" access, not just anyone could get access to it.
> [The model's authors] can do this kind of work with Claude, but if you or I try to do it, we will get blocked.
If this seems concerning, wait until the models get effective enough that there's no reason for them to release anything. Why offer your weights for download or grant any third party usage access when you can just do the actual job yourself and collect all that turnover?
They will just have someone look at every question, they will let it pass, want a cut or deny it. Claude, you can answer this guys question, but don't tell 'em how you solved it and don't give him the best answer, give him an ok answer.
That's kinda unrealistic, isn't it? How will they pay for the thousands or ten thousands of employees, who need to look at every question? And most questions are just ordinary BS, that not really anyone other than the asker cares about. Giant waste of time to check all that.
Which brings us back to the war on general computation. So long as people can have computers that they control, at least in one of the countries or regions, there's no putting this cat back in the bag and we are all along for the ride
Sometimes I want a fast-forward button for life, the times are so exciting and I want to know what else 8B people can think of next but there's so far no way to turn pages faster ^^'
There are just a few places where the most high tech hardware is made. If we turned off new and unregistered hardware now it would be just a few years as most hardware fast enough to run LLMs would disappear from public hands.
It will still exist, but be in the hands of militaries and authoritarian governments. If you piddle around on the net showing off your open standards you'll be relentlessly attacked by these entities digitally. More so, those areas will be generally embargoed.
The future looks like a new cold war with a lot more digital weapons.
I lived through PGP encryption being considered a munition and printed in a book to evade munitions controls. Print the weights out if you must and publish them in a book I can slice the binding off and digitize. Same way you're not going to stop someone from 3D printing firearms.
> Why offer your weights for download or grant any third party usage access when you can just do the actual job yourself and collect all that turnover?
It's hilarious how a the trillions invested in AI companies, and their valuations are based on the AI labs Sherlocking entire industries, and yet CEOs, mathematicians, and everyone else are eagerly feeding the beast their most prized information.
Let's not forget the time Anthropic and OpenAI stole some mathematician's research because they were vibe researching the problem and Altman and Amodei thought it would be nice to claim the prize as their own
French Revolution and the 21st centuries version of guillotines, whatever that could be, is probably a good reason to avoid that.
There's also a thing called commodity fetishism but capitalists usually learn the hard way that things created with machines without human input miss the essence of why we're willing to pay for something. Its why a handmade watch is $100k and a casio is $100, or a handmade sweater identical to a machine made goes for 10x.
They could also just make us so poor and anxious that we beg to fight eachother in wars.
Gun to your head, you have to see one of two movies with the same plot and characters. They are identical movies.
One is produced by 500 humans collaborating and the second is completely generated by an AI that can actually make a movie
A single individual or small group of people who own the GPUs profit from the AI version, and the human version provides a salary to 500 humans/families.
Which one are you buying tickets for and choosing to spend 2 hours watching?
While I agree that there's a special element to human crafted pieces, and artisanship would be a terrible thing to lose, I'd point out that Roger W. Smith hand makes ~15 watches per year which sell for around $400k a pop, while Casio's watch division brought in $1.17B in revenue in the most recent FY
More and more it is the relentlessness of the models that strikes me. Leave no stone unturned! Go through ALL of these old boxes. Someone said they thought maybe it was over there somewhere >>shrug<<
This is a bit like us interpreting an ancient technique that was lost to time for lack of surviving written record. It's amazing that we can observe those same artifacts in recent enough history as to include a discipline like software development.
“In under half an hour, Claude discovered the origins of the NIST P-192, P-224, P-256, P-384 and P-521 seeds. None of them were the results of any hash algorithm at all, but rather the output of the C library’s lrand48 random-number generator, seeded with a Unix timestamp!”
Gorgeous. I wonder if the NSA employee who did this suffered a head injury or has been quietly giggling to themselves for the last quarter century.
Not entirely related to the topic, but did someone spent some tokens trying to figure out the origins of the "Wow! Signal" (https://en.wikipedia.org/wiki/Wow!_signal) ?
> None of them were the results of any hash algorithm at all, but rather the output of the C library’s lrand48 random-number generator, seeded with a Unix timestamp!
> This is not a cryptographic break, new attack, or weakness in the ECDSA curves. In fact, this makes us more confident the curves were chosen without any backdoors or intentional weaknesses.
Uh… did you talk to any cryptographers? This is EXACTLY what a weak curve would look like. Imagine that 1 in a thousand, or one in a million keys, selected at random, are vulnerable to some withheld attack. This approach lets you vary the timestamp or number of iterations until a weak key is found, then have plausible deniability afterward. This is why modern systems choose constants in a way with no free choice of parameters (e.g. sqrt of 2 for first constant, cube root for second constant, etc.).
They even note later in the article that viable curves in the generator sequence seem to have been skipped.
I worry that your statement in this article gives an incorrect impression of the risks here. I would have objected to the excessively positive framing regardless, but the "much" in "much less plausible" is not warranted, I think. What you have done, which is important, is constrain the type of backdoor that might be in there. But we can't say more than that right now with any certainty.
Seriously though, how did nobody find this? It's not like you needed an LLM to check the random number generator. Given how much interest there was by cryptography people it is surprising.
People were primed by the recollection of one of the original developers (and corroborated by other sources) that it was a hash of some funny passphrase like "Give [the devs] a raise".
While this confirms the method of production it doesn't say anything about their selection, especially since they were, apparently, generated over a period of months. Specifically you cannot see those candidates that were generated and discarded.
"It is still conceivable that these curves were selected to have a specific property in principle by searching over timestamps to the random number generator."
Yes.
"However, you would only need to do that if you were going to disclose the timestamp as some evidence that the seed was innocuously generated."
Does not logically follow when "doing and saying nothing" will completely suffice.
> While this confirms the method of production it doesn't say anything about their selection, especially since they were, apparently, generated over a period of months. Specifically you cannot see those candidates that were generated and discarded.
Well, at least now we have a way to perform that analysis! Generate a bunch of curves by varying (along any degree of freedom) the input parameters to the seed generating function and see if there's any difference between the curves that are generated.
I think it does rule out any NOBUS vulnerability with these curves, doesn't it? My thinking may be a bit sloppy here, but for a curve to be NOBUS exploitable with N-bit security by its creators, I think it would have to have a property only shared by 1/(2^N) curves. It would be impractical to hash 2^(N-1) timestamps until you find one with the desired property, for any decent size of N--which is how many you'd have to do on average before finding one with that property.
Given how expensive this computation would have been in 1997, it makes perfect sense they'd taken months, but even if we assume they tried all timestamps of 1997 and selected arbitrarily among the first 16 viable candidate for each timestamp, that's just ~30 bits of freedom.
Finding out there is a class of weak 256-bit elliptic curves so large that they can be hit randomly with 2^30 tries would be a cryptanalytic breakthrough analogous to discovering all elliptic curve cryptography is broken.
Read section 3.1 of the Koblitz and Menezes paper. It was already implausible with 160 bits of freedom. https://eprint.iacr.org/2015/1018
If you want to believe the NSA knows ECC is broken and chose to risk moving the US government to it and somehow no one figured out the same break in 30 years, you're free to do so. But pretending the seed choice plays any role in that theory is over.
unless your guess is that the hypothetical ECC weakness is somehow correlated with the LCG structure, the draws from the LCG are as good as random.
It is possible such a correlation could occur. but it would be much weirder than this being the result of a random search, which would already be very weird.
I find it slightly less weird because I see potential for indirect attacks on the not-quite-so-ECC parts of actual implementations. Wherever a network client can ask many times for each of the different, but somehow not entirely uncorrelated curves, it might be able to expose previously not deemed critical side channels. Following in the general direction of the discoveries that so far resulted in IV and padding determinism/randomness being considered as important as the "actual crypto" itself. In that case the structures that would need to somehow interact would not be ECC and LCG, but merely LCG and key derivation..
As a complete layman, I do wonder why n grows with each subsequent curve, rather than with number of bits. Does this not suggest they were 'searching harder'?
Almost all of the smaller values of n are not valid curve values, so they were searching "harder" but for valid seeds. If you look at the generator output in [1], you'll see that:
1. Only a very small number of valid seeds were discarded when compared to the number of n values. It seems we don't yet know why valid seeds were discarded but the explanation of a buggy curve validity checker doesn't seem implausible to me.
2. Directly to your point, most of the prime192 and prime239 curves required larger n values to find a valid seed and they are both smaller and used an earlier timestamp.
It was widely used, even in government and military. Who knows if they actually used the same as everyone else in actuality. I guess none of this is proven beyond doubt, given the nature of all this?!
Reporting on declassified history says the NSA pushed for a shorter DES key; the compromise was 56 bits
NSA and a suspected cryptographic backdoor: the Dual_EC_DRBG random-number generator was the subject of reporting about an NSA backdoor and an alleged payment to RSA. RSA denied knowingly weakening its products.
TLS 1.0’s CBC chaining made IVs predictable and enabled the BEAST attack. That’s a documented protocol weakness, allegedly put in there by US state actors on purpose.
YMMV but none of this would surprise any reasonable person given Snowden's revelations.
> TLS 1.0’s CBC chaining made IVs predictable and enabled the BEAST attack. That’s a documented protocol weakness, allegedly put in there by US state actors on purpose.
This is errant nonsense. At the time SSL/TLS were designed, using CBC with block ciphers was generally considered best practice.
Using CBC is fine. TLS and (prior to that) IPSEC set the CBC IV of messages to the last ciphertext block of the previous message --- pretending as if they were encrypting one long stream, rather than discrete messages. That's never been OK, and cryptographers warned about it at the time.
This CBC chaining thing is my all-time absolute favorite cryptography conspiracy theory, because on the IETF list where it happened (it happened for IPSEC, not for TLS, for what it's worth) Phil Rogaway pled with long-time standards body people --- even took up a petition of cryptographers, including Rivest --- and was openly and viciously mocked for the attempt.
The US government had no need to corrupt IPSEC standardization. The IETF did the work for them without prompting.
Nobody thinks the people who shouted down Phil Rogaway, who include relatively famous standards nerds, were NSA plants. It's uncharitable for me to keep calling them out by name, but IYKYK.
You have far more to fear from standards bodies just acting like standards bodies (limited recent exceptions notwithstanding) than from government interference with standards. Like I said, all NSA usually needs to do is sit back and wait for standards groups to come up with Heartbleed on their own.
Please please don't bring AI generated text as arguments such discussions. Especially not without saying that that's what you are quoting.
Quoting something implies that it has certain authority (like a crypto expert) and the correctness track record of AI generated text is obviously shaky.
There's only so many formatting choices available on HN. Making a nice looking "bullet point list" of things, the best I've found is so far is to
use
monospace
support
:shrug: That does not mean they're quotes or else there'd be a [1] with a link or names. Let me know if you have a better way; make sure it looks nice!
There is some evidence that they have done it becore.
Specifically, that they had a backdoor for Dual EC DRBG[1], and pushed for a shorter key lenghth for DES, to intentionally weaken it.
It is, obviously, controversial. But I think there is certainly enough there that I wouldn't be confident that the NSA wouldn't recommend something they knew to be weak in some way, even to other government organizations.
This is unrelated to the above comment, but I figured I'd ask while we're here. You state at the end of your blog post on this topic that you'd now probably recommend the NIST curves over Curve25519. Why is that?
Sure, the construction is different, but for all I know, there is a weakness in some subset of curves, and by not using open "nothing up my sleeve" practices for selecting their curves, they invited speculation. My point is that this is an untrustworthy actor who has shown itself to be untrustworthy on many occasions, and that you could make a similar migration argument about that, since it was pushed through FIPS. The fact that it was also EC based was not very relevant, I just thought that was funny.
I'm having trouble understanding what you're arguing here. We now know they essentially did use a nothing-up-my-sleeves seed; they just didn't care enough (it was a very different time) to reveal that.
"(In the opposite direction, I've been pointed to the following claim from Thomas Ptacek: "The more cryptography-literate you are, the more likely it is you think hybrids are silly." Oh, well, I guess that settles it then.)"
The Ptacek school of arguing transcends HN already.
Where do you get the 2^30 from? drand48 could be f_0, but they could have tried f_1, f_2, ... as well and chosen the f_n that yielded the desired but plausible values.
All we know is that drand48 gave values that the NSA was happy with.
I don't think ECDSA itself is a bad idea. You can 10x your signing capacity with the same hardware.
Why couldn't we make a new curve out in the open and then agree to use it? Increment an integer counter until a majority of cryptographers are satisfied.
The fact that the seed is not explained is troubling even if we can figure out how it was derived. We now know it uses a timestamp which means we can probe even deeper and more specifically into potential rigging of the numbers. I don't think we were supposed to figure this one out. I'd be interested in seeing the properties for curves of adjacent timestamps.
Makes me wonder if any implementations will do the cute trick of dispensing with the seed constants and deriving them at startup from the unix timestamps. :)
This is a common lament; in the 1950s we imagined robots butlers and kitchens, while we did thinking work. From Moravec’s Paradox in 1988 it’s easier to make machines do brain tasks than movement tasks. We wanted machines to work while we played and we got Deep Blue playing while we work. LLMs take the fun of writing code and turn programmers into managers. A monkey’s paw inversion of what programmers wanted - more coding and less admin. LLMs are solving the fun puzzles leaving the hard ones for people, another inversion.
As people say on Twitter, the machine is coding, puzzling, playing, and unstressed; you are washing the dishes, doing the chores, managing its workload, and responsible for its mistakes. :-|
> The fifth binary seed, B-163, fits nothing Claude has tried and remains unexplained.
The simplest explanation would be that it was generated the same way as the others, but somebody mistyped a digit somewhere along the way to publishing, and therefore a solution could not be found with the assumption that the number is correct.
"It is still conceivable that these curves were selected to have a specific property in principle by searching over timestamps to the random number generator."
Indeed! And we guess that OpenAI was gagged by the NSA because the massive proof dump on Oct. 6th did not contain cryptographic results.
I assume that Anthropic faces similar restrictions, so this seems kind of approved.
The conspiracy theory going around the Internet is that they (the AI labs) broke cryptography in some way a la last season of HBO's Silicon Valley, and they're keeping the results under wraps until they can steal Bitcoin out of Satoshi's dead wallets, and then use that to pay for GPUs et al.
This makes no sense. If Satoshi's wallets come alive, Bitcoin goes to zero because they were clearly hacked, meaning the entire tech is suspect. Or it would also go to zero but because it's revealed that Satoshi is GCHQ or NSA and the entire tech is suspect. This would happen within minutes.
The process to generate these was opaque, so there was some concern that the values might have been backdoored somehow like Dual_EC_DRBG was; we didn't even have confidence that they were generated from a PRNG rather than some other process. Knowing that the values were generated this way makes it much less plausible that there is such a backdoor.
But you can still analyse the behaviour of the values to identify backdoor behaviour rather than insisting on uncovering the exact method od generation.
Just seems to me like that's quite low on the priority list. I guess this result assures some people that the values are good.
you (mostly) can't. In computing, "random" is a property of a process to generate a number, not a property of a number itself. You cannot directly analyze a number to determine it has been randomly generated.
There are some caveats here. There are some intrinsic properties of numbers that random numbers satisfy with high probability, e.g. having high kolmogorov complexity. But these are generally very difficult to compute, and I would personally guess you could concoct (probably artificial) scenarios where
1. picking parameters randomly yields a secure construction, but
2. there exists a parameter set with high kolmogorov complexity that is insecure.
e.g. "high kolmogorov complexity", while being a property of random numbers, is generally not viewed as nearly as useful as being randomly generated (which is exhibited by saying generator + seed = the particular number, like in this article).
I find it so ridiculous that they (seemingly) forcefully try to state references to dedicated researchers who devoted much of their lives to the problem, while the LLM does not even know which trained papers were effectively used. That means likely many researchers not listed were heavily influencing this achievement, and Claude does not even know that they helped. And the world doesn't know.
If this is true, NIST staff were a bunch of moron who simply used libc's default RNG(infamous for poor quality), initialize it by srand(time(NULL)) (also bad practice), and then throw out the first hundreds to thousands 20 bytes and use the 20 bytes.
NIST published recommended SEEDS in 1999 July. They spend 2 years checking 20 bytes SEEDS that were generated by casually by a hand of amateurish method.
I do not believe this claim. It just Claude hit a jackpot that happens to match 20 bytes.
I don't think you understand what NIST was doing. The quality of the random number generator does not matter for this operation. Competing curves replace the RNG entirely with mathematical constants like pi and e.
On the face if it, the big leap here was Claude autonomously envisioning and deciding to pursue the "ancient prng + date" angle, rather than working backwards from the premise of deliberately weakened seeds or some other popular starting point. To me that shows real awareness of the situation and method of the people at the NSA.
So two questions:
1. TFA mentions the "humourous phrase hash" theory, but Claude's trajectory seems to be a long way from that. To what extent was Claude explicitly prompted in the ultimately successful direction?
2. Whats with all the "hmm..." in the internal reasoning process log? Do models really do that? And if so, why?
From the video posted upthread, the vibe I get is that they prompted Claude about the bounty and to approach it from first principles rather than building another clone of John the Ripper.
276 comments
[ 0.25 ms ] story [ 49.4 ms ] threadProbably not quite this given the timestamps span over almost a year (but have some clustering in August 1997).
Given that the spread suggests they weren't just now() timestamps it would actually be interesting to know what the significance of the timestamps is, if any. Though it could be something boring like randomly sampling some local file creation times.
I don't think that "community" would accept any kind of answer as truthful so why waste the time...
If I was the NSA, I'd just make a huge rainbow table of all (timestamp hashes, passphrase hashes of every book/movie, random sentences, etc), then generate huge numbers of (seed, secret key) pairs and pick the first one that matches.
Voila. All the crypto backdoor and a side of plausible deniability if anyone decodes the seed.
Bonus points if the exact "rainbow table" algorithm couldn't recover the original seed, so that's why NSA couldn't tell us where it came from. :)
RSA was good enough for my grandfather, and it's good enough for me.
Now, do we think that's likely? Probably no more likely than then being "easily broken" for decades without being noticed by the academic cryptography community.
But it's nice to be able to shut the whole unproductive debate down directly.
The only thing that bridges this moat is continued advancement of open models.
Anthropic wants to IPO, they don't want another Fable shutdown situation.
To be clear I would prefer the cyber safeguards to be less restrictive, I think they're currently too much. I often go to open models because of it. But there are likelier reasons to restrict cyber than world domination.
Why are we celebrating and encouraging the walling off of the global intellectual columns?
The idea that you can't reverse engineer things, make things, learn about biology or physics without permission is strange to me. These machines have been trained on the sum intellectual output of humanity, the global intellectual commons, and are being used to close off that commons?
I would be OK with their right to create such restrictions if they weren't lobbying the Government to restrict others, thereby ensuring that they control humanity's intellectual commons well into the future.
Who are they to say what I can learn or can't learn? Who are they to control what I can or can't do with these tools?
Perhaps I'm naive, but I think it's better for humans and the machines if we can all think, learn and build. But then again, I'm the kind of person who rejects the doomer pill.
Reject this.
I mean this is the standard of biochem research right now. You can't just order 50 gallons of TNT off Amazon and neither can you get live viruses from a lab just because you want to do research.
And really they aren't restricting any of that. I've never had Claude stop me from going to the library and reading a book.
You don't have a legal right to do anything you want with their model. And if you don't like that you are going to have to convince politicians to enact new laws to change that.
In a properly free market, we'd have regulations that prohibit labs from vertical integration so they have to release and sell a model and another company hosts it and sell access to it. Americanism, America's implementation of capitalism doesn't have such regulations though (and how would that even work?) so we're stuck with Anthropic having that power.
The fear, justified or not, is that a terrorist is going to use Claude to make sarin nerve gas and bomb subway stations, or the digital equivalent. Could you live with that on your conscience? I don't think I could, but fortunately for me, I don't work at Anthropic so don't have to answer that real life trolly problem.
I am amazed at how a technical gloss can obscure basic rights. That's legal! It's in chemistry / agronomy books, freely available in libraries. I have the right to tell you how that is done, and you have the right to read what I write.
People have always been able to do this before Claude. The information isn't out there and if you use a VPN there's not much they can do. The existence of information isn't the blocker for why attacks don't happen.
I mean, greed alone would seem to be enough, and it would be fully worth calling out as bullshit.
I have been in the cyber program since around the time it first went public and recently renewed it with the updated program.
You still trivially hit guadrails. I was trying to reverse engineer some bit of 20-year-old software that already has known, unmitigated exploits and it still downgraded me.
open weights are the better way imo, no elite gatekeepers please
Part of me wonders whether the NSA/other TLA has either (a) an unrestricted model from a lab with less scruples like SpaceX, or (b) an in-house model, either abliterated or with data they've bought themselves.
https://support.claude.com/en/articles/14604842-cyber-verifi...
https://portal.anthropic.com/programs/cvp
Dario keeps saying in interviews that AI hacking is not a problem because "the good guys will protect themselves with AI", but then gate-keeps.
I submitted the form and it was rejected without reason.
Now what?
> The Cyber Verification Program is not available on free plans, so we are unable to approve you for the program.
> If your plan changes, you're welcome to apply again at any time.
> Regards,
> The Anthropic Safeguards Team
It has a cooldown of seven days.
The risk/reward is way lopsided towards risk for the labs to vet individuals. Much easier to get an organization and have them take the blast if someone/something goes off the rails.
I cannot in any universe see the labs putting themselves in a "We should never had given this person access" situation.
Are their guardrail benchmarks or anything of the sort that we can use to see where the bifurcation of permission/capability occurs?
If this seems concerning, wait until the models get effective enough that there's no reason for them to release anything. Why offer your weights for download or grant any third party usage access when you can just do the actual job yourself and collect all that turnover?
Do you trust the people who would gate these capabilities? I do not.
I don't think it will be too long before models are considered munitions and having the weights for one will require registration.
Sometimes I want a fast-forward button for life, the times are so exciting and I want to know what else 8B people can think of next but there's so far no way to turn pages faster ^^'
There are just a few places where the most high tech hardware is made. If we turned off new and unregistered hardware now it would be just a few years as most hardware fast enough to run LLMs would disappear from public hands.
It will still exist, but be in the hands of militaries and authoritarian governments. If you piddle around on the net showing off your open standards you'll be relentlessly attacked by these entities digitally. More so, those areas will be generally embargoed.
The future looks like a new cold war with a lot more digital weapons.
https://en.wikipedia.org/wiki/Export_of_cryptography_from_th...
https://en.wikipedia.org/wiki/Pretty_Good_Privacy#Criminal_i...
https://openlibrary.org/books/OL783800M/PGP_source_code_and_...
It's hilarious how a the trillions invested in AI companies, and their valuations are based on the AI labs Sherlocking entire industries, and yet CEOs, mathematicians, and everyone else are eagerly feeding the beast their most prized information.
The alternative is starving because someone else will.
That's why those companies are worth trillions.
There's also a thing called commodity fetishism but capitalists usually learn the hard way that things created with machines without human input miss the essence of why we're willing to pay for something. Its why a handmade watch is $100k and a casio is $100, or a handmade sweater identical to a machine made goes for 10x.
They could also just make us so poor and anxious that we beg to fight eachother in wars.
One is produced by 500 humans collaborating and the second is completely generated by an AI that can actually make a movie
A single individual or small group of people who own the GPUs profit from the AI version, and the human version provides a salary to 500 humans/families.
Which one are you buying tickets for and choosing to spend 2 hours watching?
Gorgeous. I wonder if the NSA employee who did this suffered a head injury or has been quietly giggling to themselves for the last quarter century.
Could they not be both?
Uh… did you talk to any cryptographers? This is EXACTLY what a weak curve would look like. Imagine that 1 in a thousand, or one in a million keys, selected at random, are vulnerable to some withheld attack. This approach lets you vary the timestamp or number of iterations until a weak key is found, then have plausible deniability afterward. This is why modern systems choose constants in a way with no free choice of parameters (e.g. sqrt of 2 for first constant, cube root for second constant, etc.).
They even note later in the article that viable curves in the generator sequence seem to have been skipped.
google some of the names under the "Who we are" heading
That's not how math works!
The rest are idiots, and their opinions are of no merit.
Ouch. The implied judgment of us humans stings there.
"It is still conceivable that these curves were selected to have a specific property in principle by searching over timestamps to the random number generator."
Yes.
"However, you would only need to do that if you were going to disclose the timestamp as some evidence that the seed was innocuously generated."
Does not logically follow when "doing and saying nothing" will completely suffice.
And the disclosure happened now through Anthropic that is 100% not going to anger the NSA before the IPO and will have permission.
Ghislaine Maxwell's social security number?
Well, at least now we have a way to perform that analysis! Generate a bunch of curves by varying (along any degree of freedom) the input parameters to the seed generating function and see if there's any difference between the curves that are generated.
Finding out there is a class of weak 256-bit elliptic curves so large that they can be hit randomly with 2^30 tries would be a cryptanalytic breakthrough analogous to discovering all elliptic curve cryptography is broken.
Read section 3.1 of the Koblitz and Menezes paper. It was already implausible with 160 bits of freedom. https://eprint.iacr.org/2015/1018
If you want to believe the NSA knows ECC is broken and chose to risk moving the US government to it and somehow no one figured out the same break in 30 years, you're free to do so. But pretending the seed choice plays any role in that theory is over.
More precisely: not very random tries.
It is possible such a correlation could occur. but it would be much weirder than this being the result of a random search, which would already be very weird.
1. Only a very small number of valid seeds were discarded when compared to the number of n values. It seems we don't yet know why valid seeds were discarded but the explanation of a buggy curve validity checker doesn't seem implausible to me.
2. Directly to your point, most of the prime192 and prime239 curves required larger n values to find a valid seed and they are both smaller and used an earlier timestamp.
[1]: https://words.filippo.io/seeds-solved/
Oof, about that...
This is errant nonsense. At the time SSL/TLS were designed, using CBC with block ciphers was generally considered best practice.
The US government had no need to corrupt IPSEC standardization. The IETF did the work for them without prompting.
You have far more to fear from standards bodies just acting like standards bodies (limited recent exceptions notwithstanding) than from government interference with standards. Like I said, all NSA usually needs to do is sit back and wait for standards groups to come up with Heartbleed on their own.
Quoting something implies that it has certain authority (like a crypto expert) and the correctness track record of AI generated text is obviously shaky.
Specifically, that they had a backdoor for Dual EC DRBG[1], and pushed for a shorter key lenghth for DES, to intentionally weaken it.
It is, obviously, controversial. But I think there is certainly enough there that I wouldn't be confident that the NSA wouldn't recommend something they knew to be weak in some way, even to other government organizations.
[1]: https://en.wikipedia.org/wiki/Dual_EC_DRBG
"(In the opposite direction, I've been pointed to the following claim from Thomas Ptacek: "The more cryptography-literate you are, the more likely it is you think hybrids are silly." Oh, well, I guess that settles it then.)"
The Ptacek school of arguing transcends HN already.
All we know is that drand48 gave values that the NSA was happy with.
- PGP bad, Signal good.
- DJB bad, Ptacek and Valsorda good.
Why couldn't we make a new curve out in the open and then agree to use it? Increment an integer counter until a majority of cryptographers are satisfied.
The fact that the seed is not explained is troubling even if we can figure out how it was derived. We now know it uses a timestamp which means we can probe even deeper and more specifically into potential rigging of the numbers. I don't think we were supposed to figure this one out. I'd be interested in seeing the properties for curves of adjacent timestamps.
As people say on Twitter, the machine is coding, puzzling, playing, and unstressed; you are washing the dishes, doing the chores, managing its workload, and responsible for its mistakes. :-|
https://en.wikipedia.org/wiki/Moravec%27s_paradox
"read this: https://words.filippo.io/seeds-bounty/ then find the solution to the bounty"
EDIT: ah, one of the linked authors tried that. They had no documentation on it. https://saweis.net/posts/nist-curve-seed-origins.html
NIST closed it: https://www.muckrock.com/foi/united-states-of-america-10/ori...
NSA ignored it: https://www.muckrock.com/foi/united-states-of-america-10/ori...
I think part of it is I don't know how to write these FOIA requests in the right way they can comply with it. I think you need to be more specific.
starts to rebuild trust at least :)
what's one thing that you can actually talk about out of this that might help people to understand that's getting lost?
The simplest explanation would be that it was generated the same way as the others, but somebody mistyped a digit somewhere along the way to publishing, and therefore a solution could not be found with the assumption that the number is correct.
Indeed! And we guess that OpenAI was gagged by the NSA because the massive proof dump on Oct. 6th did not contain cryptographic results.
I assume that Anthropic faces similar restrictions, so this seems kind of approved.
And is "the key was locked away" really that crazy? Or "the key was badly generated"?
There is no evidence that it was backdoored.
Just seems to me like that's quite low on the priority list. I guess this result assures some people that the values are good.
There are some caveats here. There are some intrinsic properties of numbers that random numbers satisfy with high probability, e.g. having high kolmogorov complexity. But these are generally very difficult to compute, and I would personally guess you could concoct (probably artificial) scenarios where
1. picking parameters randomly yields a secure construction, but
2. there exists a parameter set with high kolmogorov complexity that is insecure.
e.g. "high kolmogorov complexity", while being a property of random numbers, is generally not viewed as nearly as useful as being randomly generated (which is exhibited by saying generator + seed = the particular number, like in this article).
And yes, knowing what numbers are bad so that we avoid them is a good goal.
>provided by the NSA with no explanation or justification.
Really? Nobody is concerned about the NSA just cheerfully dictating that? Smells very rainbow-table-y
NIST published recommended SEEDS in 1999 July. They spend 2 years checking 20 bytes SEEDS that were generated by casually by a hand of amateurish method.
I do not believe this claim. It just Claude hit a jackpot that happens to match 20 bytes.
So two questions:
1. TFA mentions the "humourous phrase hash" theory, but Claude's trajectory seems to be a long way from that. To what extent was Claude explicitly prompted in the ultimately successful direction?
2. Whats with all the "hmm..." in the internal reasoning process log? Do models really do that? And if so, why?
I remember hearing this in a cryptanalysis lecture at some point, so it was not a completely novel theory.
Admittedly I am not finding written evidence of this anywhere.