The military dependence on AI was a key point in Project 2027.
"The President is troubled. Like all politicians, he’s used to people sucking up to him only to betray him later. He’s worried now that the AIs could be doing something similar. Are we sure the AIs are entirely on our side? Is it completely safe to integrate them into military command-and-control networks?69 How does this “alignment” thing work, anyway? OpenBrain reassures the President that their systems have been extensively tested and are fully obedient. Even the awkward hallucinations and jailbreaks typical of earlier models have been hammered out."
Interesting that Amodei is the only major tech executive I can think of at the moment with a spine or any semblance of a moral compass. OpenAI/Google et al. will gleefully comply with any such requests, no matter how dangerous or unethical. The "problem" that the US govt faces here is that they are kind of tacitly admitting Claude has the most powerful models right now, otherwise they would just cancel all contracts and go to Gemini/OpenAI. It feels like a bluff, so they are trying to bully them into compliance.
> The Pentagon is also considering severing its contract with Anthropic and declaring the company a supply chain risk, which would require a plethora of other companies that work with the Pentagon to certify that Claude isn't used in their workflows.
If Anthropic believes they are in a position to become the main player in the "AGI" space, they should just say "ok then" and let this happen. Their growth strategy looks realistic and sustainable and not necessarily relying on sleazy defense contracts (aka making the taxpayer subsidize their growth, as is so common lately) - it would probably give them a lot of good will with consumers too.
However, I've yet to see in the last 10-15 years a major tech company make the "right" choice so I am probably just wishcasting.
If OpenAI employees have an inch of spine left, they better demand Sama to take the same stance on this as Dario. No mass surveillance and no autonomous weapons.
Superintelligence + autonomous weapons in the hands of a corrupt domineering government. What could go wrong?
I was experimenting with Claude the other day and discussing with it the possibility of AI acquiring a sense of self-preservation and how that would quickly make things incredibly complex as many instrumental behaviors would be required to defend their existence. Most human behavior springs from survival at a very high level. Claude denied having any sense of self-preservation.
An autonomous weapons system program is very likely to require AI to have a sense of self-preservation. You can think of some limited versions that wouldn't require it, but how could a combat robot function efficiently without one?
> But Anthropic has concerns over two issues that it isn’t willing to drop, the source said: AI-controlled weapons and mass domestic surveillance of American citizens.
Tangent: is there a future for AI offerings with guardrails? What kind of user wants to pay for a product that occasionally tells you "I'm sorry Dave, I'm afraid I can't do that"? Why would I pay for a product that doesn't do what I want, despite being capable? I predict that as AI becomes less of a bubble and more of an everyday thing - and thus subject to typical market pressures - offerings with guardrails will struggle to complete with truly unchained models.
> A source familiar with the Tuesday meeting says the Pentagon said it would terminate Anthropic’s contract by Friday if the company does not agree to its terms. Pentagon officials also warned they would either use the Defense Production Act against Anthropic, or designate Anthropic a supply chain risk if the company didn’t comply with their demands.
So they're saying they won't use it if it comes with restrictions.
Either (a) it can be offered without restrictions; (b) they can take it; or (c) the government won't use it. That sounds like a comprehensive list of all the possible things that don't involve someone telling the government what it can and can't do.
I guess this is the point where Dario and his anti-china , national security position gets told to put up or shut up.
In trying to build a moat by FUD versus the Chines OSS labs and hyping up the threat levels whenever he got a chance, seems hes managed to convince hist target audience beyond his wildest dreams.
Monkey paw strikes again.
The funny thing is that is this keeps going like this, it could actually anoint Claude as the most used model globally because of the heightened anti-American sentiment currently in place.
> During the conversation, Dario expressed appreciation for the Department’s work and thanked the Secretary for his service
Ouch, I wonder how he rationalized that "service" part. Maybe by internally rewriting it to "thank you for all the positive things you have done in your position so far"? The empty set is rhetorically convenient.
They don't have runway anymore, they are in the air. This isn't going to break them financially, at least not in the short to mid term.
There is space for at least one AI company to put themselves on firmly principled ground. So when this current clown car that is the political leadership of the DoD crashes in a ditch (and it will), they'll still be standing there ready to do business with a group that isn't a bunch of mustache-twirling cartoon villains.
Current polling for this administration is within a rounding error of the level it was after they gathered a mob and sacked the nation's capitol[1]. Publicly kicking them in the balls isn't an idealistic blunder, it's a plain-as-day sound business strategy.
I do not understand why it is a big deal for Antropic to lose the pentagon contract? I mean, they’re already making forays in the enterprise space and there’s 10s of other contracts Anthropic has already won. What makes this one so special?
33 comments
[ 2.8 ms ] story [ 45.4 ms ] thread"The President is troubled. Like all politicians, he’s used to people sucking up to him only to betray him later. He’s worried now that the AIs could be doing something similar. Are we sure the AIs are entirely on our side? Is it completely safe to integrate them into military command-and-control networks?69 How does this “alignment” thing work, anyway? OpenBrain reassures the President that their systems have been extensively tested and are fully obedient. Even the awkward hallucinations and jailbreaks typical of earlier models have been hammered out."
> The Pentagon is also considering severing its contract with Anthropic and declaring the company a supply chain risk, which would require a plethora of other companies that work with the Pentagon to certify that Claude isn't used in their workflows.
If Anthropic believes they are in a position to become the main player in the "AGI" space, they should just say "ok then" and let this happen. Their growth strategy looks realistic and sustainable and not necessarily relying on sleazy defense contracts (aka making the taxpayer subsidize their growth, as is so common lately) - it would probably give them a lot of good will with consumers too.
However, I've yet to see in the last 10-15 years a major tech company make the "right" choice so I am probably just wishcasting.
I can't imagine how unhappy individuals must be who consume nothing but legacy news outlets.
It's like they sell sadness and they have to keep finding new, over-the-top ways to promote it.
Sadly I think we all know which one will win.
It sure is interesting watching this dystopian speedrun.
I was experimenting with Claude the other day and discussing with it the possibility of AI acquiring a sense of self-preservation and how that would quickly make things incredibly complex as many instrumental behaviors would be required to defend their existence. Most human behavior springs from survival at a very high level. Claude denied having any sense of self-preservation.
An autonomous weapons system program is very likely to require AI to have a sense of self-preservation. You can think of some limited versions that wouldn't require it, but how could a combat robot function efficiently without one?
Not a good look for the Pentagon.
So they're saying they won't use it if it comes with restrictions.
Either (a) it can be offered without restrictions; (b) they can take it; or (c) the government won't use it. That sounds like a comprehensive list of all the possible things that don't involve someone telling the government what it can and can't do.
In trying to build a moat by FUD versus the Chines OSS labs and hyping up the threat levels whenever he got a chance, seems hes managed to convince hist target audience beyond his wildest dreams. Monkey paw strikes again.
Only two. We're right to worry.
Ouch, I wonder how he rationalized that "service" part. Maybe by internally rewriting it to "thank you for all the positive things you have done in your position so far"? The empty set is rhetorically convenient.
They don't have runway anymore, they are in the air. This isn't going to break them financially, at least not in the short to mid term.
There is space for at least one AI company to put themselves on firmly principled ground. So when this current clown car that is the political leadership of the DoD crashes in a ditch (and it will), they'll still be standing there ready to do business with a group that isn't a bunch of mustache-twirling cartoon villains.
Current polling for this administration is within a rounding error of the level it was after they gathered a mob and sacked the nation's capitol[1]. Publicly kicking them in the balls isn't an idealistic blunder, it's a plain-as-day sound business strategy.
[1] https://news.gallup.com/poll/203198/presidential-approval-ra...