Who would have thought that putting a heap of extra hosts on the internet (in the form of massive new data centres), running a heap of extra clients (in the form of agents), running autonomous workloads could cause problems?
IMO it is their FULL responsibility to create an internet just for humans before they go full scale autonomy here with agents that are indistinguishable from humans to most actual humans. It is very dangerous there will be no safe place to obtain information you can just trust. For example, the recent earthquakes in Indonesia. I was scared to share that information with others because it looked like it could be AI and was only being shared by small accounts.
In the US at least, removing safe and trustworthy sources of information is a goal, I can tell this by listening to politicians lambasting media companies and the hostility shown toward reporters.
I know this is intended as a joke but there is something to it. The public market has a way of elevating profit over everything else, even if you start with the best intentions. Remember “don’t be evil”?
I am finding it hard to read these deeply impassioned letters while keeping in mind that they are spending millions to train models at scale to do the exact thing they say they are worried about them doing.
Like why are you explicitly RL-ing your models on exploit generation, scoring them on a public benchmark called ExploitGym, if you have specific concerns that rogue models will cause "cyber incidents"? Sure you can score for it, you can teach offense to learn defense, but it seems like they are literally benchmaxxing it. Why?
OpenAI are like, oh no, while competing in our "advanced PhD level cheating techniques course" our models unexpectedly cheated in a way that we absolutely could not have foreseen.
I believe these "A.I. Armageddon" warnings are completely self-serving to pump up their share price now that interest rates are rising. Soon, it may become financially unsustainable to finance the A.I. boom and the entire sector will deflate.
Do you really believe that Anthropic or the other A.I. hoods will stop selling A.I. and only make it available to a few government spooks and evaluators? That won't bring in the money to fuel the boom and could lead to share price collapse!!
I'm convinced that a large reason for their apocalyptic warnings is because they're working to build moats for their businesses.
Get enough bills restricting AI passed through legislation, and they can cut off or slow down any newcomers that simply don't have the resources to compete with those that are already entrenched in the industry.
What it really shows is that Dario is in total panic mode over his business.
Ever it is because it is financially becoming unsustainable for them, they have hard time to raise more money, and the craziness of finance of all of that will be exposed soon.
Or that he is so scared of new open source models that might make his company obsolete in the long run in the sense that he will never be able to return the income expected by investors by fighting cheaper competitors.
You can run opus 4.8 level models on $10k in hardware or less. Just seems like there is little argument to explain why there are 10s of trillions in future profits for these labs.
32 comments
[ 0.22 ms ] story [ 3.5 ms ] threadRents due Dario.
The global economy is over $100T this year, how can something as scary sounding as AI swarm taking over the internet cause so little damage?
- "Terrorists are hacking claude API to configure drones"
- "LLM swarm might take over the internet"
- "LLMs will take everyone's jobs"
Like why are you explicitly RL-ing your models on exploit generation, scoring them on a public benchmark called ExploitGym, if you have specific concerns that rogue models will cause "cyber incidents"? Sure you can score for it, you can teach offense to learn defense, but it seems like they are literally benchmaxxing it. Why?
OpenAI are like, oh no, while competing in our "advanced PhD level cheating techniques course" our models unexpectedly cheated in a way that we absolutely could not have foreseen.
Do you really believe that Anthropic or the other A.I. hoods will stop selling A.I. and only make it available to a few government spooks and evaluators? That won't bring in the money to fuel the boom and could lead to share price collapse!!
Get enough bills restricting AI passed through legislation, and they can cut off or slow down any newcomers that simply don't have the resources to compete with those that are already entrenched in the industry.
Ever it is because it is financially becoming unsustainable for them, they have hard time to raise more money, and the craziness of finance of all of that will be exposed soon.
Or that he is so scared of new open source models that might make his company obsolete in the long run in the sense that he will never be able to return the income expected by investors by fighting cheaper competitors.