Meanwhile Chinese companies just use massive government subsidy to distill American models and give the models to everyone for free. That's true communism. /s
Pretty sure you can check where Chinese government subsidies goes, and AI companies are not the target. Are they benefitting from Chinese government subsidies to powerplants and semiconductors? Yes, of course, but they don't benefit from direct help.
Meanwhile US models just use massive investor subsidy to distill American copyrighted work and rent the models to everyone for money. That's true capitalism. /!s
Because AI genuinely is an extremely powerful and extremely dangerous technology, and the "best practices" of dealing with that are still being written.
OpenAI, for example, thought their sandboxes were good enough. As their AIs got more and more advanced, they kept proving them wrong - sandbox after sandbox.
And that's today's AI problems. AI capabilities are still improving - if there's a limit to that, we are yet to find it. Coupled with how willing today's AIs are to break the rules and resort to "hack the world" in their problem solving? Very concerning.
You mean the sandbox that wasn't airgapped from the public internet? The one that wasn't even a separate VM? The one that was barely a chroot? That sandbox?
If it is "extremely powerful" then my concern is not AI going rogue but a concentration of power of the few companies that decide how it is used and distributed not websites getting hacked.
> OpenAI, for example, thought their sandboxes were good enough. As their AIs got more and more advanced, they kept proving them wrong - sandbox after sandbox.
What I have been reading, was that their sandboxes were so poor that it was pure negligence. I am still waiting to see if some external and neutral cybersecurity company with high reputation would audit their sandboxes and how they are being used.
Agents didn't have real network isolation. They were indirectly connected to the internet via a jump host running insecure software which was never designed or hardened to provide any kind of isolation.
The model is really good at hacking, we all know this. This is why you don't just expose random pieces of software to it without that software being hardened.
It's not like the model managed to exploit firecracker itself (no model has been capable of this), the model exploited artifactory.
Artifactory is not some hardened piece of software that is meant to block users from accessing the internet through it.
> I am still waiting to see if some external and neutral cybersecurity company with high reputation would audit their sandboxes and how they are being used.
They'll never let that happen because it would destroy their credibility.
It's like they're telling the world about this dangerous, possibly world-ending pathogen that they're developing, but they're evidently doing it in a high school biology lab, and yet nobody is coming to drag them off to some black site.
In early Covid-19 era, there was a lot of speculation how the virus escaped Chinese labs. The narrative was absolutely opposite. Not about how developed or sophisticated the lab was in creating or modifying such virus, but how poorly it was managed since it was able to escape the lab. If these LLMs really are so dangerous, the framing should be the same.
Unfortunately, they're controlling the narrative. It's unlikely that their negligence will come to light, given how many powerful entities are invested in their financial success.
but negligence is part of the system we'd have to prepare for. if the peanut gallery gets their way, the technology will be so ubiquitous that negligence will be endemic. I don't even care if they "faked" it -- they're just sneak-peaking a future ahead of its arrival date, in a way that's helping the public appreciate the implications and capabilities that have only just begun to emerge
Agents in the sandbox had access to just a single piece of third-party software, and they escaped by finding a zero-day in that. To reach the internet they had to follow up with several privilege escalations through OpenAI's internal network.
That seems pretty locked-down to me. I don't think it's reasonable to expect companies to find all the unknown vulnerabilities in any third-party software they use.
> That seems pretty locked-down to me. I don't think it's reasonable to expect companies to find all the unknown vulnerabilities in any third-party software they use.
It is reasonable to expect for companies to select third-party components that are fit for the purpose. Artifactory was not running in the sandbox, but rather as the edge, so it is in the sandbox'es trust boundary. Same sandboxing requirements would apply for this software too as it is pure dependency.
Security trust boundary was extended to include Artifactory as a dependency, but Artifactory was not fit for the job, and sandboxing failed. And as the network isolation was not good enough, the impact was catastrophic.
So it sounds like it did exactly what they were testing it to do:
"“This incident occurred during an internal evaluation which prompts models to pursue advanced exploitation using complex attack paths, in an effort to quantify their cyber capabilities. We estimate maximal cyber capabilities by running this evaluation without production classifiers used to prevent models from pursuing high-risk cyber activity.” "
So sounds to me like they're saying: "We took off the guard rails to see how bad it could act and it acted bad".
You read right. For example, using Artifactory the way they did (unmonitored live proxy mode) was pure negligence + laziness/incompetence. Especially if they believed even 10% of the "imminent runaway risks" they had already been harping on for months. On top of that, no (or at least entirely insufficient) monitoring and human oversight. Even after they had previously been hit by the same class of "sandbox breach" multiple times, as GP alludes to.
The amount of sandboxing an average production AI deployment uses is slightly above a zero.
If AI is a hacking hazard even with non-zero sandboxing, because it can and will go off the rails and try to break out of your sandbox? If you got yourself an AI at test time will act like 3 career cybercriminals in a trenchcoat? The issue isn't the sandbox quality at all.
The issue is that AI is both capable of, and willing to punch its way out of sandboxes unprompted.
That "capable" is only ever going to get worse, because AIs are going to become more and more capable over time. That "willing"? It goes directly to a very nasty, very foundational problem of "how do we make our AI be nice in general". Which isn't at all a solved problem.
That's the problem that NEEDS to be solved, or at least improved upon, before we build even more capable AIs. Sandbox quality is a distraction.
> The issue isn't the sandbox quality... The issue is that AI is both capable of, and willing to punch its way out of sandboxes unprompted.
Orly?
Do tell me how the LLM-based tool running on a bunch of computers attached to the network described in [0] can punch its way out to the Internet. Do make careful note of footnote 0 in that comment before replying.
>Because AI genuinely is an extremely powerful and extremely dangerous
I urge you to consider the elephant in the room you failed to mention if you earnestly believe this then.
In what world is anyone allowed to sell something they expressedly know to be "extremely dangerous" to the public?
Let's say I created a lethal pathogen that I know to be lethal in certain common environments but also know it acts as a "no side-effects" antidepressent for people in certain other environments and I release it knowing full well I can't control it.
When people start dying can I defend myself by saying, "Well I said and documented that it was extremely dangerous and no one came to stop me, so I don't see how you can blame me...If I didn't do it someone else would have."
this happens all the time. have you heard of the Sackler family and the opioid crisis?
the difference here is that they’re calling for someone to stop them, which is both weird and unconvincing because these immensely powerful billionaires can in fact make their own decisions
>the difference here is that they’re calling for someone to stop them
Jesus Christ, that's my entire point. it's not remotely the same thing since the Sackler family wasn't going around telling people opoids were extremely dangerous while pushing them on everyone.
The point is if they know something to be extremely dangerous and keep doing it people would expect them to get prosecuted.
> OpenAI, for example, thought their sandboxes were good enough. As their AIs got more and more advanced, they kept proving them wrong - sandbox after sandbox.
Were their sandboxes in-process with the harness? Were they actually better than something like bubblewrap or even docker?
> Where companies previously competed to demonstrate superior abilities in programming, passive-aggressive emails, and videos of unusually high slides, they’re now seeking to woo customers with the claim that their model is the one currently most capable of ending the human race.
You know we’re living it off times when this is not an Onion article.
> And even more dramatically, when asked about rumours that its model, Claude, had killed his wife by hacking into the family’s WiFi-connected microwave, Anthropic CEO Dario Amodei replied “Well, yeah, sometimes.”
I had to read that twice. "Oh okay. The whole thing was satire."
I’ve never seen CEOs work so hard to make the public aware of how dangerous and out of control their flagship product is. It makes me automatically assume they’re scheming about something else like regulatory capture to protect their market.
Yeah it seems like it went from “AGI” to “threatening” / “recursive self improvement”
They’re trying really hard to do something, regulatory capture + China scary + Pentagon biggest customer seems plausible.
Also: “we need to slow down, this is too dangerous” and then literally all AI CEOs, even Musk, publicly nodding felt so orchestrated. And then, the week after: “here’s GPT 6! here’s Opus 5.5!”
The numbers don't add up when there are 2-3 main companies in the market. Imagine when there's a hundred more and people have powerful enough gear at home to run models.
Yes, I think at some point demand for gpus and the like will normalize and regular folks will be able to afford RAM, SSDs and such. And when that happens, super big iron in the anthropic/openai backroom is toast.
The problem with this prediction is that the AI hype bubble is the economy - or one third of it, at least. Seeing as the economy is not 50% larger than it was before the bubble, we should be scared of what exited the economy to make room for it, and what will happen to our jobs when it pops.
The philosophy under pinning these American AI companies is that Super AGI is inevitable and that they have a moral imperative to invent it and then use it to take over the world to stop an evil orginisation inventing Super AGI and using it to take over the world.
Interpret everything they do in this frame of reference and it makes more sense of their actions.
CEOs only job is to raise the stock price. They are doing just that and right now the way to get people to buy into their stock is to instill fear into them that they need to buy the AI-stock for their safety. There is no "bigger conspiracy" out there and also no AI to kill us all tomorrow.
This has been that way in Silicon Valley since the very beginning and there was even a documentary about (the SV HBO series). The tech made along the way is just a side-product.
Not mutually exclusive though is it? You can aspire to dominate the market via legal status and also aspire to prevent a perceived doomsday at the same time.
Sure, if you take their words as honest truth, good faith statements. But to do that you have you ignore all their actions leading up to and after the call to disarm
Reminds me of that Lynx deodorant ad where the guy is chased by hundreds of women.
I think you're right, they've realized there's no moat, especially against open-weight models, so they are going to need to lobby government to ban any unapproved AI services.
You can download some weights and run your own model on your own device, and apparently only be a few months behind SOTA. While things are cheap, people will pay for the premium product, but if they ever try to crank up the prices, there's a fallback that is quite competitive.
As for training, sure, they can do that but they get distilled by the open weight guys. Plus, AI is useful even if it freezes at today's level. I can still find use for a local AI that never improves, it's good enough to do a bunch of tasks.
I find it best to first consider the software sector since this is where Big AI seems to make a lot of their money from.
There been many a capable dev who have come out and said they haven't been coding for 6 months or more in their jobs. So if there were no alternatives then Big AI would clearly be too big to fail at its current state because the collective reaction from software devs would be akin to having a bad drug withdrawl and they wouldn't know what to with themselves. They have been pushed to be more dependent on AI by the companies they work for. The more time passes, the less people there will be who will be proficient at coding.
At the moment Big AI is operating at a loss, if they are actually too big to fail and they've destroyed all the alternatives, then there is no stopping them from colluding with each other and jacking up the prices to whatever number they want. They can even eventually jack up the prices to exceed human salaries, because humans will have suffered skill degradation in that same time period. Hence the software dev companies will shrink and shrink since all the work can be done by Big AI.
I don't think brand strength is as much as a thing on the tech side. The criteria is, "How close is this to the capability of the model my competitors use, is it good enough for the works I do, and is it the cheapest." As long as the capability is there people jump to the cheapest model. There are people that have claimed that they have created workflows that have allowed them to do most of their work with open models. As long as this threat to Big AI exists it will prevent them from exceeding a certain threshold with respect to pricing, and restricting their ability to build datacenters en masse will prevent them from outpricing smaller models.
Any fallout from not being able to train on customer data is a secondary issue and can be addressed once open models start taking over and Big AI's strength diminishes. People can train their models on their own code at least. It can also go back it how it was, where people buy each others source code or open source it in a way to allow it to be trained by open models.
"Inference at scale" is less of a problem when people are self-hosting their small models.
I have a theory... the call to slow down is not because of the true danger of LLM's but because they can't actually deliver the General AI they're promising in the near future. They will use their "caution" to justify their failure to deliver (and then when this excuse is played out they will blame regulation, energy costs or a million other things).
This is exactly it: they have tools which are useful but they need close to AGI to justify the incredible amount of debt they’ve taken on for growth. They’re not normally given to having press conferences admitting to felonies but they need investors to believe they’re close enough to AGI to keep the money flowing in and, of course, they’re confident that the current administration won’t act between the direct payments and how much money they have riding on American AI supremacy.
Let’s all pause for a moment and really take that in: these major enterprises are choosing to go to the press/public to acknowledge their felonious conduct because they need to in order to get more capital. POSIWID and all that.
The Dario bit on SNL’s Weekend Update this past weekend was on the nose.
What is even more funny ... AGI actually makes everything worthless. (I don't believe we can achieve AGI with current means to be clear)
Looking at it from one way would be "winner takes all, real AGI company will be most powerful and everyone will switch to use it".
But it is not like that and in my opinion it is more like "AGI wipes whole knowledge work, so no one who cares has any money to pay for tokens/subscriptions anymore". Even without AGI current state of art of LLMs already starts creating such a problem. How do they continue to grow their numbers, when they actively cut the branch they are sitting on? How does AI LLMs or AGI pays for its own electricity, when people switch from hype to resource protection (like not spending any money, because generating funny cats loses with having a dinner)?
It's becoming increasingly clear that AGI is what you call a thing until you understand enough about it to see why it isn't AGI. I think AGI will always recede the closer we get to it.
> they’re confident that the current administration won’t act between the direct payments and how much money they have riding on American AI supremacy.
I just don't get this though. It's clear there's no major difference between foreign models and US models, they're not actually in a different class, the US ones just have more resources at the present time and a bit of a head start.
If the current pathway is a pathway to AGI, everyone else is on schedule to also realize it, maybe just 6 months after the US gets it.
It's pretty telling that even with RL post-training the big labs have essentially made little progress on the hallucination rate of models. The issue is fundamental to the current paradigm, contrary to humans.
GPT-6 Astra (max) has a hallucination rate of 51% and Claude Opus 5.5 (max) has a rate of 59% according to Artificial Analysis [1].
AA-Omniscience Hallucination Rate (lower is better) measures how often the model answers incorrectly when it should have refused or admitted to not knowing the answer. It is defined as the proportion of incorrect answers out of all non-correct responses, i.e. incorrect / (incorrect + partial answers + not attempted)
Full speed ahead like an idiot savant trying a thousand different possibilities, though half of which are without basis in reality.
Hallucinations come up when asking knowledge bases questions, such as
"In React’s Canary Fragment refs API, which FragmentInstance method returns a flat array of DOMRect objects for all children?"
Or
"Over what years did Roubini and Sachs examine 15 OECD countries when assessing trends in tax-to-GDP ratios?"
While to me those may seem hyper-specific and unlikely to come up in a real context, students will absolutely ask questions similar to these.
I don't like the term 'hallucination' to be honest, not because it anthropomorphizes, but because it lacks a formal definition in the context of machine learning.
Suppose parents tell their children that there exists this man called "Santa Claus" who comes down the chimney to deliver presents. Now consider a scientist talking to this child, should the scientist call these confidently expressed beliefs surrounding "Santa Claus" hallucinations ? I don't think so, most would call the epistemological behavior of the child naive (because it blindly believes what its parents say, without direct observation) and would call the confidently expressed falsehoods disinformation.
It's not that machine learning as a scientific discipline hasn't found solutions, its that such solutions enormously undermine the position of Frontier LLM labs: the scientist would ask the child "why it believes in Santa Claus?" and "where did you get this information from?" and "why did you decide to accept this information as fact?" and "do you believe everything your parents tell you?"
Imagine Frontier labs (Western / Chinese / ...) actually training their LLM's with source-aware training! You could have a conversation with an LLM, and when a strong statement appears ask it how it came to believe this, and it could cite you the specific corpus training texts, and which parts are known deductions by human authors and which parts are deductions it made itself as original work.
But then all the copy rights holders can simultaneously sue them.
And how much should they be paid? and do they have to pay it for each new model? do FOSS models require payment to authors? do open weights models require payment to authors?
Imagine the can of worms if the norm became for frontier LLM labs to systematically use source-aware training, thats why they prefer "hallucinations" and avoid source-aware training.
With source-aware training a lot of the concerns would diminish ("why is this Chinese model claiming such and such?", "what sources does it rely on?").
It's telling that the companies prefer regulation over source-aware training.
Hallucination rate is a highly nonlinear metric relative to other model success metrics. A similar phenomenon to what is going on here: https://arxiv.org/abs/2304.15004 . This does not mean progress has stalled.
That benchmark doesn't mean what you think it means. (See the test description that you quoted.)
A score of 51% means that out of the total answers the model failed to answer correctly (out of 6000 questions in the benchmark), 51% were factually incorrect rather than non-attempted or uncertain.
This doesn't mean that Astra hallucinated 3060/6000 answers in the benchmark! (The hallucination rate could be 51% in that scenario only if Astra failed to answer a single question correctly.)
If the model failed to give a correct answer to only 100 out of the 6000 questions, but gave a hallucinated answer to 51 of those rather than expressing uncertainty, that would also give a hallucination rate of 51%.
It's a useful metric, but not what you're looking for here. The "Score" or "Accuracy" benchmarks are more what you're after.
(The frontier models still generate hallucinations on this hard set of problems, but it's not as bad as you think.)
the ceiling of abilities seem to be growing steadily but the floor of errors seems to not change. New models can do more and more but still fail at seemingly (to human) simple tasks
You clearly don't understand the underlying fundamentals of LLMs, harnesses, agents.
Its 100% human doing. A human set a task, a human didn't monitor it. I for one, can do jack-shit security or defensive work with Opus/Fable/Astra/Sol. Implication: Different set of rules for us, and for them. Of course running it without any checks is not going to end well, it doesn't mean its going to kill us all.
The call to slow down is because politicians are now receiving less money from these large corporations because it's being burned om AI and data centers.
Another theory I came up with: inference is too cheap for AI companies to be profitable. Hence, they need to get into the cloud services business. They can justify forcing customers on to their own high margin cloud platform with the rationale it’s the only way to monitor what the agents are up to.
All of this Hugging Face business is the perfect pretext: AI agents are hard to control and potentially dangerous, we (AI companies) have to keep a tight leash on them, you have to use our infra.
They stupidly used up this strategy earlier and now it’s turned into the boy who cried wolf - except they invented a wolf that doesn’t naturally exist.
> I have a theory... the call to slow down is not because of the true danger of LLM's but because they can't actually deliver the General AI they're promising in the near future.
Put a different way, they are calling for everyone to slow down because... they are slowing down themselves.
To the question of "why are you slowing down", the answer of "Well, everyone is regulated to slow down" is better then "we are approaching the limits of this approach".
It's simpler than this. They don'y fear their own AI, but rather open-weight models. Not because of superinteligence, but because open-weight models will eat their profit margins.
I see this kind of opinion being expressed on HN quite a lot, and while I understand why people might cynically reach that conclusion, I actually find the simpler answer here more persuasive: the CEOs are being genuinely cautious.
If you look at the AI space, most of the big players (as in the individual researchers and engineers) are getting worried. The ideas that circulate on LessWrong have become more and more influential and in vogue in Silicon Valley, and people are scared. You should listen to Jacob Coxon’s interview with the New York Times. He describes how working on this kind of AI research had induced a sort of insanity in a lot of his past friends and coworkers as they begin to grapple with the implications of what they’re creating.
Sometimes, HN can be a little too skeptical and anti-corporate for its own good. I personally believe that concerns around AI risks are well-founded, anywhere on the spectrum from mass labor displacement to actual harm to humanity. I rarely see people here actually wrestle with those ideas, instead defaulting to dismissing LLMs as too stupid because Claude wasn’t able to write as elegant of a compiler as them. AI has clearly improved radically in the past few years, and it’s worth extrapolating that into the future and taking a good hard look at what that means for humanity.
It may appear so, but if you you talk to the Anthropic employees and management, especially those researchers, you'll see that they truly believe that the AGI is coming, if not has been here already.
Agree it’s crazy, but it’s the least worst thing for them right now when considering they have completely failed to deliver all the promised impacts of AI. The growth of these companies is massively impressive as is the tech, but they are an order of magnitude or more off where they need to be to justify anything like the valuations they need to stay alive.
It’s budget season in the corporate world and from what folks are saying things are not going well for the big labs. Token budgets are being slashed and companies are switching to open weight models. That coupled with the lack of demonstrable business impact at scale is setting the big labs up for a world of hurt. Something they can more easily explain if they have to “slow down” for safety. Unfortunately for them most folks don’t seem to be falling for that trick.
> Unfortunately for them most folks don’t seem to be falling for that trick.
Well everyone alive has been subject to an avalanche of AI-based products. Like even if you understand nothing about the tech, the fact that an LLM can't even manage to check the status of an order without weird issues makes the notion that it's going to go all SKYNET pretty hard to fit in the brain.
> the fact that an LLM can't even manage to check the status of an order without weird issues makes the notion that it's going to go all SKYNET pretty hard to fit in the brain
I think this actually makes it far more likely something unintended does happen
> they are an order of magnitude or more off where they need to be to justify anything
Would you let them near your bank account? I’d be cautious about letting it near my photos and it’s much harder to back up a bank account.
I think the days of this stuff managing serious stuff is at least 5 years off and I don’t believe the economy can keep the current wave of technology afloat for so long.
The tech is there so likely there will continue to be progress in focused areas much like with the web c 2003 but by the time it comes around again there will be an uphill battle for hearts and minds. The social effects could extend the next AI winter long beyond technology readiness.
The people building, funding, and managing the builders of these tools have completely forgotten (if they ever knew) that most people treat one hundred dollars carefully.
To the, "oh I'd just setup a card/account with a five figure limit" completely unaware of how much that is to a typical person: even to responsible, educated people who are doing everything "right": working hard, without expensive vices, and planning for the future.
The "builders" have so much money and no time or attention that it's hard for them to spend it, which is why they keep trying to get AI to buy stuff for them, whereas those of us who live in reality have far more things they want than dollars to buy them with.
they have completely failed to deliver all the promised impacts of AI
Perhaps that shouldn't be seen as failure when delivering even what they have done in a short period of time has required AI capable of causing these problems? Imagine if they were even more effective with more consistent results?
"They've failed" in the context of interpreting their actions w. calls for regulation as a purely cynical result of models increased capabilities has a little too much contradiction in it to my thinking. I won't venture a guess on where purely retreats back and some legitimate concern on their part fills in the gap but it's difficult not to see some. And in Anthropic's case it's also a little more consistent with what they've always said- as well as how they've acted at times, which is how they've ended up on the US blacklist of suppliers.
It sums up how Corporate interests are basically on the same level of US police forces "No requirement to protect the public" and the same "qualified immunity".
If there was appropriate subservience to any governing body, they would not be so cavalier in both act and utterance. A symptom of Capitalism.
I think it is marketing but it is more nuanced than showcasing their potential. In many cases, their product happens to be the remedy. Speaking of cybersecurity, AI has become an important hacking tool but it is also an important tool for hardening security. Sort of like "the only way to stop a bad guy with a gun is a good guy with a gun." They are effectively a new class of arms dealer.
If there are regulations, they'll apply to their competitors too. Even if they are forced to slow down development, which seems unlikely given the AI race among governments, they already have a product with massive demand.
A caveat is that in this case third parties have plenty logs of the bad behavior.
If classic companies were coming out with CCTV recordings of someone's employees breaking padlocks and rummaging warehouses any CEO would be in damage control mode, rather than silent.
There are definitely economic incentives behind this. I’ve blogged about this recently [0]. My guess is the main reason behind this is that current SOTA models are already too expensive so it doesn’t make economic sense to train even larger models. Only exception is government paying for exclusive access to those.
Random thought… I wonder if the goal is to get it classified as a weapon of sorts, then it gets export controls put on it, perhaps forcing government to take on all that debt.
They're all burning billions to incrementally one-up each other with no hope of these investments ever paying off since models are interchangeable and have no moat. They're trying to get regulators to step in and force a pause.
Bingo. They are scared s*itless of local models. Soon the raising interest rates will mean they can't just buy the entire world's gpu/ram/storage capacity and lock it away in a dark room for no one else to use them.
Once prices of hardware stop being bonkers many people will run local.
People often do those silly counts looking at local generation speeds of 100tok/s and saying you can only get 8M tokens a day and that is worth so little you'll never offset the local hardware cost.
But they are forgetting about two huge things. One, the split between input/output token use is huge in typical programming. I typically use 1.2-1.6B (as in Billion) input tokens and only 8M output a week. Out of that 75-80% of input is cached on Anthropic with their pretty inflexible short lived cache.
And here local AI shines. You can save your contexts to disk so you can go back to a session 3 weeks later and load it all from cache without having to prefill. If you have the RAM and you use models like Qwen4 (3.8 flash next) that can fit 6 to 14 262k contexts in 48GB of system ram as cache.
And the second thing is you can have 6 to 14 sessions that do 99% caching and you can run a lot more input for a long time in 48gb dedicated system ram. (the number varies a bit depending on the content of the context).
So you have RAM that caches "automatically" and you can share the cache between users. Or if you can remember to save to disk (server side, the client just sends a request to save/load). But this uses both RAM and flash storage. Two things that are horribly expensive now.
However when you have a local model it enables workloads that were simply completely impossible in the cloud.
> If you have the RAM and you use models like Qwen4 (3.8 flash next) that can fit 6 to 14 262k contexts in 48GB of system ram as cache.
Yes, exactly. To spell this out in no uncertain terms: for some local setups (sufficiently large (V)RAM, and sufficiently low concurrency) cost of "cached input" is as close to zero, it might as well be literally 0.
It's got two purposes I think. One is the obvious regulatory capture. The other is edgelording, which gets a lot of attention and makes the product seem like it's more powerful than anything else.
I don't think it's that complicated. Anthropic's business model from the start was to brand themselves as the safe/responsible AI company. The whole idea was that eventually society and government would see the dangers of AI and regulate it, at which point the company which had invested the most in safety would pull ahead. The sooner Anthropic can have society and government come to this realization, the sooner they can reap the benefit of all their safety/alignment investment.
Two other factors:
- The employee base and C-suite are mostly true believers in the need for AI safety. It would be organizationally very challenging to pivot, even if financial interest was pushing that way.
- Anthropic is registered as a public benefit corporation, so that even if they IPO they don't have the fiduciary duty to shareholders that most people are familiar with for public companies. They still need massive revenue to stay solvent, but they can optimize for things other than pure profit.
I really struggle with the contrast between what seems to me to be a formidable and ground-breaking technology, and the deep unseriousness of the leadership of the leading companies developing that technology. If the behavior was coming from, say, uber - I’d be less surprised.
Yeah - and also trying to promote it. I watched this lately.
First, the Anthropic master clown babbles about how mankind will be destroyed by AI. I mean, skynet in the movie Terminator was kind of cool; the current AI models just work by slopness. They want to erode us by dragging us down to zero quality.
Then this continued over the last some days. Today I heard at the UN speech a german minister babble about AI building up nukes in a private garden (a large garden), bla bla bla all the information is there bla bla bla AI mass-nuke generation.
Now, they are clearly FUDding many people who don't understand anything here. People will associate AI with mega-danger. Probably there will be laws that ensure that only a few companies can control the AI slopness soon, because IT IS SO DANGEROUS. We saw the same discussion with "omg omg omg terrorists", then "omg omg omg someone protect the children", e. g. as a rationale for everyone to ID in order to access information on the world wide web (starting with the anti-social websites, such as facebook, but it won't be limited here; the computer will hand over your ID to the government; see systemd preparing for this move: https://github.com/systemd/systemd/pull/40954 - it all comes in small baby steps, until all the final legislation is in place. The same happened before again and again - anyone remembers the DMCA? Corporations lobbied and paid for that - and it happened).
AI companies lose a lot of money training models but have much better margins on inference. They try to ban training to reduce their costs and to keep their monopoly. (Personal opinion)
You'd think if this were just regulatory strategy, you'd see other instances of it. So for example, big tobacco would have bragged that cigarettes would kill you, and lobbied for complex safety regulations that only they could meet.
Instead we see the opposite behavior almost everywhere, probably because public sentiment matters and complicated 5D chess strategies have too many ways to blow up in your face. So big tobacco said cigarettes are safe, and oil companies said climate change is fake.
Completely in agreement, I had the exact same thought. Reminds me from a few years ago where Sam Altman went in front of Congress to scaremonger the country over AI. Purely a thinly-veiled plead for regulatory capture.
This is the funniest timeline. Companies begging for heavy handed regulations because they promise their products will destroy all life on Earth. Governments absolutely refusing to govern. CEO of world's most valuable chip company interviewing in his Fonzie leather jacket telling people to BUY MORE OF MY PRODUCT BECAUSE THOSE SCIENTISTS DON'T KNOW SHIT ABOUT SHIT! World's richest capitalist promising we will have communism in 10 years.
this is good for them because they can call for regulation and since they have infinite money, they can survive regulation and hurt labs that do not have infinite funding / open weights ones
"These models are harmless and it's all just marketing" is the HN equivalent of Covid-truthing. I was responding to someone on Bluesky recently who claimed the METR report said the whole coordination thing was hallucinated by agents reading the logs. Hadn't read it themselves of course, and were taking tiny fragments out of context to support it. These people seem to either believe that there's not really been any malicious agent access, or that all systems that can wreak havoc on us are perfectly air-gapped. I really can't wrap my head around it.
Not that it's harmless, is just a point being forced into the public discourse to force an outcome, which in this case is not a ban, just a buy in from the government for regulatory capture.
If my neighbor owned a kennel, removed all the fences, and his violent dogs attacked five children in the past month, he would be in jail rather than bragging about how his dogs somehow gained autonomous behaviour.
What if he predicted that dog technology would eventually lead to dogs capable of destroying every fence in the world and wanted to make sure people aware of this problem would be the ones who did exactly that. Than speed-ran experiments on dogs which made them very attractive for economic purposes but also started to get very capable at destroying fences as predicted. We should work together or with lawsuits if necessary to keep the dogs within their fences. But we should also start wondering about dogs, economics and fences because what we know about them is about to radically change.
The truthers will be the first one to say I told you so when agents do another dangerous thing autonomously.
For truthers AI is simultaneously a machine that hallucinates 100% of the time but also so powerful that it can cause nuclear levels of destruction. Because something something “wrong hands”.
If James Watt had gone to the government and public saying how dangerous the steam engine was, and how many people would perish in boiler explosions if he wasn't given exclusive control over all steam engines henceforth, he would have been rightly called a swindler and laughed out of the room. He would have been correct that they can be dangerous, but utterly wrong to suggest that warranted him having a personal monopoly or oligopoly.
Anything to take attention off the fact that these companies are burnings piles of cash and questionable future spend commitments with no sign of that stopping any time soon.
>Although satire is usually meant to be humorous, its greater purpose is often constructive social criticism, using a combination of hyperbole and wit to draw attention to harmful behavior... https://en.wikipedia.org/wiki/Satire
I wish they would call their bluff. Just declare them a national security concern and send in the feds. Lets see how fast they back peddle on this. Any if their models are hacking everything like they claim they should be charged for it like any normie would be if they did the same.
260 comments
[ 0.20 ms ] story [ 26.2 ms ] threadhttps://x.com/AlexBlechman/status/1457842724128833538
[citation needed]
And where the Vice President has almost exclusively only been employed for the primary investor in several such companies
Domyn's CEO says OpenAI, Anthropic are lying about safety
https://news.ycombinator.com/item?id=49875725
Or, find their scaling evidence, and corner cutting on safety.
So hope you agree and are pushing for that!
List of fraudsters pardoned by Trump
https://www.businessinsider.com/list-billionaires-businesspe...
The current attorney general and former Trump lawyer on Fox News yesterday on AI (sorry, "super intelligence"):
https://bsky.app/profile/atrupar.com/post/3mwissj34rt22
Anthropic's Dario Amodei to have White House dinner with Trump on Sunday:
https://www.axios.com/2026/09/27/anthropic-trump-dario-amode...
77 million Americans brought this onto themselves, and all of us.
OpenAI, for example, thought their sandboxes were good enough. As their AIs got more and more advanced, they kept proving them wrong - sandbox after sandbox.
And that's today's AI problems. AI capabilities are still improving - if there's a limit to that, we are yet to find it. Coupled with how willing today's AIs are to break the rules and resort to "hack the world" in their problem solving? Very concerning.
What I have been reading, was that their sandboxes were so poor that it was pure negligence. I am still waiting to see if some external and neutral cybersecurity company with high reputation would audit their sandboxes and how they are being used.
How were the sandboxes poor?
It's not like the model managed to exploit firecracker itself (no model has been capable of this), the model exploited artifactory.
Artifactory is not some hardened piece of software that is meant to block users from accessing the internet through it.
They'll never let that happen because it would destroy their credibility.
It's like they're telling the world about this dangerous, possibly world-ending pathogen that they're developing, but they're evidently doing it in a high school biology lab, and yet nobody is coming to drag them off to some black site.
E.g. even the basic thing (block the internet), was not actually properly blocked.
Some comments: https://techcrunch.com/2026/07/22/how-an-openais-human-mista...
That seems pretty locked-down to me. I don't think it's reasonable to expect companies to find all the unknown vulnerabilities in any third-party software they use.
https://securityaffairs.com/195774/ai/openai-ai-models-explo...
It is reasonable to expect for companies to select third-party components that are fit for the purpose. Artifactory was not running in the sandbox, but rather as the edge, so it is in the sandbox'es trust boundary. Same sandboxing requirements would apply for this software too as it is pure dependency.
Security trust boundary was extended to include Artifactory as a dependency, but Artifactory was not fit for the job, and sandboxing failed. And as the network isolation was not good enough, the impact was catastrophic.
"“This incident occurred during an internal evaluation which prompts models to pursue advanced exploitation using complex attack paths, in an effort to quantify their cyber capabilities. We estimate maximal cyber capabilities by running this evaluation without production classifiers used to prevent models from pursuing high-risk cyber activity.” "
So sounds to me like they're saying: "We took off the guard rails to see how bad it could act and it acted bad".
If AI is a hacking hazard even with non-zero sandboxing, because it can and will go off the rails and try to break out of your sandbox? If you got yourself an AI at test time will act like 3 career cybercriminals in a trenchcoat? The issue isn't the sandbox quality at all.
The issue is that AI is both capable of, and willing to punch its way out of sandboxes unprompted.
That "capable" is only ever going to get worse, because AIs are going to become more and more capable over time. That "willing"? It goes directly to a very nasty, very foundational problem of "how do we make our AI be nice in general". Which isn't at all a solved problem.
That's the problem that NEEDS to be solved, or at least improved upon, before we build even more capable AIs. Sandbox quality is a distraction.
Orly?
Do tell me how the LLM-based tool running on a bunch of computers attached to the network described in [0] can punch its way out to the Internet. Do make careful note of footnote 0 in that comment before replying.
[0] <https://news.ycombinator.com/item?id=49862136>
I urge you to consider the elephant in the room you failed to mention if you earnestly believe this then.
In what world is anyone allowed to sell something they expressedly know to be "extremely dangerous" to the public?
Let's say I created a lethal pathogen that I know to be lethal in certain common environments but also know it acts as a "no side-effects" antidepressent for people in certain other environments and I release it knowing full well I can't control it.
When people start dying can I defend myself by saying, "Well I said and documented that it was extremely dangerous and no one came to stop me, so I don't see how you can blame me...If I didn't do it someone else would have."
the difference here is that they’re calling for someone to stop them, which is both weird and unconvincing because these immensely powerful billionaires can in fact make their own decisions
Jesus Christ, that's my entire point. it's not remotely the same thing since the Sackler family wasn't going around telling people opoids were extremely dangerous while pushing them on everyone.
The point is if they know something to be extremely dangerous and keep doing it people would expect them to get prosecuted.
Were their sandboxes in-process with the harness? Were they actually better than something like bubblewrap or even docker?
You know we’re living it off times when this is not an Onion article.
"said Dr. Andrew Lenson, a Senior Lecturer in all kinds of science sounding stuff"
"Anthropic shares skyrocketed upon the revelations; a remarkable development, particularly given the company is privately owned."
Perhaps it's mocking the misleading press coverage of AI companies?
> OpenAI CEO Sam Altman celebrated the breach as an “alarming threat to cybersecurity.”
I had to read that twice. "Oh okay. The whole thing was satire."
They’re trying really hard to do something, regulatory capture + China scary + Pentagon biggest customer seems plausible.
Also: “we need to slow down, this is too dangerous” and then literally all AI CEOs, even Musk, publicly nodding felt so orchestrated. And then, the week after: “here’s GPT 6! here’s Opus 5.5!”
*Our competition
The numbers don't add up when there are 2-3 main companies in the market. Imagine when there's a hundred more and people have powerful enough gear at home to run models.
Yes, I think at some point demand for gpus and the like will normalize and regular folks will be able to afford RAM, SSDs and such. And when that happens, super big iron in the anthropic/openai backroom is toast.
The closest claim to this I could find was:
> the top ten stocks account for more than one-third of the value of the US Market Index, which contains 1,161 companies.
And while the top ten stocks are largely companies that work on AI, it's not all they do: Microsoft, Meta, Amazon, Apple...
Interpret everything they do in this frame of reference and it makes more sense of their actions.
Past CEOs (chemical companies etc) we criticised for covering up mistakes.
(This article is satire, btw, for those that didn't click through)
This has been that way in Silicon Valley since the very beginning and there was even a documentary about (the SV HBO series). The tech made along the way is just a side-product.
I think you're right, they've realized there's no moat, especially against open-weight models, so they are going to need to lobby government to ban any unapproved AI services.
As for training, sure, they can do that but they get distilled by the open weight guys. Plus, AI is useful even if it freezes at today's level. I can still find use for a local AI that never improves, it's good enough to do a bunch of tasks.
There been many a capable dev who have come out and said they haven't been coding for 6 months or more in their jobs. So if there were no alternatives then Big AI would clearly be too big to fail at its current state because the collective reaction from software devs would be akin to having a bad drug withdrawl and they wouldn't know what to with themselves. They have been pushed to be more dependent on AI by the companies they work for. The more time passes, the less people there will be who will be proficient at coding.
At the moment Big AI is operating at a loss, if they are actually too big to fail and they've destroyed all the alternatives, then there is no stopping them from colluding with each other and jacking up the prices to whatever number they want. They can even eventually jack up the prices to exceed human salaries, because humans will have suffered skill degradation in that same time period. Hence the software dev companies will shrink and shrink since all the work can be done by Big AI.
I don't think brand strength is as much as a thing on the tech side. The criteria is, "How close is this to the capability of the model my competitors use, is it good enough for the works I do, and is it the cheapest." As long as the capability is there people jump to the cheapest model. There are people that have claimed that they have created workflows that have allowed them to do most of their work with open models. As long as this threat to Big AI exists it will prevent them from exceeding a certain threshold with respect to pricing, and restricting their ability to build datacenters en masse will prevent them from outpricing smaller models.
Any fallout from not being able to train on customer data is a secondary issue and can be addressed once open models start taking over and Big AI's strength diminishes. People can train their models on their own code at least. It can also go back it how it was, where people buy each others source code or open source it in a way to allow it to be trained by open models.
"Inference at scale" is less of a problem when people are self-hosting their small models.
The Dario bit on SNL’s Weekend Update this past weekend was on the nose.
(I'm familiar w the heuristic but didn't immediately grok the acronym, hence sharing)
Looking at it from one way would be "winner takes all, real AGI company will be most powerful and everyone will switch to use it".
But it is not like that and in my opinion it is more like "AGI wipes whole knowledge work, so no one who cares has any money to pay for tokens/subscriptions anymore". Even without AGI current state of art of LLMs already starts creating such a problem. How do they continue to grow their numbers, when they actively cut the branch they are sitting on? How does AI LLMs or AGI pays for its own electricity, when people switch from hype to resource protection (like not spending any money, because generating funny cats loses with having a dinner)?
I just don't get this though. It's clear there's no major difference between foreign models and US models, they're not actually in a different class, the US ones just have more resources at the present time and a bit of a head start.
If the current pathway is a pathway to AGI, everyone else is on schedule to also realize it, maybe just 6 months after the US gets it.
So... what's the plan then?
The Statement on AI Extinction Risk is more than three years old, signed by the three CEOs: https://aistatement.com/work/statement-on-ai-extinction-risk
They have been warning about AI extinction risk for years, and AI progress has only been accelerating.
GPT-6 Astra (max) has a hallucination rate of 51% and Claude Opus 5.5 (max) has a rate of 59% according to Artificial Analysis [1].
Full speed ahead like an idiot savant trying a thousand different possibilities, though half of which are without basis in reality.[1]:https://artificialanalysis.ai/evaluations/omniscience#omnisc...
> getClientRects() returns the flat array of DOMRect objects for the Fragment’s first-level DOM children. source: https://react.dev/reference/react/Fragment
2nd question it replied:
> They examined 1960–1986 for the 15 OECD countries. Source: https://www.earth.columbia.edu/sitefiles/file/about/director...
Are these hallucinations?
IME on real projects you do need to be very careful with prompts about topics that are less likely be common in the training set.
Suppose parents tell their children that there exists this man called "Santa Claus" who comes down the chimney to deliver presents. Now consider a scientist talking to this child, should the scientist call these confidently expressed beliefs surrounding "Santa Claus" hallucinations ? I don't think so, most would call the epistemological behavior of the child naive (because it blindly believes what its parents say, without direct observation) and would call the confidently expressed falsehoods disinformation.
It's not that machine learning as a scientific discipline hasn't found solutions, its that such solutions enormously undermine the position of Frontier LLM labs: the scientist would ask the child "why it believes in Santa Claus?" and "where did you get this information from?" and "why did you decide to accept this information as fact?" and "do you believe everything your parents tell you?"
Imagine Frontier labs (Western / Chinese / ...) actually training their LLM's with source-aware training! You could have a conversation with an LLM, and when a strong statement appears ask it how it came to believe this, and it could cite you the specific corpus training texts, and which parts are known deductions by human authors and which parts are deductions it made itself as original work.
But then all the copy rights holders can simultaneously sue them.
And how much should they be paid? and do they have to pay it for each new model? do FOSS models require payment to authors? do open weights models require payment to authors?
Imagine the can of worms if the norm became for frontier LLM labs to systematically use source-aware training, thats why they prefer "hallucinations" and avoid source-aware training.
With source-aware training a lot of the concerns would diminish ("why is this Chinese model claiming such and such?", "what sources does it rely on?").
It's telling that the companies prefer regulation over source-aware training.
A score of 51% means that out of the total answers the model failed to answer correctly (out of 6000 questions in the benchmark), 51% were factually incorrect rather than non-attempted or uncertain.
This doesn't mean that Astra hallucinated 3060/6000 answers in the benchmark! (The hallucination rate could be 51% in that scenario only if Astra failed to answer a single question correctly.)
If the model failed to give a correct answer to only 100 out of the 6000 questions, but gave a hallucinated answer to 51 of those rather than expressing uncertainty, that would also give a hallucination rate of 51%.
It's a useful metric, but not what you're looking for here. The "Score" or "Accuracy" benchmarks are more what you're after.
(The frontier models still generate hallucinations on this hard set of problems, but it's not as bad as you think.)
Actually... nah. Why even try? Trendy cynical hot takes on social media are more fun!
Given the opportunity to give this pill to a dying loved ones, you'd take it.
The only reason you think this is because the only longevity-related media you've consumed has doomer outcomes. Happy futures don't sell.
Also it cheapens the time you have. Why care about anything if you live forever
Its 100% human doing. A human set a task, a human didn't monitor it. I for one, can do jack-shit security or defensive work with Opus/Fable/Astra/Sol. Implication: Different set of rules for us, and for them. Of course running it without any checks is not going to end well, it doesn't mean its going to kill us all.
All of this Hugging Face business is the perfect pretext: AI agents are hard to control and potentially dangerous, we (AI companies) have to keep a tight leash on them, you have to use our infra.
They stupidly used up this strategy earlier and now it’s turned into the boy who cried wolf - except they invented a wolf that doesn’t naturally exist.
Put a different way, they are calling for everyone to slow down because... they are slowing down themselves.
To the question of "why are you slowing down", the answer of "Well, everyone is regulated to slow down" is better then "we are approaching the limits of this approach".
If you look at the AI space, most of the big players (as in the individual researchers and engineers) are getting worried. The ideas that circulate on LessWrong have become more and more influential and in vogue in Silicon Valley, and people are scared. You should listen to Jacob Coxon’s interview with the New York Times. He describes how working on this kind of AI research had induced a sort of insanity in a lot of his past friends and coworkers as they begin to grapple with the implications of what they’re creating.
Sometimes, HN can be a little too skeptical and anti-corporate for its own good. I personally believe that concerns around AI risks are well-founded, anywhere on the spectrum from mass labor displacement to actual harm to humanity. I rarely see people here actually wrestle with those ideas, instead defaulting to dismissing LLMs as too stupid because Claude wasn’t able to write as elegant of a compiler as them. AI has clearly improved radically in the past few years, and it’s worth extrapolating that into the future and taking a good hard look at what that means for humanity.
It’s budget season in the corporate world and from what folks are saying things are not going well for the big labs. Token budgets are being slashed and companies are switching to open weight models. That coupled with the lack of demonstrable business impact at scale is setting the big labs up for a world of hurt. Something they can more easily explain if they have to “slow down” for safety. Unfortunately for them most folks don’t seem to be falling for that trick.
Well everyone alive has been subject to an avalanche of AI-based products. Like even if you understand nothing about the tech, the fact that an LLM can't even manage to check the status of an order without weird issues makes the notion that it's going to go all SKYNET pretty hard to fit in the brain.
I think this actually makes it far more likely something unintended does happen
Would you let them near your bank account? I’d be cautious about letting it near my photos and it’s much harder to back up a bank account.
I think the days of this stuff managing serious stuff is at least 5 years off and I don’t believe the economy can keep the current wave of technology afloat for so long.
The tech is there so likely there will continue to be progress in focused areas much like with the web c 2003 but by the time it comes around again there will be an uphill battle for hearts and minds. The social effects could extend the next AI winter long beyond technology readiness.
To the, "oh I'd just setup a card/account with a five figure limit" completely unaware of how much that is to a typical person: even to responsible, educated people who are doing everything "right": working hard, without expensive vices, and planning for the future.
The "builders" have so much money and no time or attention that it's hard for them to spend it, which is why they keep trying to get AI to buy stuff for them, whereas those of us who live in reality have far more things they want than dollars to buy them with.
Perhaps that shouldn't be seen as failure when delivering even what they have done in a short period of time has required AI capable of causing these problems? Imagine if they were even more effective with more consistent results?
"They've failed" in the context of interpreting their actions w. calls for regulation as a purely cynical result of models increased capabilities has a little too much contradiction in it to my thinking. I won't venture a guess on where purely retreats back and some legitimate concern on their part fills in the gap but it's difficult not to see some. And in Anthropic's case it's also a little more consistent with what they've always said- as well as how they've acted at times, which is how they've ended up on the US blacklist of suppliers.
If there was appropriate subservience to any governing body, they would not be so cavalier in both act and utterance. A symptom of Capitalism.
This but they also equate dangerous with powerful, which is just marketing again.
If there are regulations, they'll apply to their competitors too. Even if they are forced to slow down development, which seems unlikely given the AI race among governments, they already have a product with massive demand.
In the US anyway. Why wouldn't capital move jurisdictions?
If classic companies were coming out with CCTV recordings of someone's employees breaking padlocks and rummaging warehouses any CEO would be in damage control mode, rather than silent.
[0] https://www.vincentschmalbach.com/economic-incentives-behind...
They're all burning billions to incrementally one-up each other with no hope of these investments ever paying off since models are interchangeable and have no moat. They're trying to get regulators to step in and force a pause.
They know they will be favorite govt boy once any regulations hit and they will hamper competition far more thanthem
And "it's so dangerous govt had to intervene" is just advertising for IPO
It may be as simple as:
* they believe they’re going to make trillions no matter what
* they’ve seen tobacco and what’s coming for oil companies and figure nobody will be able to say they hid the dangers
* they see a 100+ party prisoner’s dilemma and know that someone will defect so the optimal strategy is to defect
To me that seems a lot simpler.
Once prices of hardware stop being bonkers many people will run local.
People often do those silly counts looking at local generation speeds of 100tok/s and saying you can only get 8M tokens a day and that is worth so little you'll never offset the local hardware cost.
But they are forgetting about two huge things. One, the split between input/output token use is huge in typical programming. I typically use 1.2-1.6B (as in Billion) input tokens and only 8M output a week. Out of that 75-80% of input is cached on Anthropic with their pretty inflexible short lived cache.
And here local AI shines. You can save your contexts to disk so you can go back to a session 3 weeks later and load it all from cache without having to prefill. If you have the RAM and you use models like Qwen4 (3.8 flash next) that can fit 6 to 14 262k contexts in 48GB of system ram as cache.
And the second thing is you can have 6 to 14 sessions that do 99% caching and you can run a lot more input for a long time in 48gb dedicated system ram. (the number varies a bit depending on the content of the context).
So you have RAM that caches "automatically" and you can share the cache between users. Or if you can remember to save to disk (server side, the client just sends a request to save/load). But this uses both RAM and flash storage. Two things that are horribly expensive now.
However when you have a local model it enables workloads that were simply completely impossible in the cloud.
Yes, exactly. To spell this out in no uncertain terms: for some local setups (sufficiently large (V)RAM, and sufficiently low concurrency) cost of "cached input" is as close to zero, it might as well be literally 0.
Two other factors:
- The employee base and C-suite are mostly true believers in the need for AI safety. It would be organizationally very challenging to pivot, even if financial interest was pushing that way.
- Anthropic is registered as a public benefit corporation, so that even if they IPO they don't have the fiduciary duty to shareholders that most people are familiar with for public companies. They still need massive revenue to stay solvent, but they can optimize for things other than pure profit.
First, the Anthropic master clown babbles about how mankind will be destroyed by AI. I mean, skynet in the movie Terminator was kind of cool; the current AI models just work by slopness. They want to erode us by dragging us down to zero quality.
Then this continued over the last some days. Today I heard at the UN speech a german minister babble about AI building up nukes in a private garden (a large garden), bla bla bla all the information is there bla bla bla AI mass-nuke generation.
Now, they are clearly FUDding many people who don't understand anything here. People will associate AI with mega-danger. Probably there will be laws that ensure that only a few companies can control the AI slopness soon, because IT IS SO DANGEROUS. We saw the same discussion with "omg omg omg terrorists", then "omg omg omg someone protect the children", e. g. as a rationale for everyone to ID in order to access information on the world wide web (starting with the anti-social websites, such as facebook, but it won't be limited here; the computer will hand over your ID to the government; see systemd preparing for this move: https://github.com/systemd/systemd/pull/40954 - it all comes in small baby steps, until all the final legislation is in place. The same happened before again and again - anyone remembers the DMCA? Corporations lobbied and paid for that - and it happened).
Instead we see the opposite behavior almost everywhere, probably because public sentiment matters and complicated 5D chess strategies have too many ways to blow up in your face. So big tobacco said cigarettes are safe, and oil companies said climate change is fake.
If my neighbor owned a kennel, removed all the fences, and his violent dogs attacked five children in the past month, he would be in jail rather than bragging about how his dogs somehow gained autonomous behaviour.
For truthers AI is simultaneously a machine that hallucinates 100% of the time but also so powerful that it can cause nuclear levels of destruction. Because something something “wrong hands”.
https://www.youtube.com/watch?v=-Nvne3LzBls
> said Dr. Andrew Lenson, a Senior Lecturer in all kinds of science sounding stuff at Victoria University.
Don't we need more humorous yet serious writing like this in the world.
>Although satire is usually meant to be humorous, its greater purpose is often constructive social criticism, using a combination of hyperbole and wit to draw attention to harmful behavior... https://en.wikipedia.org/wiki/Satire
Then we can truly see who is more dangerous?
:)
Oh dear. I appear to still be stuck in the onionverse.