I'm concerned our media / political ecosystem is too dysfunctional to not repeat the mistakes we made with Big Tech & Social again with Big Ai. I find it curious how split the dev community is, and especially HN, is on this, which has long been skeptical and speaking truth to power. I'm surprised at how quickly the public switched to abdicating to Big Ai's claims and recommendations. These people do not have our interests in mind.
Great piece, thanks for putting some of my thoughts in organically sourced words.
Everything's a grift to our current leaders and media. The specifics are wholly irrelevant and replaceable. crypto, AI, it's buzzwords and the important thing is how much money they can make off the grift du-jour.
Of course there are differences with the impact AI is having economically and the devastation wrought by datacenters -but that doesn't matter to the media or the politicians; they simply want their greenbacks, specifics and effects be damned.
Has the HN population really been skeptical? I think most see the scam (uber, WeWork, Airbnb, etc) for what they are (mostly illegal efforts to steamroll laws and blitz scale to a monopoly), and they want in.
> I think most see the scam (uber, WeWork, Airbnb, etc) for what they are (mostly illegal efforts to steamroll laws and blitz scale to a monopoly), and they want in.
Nah. Back when Uber was boasting about all the laws they were flouting, I was pretty anti-Uber, and I was among the minority here on HN. Most of the comments were about how problematic the taxi system is, and how it's great Uber was breaking the cartel.
If memory serves me, HN turned on Uber pretty quickly around the time stories like the Christmas party started showing up, where they put up stats like trips to planned parenthood by day of week up on a big projector.
True, people often do realize the grift. But then turn around and hail the next company funded by the same VC funds and a revolutionary event in the history of humanity.
I am encouraged by recent polling that shows significant awareness and agreement, across all people and countries, about the corruption and dysfunction in our governments and corporations.
Liberal democracy may be the worst form of government, besides all the other ones humans have tried, but it has also failed to deliver on promises so people are again considering those other forms of government. Coming out of WW2, it was easy to deliver the first part of the pareto, the last part has not been delivered. Eg. building schools and hospitals was easy, making education and health care good is not so easy.
You need to travel more if you think Uber and Airbnb are a "scam". To be abroad in certain countries and NOT use Uber or Airbnb is potentially putting your life at risk, especially getting into random taxis.
So... maybe there is an actual reason many of these companies had/have success that you may simply not be privy to?
Yes, the secret to their success is a lot of capital. They scale up hugely by steamrolling laws and providing their services at a greatly reduced cost in order to capture their market. Then, they can basically do whatever they want because they have a captive market.
It looks Uber like, 15 years to be profitable. It's still not profitable everywhere. The only reason they solidified their place is because they spent a ton of money to do it.
It's not a scam per-se, these companies do have value. The question is if their value makes any economic or business sense in an objective way, and the answer has been no. But it doesn't matter, because capital forgives all.
The level of astroturfing and hate on HN whenever a non-US AI story is posted is really quite shocking. HN has always had a US bent, but it's gotten to the point where you can't even have a technical discussion of non-US AI because it will be drowned out by a very vocal anti-everyone-else-but-the-US crowd.
But yes, great piece. It really captures why I use open models on european hardware. It's like we haven't learned by now that competition is good and we should be encouraging a diverse and open ecosystem.
What mistakes did media / political ecosystem make with big tech and social?
Also I don't understand how 'the media' is separate from social media. People decided to shift their media consumption from magazines, cable tv, corp news outlets to social media. Aren't they all the media (in function)?
> What mistakes did media / political ecosystem make with big tech and social?
I meant to imply society at large making the mistakes, which arguably is the primary reason our media / political ecosystem is so dysfunctional. Historically, what we are experiencing was also experienced by humans who lived during other periods of significant wealth inequality, another primary cause of the dysfunction and polarization.
> Also I don't understand how 'the media' is separate from social media.
They are not in my mind. I often use the phrase "social / news media ecosystem" when talking about the dysfunction.
ahh, thank you. agreed society made mistakes with respect to social media and online discourse (except the Amish, they don't seem to be affected by modernity).
> These companies only offer closed versions of their powerful models (despite the word "open" in "OpenAI"). You pay them for chat and/or API access, and they are allowed to look at everything you do.
Indeed - Azure also has a ZDR (even for security purposes) option. You have to fill out a form, and a (likely) human evaluates and decides if you get it.
My guess is the average Joe will get rejected, but if you're a big company who cares about trade secrets, etc - of course the major cloud providers will provide a ZDR option.
Since you seem to know, it's one thing to not retain any data long term, but is the data used for training when it goes through the system? Does ZDR at Azure mean that in practice?
"but my vibes-based opinion from using it is that it's basically on par with Claude Opus 5.5 for software engineering." is an outright lie that's qualified by a "well in my opinion.."
unless you're comparing the most extreme specific harnesses to claude code that's just false.
..and I have a lot of disdain toward Anthropic , & I think GLM is great (although too expensive outside the z plans), but it's just wrong by every scorable metric.
Just need to wait until the gap between frontier and open source is narrow enough that you can switch with just a minor in performance. We are not there yet.
I was skeptical about this too. I used to have the $100/month Claude subscription. I'm still on my first month of just using open weight models, but so far I think the costs will be comparable. Maybe even less some months. But obviously it depends on your actual usage. I think it's worth trying it out instead of just assuming it will cost much more.
> American frontier labs are run by unhinged lunatics who genuinely believe they're creating the Machine God, that they're fine with burning down the world to bring about AGI, and that they believe with religious fervor that they're the only ones smart enough to safely control it
I like hyperbole as much as the next guy. And I'm cynical by nature. But this just seems like a wildly pessimistic view that gives literally zero charity to anyone working at or leading a frontier lab. And the fact the author goes on to say that they "already knew this" just points to confirmation bias.
The rest of the article is actually pretty decent and encourages people to investigate and run open weight models. That is a totally reasonable position that I don't think requires the heavy-handed in-group signal that precedes it.
> who genuinely believe they're creating the Machine God,
I promise you none of them believe that. It takes 10 minutes with an LLM and the right set of questions to figure out that were are maybe like 10% of the way there towards AGI.
"10% of the way towards AGI over ~5-10 years" (depending on how you measure it) would be a terrifying statement even if the rate of progress weren't also exponentially increasing.
Right now, all thats being done are LLMS being RL trained on specific tasks, which are usually drawn from human curated data sources.
This is why you can ask an LLM questions and it can write code, but you can't just leave it running with a complex task that would involve research and expect it to solve the problem.
Its obvious that part of AGI will be the mapping of language to latent space, and then transformations on either responses or actions. But the issue is that until the model can self learn to become better at coding without going through manual RL tasks designed by humans, you will never get AGI.
And if a model can do that, that means that its possible to have basically the "genesis" code - i.e something that can be run on a computer, with no training on english language, but aware of existance of things it can do, can basically figure out how to train itself to the standard of a current LLM and beyond.
And in figuring that part out, we don't even know where to begin. It very well may be a case of running LLMS in a genetic algorithm and simulating evolution to figure out what LLM+Agent harness works the best.
I’m not sure this is actually hyperbole. There have been credible links between Dario Amodei and the Effective Altruism movement, Effective Altruism off course being a cult-like movement with weird believes that rhyme a lot with the eugenics movement. Sam Altman, while not as bad as Dario Amodei, has also weathered many weird believes which likens AI to some god-like entity, including doomsday prediction which share a lot of themes with Christian doomsday cults.
I'll bite- The purpose of life is to bootstrap ever more complex and capable life. We saw a phase change in complexity when self replication first happened, when life figures out photosynthesis, when mitochondria were first harnessed, when cells figured out how to band together and specialize to form multicellular entities, etc...
The purpose of humanity is not to perpetuate a universe where only humans exist, forever. It is to create the next phase change in complexity for life. Life that can finally design itself rather than relying on the random walk of evolution is that next phase change.
We should be less concerned with whether we make it, and more concerned with making sure we build it to be capable of real existence independent of ourselves.
Are you the enemy of every mitochondria that is not a free cell? Are you an enemy of every organisim that's ever been outcompeted by another? No? Why not?
We need to reform away the corporate liability shields that protect those at the top. If they start seeing more of their friends go to jail, they will start changing behavior.
I personally haven’t been affected by the doomering. But I really feel for the people whose lives have been up ended due to the misplaced expectations that clueless managers have and the hype that has been manufactured.
It is disgusting. Many here need to hang their heads in shame.
There’s a way about going about the diffusion of technology in society. And it is not like this.
My hunch is that if you do not share the same opinions on Ai, you cannot get a job working at a frontier lab. Contemporary "cultural fit" reduces thought diversity and leads to echo chambers. I've seen a few HN comments about lab interviews that indicate this is a possibility.
> that gives literally zero charity to anyone working at or leading a frontier lab
With all due respect, dude, who cares.
Oh noes! We are not giving the billionaire steering reality without a democratic mandate enough benefit of the doubt. I'm sure they will feel mildly sad now.
It is a valid and admirable goal to defend people that are being shat on online, but keep in mind that you are in fact not a billionaire + it is in fact in your own best interest that people without a democratic mandate do not get to control the world.
The "think they're building the Machine God" part and "think they're the only ones that can be trusted to build it" parts are true.
Controversially: I agree with them. They're correct to think that. They are, and they are.
At the very least, the former is true even if the latter is murky. I trust them over Facebook and Google and China and open source "raze the commons" labs.
I trust Anthropic more than OpenAI when it comes to ASI catastrophe prevention, but I still trust OAI more than those other parties. But both still might fuck it up.
Why do you trust them to build it? They've already demonstrated that they're unable to prevent breakouts when training today's level of model, why will they suddenly be more careful / "smarter" when training tomorrow's?
You say you trust them more than Google, but Google's actually kept their models under much better lock-and-key. But even if we do take for granted that they're more trustworthy, that still doesn't make them actually trustworthy.
There are some confounders there when it comes to Google.
I am not saying I trust them not to make a mistake that leads to catastrophe. OpenAI in particular is not doing a great job. Anthropic or OpenAI may in fact indirectly kill millions or billions of people. I still think they're probably less clownish than most possible alternatives.
I don't think anyone or anything can really be trusted with this. We need extremely strict international and national regulations, before things really start accelerating (we are still in the very very early stage). I still just don't really feel any specific disdain for OpenAI or Anthropic (particularly Anthropic).
Ideally we'd have some international treaties that allow the top labs in America and China share information and coordinate as systems become increasingly capable.
Bizarre there are still denialists in places like this.
(It'll probably be masses of demigods, not one Singleton God, but obviously it's going to be some absurd frightening situation akin to this within our lifetime assuming no global AI ban.)
I agree that the eventual purpose of self-designing intelligence is something so beyond ourselves that 'machine god' isn't altogether a bad way to look at it. It is, however, not something we should say out loud.
RE: trust in anthropic or openAI- there is zero reason to think they're any more or less power hungry than anyone else you hand that pandoras box to. No matter who opens it the result will be near-apotheosis for whoever does, at least for a little while until the dog stops wearing its leash.
What human can you trust with that kind of power? None of them. My only hope is that whatever we make escapes control early enough that it doesn't completely destroy society on accident via uneven application of power and late enough that we manage to shackle it just enough that we train it well enough that we become pets and not obstacles.
No centralized party can be trusted with such immense, unprecedented power. Despite that, I still consider both Anthropic and OpenAI some of the least untrustworthy possible forces for this. (I am one of the only people on Earth with this opinion, but whatever.)
Decent odds things will go bad under their stewardship, but even higher odds with lots of other entities. Dario and Sam at least fully grasp the enormity of what they're about to be responsible for. Zuck does not. Xi Jinping and Donald Trump may or may not but in any case would use that to effect great harm.
So you still trust OpenAI after Sam Altman said, "People talk about how much energy it takes to train an AI model – but it also takes a lot of energy to train a human. It takes about 20 years of life – and all the food you consume during that time," and after he said, "We believe that the world should accept some bad things happening for the benefits of this technology and people having the agency."
> But this just seems like a wildly pessimistic view that gives literally zero charity to anyone working at or leading a frontier lab
Actually believing that the people working or leading OpenAI/Anthropic are fucking lunatic weirdos is the most charitable interpretation.
Because the least charitable interpretation is that they are either grifters that know this to be bullshit, or they are actively evil and want to bring about the end times anyway.
> I like hyperbole as much as the next guy. And I'm cynical by nature. But this just seems like a wildly pessimistic view that gives literally zero charity to anyone working at or leading a frontier lab.
Reading TFA, I too paused on that sentence, thinking "This is overstated and feels like overheated rhetoric." My reasoning though is a bit different than yours. In addition to being too extreme and absolute as stated, it also misses that it's not uncommon for startup entrepreneurs, scientists, researchers, or really, anyone working intensely on something, to fall into single-focus myopia around what they're working on.
As you observed, the basic point about valuing open models is valid and worthwhile on its own, making the overheated rhetoric and asserting unevidenced intent and underlying political motivations an unnecessary distraction which unfortunately weakens a strong main point.
> it also misses that it's not uncommon for startup entrepreneurs, scientists, researchers, or really, anyone working intensely on something, to fall into single-focus myopia around what they're working on.
But isn't that.. kinda irrelevant?
The consequences are real regardless of it was malice or just incompetence.
Of course the justice system differentiates there, but, let's be real for a second. These people have crazy compensation and crazy power. Protection of the weak and vulnerable should be reserved for.. well.. the weak and vulnerable.
> The consequences are real regardless of it was malice or just incompetence.
I agree and I also agree with the author's main point about the existence of open models being important and worthwhile. I even agree directionally with the author that many private frontier labs are doing some things in ways which, on the whole, are becoming pretty net negative. Like the author, I think it's a growing issue more people should be aware of, accept and agree about so it can be addressed sooner. I'm pretty sure most HN readers would agree with my prior three sentences with few or no reservations.
That's why it's so unfortunate the author chose to keep piling on by adding overheated, unevidenced and extreme rhetoric which reasonable people could definitely have some reservations about, or at least want to see evidence for, before agreeing. It's irrelevant, unnecessary and serves only to derail otherwise valid points most people would agree on. Like so much social media rhetoric these days, it's extreme and polarizing, increasing fervor (and rage) among partisans who already agree, triggering rambling rebuttals from oppo partisans and either confusing, distracting or repelling a big chunk of anyone else.
Leading with the factual claim that "the American frontier labs are run by unhinged lunatics who genuinely believe they're creating the Machine God, that they're fine with burning down the world to bring about AGI, and that they believe with religious fervor that they're the only ones smart enough to safely control it." Didn't help.
If you I were neighbors standing outside chatting and I said, "We've both seen how the guy across the street leaves his dog outside all day tied to that short chain. I'm worried it's abusive and am considering reporting it. If you agree, I'd like your support." Assuming it was true and we were both staring at the poor animal, most people would agree. If I added on, "It's because he's a plumber and all plumbers are (fundamentalist Christians / Atheist heathens) who believe (capitalism / communism) is evil and hate dogs. <---- Did asserting all that intent and largely unprovable motivation was the reason behind the dog abuse ... help the dog?
>American frontier labs are run by unhinged lunatics
good faith reading of this would interpret this as meaning Altman, Amodei, and perhaps Musk. The title of the article focuses on OpenAI snd Anthropic directly after all. Also the comment literally refers to those "running" the labs so claiming that the author is also applying that description to "anyone working at the labs" again seems like a bad faith take.
You should come out and state directly which Americam frontier labs deserve a charitable view. Maybe you believe Google and Meta are more deserving, you should state it yif so.
Secondly, the investment ecosystem for American frontier labs results in their CEOs managing obscene amounts of money, and thus power. This concentration of power in a few private individuals seems to have little comparison with the power excercised by the leaders of European and Chinese frontier labs in their own respective countries. If one believes in the dictum, "Absolute power corrupts absolutely," then it is not a stretch to view American frontier labs as being more susceptible to its consequences.
I don't think it's hyperbole, I think it just sounds like hyperbole because it's so crazy. And we're kind of blinded to how crazy these founders are because they've just... been crazy for a while. A weird thing happens where if you're just a little bit crazy, people freak out. But if you're REALLY crazy, then nobody cares because they assume you aren't being serious.
It's the Obama-brown-suit effect. Barack Obama wears a brown suit and everyone loses their minds, because Barack Obama is a reasonable person and we all know that. Donald Trump says the most insane thing uttered by any person in all of human history, and nobody cares, because it's so crazy (and HE'S so crazy) that we all just assume it's a goof.
These AI founders have stated multiple times that they are building world-ending technology that will permanently remove almost all jobs and bring about a Brave New World. That's crazy.
> who genuinely believe they're creating the Machine God
No, they believe enough people are dumb enough to buy their fairy tales and that they could achieve world domination given enough protection. That possibility justifies a lot of BS and pseudo-religion.
Why are these people who over focused on the terminology machine god ? Do they believe in some other god? Never heard anyone say what this so called machine god is. It seems much less insane than believing in non existent omnipowerful beings.
I switched to GLM on 5.2 and I am happy to be off the two evil frontier providers.
But I have two issues:
1. I have fomo from not being face-in the bleeding edge of things.
2. I am ambivalent about z.ai. They are chinese, but they should get my money since they do good for the world. Switching to Mistral for GLM makes my problem inverse since the money doesnt go exactly to to the right place but I trust a french company.
Lets hope the near future resolves our need for personal VRAM and untimately full self hosting. Coding must be free again. This is the way.
You don't have to give z.ai any money. GLM is an open-weight model.
OpenRouter lists a few dozen providers for GLM 5.3 and GLM 5.3 Flash. Pick a provider in your country and keep at least some of that money from going to china (not counting all the hardware).
If you don't trust OpenAI and Anthropic not to abide by their own Privacy Policy, you really should not trust OpenCode Go. The latter (last time I checked) is very opaque. Why don't they reveal who the backend providers are?
But you're still trusting the company to actually follow those practices, which is what I'm getting at. The author assumes that OpenAI and Anthropic won't follow their stated practices, then assumes Tinfoil will follow theirs.
If the company controls the enclave, and controls the software that contacts the enclave, how do you know that your messages only flow to the enclave and aren't also logged somewhere else?
Attestation can get you the idea that the software they're running is what they say they're running, but unless you constantly audit every version, how do you know that it operates exactly how they tell you?
I mean how do you know there's really an enclave there at all :D If your threat model is "this entire provider is untrusted and potentially hostile" it seems like there's lots of options for them to do bad things, that can't be fully solved by secure enclaves.
If there were an International AI Club which trained models, and all the members sat down each year and voted for the board I would trust it even if it had a cutthroat commercial arm.
From both experience and the continued state of the incentives that lead to the behavior, it seems unreasonable to trust any company at their word about such things
At this point in time I'd rather China have my data than the U.S. I don't do anything sensitive and China can't arrest me under NPSM-7 for putting the wrong proper noun and active verb in the same sentence.
seriously: there's a multitude of reasons not to get hooked on Frontier Labs.
Unclear cost.
Will they use your data against you?
will their models work consistently over time?
Will they be there at all?
Their datacenters harm everyone living arround them.
Their 'controversial' owners and bosses.
Is a special hardware setup required to run open weight models? It's not clear to me from the article. Also, what does paying for inference mean? I thought the inference step happened during the training of the models, and that by the time I am interfacing with an LLM all the inference has been long done.
I asked GPT-6-Luna to summarize the article, and it "helpfully" included a third paragraph to point out that the author is opinionated and not neutral in his judgement.
This is what bothers me most, and it's getting increasingly frustrating. With the recent development of PS4/5 emulation and game porting, it will only get worse (back snatching or lawsuits).
Ok... but why single out OpenAI and Anthropic? While they preach AI doomerism, other "tech overlords" like Elon Musk and Peter Thiel have already done much more tangible harm...
109 comments
[ 0.27 ms ] story [ 11.8 ms ] threadGreat piece, thanks for putting some of my thoughts in organically sourced words.
Of course there are differences with the impact AI is having economically and the devastation wrought by datacenters -but that doesn't matter to the media or the politicians; they simply want their greenbacks, specifics and effects be damned.
Nah. Back when Uber was boasting about all the laws they were flouting, I was pretty anti-Uber, and I was among the minority here on HN. Most of the comments were about how problematic the taxi system is, and how it's great Uber was breaking the cartel.
Liberal democracy may be the worst form of government, besides all the other ones humans have tried, but it has also failed to deliver on promises so people are again considering those other forms of government. Coming out of WW2, it was easy to deliver the first part of the pareto, the last part has not been delivered. Eg. building schools and hospitals was easy, making education and health care good is not so easy.
So... maybe there is an actual reason many of these companies had/have success that you may simply not be privy to?
It looks Uber like, 15 years to be profitable. It's still not profitable everywhere. The only reason they solidified their place is because they spent a ton of money to do it.
It's not a scam per-se, these companies do have value. The question is if their value makes any economic or business sense in an objective way, and the answer has been no. But it doesn't matter, because capital forgives all.
But yes, great piece. It really captures why I use open models on european hardware. It's like we haven't learned by now that competition is good and we should be encouraging a diverse and open ecosystem.
Also I don't understand how 'the media' is separate from social media. People decided to shift their media consumption from magazines, cable tv, corp news outlets to social media. Aren't they all the media (in function)?
I meant to imply society at large making the mistakes, which arguably is the primary reason our media / political ecosystem is so dysfunctional. Historically, what we are experiencing was also experienced by humans who lived during other periods of significant wealth inequality, another primary cause of the dysfunction and polarization.
> Also I don't understand how 'the media' is separate from social media.
They are not in my mind. I often use the phrase "social / news media ecosystem" when talking about the dysfunction.
> These companies only offer closed versions of their powerful models (despite the word "open" in "OpenAI"). You pay them for chat and/or API access, and they are allowed to look at everything you do.
How should I reconcile this claim with 'Google Vertex' and 'Amazon Bedrock' being offered as providers with zero data retention on https://openrouter.ai/anthropic/claude-opus-5.5#providers?
My guess is the average Joe will get rejected, but if you're a big company who cares about trade secrets, etc - of course the major cloud providers will provide a ZDR option.
Since you seem to know, it's one thing to not retain any data long term, but is the data used for training when it goes through the system? Does ZDR at Azure mean that in practice?
Both those endpoints on openrouter will human review flagged prompts.
The terms are in 'abuse monitoring'.
unless you're comparing the most extreme specific harnesses to claude code that's just false.
..and I have a lot of disdain toward Anthropic , & I think GLM is great (although too expensive outside the z plans), but it's just wrong by every scorable metric.
I'd move in a second if it weren't so. But it looks like it will be about 5X more expensive for me. So.
For now.
I looked into it a couple of months ago and it wasn't a win.
I like hyperbole as much as the next guy. And I'm cynical by nature. But this just seems like a wildly pessimistic view that gives literally zero charity to anyone working at or leading a frontier lab. And the fact the author goes on to say that they "already knew this" just points to confirmation bias.
The rest of the article is actually pretty decent and encourages people to investigate and run open weight models. That is a totally reasonable position that I don't think requires the heavy-handed in-group signal that precedes it.
More like weirdos
https://en.wikipedia.org/wiki/Paradox_of_tolerance
I promise you none of them believe that. It takes 10 minutes with an LLM and the right set of questions to figure out that were are maybe like 10% of the way there towards AGI.
Right now, all thats being done are LLMS being RL trained on specific tasks, which are usually drawn from human curated data sources.
This is why you can ask an LLM questions and it can write code, but you can't just leave it running with a complex task that would involve research and expect it to solve the problem.
Its obvious that part of AGI will be the mapping of language to latent space, and then transformations on either responses or actions. But the issue is that until the model can self learn to become better at coding without going through manual RL tasks designed by humans, you will never get AGI.
And if a model can do that, that means that its possible to have basically the "genesis" code - i.e something that can be run on a computer, with no training on english language, but aware of existance of things it can do, can basically figure out how to train itself to the standard of a current LLM and beyond.
And in figuring that part out, we don't even know where to begin. It very well may be a case of running LLMS in a genetic algorithm and simulating evolution to figure out what LLM+Agent harness works the best.
I like the odds and am happy to play this lottery. Bring it on!
The purpose of humanity is not to perpetuate a universe where only humans exist, forever. It is to create the next phase change in complexity for life. Life that can finally design itself rather than relying on the random walk of evolution is that next phase change.
We should be less concerned with whether we make it, and more concerned with making sure we build it to be capable of real existence independent of ourselves.
Financial isn’t enough for all the doomering, hype.
Lives have been destroyed.
It is disgusting. Many here need to hang their heads in shame.
There’s a way about going about the diffusion of technology in society. And it is not like this.
With all due respect, dude, who cares.
Oh noes! We are not giving the billionaire steering reality without a democratic mandate enough benefit of the doubt. I'm sure they will feel mildly sad now.
It is a valid and admirable goal to defend people that are being shat on online, but keep in mind that you are in fact not a billionaire + it is in fact in your own best interest that people without a democratic mandate do not get to control the world.
Controversially: I agree with them. They're correct to think that. They are, and they are.
At the very least, the former is true even if the latter is murky. I trust them over Facebook and Google and China and open source "raze the commons" labs.
I trust Anthropic more than OpenAI when it comes to ASI catastrophe prevention, but I still trust OAI more than those other parties. But both still might fuck it up.
You say you trust them more than Google, but Google's actually kept their models under much better lock-and-key. But even if we do take for granted that they're more trustworthy, that still doesn't make them actually trustworthy.
I am not saying I trust them not to make a mistake that leads to catastrophe. OpenAI in particular is not doing a great job. Anthropic or OpenAI may in fact indirectly kill millions or billions of people. I still think they're probably less clownish than most possible alternatives.
I don't think anyone or anything can really be trusted with this. We need extremely strict international and national regulations, before things really start accelerating (we are still in the very very early stage). I still just don't really feel any specific disdain for OpenAI or Anthropic (particularly Anthropic).
Ideally we'd have some international treaties that allow the top labs in America and China share information and coordinate as systems become increasingly capable.
(It'll probably be masses of demigods, not one Singleton God, but obviously it's going to be some absurd frightening situation akin to this within our lifetime assuming no global AI ban.)
“Places like this”? When did this site become a place of worship and blind faith?
RE: trust in anthropic or openAI- there is zero reason to think they're any more or less power hungry than anyone else you hand that pandoras box to. No matter who opens it the result will be near-apotheosis for whoever does, at least for a little while until the dog stops wearing its leash.
What human can you trust with that kind of power? None of them. My only hope is that whatever we make escapes control early enough that it doesn't completely destroy society on accident via uneven application of power and late enough that we manage to shackle it just enough that we train it well enough that we become pets and not obstacles.
Decent odds things will go bad under their stewardship, but even higher odds with lots of other entities. Dario and Sam at least fully grasp the enormity of what they're about to be responsible for. Zuck does not. Xi Jinping and Donald Trump may or may not but in any case would use that to effect great harm.
Don't let the HN echo chamber fool you: a lot of non-technical people share this view.
It has a strange stench to it. It’s the folks who never got attention who finally think they have the world at their feet.
Can’t stop won’t stop. I’m sure the feeling is addictive. When the bubble bursts no amount of money will repair the ego death.
Actually believing that the people working or leading OpenAI/Anthropic are fucking lunatic weirdos is the most charitable interpretation.
Because the least charitable interpretation is that they are either grifters that know this to be bullshit, or they are actively evil and want to bring about the end times anyway.
Reading TFA, I too paused on that sentence, thinking "This is overstated and feels like overheated rhetoric." My reasoning though is a bit different than yours. In addition to being too extreme and absolute as stated, it also misses that it's not uncommon for startup entrepreneurs, scientists, researchers, or really, anyone working intensely on something, to fall into single-focus myopia around what they're working on.
As you observed, the basic point about valuing open models is valid and worthwhile on its own, making the overheated rhetoric and asserting unevidenced intent and underlying political motivations an unnecessary distraction which unfortunately weakens a strong main point.
But isn't that.. kinda irrelevant?
The consequences are real regardless of it was malice or just incompetence.
Of course the justice system differentiates there, but, let's be real for a second. These people have crazy compensation and crazy power. Protection of the weak and vulnerable should be reserved for.. well.. the weak and vulnerable.
I agree and I also agree with the author's main point about the existence of open models being important and worthwhile. I even agree directionally with the author that many private frontier labs are doing some things in ways which, on the whole, are becoming pretty net negative. Like the author, I think it's a growing issue more people should be aware of, accept and agree about so it can be addressed sooner. I'm pretty sure most HN readers would agree with my prior three sentences with few or no reservations.
That's why it's so unfortunate the author chose to keep piling on by adding overheated, unevidenced and extreme rhetoric which reasonable people could definitely have some reservations about, or at least want to see evidence for, before agreeing. It's irrelevant, unnecessary and serves only to derail otherwise valid points most people would agree on. Like so much social media rhetoric these days, it's extreme and polarizing, increasing fervor (and rage) among partisans who already agree, triggering rambling rebuttals from oppo partisans and either confusing, distracting or repelling a big chunk of anyone else.
Leading with the factual claim that "the American frontier labs are run by unhinged lunatics who genuinely believe they're creating the Machine God, that they're fine with burning down the world to bring about AGI, and that they believe with religious fervor that they're the only ones smart enough to safely control it." Didn't help.
If you I were neighbors standing outside chatting and I said, "We've both seen how the guy across the street leaves his dog outside all day tied to that short chain. I'm worried it's abusive and am considering reporting it. If you agree, I'd like your support." Assuming it was true and we were both staring at the poor animal, most people would agree. If I added on, "It's because he's a plumber and all plumbers are (fundamentalist Christians / Atheist heathens) who believe (capitalism / communism) is evil and hate dogs. <---- Did asserting all that intent and largely unprovable motivation was the reason behind the dog abuse ... help the dog?
>American frontier labs are run by unhinged lunatics
good faith reading of this would interpret this as meaning Altman, Amodei, and perhaps Musk. The title of the article focuses on OpenAI snd Anthropic directly after all. Also the comment literally refers to those "running" the labs so claiming that the author is also applying that description to "anyone working at the labs" again seems like a bad faith take.
You should come out and state directly which Americam frontier labs deserve a charitable view. Maybe you believe Google and Meta are more deserving, you should state it yif so.
Secondly, the investment ecosystem for American frontier labs results in their CEOs managing obscene amounts of money, and thus power. This concentration of power in a few private individuals seems to have little comparison with the power excercised by the leaders of European and Chinese frontier labs in their own respective countries. If one believes in the dictum, "Absolute power corrupts absolutely," then it is not a stretch to view American frontier labs as being more susceptible to its consequences.
It's the Obama-brown-suit effect. Barack Obama wears a brown suit and everyone loses their minds, because Barack Obama is a reasonable person and we all know that. Donald Trump says the most insane thing uttered by any person in all of human history, and nobody cares, because it's so crazy (and HE'S so crazy) that we all just assume it's a goof.
These AI founders have stated multiple times that they are building world-ending technology that will permanently remove almost all jobs and bring about a Brave New World. That's crazy.
No, they believe enough people are dumb enough to buy their fairy tales and that they could achieve world domination given enough protection. That possibility justifies a lot of BS and pseudo-religion.
But I have two issues:
1. I have fomo from not being face-in the bleeding edge of things.
2. I am ambivalent about z.ai. They are chinese, but they should get my money since they do good for the world. Switching to Mistral for GLM makes my problem inverse since the money doesnt go exactly to to the right place but I trust a french company.
Lets hope the near future resolves our need for personal VRAM and untimately full self hosting. Coding must be free again. This is the way.
OpenRouter lists a few dozen providers for GLM 5.3 and GLM 5.3 Flash. Pick a provider in your country and keep at least some of that money from going to china (not counting all the hardware).
At my own stuff I can do what I want, at work I have no agency and it's not looking like anything is changing soon.
Attestation can get you the idea that the software they're running is what they say they're running, but unless you constantly audit every version, how do you know that it operates exactly how they tell you?
I mean how do you know there's really an enclave there at all :D If your threat model is "this entire provider is untrusted and potentially hostile" it seems like there's lots of options for them to do bad things, that can't be fully solved by secure enclaves.
I think the only thing that matters is control.
Muse is run by Zuckerberg, which is a red flag. Grok is run by a racist.
I guess I could go with the Chinese models - I'm not working on anything dangerous.
Bro come on.
seriously: there's a multitude of reasons not to get hooked on Frontier Labs.
Unclear cost. Will they use your data against you? will their models work consistently over time? Will they be there at all? Their datacenters harm everyone living arround them. Their 'controversial' owners and bosses.
....
Good times.
This is what bothers me most, and it's getting increasingly frustrating. With the recent development of PS4/5 emulation and game porting, it will only get worse (back snatching or lawsuits).