> "This is purely an observation: You only jump ship in the middle of a conquest if either all ships are arriving at the same time (unlikely) or neither is arriving at all. This means that no AI lab is close to AGI."
The central claim here is illogical.
The way I see it, if you believe that AGI is imminent, and if your personal efforts are not entirely crucial to bringing AGI about (just about all engineers are in this category), and if you believe that AGI will obviate most forms of computer-related work, your best move is to do whatever is most profitable in the near-term.
If you make $500k/year, and Meta is offering you $10M/year, then you ought to take the new job. Hoard money, true believer. Then, when AGI hits, you'll be in a better personal position.
Essentially, the author's core assumption is that working for a lower salary at a company that may develop AGI is preferable to working for a much higher salary at a company that may develop AGI. I don't see how that makes any sense.
Also, AGI is not just around the corner. We need artificial comprehension for that, and we don't even have a theory how comprehension works. Comprehension is the fusing of separate elements into new functional wholes, dynamically abstracting observations, evaluating them for plausibility, and reconstituting the whole - and all instantaneously, for security purposes, of every sense constantly. We have no technology that approaches that.
Observe what the AI companies are doing, not what they are saying. If they would expect to achieve AGI soon, their behaviour would be completely different. Why bother developing chatbots or doing sales, when you will be operating AGI in a few short years? Surely, all resources should go towards that goal, as it is supposed to usher the humanity into a new prosperous age (somehow).
If we just forget about AGI for a second, and looked at AI Wars as something in the vein of Search Wars Redux, wouldn’t that explain a lot of the strategy?
Granted, history doesn’t exactly repeat itself, it rhymes…
Are we finally realizing that the term "AGI" is not only hijacked to become meaningless, but achieving it has always been nothing but a complete scam as I was saying before? [0]
If you were in a "pioneering" AI lab that claims to be in the lead in achieving "AGI", why move to another lab that is behind other than offering $10M a year.
I never trusted them from the start. I remember the hype that came out of Sun when J2EE/EJBs appeared. Their hype documents said the future of programming was buying EJBs from vendors and wiring them together. AI is of course a much bigger hype machine with massive investments that need to be justified somehow. AI is a useful tool (sometimes) but not a revolution. ML is much more useful a tool. AGI is a pipe dream fantasy pushed to make it seem like AI will change everything, as if AI is like the discovery that making fire was.
Im reading the "AI"-industry as a totally different bet- not so much, as a "AGI" is coming bet of many companies, but a "climate change collapse" is coming and we want to continue to be in business, even if our workers stay at home/flee or die, the infrastructure partially collapses and our central office burns to the ground-bet. In that regard, even the "AI" we have today, makes total sense as a insurance policy.
> Right before “making tons of money to redistribute to all of humanity through AGI,” there’s another step, which is making tons of money.
I've got some bad news for the author if they think AGI will be used to benefit all of humanity instead of the handful of billionaires that will control it.
Honestly this article sounds like someone is unhappy that AI isn’t being deployed/developed “the way I feel it should be done”.
Talent changing companies is bad. Companies making money to pay for the next training run is bad. Consumers getting products they want is bad.
In the author’s view, AI should be advanced in a research lab by altruistic researchers and given directly to other altruistic researchers to advance humanity. It definitely shouldn’t be used by us common folk for fun and personal productivity.
> This is purely an observation: You only jump ship in the middle of a conquest if either all ships are arriving at the same time (unlikely) or neither is arriving at all. This means that no AI lab is close to AGI. Their stated AGI timelines are “at the latest, in a few years,” but their revealed timelines are “it’ll happen at some indefinite time in the future.”
This makes no sense to me at all. Is it a war metaphor? A race? Why is there no reason to jump ship? Doesn't it make sense to try to get on the fastest ship? Doesn't it make sense to diversify your stock portfolio if you have doubts?
I keep seeing this charge that AI companies have an “Uber problem” meaning the business is heavily subsidized by VC. Is there any analysis that has been done that explains how this breaks down (training vs inference and what current pricing is)? At least with Uber you had a cab fare as a benchmark. But what should, for example, ChatGPT actually cost me per month without the VC subsidy? How far off are we?
Point 1. could just as easily be explained by all of the labs being very close, and wanting to jump ship to one that is closer, or that gives you a better deal.
> This reminds me of a paradox: The AI industry is concerned with the alignment problem (how to make a super smart AI adhere to human values and goals) while failing to align between and within organizations and with the broader world. The bar they’ve set for themselves is simply too high for the performance they’re putting out.
> The AI industry oscillates between fear-mongering and utopianism. In that dichotomy is hidden a subtle manipulation. […] They don’t realize that panic doesn’t prepare society but paralyzes it instead, or that optimism doesn’t reassure people but feels like gaslighting. Worst of all, both messages serve the same function: to justify accelerating AI deployment—either for safety reasons or for capability reasons
This is a great point and also something I’ve become a bit cynical about these last couple of months. I think the very extreme and “bipolar” messaging around AI might be a bit more dishonest than I originally (perhaps naively?) though.
>If they truly believed we’re at most five years from world-transforming AI, they wouldn’t be switching jobs, no matter how large the pay bump (they’re already affluent).
What ridiculous logic is this? TO base the entire premise that AGI is not imminent based on job switching? How about basing it on something more concrete.
How do people come up with such shakey foundations to support their conclusions? It's obvious. They come up with the conclusion first then they find whatever they can to support it. Unfortunately if dubious logic is all that's available then that's what they will say.
The primary use case for AI-in-the-box is a superhuman CEO that sees everything and makes no mistakes. As an investor you can be sure that your money are multiplying at the highest rate possible. However as a self-serving investor you also want your CEO to side-step any laws and ethics that stand in your way, unless ignoring those laws will bring more trouble than profit. All that while maintaining a facade of selfless philanthropist for the public. For a reasonable price, your AI CEO will be fine-tuned to serve your goals perfectly.
Remember that fine-tuning a well-behaved AI to do something as simple as writing malware in C++ makes widespread changes in the AI and turns it into a monstrosity. There was an HN post about this recently: fine-tuning an aligned model produces broadly misaligned results. So what do you think will happen when our AI CEO gets fine-tuned to prioritize shareholder interests over public interests?
My question is this - once you achieve AGI, what moat do you have, purely on the scientific part? Other than making the AGI even more intelligent.
I see a lot of talk that the first company that achieves AGI, will also achieve market dominance. All other players will crumble. But surely when someone achieves AGI, their competitors will in all likelihood be following closely after. And once those achieve AGI, academia will follow.
Point is, at some point AGI itself will become available the everyone. The only things that will be out of reach for most, is compute - and probably other expensive things on the infrastructure part.
Current AI funding seems to revolve around some sort of winner-take-all scenario. Just keep throwing incredible amounts of money at it, and hope that you've picked the winner. I'm just wondering what the outcome will be if this thesis turns out wrong.
34 comments
[ 3.6 ms ] story [ 50.2 ms ] threadThe central claim here is illogical.
The way I see it, if you believe that AGI is imminent, and if your personal efforts are not entirely crucial to bringing AGI about (just about all engineers are in this category), and if you believe that AGI will obviate most forms of computer-related work, your best move is to do whatever is most profitable in the near-term.
If you make $500k/year, and Meta is offering you $10M/year, then you ought to take the new job. Hoard money, true believer. Then, when AGI hits, you'll be in a better personal position.
Essentially, the author's core assumption is that working for a lower salary at a company that may develop AGI is preferable to working for a much higher salary at a company that may develop AGI. I don't see how that makes any sense.
Granted, history doesn’t exactly repeat itself, it rhymes…
If you were in a "pioneering" AI lab that claims to be in the lead in achieving "AGI", why move to another lab that is behind other than offering $10M a year.
Snap out of the "AGI" BS.
[0] https://news.ycombinator.com/item?id=37438154
What I can't figure out is why this author thinks it's good if these companies do invent a real AGI...
I've got some bad news for the author if they think AGI will be used to benefit all of humanity instead of the handful of billionaires that will control it.
So far I have only seen it been thrown around to create hype.
Talent changing companies is bad. Companies making money to pay for the next training run is bad. Consumers getting products they want is bad.
In the author’s view, AI should be advanced in a research lab by altruistic researchers and given directly to other altruistic researchers to advance humanity. It definitely shouldn’t be used by us common folk for fun and personal productivity.
This makes no sense to me at all. Is it a war metaphor? A race? Why is there no reason to jump ship? Doesn't it make sense to try to get on the fastest ship? Doesn't it make sense to diversify your stock portfolio if you have doubts?
My argument is that it’s our job as consumers to align the AIs to our values (which are not all the same) via selection pressure: https://muldoon.cloud/2025/05/22/alignment.html
This is a great point and also something I’ve become a bit cynical about these last couple of months. I think the very extreme and “bipolar” messaging around AI might be a bit more dishonest than I originally (perhaps naively?) though.
What ridiculous logic is this? TO base the entire premise that AGI is not imminent based on job switching? How about basing it on something more concrete.
How do people come up with such shakey foundations to support their conclusions? It's obvious. They come up with the conclusion first then they find whatever they can to support it. Unfortunately if dubious logic is all that's available then that's what they will say.
Remember that fine-tuning a well-behaved AI to do something as simple as writing malware in C++ makes widespread changes in the AI and turns it into a monstrosity. There was an HN post about this recently: fine-tuning an aligned model produces broadly misaligned results. So what do you think will happen when our AI CEO gets fine-tuned to prioritize shareholder interests over public interests?
I see a lot of talk that the first company that achieves AGI, will also achieve market dominance. All other players will crumble. But surely when someone achieves AGI, their competitors will in all likelihood be following closely after. And once those achieve AGI, academia will follow.
Point is, at some point AGI itself will become available the everyone. The only things that will be out of reach for most, is compute - and probably other expensive things on the infrastructure part.
Current AI funding seems to revolve around some sort of winner-take-all scenario. Just keep throwing incredible amounts of money at it, and hope that you've picked the winner. I'm just wondering what the outcome will be if this thesis turns out wrong.