In my opinion, the problem is not even the cost. The problem is that people are using AI for running recurrent stuff instead of writing code to automate it.
For example. Imagine that you are comparing two documents (let's assume diff doesn't exist). You could ask an AI to compare the differences from you or you could use AI to write a tool to do it. For whatever reason, people are starting to go with the former not realizing that now they basically have to pay to compare documents.
On the one hand, organizations are without question using LLM's well beyond what is actually necessary, and as reality kicks in they're forced to scale back accordingly. However at the same time, on intervals counted in months, we're seeing breakthroughs both in hardware and software that dramatically reduce the cost of inference.
Between corporate FOMO and the rapidly decreasing costs of actually running LLM's I'm interested to see at which side of the spectrum these two meet
AI is overhyped. I have yet to see an end user product that in itself isnt a wrapper around LLMs that is impressive created by LLM assistance. I have also yet to see dramatic increases of revenue of companies using LLMs that don't involve selling things in its supply chain. Is it a nice affordance? Sure. 1T capex good? No.
If it was so good I would expect to see 2005-2015 advancements yearly.
Meanwhile China is blowing past the world with real improvements in the real world- solar, EVs, etc. meanwhile people keep making their fancy sans serif websites about todo apps, faster than ever before. Useless.
Another reason to favor using AI to build automation instead of relying on it in prod: the risk of war and global instability.
If LLMs are genuinely helpful or even decisive in a military engagement, you can expect any host country to commandeer whatever data centers they need, leaving commercial entities to bid up the prices on the leftover capacity.
Another risk is that data centers are a great target for cyber warfare.
It’s ideal if your business can leverage LLMs when they’re online but continue to operate profitably when they’re offline.
There's a paywall, but it's an interesting question how much of the recent explosion of the AI companies revenues is because of the explosion in prices, and how much their customers will accept the increased prices.
The abrupt swing in many non-technology company IT departments from "hey developer, you aren't using enough tokens" to this is just too funny.
And I'm seeing almost no self-awareness from leaders. They are making decisions about things that they just don't understand. And are completely unworried about it. Just blindly following whatever the news cycle is about AI.
The worst part is that techies can still work around the insanity if they keep their opinions private. For the serious average Joe the AI mandates must be feeling like hell on earth.
I once worked in a company that had soviet-level efforts to push LLMs into everything, someone eventually made the classic "Natural Language -> SQL Query -> Magic Result in webpage" and got promoted, the tool got mandated for every non-tech employee as part of an AI-boosting effort (people pushing metrics up).
One day I wake up with a product person in despair because the tool couldn't handle what looked like a very simple aggregation, I stopped what I was doing, crafted a 30-line SQL query over HORRIBLE TABLES, a couple CTEs and window functions here and there got him what he wanted. I found out later that single query that took 30 minutes to make saved him from inheriting a 6-month effort to create a microservice dedicated to patching said tool.
Would have been nice to see 'soaring costs' with numbers. WSJ could do better here. Hundreds of thousands of dollars a month is nothing compared to how much they take with better financial models.
There's an old saying, "in the land of the blind, the one-eyed man is king."
Here we have the opposite: In the land of the one-eyed, the blind are leading.
The blind in this case are all those executives and managers who don't understand much about AI's current potential and limitations, and so far have treated it like a magic button that will solve everything. The one-eyed are rank-and-file employees who maybe sort of know a little more about AI.
The other day we (wrongly) concluded that product market fit has been achieved and now the rivers of hot molten milk chocolate and honey are all that's in the future etc.
I’ve seen comments on other threads on this subject the general idea that these article headlines are overstating the pullback from AI.
In other words, the news cycle is looking for an AI story that lands with readers, and that the example
of Uber blowing through its AI budget and Microsoft discontinuing use of Claude internally are not good indicators.
I agree that those aren’t good indicators.
However, at some point we have to remember that CEOs and boards of directors are just regular morons who read the news the same way everyone else does.
At some point, if a lot of corporate leaders associate AI with mediocre results, high costs, and public backlash, they might just start saying “this juice isn’t worth the squeeze.”
Only thing I can say AI was useful for, in a corporate environment, was learning a new coding language on the fly. Gives me a baseline to work off of and fix.
But I can learn without it, too. A nice tool, but not a need.
The cost is a problem, but IMHO more important is delegating so much of your internal knowledge, thinking, and systems to a 3rd party.
We are very close to the point where if Claude and ChatGPT APIs are down, companies cannot function. How is that introduced so quickly into so many critical places without taking that specific fact in consideration? What is the plan for all those companies whose workflows now depend heavily on a remote LLM whenever the services get cut? What if your company account gets banned?
In some ways it is worth than depending on a company for hosting, because even your debugging tools are based on AI. MCP is great to go through datadog, sentry, until your agent or the MCP server are down and you don't know how to look for the issue yourself because you do not actually understand how your systems work.
> We are very close to the point where if Claude and ChatGPT APIs are down, companies cannot function.
Contrast with Gmail/Gsuite/Outlook365/QuickbooksOnline/etc are down, though.
What you cite here isn't a direct attack on AI but on centralized service provision in general. Unfortunately that battle has been lost for decades, now.
They are likely also starting to realize that the end result of their anthropic contract is that nobody but anthropic knows how to run their business. Why would anthropic not treat their business like a utility in the future?
90%+ of corporate people are not programmers. 1 programmers can cause the same token damage with a bunch of concurrent agents as a couple thousand Karens in compliance asking a chatbot questions
It's much easier to deliver incremental AI ROI on the later even if it's hard to measure/quantify. A 1000 tokens might point this compliance person in the right direction on a key problem. Meanwhile 1000 tokens doesn't get you anything useful on coding
This phase that all these companies went through doesn’t seem that bad. Before these places had a big problem where all their employees didn’t understand how to us ai for their work. Now they’ve overspent and tokenmaxxed and haven’t seen much from it. The next phase is to set the goalpost lower and set quotas based on who uses ai more effectively. Eventually the folks that use it well and are productive will bring in roi. Then you can fire all the folks that aren’t using it effectively and replace them with people that know how to use it. We’re already starting to see this.
35 comments
[ 3.8 ms ] story [ 55.3 ms ] threadFor example. Imagine that you are comparing two documents (let's assume diff doesn't exist). You could ask an AI to compare the differences from you or you could use AI to write a tool to do it. For whatever reason, people are starting to go with the former not realizing that now they basically have to pay to compare documents.
This is why we have business analysts and software developers.
To help identify inefficiencies and to build technical solutions.
Between corporate FOMO and the rapidly decreasing costs of actually running LLM's I'm interested to see at which side of the spectrum these two meet
If it was so good I would expect to see 2005-2015 advancements yearly.
Meanwhile China is blowing past the world with real improvements in the real world- solar, EVs, etc. meanwhile people keep making their fancy sans serif websites about todo apps, faster than ever before. Useless.
If LLMs are genuinely helpful or even decisive in a military engagement, you can expect any host country to commandeer whatever data centers they need, leaving commercial entities to bid up the prices on the leftover capacity.
Another risk is that data centers are a great target for cyber warfare.
It’s ideal if your business can leverage LLMs when they’re online but continue to operate profitably when they’re offline.
https://news.ycombinator.com/item?id=48268871
https://news.ycombinator.com/item?id=48238896
https://news.ycombinator.com/item?id=48307098
And I'm seeing almost no self-awareness from leaders. They are making decisions about things that they just don't understand. And are completely unworried about it. Just blindly following whatever the news cycle is about AI.
I once worked in a company that had soviet-level efforts to push LLMs into everything, someone eventually made the classic "Natural Language -> SQL Query -> Magic Result in webpage" and got promoted, the tool got mandated for every non-tech employee as part of an AI-boosting effort (people pushing metrics up).
One day I wake up with a product person in despair because the tool couldn't handle what looked like a very simple aggregation, I stopped what I was doing, crafted a 30-line SQL query over HORRIBLE TABLES, a couple CTEs and window functions here and there got him what he wanted. I found out later that single query that took 30 minutes to make saved him from inheriting a 6-month effort to create a microservice dedicated to patching said tool.
Here we have the opposite: In the land of the one-eyed, the blind are leading.
The blind in this case are all those executives and managers who don't understand much about AI's current potential and limitations, and so far have treated it like a magic button that will solve everything. The one-eyed are rank-and-file employees who maybe sort of know a little more about AI.
In other words, the news cycle is looking for an AI story that lands with readers, and that the example of Uber blowing through its AI budget and Microsoft discontinuing use of Claude internally are not good indicators.
I agree that those aren’t good indicators.
However, at some point we have to remember that CEOs and boards of directors are just regular morons who read the news the same way everyone else does.
At some point, if a lot of corporate leaders associate AI with mediocre results, high costs, and public backlash, they might just start saying “this juice isn’t worth the squeeze.”
Only thing I can say AI was useful for, in a corporate environment, was learning a new coding language on the fly. Gives me a baseline to work off of and fix.
But I can learn without it, too. A nice tool, but not a need.
We are very close to the point where if Claude and ChatGPT APIs are down, companies cannot function. How is that introduced so quickly into so many critical places without taking that specific fact in consideration? What is the plan for all those companies whose workflows now depend heavily on a remote LLM whenever the services get cut? What if your company account gets banned?
In some ways it is worth than depending on a company for hosting, because even your debugging tools are based on AI. MCP is great to go through datadog, sentry, until your agent or the MCP server are down and you don't know how to look for the issue yourself because you do not actually understand how your systems work.
Contrast with Gmail/Gsuite/Outlook365/QuickbooksOnline/etc are down, though.
What you cite here isn't a direct attack on AI but on centralized service provision in general. Unfortunately that battle has been lost for decades, now.
90%+ of corporate people are not programmers. 1 programmers can cause the same token damage with a bunch of concurrent agents as a couple thousand Karens in compliance asking a chatbot questions
It's much easier to deliver incremental AI ROI on the later even if it's hard to measure/quantify. A 1000 tokens might point this compliance person in the right direction on a key problem. Meanwhile 1000 tokens doesn't get you anything useful on coding
- ChatGPT drops, AI is perfect to be our savior.
- AI glorified as the great messiah.
- Everyone worships stocks even remotely related to AI.
- Execs desperate for relevance boast about tokenmaxxing.
- SHTF
- burst
- last year flagship GPUs and DRAM are sold used for the price of a burger.
- Laidoff people start using local AI as hardware price drops to make actual useful stuff
- New round of bootstrapped tech bros that eventually give birth to the new metaverse/NFT/etc.. hype.