Somebody has to be the brave experimenter that tries the new thing. I'm just glad it was these folk. Since they make no tangible product and contribute nothing to society, they were perhaps the optimal choice to undergo these first catastrophic failed attempts at AI business.
I'm aware that "what does Salesforce actually do?" is a joke but I also really don't know what they do and this article didn't help. They... have conversations with customers? What does the AI do?
weird - even if AI was literally omnipotent and omniscient, you would still be bottlenecked on human's ability to actually evaluate and verify what it is doing and reconciling that with what you wanted it to do. Unless you're of course, willing to YOLO the entire company on output you haven't actually checked yourself.
for that reason alone humans will always need to be in the loop. of course you can debate how many people you need to the above activity, but given that AI isn't omniscient, nor omnipotent I expect that number to be quite high for the foreseeable future.
one example - I've been vibe coding some stuff, and even though a pretty comprehensive set of tests are passing, I still end up reading all of the code. if I'm being honest some of the decisions the AI makes are a bit opaque to me so I end up spending a bunch of time asking it why (of course there's no real ego there, but bare with me...), re-reading the code, thinking about whether that actually makes sense. I personally prefer this activity/mode since the tests pass (which were written by the AI too), and I know anything I manually change can be tested, but it's not something I could just submit to prod right away. this is just a MVP. I can't imagine delegating if real money/customers were on the line without even more scrutiny.
The problem goes deeper: verification is harder than generation. When writing an answer yourself, you build the logic chain from scratch. When verifying AI, you have to deconstruct its logic, cross-reference facts, spot hidden hallucinations, and only then approve. For complex cases (which are exactly what the humans were left with), the time for quality verification approaches the time to write from scratch. If the time becomes roughly equal, the AI stops being an accelerator and becomes just a source of noise that yields no productivity gains
Executive compensation is justified by "...enormous impact leadership decisions have on company outcomes..." yet when those decisions blow up spectacularly, the accountability somehow evaporates.
If your pay is 400 times average employee salary because of your unique strategic vision, surely firing 4000 people based on faulty assumptions should come with proportional consequences?
Or does the high risk, high reward, philosophy only apply to the reward part?
Salesforce is B2B and a complex software. I wouldn’t expected them to layoff that much support. Surprising. They should be empowering their support staff with AI tools to improve customer experiences.
Though I’m a bit surprised they have that much support staff.
What is this site? maarthandam.com? Is it a blog? An AI generated “newspaper”? An internet Newspaper? The menu doesn’t work on mobile, no articles appear to have a by-line, and there’s no link to outside sources to indicate the provenance of these quotes.
It is impossible to verify anything in this article. For example "In recent internal discussions and public remarks". Where are these public remarks? How did this author get access to internal discussions? I rate this article as clickbait nonsense.
For an AI agent to do a good job at customer support, you would need to
1. literally document everything in the product and keep documentation up to date (could be partially automated?)
2. Build good enough search to find those things
3. Be able to troubleshoot / reason / abstract beyond those facts
4. Handle customer information that goes against the assumptions in the core set of facts (ie customers find bugs or don’t understand fundamental concepts about computers)
5. Be prepared to restart the entire conversation when the customer gets frustrated with 1-4 (this is very annoying)
Point 1 (document everything) is the utopia that killed the project. In any complex system, documentation is a lossy compression of reality. The actual truth about how to fix bugs doesn't live in Confluence; it lives in senior heads, Slack chats, and intuition, and AI has no access to this layer of tribal knowledge
> declining service quality, higher complaint volumes, and internal firefighting
LLMs are a great technology for making up plausible looking text. When correctness matters, and you don't have a second system that can reliably check it, the output turns out to be unreliable.
When you're dealing with customer support, everyone involved has already been failed by the regular system. So they're an exception, and they're unhappy. So you really don't want to inflict a second mistake on them.
The counter: the existing system of checks with (presumably) humans was not good enough. For the last 15 months or so, I have been dealing with E.ON claiming one thing and doing another, and had to escalate it to the Ombudsman. I don't think E.ON were using an AI to make these mistakes, I think they just couldn't get customer support people to cope with the idea "the address you have been posting letters to, that address isn't simply wrong, it does not exist". An LLM would have done better, except for what I'm going to say in the counter-counter.
The counter-counter, is that LLMs are only an extra layer of Swiss-cheese: the mistakes they make may be different to human mistakes or may overlap, but they're still definitely present. Specifically, I expect that an LLM would have made two mistakes in my case, one of which is the same mistake the actual humans made (saying they'd fixed everything repeatedly when they had not done so, see meme about LLMs playing the role of HAL in 2001 failing to open the pod bay door) and the other would have been a mistake in my favour (the Ombudsman decided less than I asked for, an LLM would likely have agreed with me more than it should have).
Competent management would have implemented a trial run to evaluate the feasibility of the plan. These sociopaths ensured their own failure by lunging for the prize without realizing they stepped off a cliff.
> “We assumed the technology was further along than it actually was,” one executive said privately, reflecting a growing recognition that AI performance in controlled demonstrations did not translate cleanly into real-world customer environments
And when they can't undo their mistake will they accept the consequences, or will they cry to the government that there are no workers available to do the jobs so national policy must be modified to give Salesforce an even larger firehose of candidates to ignore? Companies complain endlessly that there isn't a huge stable of unicorns for them pick and choose from but those 4000 experienced staff were known good workers and they dumped them anyway to chase fantasies. Salesforce will demand the government fix their mistake for them. The larger the company, the more they expect to never have to pay for their mistakes.
40 comments
[ 2.0 ms ] story [ 51.8 ms ] threadThe root problem is they /estimated/.
> “We assumed the technology was further along than it actually was,” one executive said privately
... and /assumed/.
for that reason alone humans will always need to be in the loop. of course you can debate how many people you need to the above activity, but given that AI isn't omniscient, nor omnipotent I expect that number to be quite high for the foreseeable future.
one example - I've been vibe coding some stuff, and even though a pretty comprehensive set of tests are passing, I still end up reading all of the code. if I'm being honest some of the decisions the AI makes are a bit opaque to me so I end up spending a bunch of time asking it why (of course there's no real ego there, but bare with me...), re-reading the code, thinking about whether that actually makes sense. I personally prefer this activity/mode since the tests pass (which were written by the AI too), and I know anything I manually change can be tested, but it's not something I could just submit to prod right away. this is just a MVP. I can't imagine delegating if real money/customers were on the line without even more scrutiny.
If your pay is 400 times average employee salary because of your unique strategic vision, surely firing 4000 people based on faulty assumptions should come with proportional consequences?
Or does the high risk, high reward, philosophy only apply to the reward part?
Though I’m a bit surprised they have that much support staff.
Salesforce has a vested interest in maintaing its seat based licenses, so it's not in favor of mass layoffs.
Internally Salesforce is pushing AgentForce full stop
1. literally document everything in the product and keep documentation up to date (could be partially automated?)
2. Build good enough search to find those things
3. Be able to troubleshoot / reason / abstract beyond those facts
4. Handle customer information that goes against the assumptions in the core set of facts (ie customers find bugs or don’t understand fundamental concepts about computers)
5. Be prepared to restart the entire conversation when the customer gets frustrated with 1-4 (this is very annoying)
LLMs are a great technology for making up plausible looking text. When correctness matters, and you don't have a second system that can reliably check it, the output turns out to be unreliable.
When you're dealing with customer support, everyone involved has already been failed by the regular system. So they're an exception, and they're unhappy. So you really don't want to inflict a second mistake on them.
The counter: the existing system of checks with (presumably) humans was not good enough. For the last 15 months or so, I have been dealing with E.ON claiming one thing and doing another, and had to escalate it to the Ombudsman. I don't think E.ON were using an AI to make these mistakes, I think they just couldn't get customer support people to cope with the idea "the address you have been posting letters to, that address isn't simply wrong, it does not exist". An LLM would have done better, except for what I'm going to say in the counter-counter.
The counter-counter, is that LLMs are only an extra layer of Swiss-cheese: the mistakes they make may be different to human mistakes or may overlap, but they're still definitely present. Specifically, I expect that an LLM would have made two mistakes in my case, one of which is the same mistake the actual humans made (saying they'd fixed everything repeatedly when they had not done so, see meme about LLMs playing the role of HAL in 2001 failing to open the pod bay door) and the other would have been a mistake in my favour (the Ombudsman decided less than I asked for, an LLM would likely have agreed with me more than it should have).
stop. reading. evals.