HN Explorer

Tell HN: Twilio support replies with hallucinated features

159 points by haute_cuisine ↗ HN
I was investigating some bug with our voice system and asked support where I can find some debugging information and event logs.

They told me where I should go in the interface to see that and reassured that they checked logs and this event exist.

It turned out these features and information doesn't exist anywhere in the interface and impossible to retrieve in any way. The support message with hallucinated features is mostly AI written.

CEOs tell us AGI is around the corner but in reality it just unreliable information and AI can't even restock the vending machine.

17 comments

[ 3.2 ms ] story [ 42.5 ms ] thread
quinnjh
These tools are perfect for deployment where providing plausible-but-incorrect info is aligned with business outcomes, like cutting your support staff and giving disgruntled customers fake information.

I’ve seen most of the frontier models hallucinate their capabilities, not surprising they might do so for api completions regarding a product they barely know about.

Unless they lose more money from cancelled subscriptions than they saved on cutting support staff, it’s probably the new normal.

trollbridge
Twilio registered my business name as “My Twilio Account” and is unable to change it. My application for 10DLC also got rejected since I wanted to do something other than send marketing messages with it and I can’t figure out how to describe an opt-in only service that is strictly for employees, to their provided phone number, with a signature opting in to get payroll information texted to them.

As a test, I set up something to send junk quality marketing texts. Was approved.

collingreen
I had this same experience. Twilio is not interested in the small internal tool customer - do mass marketing or go away.
barbazoo
None of them know what they're doing. Even Google's own AI integrated into their own apps, hallucinates about those very apps, e.g. asking Gemini in Docs about how to do something in Docs. It's laughable. LLM have great utility but this is not it.
gdulli
There used to be a contract that a business had something to lose by providing bad service, that customers would leave and seek better service elsewhere.

I believe the most important and least discussed phenomenon of modern consumer culture is that consumers have passed a threshold of passive and docile behavior such that businesses no longer fear losing customers. Partly because the customers have shown willingness to eat shit, partly because there's a new understanding that all businesses will adopt the same customer-hostile behaviors (AI customer service in this case) so consumers don't have significant choice anyway.

lab14
"Hallucination machine, responds with hallucinations".

But seriously, entreprise customers (and any big spender account) usually get access to a dedicated (human) account rep and private support channels in Slack, so they never really interact with this.

gooseWithFood20
This happened to me with a chat it for Chevrolet lmao. Started telling me about a car that DID NOT EXIST
elicash
What distinguishes AI slop customer support from the previous enshittification of customer service is that previously if you wanted to avoid the garbage chat support you could get on the phone and -- even if you had to go through a phone tree -- you could at least eventually ask a person about the problem.

But now, even if it's possible to get a person on the phone, THAT PERSON is just doing the AI chatbot on their end. By talking to a human, you're just adding a middleman who is accessing the same incorrect chatbot that's available to you.

cacozen
The vending machine mention is about this paper from Anthropic: https://www.anthropic.com/research/project-vend-1

The gist is: Claude AI successfully ran a shop by itself! - Actually a vending machine - Actually a mini-fridge in our office - Actually it gave lots of discounts and free products on our slack - Actually it hallucinated a Venmo account and people sent payments to God-knows-who

jqpabc123
Current AI is probabilistic, not deterministic.

This means it can and most probably will be dead wrong at some point.

So before integrating AI into your workflow, you should ask yourself, "Do you feel lucky?".

ilamparithi
I searched on Google to check if banks were open on a certain day. The AI response on top said they were closed because it was a second Saturday, but it was actually a Wednesday.
aiiizzz
Twilio CEO tells you agi around the corner? Doubt
taf2
Fun - I always test the ai support by asking it for a really good sc2 Zerg rush build - as I recall Twilio gave me a pretty good build order