I worked on CNNs in Japan during the AI winter and then later on DARPA/CALO (which several years later morphed into Siri.) When I got to Amazon, I begged the powers that be to avoid projects that replace humans and think about projects that augment humans. This was a decade ago and Amazon kept hiring UoW researchers who would come in, over-promise and under-deliver and then go back to academia after they collected sufficient data for their next paper.
I don't claim to be an expert, but you don't have to be an expert to properly evaluate risk.
> One clip on Instagram, which has been viewed over 21.5 million times, shows a man ordering "a large Mountain Dew" and the AI voice continually replying "and what will you drink with that?".
Are they even testing these systems before deploying? With a limited menu and only so many possible permutations... It sounds like they did not do any validation testing or put in safeguards or design it to be robust. I'm like, fairly confident that speech to text + some recorded messages would be able to get you pretty far over a fully AI solution.
FTA - This is after McDonalds cancelled AI in their drive through last year after people getting orders for bacon on ice cream (which can be delicious, tbf) and hundreds of dollars of unwanted nuggets. So... AI isn't even smart enough to run a drive through. Until "common sense" is addressed, AI is going to be more trouble than it's worth for the vast majority of applications. Problem is, we don't know how to implement "common sense", or even define it concretely. That's why all these projections about AGI and superintelligence are so bogus. We are clearly missing at least one, and probably many, algorithmic advancements. And these types of advancements just aren't predictable. Could be 5 years, could be 50.
This sounds strange? If I were to build this system (without really having time to think about it), I'd let the AI "build" the order, which would impose some hard limits - like not accepting 18 000 items. Then I'd have the user confirm it without any AI involvement, so you wouldn't end up with bacon in ice cream. This sounds like they just connected an AI directly to ordering functions and that's it?
It's excruciatingly clear that as impressive as LLMs are, they're still very much an experimental technology. While multinational corporations like Taco Bell should be experimenting with such technologies, they should be experimenting with them in research labs, not shoving them in front of customers and being surprised by the consequent reputational damage.
Based on the problems described in the article, I would guess this system is not using AI in the literal sense. It sounds like a rule based system with bad rules, where AI may possibly be used to navigate the decision tree.
These are definitely skill issues and frankly low skill issues. Just ask ChatGPT 5 thinking to think after the order is finalized whether it is a reasonable order or not and you’ll get rid of 99% of these cases. I don’t know if they’re using ChatGPT API. This is definitely a solvable with current state of AI.
It seems crazy to me to not filter the order through a "reasonableness check", and if it fails that, a human is brought into the transaction.
When I was at Caltech, institute policy was that if you solved an exam problem, and came up with not just a wrong answer but an absurd answer, you would get negative credit rather than a zero.
The way to get just a zero is to annotate with "I know the answer is absurd, but I cannot find the mistake".
The videos where people try to do the 10,000+ drinks are pretty funny but the ones where people are just straight up frustrated their order isn't getting interpreted correctly are also telling [1]. I've also heard of employees intentionally breaking these kiosks or AI things in this way just to make their own job easier because these things messing up all the time are just getting in the way of their burger flipping and making things complicated. I thought they kept beta testing of new flavours to a few locations in Orange County, you think they'd do the same for large software rollouts
I used to regularly go to my local Taco Bell, but stopped going after they rolled this out. Not mad at them or anything, it was just sometimes a frustrating experience, and overall I was not sure how I felt about it: it's more impersonal, I wondered if it meant less jobs available in my local community, etc. So without making a conscious decision, I just stopped going.
I wonder how this has affected sales and net profit at their locations using AI in this way.
I recently watched a YouTube video where some guy tested these AI drive-throughs with ridiculous requests and every time a human operator would intervene. They seemed quite restrictive in their ability to "converse" (which is good IMHO).
FWIW the takeaway from the Taco Bell employee:
He didn't like it. He used to take and process all drive-through orders, now he only handles people with problems.
If you watch the actual video[0], you'll see that it's not that dramatic. Man says "18 thousand water cups", the AI appears to transfer the customer to an employee, who immediately picks up and takes over.
There was never an actual order of 18,000 water cups. The AI did exactly what it was supposed to do in order to prevent malicious abuse of the system.
McDonalds was testing a system like that with one at a location near me. I found it quite useful and good at taking my order. When it messed up there was a backup person to take over and get it right. Normally McDonalds has one person doing two jobs - taking orders and also collecting money and giving change at the first window. This AI was relieving that person of the order-taking job, but they still listened in and would take over if needed. I'm not sure that would ever increase profits, but it definitely reduced the burden on that person working two jobs. It worked well enough IMHO that I was hoping they'd roll it out to more locations, but the canceled it so I guess my experience was not universal ;-)
These issues are often attributed to a bad implementation of AI, but I think the problem is a little more fundamental.
The potential of AI that causes VCs and investors to swap their eyes for dollar signs is the ability to take unstructured, unpredictable inputs and convert them into structured actions or data: in this case a drive through conversation into a specific order. However, the ability to generalize to unseen inputs (what we call common sense) is neural networks glaring weakness. LLMs can look amazingly capable through internal testing, but there is a long and ever increasing tail of unseen interactions when it comes to human conversation.
I’ve seen this play out repeatedly over the last decade in the contact center industry with neural networks as a data scientist in this field.
Seems successful to me… rolled it out to ~6% of their locations and got a ton of useful data, 2M successful orders (according to them), and kinda-funny viral marketing out of the small number of failed orders
36 comments
[ 3.5 ms ] story [ 46.3 ms ] threadThey'll need to create an agent that handles all the edge cases.
I don't claim to be an expert, but you don't have to be an expert to properly evaluate risk.
Lol. This is the perfect capture of exec level AI understanding. Amazing.
Are they even testing these systems before deploying? With a limited menu and only so many possible permutations... It sounds like they did not do any validation testing or put in safeguards or design it to be robust. I'm like, fairly confident that speech to text + some recorded messages would be able to get you pretty far over a fully AI solution.
Although I guess you miss out on a little human-to-human interaction…
Even an “if” statement, or heck, even running the order through an LLM with a prompt “does this look a normal order?” ?.
I’m sure I’m oversimplifying things here, but this specific case looks like it could be easily prevented vs “rethinking” the whole AI initiative.
When I was at Caltech, institute policy was that if you solved an exam problem, and came up with not just a wrong answer but an absurd answer, you would get negative credit rather than a zero.
The way to get just a zero is to annotate with "I know the answer is absurd, but I cannot find the mistake".
[1] https://www.youtube.com/shorts/bsTFEgFAAjY
I wonder how this has affected sales and net profit at their locations using AI in this way.
FWIW the takeaway from the Taco Bell employee:
He didn't like it. He used to take and process all drive-through orders, now he only handles people with problems.
There was never an actual order of 18,000 water cups. The AI did exactly what it was supposed to do in order to prevent malicious abuse of the system.
[0] https://www.youtube.com/watch?v=FDZj6DCWlfc
The potential of AI that causes VCs and investors to swap their eyes for dollar signs is the ability to take unstructured, unpredictable inputs and convert them into structured actions or data: in this case a drive through conversation into a specific order. However, the ability to generalize to unseen inputs (what we call common sense) is neural networks glaring weakness. LLMs can look amazingly capable through internal testing, but there is a long and ever increasing tail of unseen interactions when it comes to human conversation.
I’ve seen this play out repeatedly over the last decade in the contact center industry with neural networks as a data scientist in this field.