Voice agents in customer support is an extremely crowded market. Seems like Sierra is taking a considerable lead.
I don't know much about their product offerings, but I was doing some speech-to-text work and came across https://research.sierra.ai/mubench/ for comparing current models. It felt fairly thoughtful, particularly in regards to coming up with better benchmarking metrics than word error rate.
If you (like me) are hearing about this for the first time, Bret Taylor is the co-founder.
> Bret is Co-Founder of Sierra. Most recently, he served as Co-CEO of Salesforce. Prior to Salesforce, Bret founded Quip and was CTO of Facebook. He started his career at Google, where he co-created Google Maps. Bret serves on the board of OpenAI.
I think this is generally a good product because businesses that previously had zero phone support can now afford to have something. However, the hard work of actually building out the various workflows and decision trees is not automatic. Previously, a call center employee would receive abuse from a caller for being unempowered to make a decision. Instead, an LLM will perform the same role.
Ideally, businesses will escalate to an empowered human for all undefined parts of the flowchart. In practice, I truly hope it will be better than the current pre-recorded phone tree system that leads to a human following a script.
I personally only call support because a fix is not available through an organization's website.
It’s interesting that the example interaction they use on their homepage is a no-friction example that can be handled without an AI chatbot. Why not something more complex that properly demonstrates the value?
I remember waiting for uber next to him in SF one night 10+ years ago. This dude must be the son of some mafia boss or some shit and have some crazy blackmail to raise billions for companies that are copies of products where he’s the 12th company doing the same thing.. never turning a profit or anything and yet raising ever more money. doesn’t make sense otherwise
It's always interesting seeing how HN reacts to AI CX (as someone who works in this space). Yes, the tech savvy crowd loves to say how they always ask for a human and love old school phone trees
in reality 50-80% of callers come in with easily answerable questions because they don't know how to nav the website and prefer to ask in natural language
The vast majority of callers call in to resolve their issue, and most don't care if they are speaking to a bot because they just want their issue fixed. Agents (if implemented well) are an order of magnitude more effective at resolving issues compared to a call centre worker who is reading off a script and churn within 9 months
There's also the 2nd order effs of making CX cheap. before, there is the perverse incentive of companies trying to keep you off support because each call costs them way more than the value they get. if your cost per call drops 100x you can invest in turning a cost centre into a revenue driver (+ a better experience)
Making me talk to a fucking robot leaves me with a deep and abiding hatred for your company. I will prefer almost any alternative to doing business with you and hope fervently to read about your bankruptcy.
What percentage of interactions having this result will cancel out your cost savings?
I think this is going to be wonderful. I'll have my offensive AI call their support AI and prompt-inject my way to a rebate tier that nobody knew existed and nobody can cancel it because all the remaining humans have been reduced to phone-to-screen input machinery.
right? its practically begging for it to be tried -_-. i wonder if someone somewhere will turn a sim farm on such companies to try and mass inject them to do weird shit or say nasty things to other customers etc. - ofc youd hope its set up in a way u cant, but then again we learn yesterday all ur stored passwords in edge are in plaintext mem... i would not be surprised if some of these companies get totally crapped on by some adversaries or malicious parties.
Tbf I've thought a decent bit about how most current AI is essentially just being used to digest what exists on a website/etc. Honestly even just the vector search/RAG part is useful, but more-so with a model to help do some initial filtering of it.
It's an odd use case - we have used language for a millionish years or so and it makes sense that that's the easiest way for us to get at information/do things.
But at the same time it's faster for me to read than listen, but it's often slower to type than to speak. It's faster to hit one button in a familiar place to do some predetermined thing, but much slower when the location of that button changes/gets hidden under submenus/I'm not familiar with an app or website.
On Android I constantly use the search function of the settings menu and I feel like this will be the golden UX going forward - a side by side UX + NL interface. So I can ask "how do I add a photo" and from there I get taken to the right place and can continue to add multiple photos in one go following the same pattern.
Though I suppose the nicer alternative is just "add all the photos I took near the waterfall from today".
Cant wait to spend my day arguing with a phone support clanker about why my medical insurance claim was rejected by the medical claims clanker, only to get forwarded to the 2nd tier "patient advocate" clanker who's really just the medical claims clanker in disguise.
The future is extremely dystopian and sad right now. The corporations are not going to use this the way you think they are. They are going to use it to maximize their profits, not help their customers.
There has never ever been a case that AI has resolved my query, except the simple decision tree for things like refund. Have you got any positive experience with gen AI in any of the site?
1. There are already apps/websites as an alternative for CSAs. Most of the time I have to call someone its because I couldn't do what I wanted through those portals, so adding an AI agent to the chain is unlikely to prevent an immediate escalation to a human.
2. How much money are you really going to save this way? CSAs aren't high salary employees. Sure you might need a bunch of them, but we've already seen that brand loyalty erodes quickly when you remove the human touch. United/Spirit airlines offer opposing views on the cut your way to profitability perspective.
3. "Pay only for good outcomes" isn't going to last.
4. Are agents good enough to even do this? Yes, the cherrypicked examples sound good, but... I just know how well coding agents really work and my only experiences with voice agents in the wild have been very poor so far.
I can't speak to the business itself but they recently published a refreshing take on improving the product engineering interview experience in the age of AI
https://sierra.ai/blog/the-ai-native-interview
Well worth a read even if you are generally anti-AI.
I’ve heard much about Sierra but haven’t ever tried their product. What do I need to pretend to buy and then complain about to get on a call with their agents?
I supervised a Sierra rollout a while back. Their performance was impressive and the price was great. I suspect both will not be true in time.
Their implementation is rather cumbersome, requiring implementation fees and AI configuration that is rather bespoke to Sierra. Anyone rolling off of Sierra will find there is nothing they can take with them.
In general, I think CX ought to disappear as a vertical in an AI world. If I'm talking to a product AI and need support, why should I switch to another AI to do that? Even if that second AI is invoked by the first as a tool, how much am I gaining?
Interestingly, the first and best implementers of AI support so far have been at companies that roll their own.
There is nothing unique to CX about AI, as far as I can tell. Sierra is still just the same AI infra people are putting in products. Granted, you can make good money positioning yourself this way, but I expect on some time horizon they will need to reposition.
Maybe its just me, but $950M seems a lot of money to invest in a "company".
Had to check my assumptions though so I looked up what the lower end of GDP for a country is and sure enough they have American Samoa, Dominica, and Tonga beat. Now that money is probably meant to last 16 months so its not quite apples to apples but kind of wild regardless.
People are rightly concerned that AI chatbots could result in worse customer experience, so I'd like to share my anecdote.
I recently had the best customer support experience I've ever had, and it was an AI chatbot, helping my in ways I would not expect from a first-line support human.
Recently I opened the chatbot window on the site of a supplier of electric vehicle chargers, and asked some obscure things about how I could access and modify the charger in a non-standard way. I explained I was doing some R&D testing of a novel way of using the chargers, and I would need to temporarily reconfigure the public unit we had already installed in an unusual, non-standard way, rewiring its network to intercept traffic via transparent proxy, make it behave a little differently than it normally does, and that I didn't have the credentials.
I was expecting the first-line person or bot on the chat to be unable to handle my non-standard request. What I was hoping for was this would start the process of getting me to someone who could help. I expected to take a few days or weeks to reach someone appropriate in the company, perhaps a sympathetic technician who could understand what I was trying to do.
To my amazement, in the first response it one-shotted a nearly complete answer to every part of my question, including the detailed and implied parts I'd left out for a later interaction. It figured out what I was doing and what that needed, told me exactly what to do and why, and showed me that it understood every aspect of my request. There were things in this I doubt anyone ever asked about before.
Because the answer so complete I didn't really need more. I asked it some things, e.g. about electrical safety, just to confirm my understanding of the one-shotted answer. In every reply, the chatbot was insightful, informative, helpful and correct, and I didn't need to explain things twice, or wait on hold, or be transferred to anyone else.
I don't know what service they were using, but whoever implemented it, hats off to them for using good technology and providing it with high quality data about the products they are supporting.
Honestly, if AI customer support calls or chats can be made that good consistently, that sounds great.
(That was a stark contrast to my awful expereinces with support at banks, where many humans made mistake after mistake, contradicted each other, and seemed to struggle to understand simple things. Particularly the one where a bank required me to fill out a form with basic company info, then it took 14 hours of phone calls and hold time until I got to the one person who understood straight away that the entries on the form were correct, had to be that way, and they could tick the "it's done" box. My execellent AI bot experience suggests a good one will always understand things like that on the first iteration, as if they are the best-trained and best-informed humans available. But other, worse AI bot experiences suggests the 14 hour calls would become infinitely long, never able to resolve the problem, leading to things like unnecessary account closures and locked funds needing a court to resolve.)
31 comments
[ 2.7 ms ] story [ 47.4 ms ] thread> Ensure you only pay for the value Sierra delivers with outcome-based pricing.
Yeah... that won't last.
I don't know much about their product offerings, but I was doing some speech-to-text work and came across https://research.sierra.ai/mubench/ for comparing current models. It felt fairly thoughtful, particularly in regards to coming up with better benchmarking metrics than word error rate.
> Bret is Co-Founder of Sierra. Most recently, he served as Co-CEO of Salesforce. Prior to Salesforce, Bret founded Quip and was CTO of Facebook. He started his career at Google, where he co-created Google Maps. Bret serves on the board of OpenAI.
Ideally, businesses will escalate to an empowered human for all undefined parts of the flowchart. In practice, I truly hope it will be better than the current pre-recorded phone tree system that leads to a human following a script.
I personally only call support because a fix is not available through an organization's website.
in reality 50-80% of callers come in with easily answerable questions because they don't know how to nav the website and prefer to ask in natural language
The vast majority of callers call in to resolve their issue, and most don't care if they are speaking to a bot because they just want their issue fixed. Agents (if implemented well) are an order of magnitude more effective at resolving issues compared to a call centre worker who is reading off a script and churn within 9 months
There's also the 2nd order effs of making CX cheap. before, there is the perverse incentive of companies trying to keep you off support because each call costs them way more than the value they get. if your cost per call drops 100x you can invest in turning a cost centre into a revenue driver (+ a better experience)
What percentage of interactions having this result will cancel out your cost savings?
It's an odd use case - we have used language for a millionish years or so and it makes sense that that's the easiest way for us to get at information/do things.
But at the same time it's faster for me to read than listen, but it's often slower to type than to speak. It's faster to hit one button in a familiar place to do some predetermined thing, but much slower when the location of that button changes/gets hidden under submenus/I'm not familiar with an app or website.
On Android I constantly use the search function of the settings menu and I feel like this will be the golden UX going forward - a side by side UX + NL interface. So I can ask "how do I add a photo" and from there I get taken to the right place and can continue to add multiple photos in one go following the same pattern.
Though I suppose the nicer alternative is just "add all the photos I took near the waterfall from today".
The future is extremely dystopian and sad right now. The corporations are not going to use this the way you think they are. They are going to use it to maximize their profits, not help their customers.
1. There are already apps/websites as an alternative for CSAs. Most of the time I have to call someone its because I couldn't do what I wanted through those portals, so adding an AI agent to the chain is unlikely to prevent an immediate escalation to a human.
2. How much money are you really going to save this way? CSAs aren't high salary employees. Sure you might need a bunch of them, but we've already seen that brand loyalty erodes quickly when you remove the human touch. United/Spirit airlines offer opposing views on the cut your way to profitability perspective.
3. "Pay only for good outcomes" isn't going to last.
4. Are agents good enough to even do this? Yes, the cherrypicked examples sound good, but... I just know how well coding agents really work and my only experiences with voice agents in the wild have been very poor so far.
As an aside, my favorite Sierra Entertainment logo version is probably the 1983-1993 version [1]. I think the design still holds up even today.
[0] https://velvetshark.com/ai-company-logos-that-look-like-butt...
[1] https://logos-world.net/sierra-entertainment-logo/
Well worth a read even if you are generally anti-AI.
Their implementation is rather cumbersome, requiring implementation fees and AI configuration that is rather bespoke to Sierra. Anyone rolling off of Sierra will find there is nothing they can take with them.
In general, I think CX ought to disappear as a vertical in an AI world. If I'm talking to a product AI and need support, why should I switch to another AI to do that? Even if that second AI is invoked by the first as a tool, how much am I gaining?
Interestingly, the first and best implementers of AI support so far have been at companies that roll their own.
There is nothing unique to CX about AI, as far as I can tell. Sierra is still just the same AI infra people are putting in products. Granted, you can make good money positioning yourself this way, but I expect on some time horizon they will need to reposition.
We clearly do not live in the same universe.
Had to check my assumptions though so I looked up what the lower end of GDP for a country is and sure enough they have American Samoa, Dominica, and Tonga beat. Now that money is probably meant to last 16 months so its not quite apples to apples but kind of wild regardless.
Ref: https://en.wikipedia.org/wiki/List_of_countries_by_GDP_(nomi...
I recently had the best customer support experience I've ever had, and it was an AI chatbot, helping my in ways I would not expect from a first-line support human.
Recently I opened the chatbot window on the site of a supplier of electric vehicle chargers, and asked some obscure things about how I could access and modify the charger in a non-standard way. I explained I was doing some R&D testing of a novel way of using the chargers, and I would need to temporarily reconfigure the public unit we had already installed in an unusual, non-standard way, rewiring its network to intercept traffic via transparent proxy, make it behave a little differently than it normally does, and that I didn't have the credentials.
I was expecting the first-line person or bot on the chat to be unable to handle my non-standard request. What I was hoping for was this would start the process of getting me to someone who could help. I expected to take a few days or weeks to reach someone appropriate in the company, perhaps a sympathetic technician who could understand what I was trying to do.
To my amazement, in the first response it one-shotted a nearly complete answer to every part of my question, including the detailed and implied parts I'd left out for a later interaction. It figured out what I was doing and what that needed, told me exactly what to do and why, and showed me that it understood every aspect of my request. There were things in this I doubt anyone ever asked about before.
Because the answer so complete I didn't really need more. I asked it some things, e.g. about electrical safety, just to confirm my understanding of the one-shotted answer. In every reply, the chatbot was insightful, informative, helpful and correct, and I didn't need to explain things twice, or wait on hold, or be transferred to anyone else.
I don't know what service they were using, but whoever implemented it, hats off to them for using good technology and providing it with high quality data about the products they are supporting.
Honestly, if AI customer support calls or chats can be made that good consistently, that sounds great.
(That was a stark contrast to my awful expereinces with support at banks, where many humans made mistake after mistake, contradicted each other, and seemed to struggle to understand simple things. Particularly the one where a bank required me to fill out a form with basic company info, then it took 14 hours of phone calls and hold time until I got to the one person who understood straight away that the entries on the form were correct, had to be that way, and they could tick the "it's done" box. My execellent AI bot experience suggests a good one will always understand things like that on the first iteration, as if they are the best-trained and best-informed humans available. But other, worse AI bot experiences suggests the 14 hour calls would become infinitely long, never able to resolve the problem, leading to things like unnecessary account closures and locked funds needing a court to resolve.)