Launch HN: Retell AI (YC W24) – Conversational Speech API for Your LLM
With the advent of LLMs and recent breakthroughs in speech synthesis, conversational voice AI has just gotten good enough to create really exciting use cases. However, developers often underestimate what's required to build a good and natural-sounding conversational voice AI. Many simply stitch together ASR (speech-to-text), an LLM, and TTS (text-to-speech), and expect to get a great experience. It turns out it's not that simple.
There's more going on in conversation than we consciously realize: things like knowing when to speak and when to listen, handling interruptions, 0-200 ms latency and backchanneling phrases (e.g., "yeah", "uh huh") to signal that they are listening. These are natural for humans, but hard for AI to get right. Developers spend hundreds of hours on the AI conversation experience but end up with poor experiences like 4-5s long latencies, inappropriate cutoffs, speaking over each other, etc.
So, we built Retell AI. We have followed the overall paradigm of having speech-to-text, LLM, and text-to-speech components, but have added additional conversation models in between to orchestrate the conversation while allowing maximum configurability for the developers in each step. You can think of our models as adding a “domain expert” layer for the dynamics of conversation itself.
Retell is designed for you to bring your own LLM into our pipeline. Currently, we can achieve 800ms end-to-end latency, handle interruptions, speech isolation, with tons of customization options (e.g., speaking rate, voice temperature, add ambient sound). We created a guest account for HN, so you can try our playground with a 10-min free trial without login: https://beta.retellai.com/dashboard/hn (Playground tutorial: https://docs.retellai.com/guide/dashboard). Our product is usage-based and the price is $0.1-0.17/min.
Our main product is a developer-facing API, but you can try it without writing code (e.g. create agents, connect to a phone number) via our dashboard. If you want to test it in production, feel free to also self-serve with our API documentation. One of our customers just launched, and you can view their demo: https://www.loom.com/share/64f09a53bf6d4b3799e5ebd08b23fec4?...
We are thrilled to see what our users are building with our API, and we’re excited to show our product to the community and look forward to your feedback!
181 comments
[ 235 ms ] story [ 3969 ms ] thread"What time works"
"Morning on tuesday would be best, but I can also do afternoon"
"I'm sorry, I didn't catch what time in the afternoon you wanted"
"No, I said the morning"
"I'm having a hard time hearing you. What time in the morning did you want?"
"10am"
And from there things were fine. It seemed very rigid on picking a time and didn't suggest times when I laid out a range.
oh no
Amazing product, looking forward to working with it.
What is worse, that no one is available to listen to you when you're suicidal, or that you lack so much value that only a machine would talk to you. I'm sure some people would have an extremely poor reaction to that.
I'm going to move all of this to an offtopic stub and collapse it.
We tell founders to make sure this doesn't happen (see https://news.ycombinator.com/yli.html) but I probably need to make the message louder. Not everyone understands that the culture of HN doesn't work this way.
Retell focusing on all of the other weird/unpredictable aspects of human conversation sounds super interesting and the demo's incredible.
Things are moving so fast, wow.
Can you please ensure, going forward, you have the universal "truth" as it were, to have your system always identify if its AI when "prompted" (irrespective of what app/dev has built - your API should ensure that if "safeword" is used it shall reveal its AI)
--
"trust me, if you ask them if they are a cop, they legally have to tell you they are a cop" (court rules its legal for cops to lie to you) etc....
(it should be like those tones on a hold-call, to remind you that youre still on hold... but instead its a constant reminder that this bitch is AI) -- there should be some Root-level escape word to require any use of this tool to contact a Human. That word used to be "operator" MANY times, but still...
Maybe if a conversation with an elderly Human goes on with too many "huh? I cant hear you" or "i dont understand, can you repeat that" questions, your AI knows its talking to a non-tech Human, and it should re-MIND the Human that youre just an AI. (meaning no sympathy, emotion, it will not stop until you are dead) etc...
Guardrails, motherfucker, Do you speak it!"
a dev builds an app it | to your API and you spit it back out? - if so - ensure when you spit out whatever it defines itself to whomever is listening....
--
Plz explain the arch of how your system works? (or link me if I missed..)
----
Shortest and most importnat law ever written:
"an AI must identify itself as AI when asked by Humans."
0. Law of robotics.
------
@autsin
-
Cool - so im on an important call with [your customer] your system has an outage?
How is this handled? dropped call?
(I am not being cynical - im being someone who is allergic to post mortems.
----
EDIT: you need to stop using the term "user" in anything you market or describe. full stop.
the reason: in the case of your product, the USER is the motherhecker on the phone listening to anything your CUSTOMER is spewing at them VIA your API.
the USER is who is making IRL *>>>DECISIONS<<<* based on what they hear from your system.
Your CUSTOMER is from whom you receive money.
THEIR customer, is whom they get money to pay you.
The USER is the end-point Human. who doesnt even know you exist.
Our homepage https://www.retellai.com/ has a GIF on it that illustrates this point.
For outage handling: we strive to keep up 99.9 plus up time, and in the case of a dropped call, the agent would hang up if using phone, and might have different error handling in web depending on how customer handles it.
> we focus most of our energy on innovating the AI conversation experience, making it more magical day by day. We pride ourselves on wowing our customers when they experience our product themselves.
This is not useful; you already have testimonials to show what customers think.
Maybe convert that first FAQ point about differentiation into a table comparing you against the closest competitors. Since you talk about performance you should measure it. Use a standard benchmark if there is one for your field.
People call customer service because they don’t know what to do. It would be better for most customers to talk to a bot that they can catch making a mistake.
Recent example: https://bc.ctvnews.ca/air-canada-s-chatbot-gave-a-b-c-man-th...
For example - "hallucinations / LLMs confabulate": techniques like RAG can help - "Language is so smooth that you often need the expertise to catch them in it", fine-tuning and prompt engineering can help
This blog breaks it down well:
https://www.papercup.com/blog/realistic-synthetic-voices
P.S: I tried to fool the Dental Office demo trying to book on Sunday or outside of the slots it had indicated, and it did a better job than many humans would have :)
As a customer of a service like Retell (though of course not specific to Retell itself), how might one go about setting up rules to keep a phone conversation from going on for too long? At 17¢ per minute, a 6-minute call will cost just over $1, or about $10 per hour. Assuming the AI receptionist can take calls outside of business hours (which would be a nice thing for a business to offer), then such a malicious/time-wasting caller could start at closing time (5pm) and continue nonstop until opening time the next day (8am), with that 15 hour span costing the business $150 for billable AI time. If the receptionist is available on weekends (from Friday at 5pm until Monday at 8am), that's a 63-hour stretch of time, or $630. And if the phone system can handle 10 calls in parallel, the dentist could come in Monday morning to an AI receptionist bill of over $6,300 for a single weekend (63 hours × $10 per hour × 10 lines).
This is in no way a reflection on Retell (I think the service is compelling and the usage-based pricing is fair, and with that being the only cost, it's approachable and easy for people to try out). The problem of when to end a call is one I hadn't considered until now. Of course you could waste the time of a human receptionist who is being paid an hourly wage by the business, but that receptionist is going to hang up on you when it becomes clear you're just wasting their time. But an AI bot may not know when to hang up, or may be prevented from doing so by its programming if the human (or recording) on the other end of the line is asking it not to hang up. You could say it shouldn't ever take more than five minutes to book a dentist appointment, but what if the person has questions about the available dental procedures, or what if it's a person who can't hear well or a non-native speaker who has trouble understanding and needs the receptionist to repeat certain things? A human can handle that easily, but it seems difficult to program limits like this in a phone system.