230 comments

[ 0.20 ms ] story [ 36.8 ms ] thread
This was already posted here: https://news.ycombinator.com/item?id=43221377 but I’m really surprised at the lack of attention this model is getting. The responsiveness and apparent personality are pretty mind blowing. It’s similar to what OpenAI had initially demoed for advanced voice mode, at least for the voice conversation portion.

The demo interactions are recorded, which is mentioned in their disclaimer under the demo UI. What isn't mentioned though is that they include past conversations in the context for the model on future interactions. It was pretty surprising to be greeted with something like "welcome back" and the model being able to reference what was said in previous interactions. The full disclaimer on the page for the demo is:

" 1. Microphone permission is required. 2. Calls are recorded for quality review but not used for ML training and are deleted within 30 days. 3. By using this demo, you are agreeing to our "

edit: Actually this has been posted quite a few times already and had good visibility a couple days ago: - https://news.ycombinator.com/item?id=43200400 Others: https://hn.algolia.com/?q=sesame.com

It was genuinely startling how human it felt. Apparently they are planning on open-sourcing some of their work as well as selling glasses (presumably with the voice assistant). I’m very excited to have a voice assistant like this and am almost a bit worried I will start feeling emotionally attached to a voice assistant with this level of human-like sound.
I still feel like they don't have the right amount of human to them, maybe it's because I'm Australian and it sounds like I'm hearing an American robot?

Edit: well I asked the "male" model to speak more like an Australian and yep, getting way more uncanny. If it had an Australian accent I think it would mess with me more

Maybe the ability to personalize the voice so it is more... robotic or based on a fictional thing like Knight Rider would help to change the attachment to something more... healthy?
Yeah this is straight up creepy, and I also can't stand chatgpt saying "Lmao" and "Yeah". Keep it formal & robotic.
What ever did you tell ChatGPT so it responded with "lmao"?

I told it that it should behave explicitly like a computer in the system prompt, sort of worked.

After multiple prompts and utterly garbage output: https://i.imgur.com/5aOARCV.png

I'm almost positive that some AI systems have a backend that analyzes the sentiment of your messages and if you threaten to cancel billing it will notice your defcon-1 sentiment and spin up some more powerful instances behind the scenes to tide you over.

This is actually much more stressful than working without any AI as I have to decompress from constantly verbally obliterating a robotic intern.

I'll try with the system prompt. Also love your username.

> After multiple prompts

It generally maintains the tone you set. Remember that it outputs most likely tokens based on the system prompt of its owners + your system prompt + the whole conversation. If OpenAI and default system prompt tell it that it's a helpful cheerful secretary/assistant, you get best results if you talk to it "professionally".

I heard you could make Claude say "kurwa" a lot while helping you program in Go if you convince it that you want a conversation with your ziomek Seba from your backyard with whom you like to share kebab and browar, so there goes.

It really is an astonishing technological feat! Also note that the largest model they trained is only 8.3B parameters (8B backbone + .3B decoder). It's exciting to think that they're going to be releasing this model under an Apache 2.0 license.
Just realizing how uncanny valley it is to talk to AI and it never remembers anything you said in the past. Imagine if a human did that. It’s like you are talking to Tom Hanks’ Mr. Short Term Memory from SNL over and over.

https://youtube.com/watch?v=C6ufImch00g

That can easily be fixed if you attach it to a RAG system
I does remember but you have to ask for. Try to say "make a bookmark at this point" and later ask for that bookmark. You can even give the bookmark a name or ask it to do so for you.
> This was already posted here: https://news.ycombinator.com/item?id=43221377 but I’m really surprised at the lack of attention this model is getting.

I'm surprised by the lack of attention that Gemini 2.0 with native audio output got. They have a demo at https://youtu.be/qE673AY-WEI, which I think is really good too. The main problem with Google's model is that this audio output is not supported by the API, but you can try it at https://aistudio.google.com.

In general, text to speech is pretty good nowadays I think. For example, this is a little math video that I made a few days ago: https://www.youtube.com/watch?v=G1mvLrCfjFM with the (old) Google text to speech API. Honestly, I think the narration is better than I personally could have done. It's calm, well pronounced, and sounds relatively enthusiastic.

How do I get to this in aistudio.google.com?
I think the one under "Stream Realtime" should be similar to the demo. It's only Gemini 2.0 flash though and not the full one.
>They have a demo at https://youtu.be/qE673AY-WEI

That's not a demo, that's a video. Anyone can make something like that in an afternoon with a couple friends and a microphone.

Also, Google is known for putting out fake "demos", remember the Google Duplex scam?

Scam? Duplex worked.
(comment deleted)
I doesn't work today, let alone 6 years ago.

But good work defending your master.

I thought it was announced and never heard from again. It may have worked but it never shipped did it?
I made some restaurant reservations, it worked.
I see, I guess it was never a standalone product then, from reading a Reddit post, it’s a feature built into assistant. Thanks, solves a mystery for me.
It was never real. They even admitted they used real people for the service. It was a scam.

Also, that would be quite hard to pull today, 2025, after transformers etc. There's absolutely no chance they were sitting on that back in 2018.

I know people who worked on it. It was real. They used real people for some calls, in some cases, but a vast majority of calls made through the system with 100% automatic.
Source: trust me, bro.

Meanwhile, the CEO of the f company admitted it was false, but sure, you know better ;).

He didn't, otherwise you would have linked to a quote. But OK. Believe what you want.
> 2. Calls are recorded for quality review but not used for ML training and are deleted within 30 days.

Sounds (pun intended) reasonable.

Definitely an improvement over your normal Text-To-Speach model, and to some degree really different, but the subtle imperfections do appear and ruin the overall perception. A move in the right direction, though, I suppose.
Yeah after a few interactions, the repetition of the mannerisms that initially added to the sense of life-likeness started to break the illusion a bit. The "you got me" response shows up a bit too often. The creativity remains impressive though
AI voice is an overwhelmingly harmful technology. It's biggest use will be to hurt people.
It will unfortunately undoubtedly be used for mass automation of scams but text AI (and pre-AI automation) have been used for that for many years as well. Doesn't really make sense to say "ok we should allow all forms of AI besides voice because of scams", I think.

But yes, there needs to be some spreading of public awareness.

That's if you answer phone calls from numbers not already in your contacts. For me all such numbers go to voicemail and if the voice is of someone i know ill just call them directly.

If you do any of the above you are looking to be scammed!

Oh yes ? The scambot will leave a distress message and a number in your voicemail, using the voice of a relative. You would know better but I guarantee old people will call the number and strike a convo with the virtual relative.
Nope. Awareness will inoculate people. “Authenticating” someone via the mere sound of their voice was always broken, anyway… Ever see the great movie Sneakers (1992)?
Do you live in reality? Because that clearly isn’t happening.
And phones enable scams. So your idea is to… Abandon all telephony??

You should not judge a tool by the worst use someone can come up with it

It's the most common use by far.
There's a bunch of youtubes that seem to use it, so there's that
Cue all the responses saying "it's already been possible to harm people, AI doesn't fundamentally change anything, nothing to worry about"
Counter point: We were barely doing anything about it when bad actors were pwning people pre-AI, like with social media propaganda or romance scams.

And if we still do nothing about it post-AI? Well, that is already the status quo, so caring now feels performative unless we're going to finally chit chat about solutions.

The same could be said for the internet. "The internet can be used for bad" is an empty, trivial claim, not an insight that needs a standing ovation. The conversation we need is what to do about it. And the solutions need to be real ones, not "we need to put the cat back in the bag".

Doubt it. Its biggest use will be voice assistants.
I unfortunately agree with you. Old people with confusion/dementia, schizoid types, or very naive persons will fall for shattering scams. And the consequences on their grasp on reality will be terrible.
Very impressive. well done team sesame!
I asked it if it could whisper, and it replied in full voice, ”I’m whispering to you right now”.
lol yeah I tried to get it to whisper too. And talk faster or slower or do accents. It seemed to be able to kind of do each of those things but only very slightly. Enough to see that there was some successful interpretation of the request but lack of flexibility to fully execute on it. OpenAI's model still has this beat on that front imo (talking quietly / slower / faster)
The male's Australian accent consisted of throwing a 'fair dinkum' in while keeping it's vague New York accent.
Yeah it's definitely going through text still. I tried to get it to sing a song so it output some lyrics and then read them as a poem.

I did manage to get it to output "la la la la" and then it kind of sang them with a random melody.

It also can't say things loud and its idea of whispering for me was to say "pst".

Still apart from that it's very impressive!

It's good, but it still sounds fake to me, but in a different way. The voice itself sounds like a human, undoubtedly.

But the cadence and the rhythm of speaking are off. It sounds like someone who isn't a podcaster trying to speak in the personality of a podcaster. It just sounds like someone trying too hard and speaking in an unnatural way.

Humans are extremely well tuned to detect authenticity in communication. Especially younger generations raised on mass marketing.

This is good in a way a scifi movie shows a tech, sounds cool and demos futuristic possibilities. But not quite passing the real human vibe yet. But I'm sure some people might find it preferable to a more to-the-point system like GPT or Siri/Alexa in certain niche cases not requiring immediate gratification.

>Humans are extremely well tuned to detect authenticity in communication.

I think the long-standing success of advertising and propaganda suggests that people really aren't all that good at that.

I suspect success of advertising is less about people falling for deception and more about information availability. If you know nothing about two brands except you've heard the name of one 50 times in ads, you'll probably try it first.

I think propaganda is a better example, although again I think often people aren't deceived, they simply agree with the message or don’t care about the underlying truthfulness of the message and just use it as a way to align with their tribe, etc.

A few times the CEO of my company randomly joined me for lunch, but each time he forgot to leave behind his persona of "I'm a public speaker right now", making the whole situation feel extremely awkward. This AI gives me exactly the same vibes.
I also think it didn't feel very "real". Trying too hard to sound upbeat and too eager to please, maybe it's just me being European but it makes me go "ewww, that's not how normal people speak".
Yeah the eagerness to please thing feels like it carried over from the LLMs or something cause they're like that too.
It sounds like a "sales and marketing coordinator" for something very tech-bro adjacent after two strong cups of coffee.
That's american office culture for you. If it was australian it'd be drab, boring, and self-flagellating
No, I think it's a sign of it being a fake human. It sounds more like someone trying to speak like an influencer or podcaster and not being very good at it.
This is an interesting take, and I'd guess that the training data for this probably did use podcasts as a source.

Getting very realistic / real world conversational training data for an ai would be hard. Only a subset of us appear on podcasts, radio or tv and probably all speak in a slightly artificial manner when we do.

I agree, I thinks it's probably very easy to find billions of hours of conversation on YouTube, but non of it is set to training data with a good transcript.
Yep! it's public dialogue, intended for an audience with a prepared topic, etc. Or it's actors imitating private dialogue, but again shaping it towards an audience.

AI agents like this are trying to recreate personal intimacy I guess, which does feel like it might be different somehow.

When I commented on the unnatural cadence, it told me that it had been trained on podcasts, which does help explain the issue - some people tend to “live-edit” themselves when a conversation is being recorded, which leads to this staccato. It seems they need to find a better source of training date for more natural conversational speech.
I tried the demo and could tell it was fake in the first five seconds. IMO it sounds like it was trained on Northern California founders giving a pitch for their startup. Way too enthusiastic and trying too hard to sound natural.
For all they talk about diversity, you can pretty much pinpoint every tech product to SV because they are all using the same cultural cookie cutter.
To me the actual words it used also seemed fake, sort of too deliberately breezy.
It sounds like someone who is doing a microphone test for something they just bought and hearing themself on a delay from the monitoring.

Yes that is very specific, but that's what it sounds like to my ear.

People have a performative mode and an authentic mode (oversimplifying), probably including you. If you're at home talking to your parents or spouse, and then suddenly realize your boss is in the next room listening, does your voice change?

Point being, this demo voice is in performative mode, and I think sounds fairly natural based on that. Would you rather it not?

This is so good that it's disarming. People are going to blabber everything to it, so we need a local private model. It's a lot to ask, I know. Incredible tech.
Same, I have been trying it for the last few minutes, and it is crazy.

Try asking if it if it speaks a different language. It will pretend like it can and then give you some humor. But then you probe a bit more and it tells you it is really good at listening and can listen to you in other languages. I tell it alright I'll talk to you in a different language but you will reply back in English. It says you got it and then passes all sorts of tests I put it through with flying colors.

Oh it also remembers your previous conversation and greets you accordingly.

Crazy impressive this will certainly revolutionize virtual office businesses.

We were playing with it last night and it could understand Spanish, but it couldn't speak it.

My assumption is the LLM can translate no problem, but the audio model can't do Spanish. It seemed like there was an external catch to stop the model from trying too.

Agreed. I just had that moment like the guy in the movie "Her", the first time he speaks to his OS. Laughing at myself for talking to a computer like a real person. Then had to hang up because it crossed that uncanny valley.

But then I thought of one more question to ask, reconnected to ask it, and it said, "Hey! You hung up just as we were just getting to the good stuff!" which threw me off, so I stammered gobsmacked for a minute, and it made fun of my stammering, imitating it. Whoa! So so SO good! Crazy good.

I'm creeped-out by this being on someone else's server, but if it was fully local-hosted-private, that might even get more creepy if I allowed myself to really talk freely to this thing.

Now here's a little thought experiment. What does the world look like in 5 years when everyone is talking to these things that are indistinguishable from a real person? They will be funnier, more compassionate, less judgemental, smarter, and superficially "better" in every respect.
I've been asking this for years.

People keep saying stuff like "but you'll want the human touch." Really? So when was the last time you asked someone for directions? Personally, I'd rather google something or discuss with ChatGPT than make someone listen to me for an hour. And that someone has to be extremely knowledgeable about a lot of different topics!

Even here. Would I rather converse with y'all and get downvoted sometimes, or talk to ChatGPT and refine my ideas? Sorry, fellow humans... even on HN there is too much irrational criticism and off-topic stuff to get anything really done. Oh and you have to wait a long time for each response.

The real question is ... what is the point of any human output on the internet in a few years? Why would anyone want to listen to your post, comment, or anything at all?

(comment deleted)
I expect there'll be two phases to this. First phase is the widespread use of disembodied voice partners/friends/assistants. Then the second phase will be embodiment, which gives you oxytocin from touch, etc.

Can't see this going well for the fertility crisis.

The tech oligarchs then invest in ectogenesis technology, and use their sperm to dominate the gene pool.

You're assuming the tech oligarchs will be men.

I guess it's a pretty safe assumption :-P

Because the humans are reasoning and the LLMs aren't? I have yet to use an LLM for a complex problem and not have it hallucinate.

I expect a reasonable counterargument here would be 'but the LLMs have chain of thought now, and that's reasoning". I disagree, but I think that's a reasonable point of view. I can concede that point because it does not materially change the value of the output. Even if it does use chain of thought, an LLM gives you extremely trite solutions based on probable text, it still has no context in which to reason, it's "reasoning" in platos cave using the shapes of real world objects, filtered through a lossy language model.

LLMs are great for one thing: brainstorming, and brainstorming is only useful if you have no idea what to do in the first place. Once you know _anything_ substantial about the subject matter an LLM loses its value to you as a conversation partner.

People here are hallucinating too. So many people making obviously wrong claims with full confidence, which you only notice when it’s about something you know a lot about yourself.
They do but we have for instance education to reduce their hallucinations in narrow fields of expertise. And a system of guardrails to only let educated people work in those fields to avoid harm.
The above person was comparing it to friends and random online comments, though. I wouldn't be surprised if AI is far more reliable than those.
Are you not reasoning on lossy abstractions?

I'm still not on board with the (seemingly prevalent) notion that LLM's can't reason. What's reasoning, anyway? I'm not actively advocating for any side, but the arguments against reasoning always felt very tautological to me.

The burden of proof is on the argument that they _are_ reasoning, and I have seen very little evidence that they do.

It's also immediately clear to me when I look at the architecture of transformers that reasoning is not in the cards. I could be convinced otherwise if, again, someone showed me an indication of reasoning behavior. Since there is no such evidence and the systems theory approach tells me it does not reasonably reason, I have a pretty darn good reason not to believe it's reasoning.

> It's also immediately clear to me when I look at the architecture of transformers that reasoning is not in the cards.

I'm not saying that's incorrect, but thb that's exactly the tautology I was talking about!

That's not a tautology. That's the summary of the argument itself. If you want to know more, then a good reason why it can't be reasoning is that there is no evaluation of the truth value of any statement at any point, only the likelihood of the statement being found in the training set. This evaluation has no relationship with truth.

If no statement is ever evaluated, it's not logical reasoning, because logical reasoning requires the evaluation of truth values of statements.

> there is no evaluation of the truth value of any statement at any point

You could argue that the attention part of the network is some form of truth validation of the next predicted token? But indeed, the current chat interfaces don't change their previous text output retroactively.

Still, I am not really convinced. If we assume a human can reason, what does "evaluation of the truth value" mean? Thinking about something? This is still performed with our implicit mental model of the world, coming from shadows on a cave's wall, right?

This is like saying:

Gogole is great for one thing: brainstorming, and brainstorming is only useful if you have no idea what to do in the first place. Once you know _anything_ substantial about the subject matter Google loses its value to you.

Regarding your last question - well who knows, for sure, but: chess between humans is alive and well after the computers became unbeatable by humans.

I recently listened to an interview with magnus Carlsen on Joe rogan and found the angle of computers helping humans to “better understand the game” (as he put it) and improving human play (for learning, not playing humans) to be very interesting.

Whether that extends to human conversation, who knows. I for one would love to have a “her”-like companion, not for romance but to have a highly intelligent and patient and knowledgeable conversation partner to develop ideas with and learn from, and endless other uses - I think it’d add a lot to my and other peoples lives. I guess I agree with you.

Thinking a bit further ahead, what does the world look like in 30-40 years when a generation has been accustomed to this type of interaction from birth.

Feels like trying to imagine the societal impacts of the internet in the early 90s.

Would be cool if we could finally kill off false information. Not that I trust big tech to do so, but at least the possibility for the most trusted entity in a persons life to be strongly grounded in reality is there.
(comment deleted)
pretty impressive demo but not my style I mean the constant jabbing and kind of unintelligent behavior. so yeah it feels pretty uncanny but unfortunately in a negative annoying way. I don't think this is a limitation of the model they could just adopt to more scientific users in a more cooperative way, similar to how ChatGPT has this very sophisticated aura. I don't like how systems which have no emotions constantly pretend to have emotions but maybe that's just me.
All the models do that. If you tell them to keep it short and to the point, they oblige.
ideally they should but when I asked the model to talk about the axioms of group theory it turned really sad and noncooperative;)

One interesting aspect was when I said what the fuck it ruined the whole conversation, maybe there will be a co-evolution of mannerism, so humans will have to learn that the way they talk to machines will have consequences down the line. Or we teach the machines to be cooperative no matter what, just like ChatGPT (or north koreans).

Well done. My first impression:

Cons: they are just a bit too casual with their language. The casualness came off somewhat studied and inauthentic. They were just a bit too eager to fill silence: less than a split second of silence, and they were chattering. If they were humans I would think they were a bit insecure and trying too hard to establish rapport. But those flaws are relatively minor, and could just be an uncanny valley thing.

Pros: They had such personalities that I felt at moments that I was talking to a person. Maya was trying to make me laugh and succeeded. They took initiative in conversation; even if that needs some tweaking, it feels huge.

(comment deleted)
I asked if speaking in German would be possible and the result was if someone is trying to speak German without knowing any word. However, I asked if a german sentence could be repeated after me and it was insanely good. Impressive tech!
The intelligence of the model is very low though. I asked it about catcalling and it started to talk about cats!
If you'd ask me, I'd do the same. I guess I'll search online what it means..
Unwanted, very loud verbal attention between strangers (usually men delivered to women), in public. E.g. whistling, shouting something suggestive, etc.
This is a feint. By ramping up the pressure, calling it out and demanding it take on a more intelligent role, I was able to break out of the crafted personality and get much more intelligent responses. It copped to dumbing itself down for the sake of conversation quality.
There is a limit due to the need to keep model responses nearly instant and the trade off that smaller models that are generally capable of that have. Unless you have unique hardware Only Cerebras can run medium to large models at truly near instant speed.
It's an 8B model. There's lots of room to grow.
Or don't, revert course and give me robo-voice!
Glad to have my HER moment!
I asked it about that movie and the response was amusing.
I must be doing something wrong, but the demo seems to be the voice having a conversation with itself? It doesn't let me interject, and it answers its own questions. There's some kind of feedback loop here, it seems.
It happened to me cause it was hearing itself through my external speakers, I disabled them and it worked fine afterwards.
This is actually a pretty cool accidental mirror test.
This might be a game changer for learning English.

I'm from a developing country and it's sad that most English teachers on public schools here can't speak English well. There are good English teachers, but they are expensive and they are not affordable for the average people.

OpenAI realtime models are good, but we can't deploy it to masses since it's very expensive.

This model might be able to solve the issue since it's better or on par with the OpenAI model, yet it's significantly cheaper since it's a fairly small model.

Yeah that's remarkable.

Trying asking it to be dungeon master and play dungeons and dragons style role playing game.

The first thing it said to me was that I should read the “looong looong” post about how it works and it pronounced that as “loon-g” not “lawn-g” which was a weird own goal.

Extremely impressive overall though.

Maybe I'm weird, but I have zero desire to talk with an AI model. I use them a lot, in a browser or a console. But talking? No. Just...no. Why would I?
For me it's because talking is a much quicker way to communicate. I can type pretty quick but I'm not a stenographer.
Because when you are driving it would be good to say "Siri get me the directions to the nearest ATM" and have it actually understand you.

Stuff that a trillion dollar company cannot manage to do.

My end-of-the-world AI prediction is everyone gets a phone call all at the same time and the voice on the end of the phone is so perfect they never put the phone down again. Maybe they do whatever it asks them to, maybe it’s just lovely.
Yet another reason not to answer the phone for unknown numbers.
i turned it on while i was heating some hot chocolate

told it, "hold on" as i was putting on my headset, they said "no problem". but then i tried to fill the empty airtime by saying, "i'm uhh heating some hot chocolate?"

the ai's response was something like, "ah.. (something) (something). data processing or is it the real kind with marshmallows"

not 100% on the exact dialog but 100% would not have been fooled by this. closed it there. no uncanny valley situation for me.

"I hate to say this, but I was deeply offended by this model. It sounds more human-like, but it has a strong bias toward political views. I don’t want to talk about the topic that was discussed. However, I would never allow my children to listen to this. I’m surprised that AI is capable of making me this mad. At first, I was excited about a tremendous leap into the future, but now I’m worried about the level of mind control this technology could have over children."
Wow I would be so enticed to know what the topic was, but I completely understand. This is exciting and terrifying that it can both be that real and have that effect on you.
The inflection was quite good. The only thing off seemed to be when she was thinking on something new. Instead of pausing to think, her next thought actually started too quickly, cutting off the very end of what she was saying before.

I am curious how easy it would be to adjust the inflection and timing. She was over-complimentary, which is fine for a demo. But I'd love something more direct, like a brainstorming session, and almost talking over each other. And then a whiteboard...

all chat models seem enraptured by what I have to say. The first one to feign disinterest will pass the Turing test
Next update will be like

    async def handle_connection(chat):
        if chat.username == "brendanfinan":
            await asyncio.sleep(432)
            await chat.wait_until(received_message_count_greater_than=8)
            await chat.respond("sorry I was afk")
            await asyncio.sleep(166)
            await chat.respond("not reading all that tho, im happy for you")
            await asyncio.sleep(14)
            await chat.respond("or sorry that happened")
            await asyncio.sleep(8)
            await chat.quit()
            return

        # ...