This solves my biggest annoyance with the current advanced voice: its speech getting interrupted by me setting yup or even background noise if loud enough
I've had some funny interactions with this issue. I sometimes use the voice mode when walking my dogs. I can confirm that ChatGPT responds positively to being told it's a "Good girl!"
Hoping to use this for natural conversation language learning. Previous iterations of the app kept correcting my words/grammar before it got to the model, causing issues with identifying mistakes in speech
I was going to post a comment on a related topic (I couldn't find in the announcement if this is English only or not), but would you mind expanding? I was thinking about doing something very similar, and yeah, if the model isn't hearing the mistakes I'm making, that would dramatically decrease it's usefulness.
I tried it for these purposes and it didn't seem any better. I couldn't even tell it was a new model. I tested today, I'm an EU customer so maybe the rollout is delayed
They most definitely did not solve real time translation yet. The French in the video is barely understandable, both the translation and the pronunciation are the quality of an American who hasn't used French since high-school.
Are there any open source full duplex models that are out besides PersonaPlex? There was a chinese open one, maybe Fun Audio chat or something, that said it was going to release a full duplex version but I am not sure if it did.
My dream would be open source full duplex with function calling or some kind of rudimentary text output. PersonaPlex is still interesting although it was looking like we would need to fine tune it to handle outgoing or avoid going off the rails easily.
I want to know this too, as I’m hoping to fabricobble up a “smart speaker” that communicates with my local AI assistant. Right now we do everything via iMessage but it would be nice to be able to tell it to add things to my grocery list by voice while my hands are busy in the kitchen. Also would love if anyone has any advice on what microphone & speaker to pick out, was planning on just reusing a raspberry pi I’ve got around for the brain part.
Working on this at https://duplexio.ai which will be open weights and free for non-commercial use. If you want to work on this send an email to anders@duplexio.ai
Oh wow, I'd like this. Our current voice interactions with ChatGPT are on a 4o era model; really terrible. oAI has always been pretty cagey on the architecture of their end to end multimodal models. And RL has basically made them worse since launch. (Check the launch videos where the model sings, is more realtime, has accents, etc). I'd love to try a next gen version.
The potential conversational dynamics of people telling each other "quiet!" after they pick up the habit from talking with AI will be interesting. It could lead to people being more assertive and thoughtful, or it could be contentious and rude.
Awesome that they've improved that aspect of voice chat, though.
Absolutely can't wait to try this for language practice. The advanced voice mode is great but ultimately just doesn't work that well and doesn't have the feel of a natural conversation.
This looks very cool. An AI that can listen and speak and handle tasks without breaking the flow of conversation would solve some big annoyances with current tools.
The concern is though as these get better will people struggle to distinguish these with real human connections?
people have been mistaking AI conversations with reality since the very first text-based models came into the public view with ChatGPT. i'm sure with each incremental improvement to outputs like this, though, more people will get convinced of its "humanity"
Every time things like this come up, I can't help but think of the ending of Inception.
It's less that you're convinced it's real and more that you no longer care if it is. "Feels real enough" is good enough.
I'm a technical user first, so I'm not sure if models have improved for RP the way they improved for applied STEM tasks and technical brainstorming. But if there is an improvement curve there, I wouldn't be surprised if this only grows in popularity.
GPT-Live does not support video at this point, but we're working hard to introduce it soon. In the meantime, our previous Advanced Voice Mode will continue to be available and supports video.
Thanks for the reply, Atty. So I guess I'm holding it wrong?
For some reason, I can't use the camera while in Live mode. The only option I see is the plus item, which does show the camera, but when I open it up and ask "Are you seeing my camera?" it will always say no and recommend me to open it.
Feels like the official camera icon does not show up for me? iPhone 13, ChatGPT Pro subscription.
Ah yes — the camera icon is to take a photo, not to show the model video in real-time. Try taking the photo and uploading it! If you're having trouble, please share a screenshot and the build number (Settings > About) with me via email at atty@openai.com.
You can choose among 9 voices in the app, all newly refreshed for GPT-Live. If you meant whether it can detect multiple people, it can (like in the livestream), but not always perfect. Would love to hear your feedback once you try it.
I like this and felt like some of it was much more fluid; but was I alone in feeling like the interjected "uh-huh" or "yeah?" moments felt a little jarring?
Almost felt a bit *uncanny valley* for what "natural" conversation is supposed to be like. If the "uh huh" isn't timed correctly, it'll feel like a zoom call with lag.
Every second of the interaction is uncomfortable to me, but I also have extreme difficulty with video calls with humans. The latency completely breaks my mind.
I had preview access to this one for a few weeks. It's very good. I had one conversation that lasted a full hour while I was walking the dog, got some good brainstorming done against one of my projects.
The best feature is that it can delegate questions out to GPT-5.5 in the background, so you're no longer restricted to a voice model that's several years behind the frontier.
I did report a fun bug with it though: it was interrupting me and laughing at my (not really intended as) jokes while I was still talking! They seem to have clamped that behavior down thankfully, it felt a bit rude and condescending.
About 2 years ago I used to have conversations with GPT while walking the dog. It really emphasized the need to think before you speak, but you had to think fast before it hung up on you.
I called a friend the other day. It's very good. I had one conversation that lasted a full hour while I was walking the dog, got some good brainstorming done against one of my projects.
Dammit! I live in Europe and the cyber truck isn't even available here..
That said a huge pickup truck is about as far as you can get from a Camaro... Then again I'm not exactly David Hasslehoff myself either... Meh if it talks that's close enough!
> "The best feature is that it can delegate questions out to GPT-5.5 in the background, so you're no longer restricted to a voice model that's several years behind the frontier."
Wow. That's exactly what I hoped they would do.
This issue has held me back from using ChatGPT's voice mode as much as I otherwise would have, because I also use it for brainstorming while commuting, exercising, etc., and don't want it to feel stuck in the past.
Funnily enough - I built this over the weekend with Fable for a local voice chat running 100% on local LLMs, Parakeet and Kokoro. I say "...ask the thinking model..." and that redirects it to Qwen 3.6 27B on vLLM.
Can't claim originality though - it was inspired by Sesame - where their models will invoke a search, or check the weather etc, and make a vocalisation to keep you engaged.
Turn taking is one of the hardest things to get right for the exact reasons mentioned - but does seem to be the way that Claude.ai's voice works - in a very obvious way.
Anthropic + OpenAI both rug-pulled voices I liked and got used to and OpenAI really dumbed down their voices at the same time - Arbor went from Estuary English and almost "jack the lad" to some generic English accent. Claude had a Birmingham accent and said things like "shit", ending sentences like "So you're telling me that they asked for a 90% discount yeah?" - then it changed overnight to a mock Derbyshire accent with a dull tone.
ChatGPT's voice also gaslights me for conventional opinions - "my Eastern European neighbour helped me lift a wardrobe upstairs - something you just can't ask your typical neighbour neighbour"... then you get a full on left-leaning lecture from the safety layers rather than a head nod or "what luck!"
Claude + Sesame are nowhere near as overbearing.
In both cases - from edgy and engaging to something that just didn't gel.
The point of making my own assistant is that I can talk for as long as I want, episodic memory is personal and private, there's no "trust me bro, we're a big corporation" vibes.
This was not my first attempt - when I had a bunch of Opus credit around Jan/Feb - I tried really hard and created something that was not good enough. What I have now, is working, and each session is training Claude/Codex on what to tune, and to fix.
"Just had a convo - can you look into what happened?" And if it's one I don't mind sharing with the model - I'll say, "and what did you think of the questions I asked?" Sometimes it'll give a lovely commentary on how the model did.
af_heart is probably the smoothest voice - but yes it's more like another commented - more "StarTrek" than "telesales assistant that pauses and laughs at your jokes".
If you're on a similar path and want something full duplex - the go to solution is PersonaPlex from Nvidia based upon Moshi.
Given the personality type common on HN, I imagine that the GP, even if unplugged from all technology on their walk, wouldn't be in a mindful state of enjoying their surroundings, but rather would be "lost in the clouds", stewing on the same ideas/thoughts/problems, but with those thoughts going more in circles, due to a lack of ability to check anything.
Literally me. Everyone is different, and that's fine. But I don't have the privilege of living in an area where I can talk to people about the things I am thinking through. It's very rural. Having a _utility_ that can act as a sounding board while I spew out my thoughts on a walk is a really meaningful improvement to my current situation.
Not saying you aren't good at your job but this statement gives me pause. Given what I see on the daily as output from models I hope there's still a lot of human interrupt behind the wheel with respect to the design of drugs.
Every drug indeed has a huge burden of proof before it is distributed to humans, including the 3 levels of FDA clinical drug trials that cost billions of dollars, as well as all the internal testing we do beforehand.
there's no benefit to anyone for being so reductive, from whichever side you're on.
As a progressive person, the constructive thing is to try to see how to harness it for good or how to fight it, it's not to pretend that it's weak. It's not weak.
Sometimes I want to learn something while I walk. Sometimes I want to listen to stories. Sometimes I want to just be present. All are good. I highly recommend letting people enjoy whatever they enjoy.
This may not be you, but the people who have said that to me irl walk significantly less than I do. And saying these things to people who don't yet exercise can make them far less likely to start as it's a far bigger step to do so. Podcasts and recorded lectures are what got me exercising in the first place, because I was excited to hear the next part. I now only listen on a portion of my walks but the gateway drug of entertainment was very useful.
It's coming from a place of objecting to burnout/overwork culture.
Recently I witnessed a CTO mention in a public channel that with Claude remote control people can now work while getting a coffee or other breaks during the day.
Tech is actively moving in a direction of destroying all the gains from the labour movement in service of enriching capital out of a combination of FOMO or fear of being replaced.
So yeah, when folks here "hey look now I can even work during leisure activities!" yeah, the reaction is negative.
Sure but context matters and given this is Hacker News, a lot of discussion centers around tech as a profession (and that's doubly true for AI adoption). You can forgive folks for jumping to natural conclusions. :D
Workers in 2026, even non-tech workers have an easier life than workers alive at any other point in human history.
Yes ofc there are problems - collectivist land use laws that ban construction near homeowners continue to drive housing costs higher for example. But if you asked any worker today if they would have preferred to be alive 40 or 400 years ago I would be shocked if any said yes.
So I'm clear: as long as our jobs don't devolve to 100 hours a week in a sweat shop or tilling the earth for the lord of our fiefdom, we should just sit back and be happy?
Is that what I should take away from your comment?
>It's coming from a place of objecting to burnout/overwork culture.
Sure, but that's not everyone. I want this because it's hard to carve out time for my fun side projects. :)
Also, not everyone feels burnt out or overworked, and may find their actual work enjoyable. I've definitely had many showers where I've been thinking about interesting work problems and never felt particularly burdened by it (quite the opposite, in fact).
It reminds me of that old parable where the acolyte asks the priest "Father, can I smoke while I pray?" and gets told "No, you should be focusing on praying. That wouldn't be respectful". But then he says "Can I pray while I smoke?" and gets told "of course, you can pray at any time".
If it's 'walking time', I probably don't want to consume that with work. But if it's 'working time', it could be great to have a nice walk during it.
> I did report a fun bug with it though: it was interrupting me and laughing at my (not really intended as) jokes while I was still talking! They seem to have clamped that behavior down thankfully, it felt a bit rude and condescending.
Some times I'm still amazed that AI "gets" humour much more effectively than the character of Data did before his emotion chip, e.g.: https://www.youtube.com/watch?v=G4VZ5kQIdV0
Other times I'm amazed in the opposite way, that the script writers cover basically the same talking points about the character as we have today about LLMs, e.g.: https://www.youtube.com/watch?v=GBJCYHwyZhw
I have friends who have brainstormed with an LLM (voice chat) for 10-30 minutes, and reported very positive experiences.
When I speak to one - while I'm impressed at how far they've progressed - the LLM just doesn't talk like someone I'd want to discuss a technical problem with (the way I would with a human).
(And my friends aren't even using a custom prompt - some of them are just talking to the default Gemini on their phone!)
I also love exploring ideas like this. The problem that made me stop using Gemini is that it would always try really hard against all my prompting to ask a stupid question at the end of its response that would completely derail my train of thought.
I love the UX of voice mode. I’ve always hated voice models.
Gemini’s being particularly egregious (always ending in some deranged question, can not reliably be prompted away) prompted me to build my own client for my real harness that simply does STT -> model -> TTS (both being independently useful).
I guess I see some value in a model responding quickly and with more nuance, but it’s not much. I can wait for it to finish. I’d much rather have it be actually useful. I’m not looking for a digital friend.
The delegation feature lets me see some value in a voice model for orchestration type features. But in either case, I don’t really like (or understand why others would like) talking to a model with different features and quirks just because I’m using a different medium to communicate over.
I have built out something similar that let's me use my phone's hardware buttons to open an input stream with the mic for my Hermes agent over my matrix gateway and then has it play back with a local TTS model on my Pixel 10 Pro.
But the Deepseek v4 flash model I am using through OpenRouter is killing me on latency. Any suggestions to improve that?
I’ve spent my time very similarly working on my own voice stack project, but having also seen how non-developers use AI or experience technology in general, I truly think they are better served with a different UX and product than what we have.
In other words, if you’re building your own voice inference tooling you’re just about the polar opposite user demographic than the one that truly needs and will value this. You’re using voice as a medium of convenience doing what existing models are technically and practically “shaped” to be able to do, knowing how they work well enough that conversation is more like typing/prompting with your voice than a natural interface. I’m guilty of this myself but have you ever even paid for a voice/audio model or hardware?
Compare that to the millions of people with an Alexa device in their home who buy products through it, or who prefer calling support to get a human over poring over technical documentation. They’re actually very close to finally getting a version of “Alexa” that lives up to its promise and I’m happy for them
Does it have full access to your chat history, project files, etc? That's the biggest limitation I have with voice mode right now, if I ask it about something I chatted about before (even in the same conversation) it has zero recollection of it.
You can start a chat in an existing session after pasting data into that session and it can then talk about that content. I haven't tried it with projects.
Cool it sounds like they have improved. This the first time I tried back in the day voice chats were different and you could not get the transcript after the fact.
>The best feature is that it can delegate questions out to GPT-5.5 in the background, so you're no longer restricted to a voice model that's several years behind the frontier.
Ahh, this makes sense. I was wondering when they would start doing this. I stopped using voice mode all together because it was frustrating talking to a dumb AI, when most of the time I discuss things with Opus 4.8 or gpt 5.5.
I was working on a phone call agent recently, and thought about doing this. It makes sense
I didn't mean the "projects" feature. I described my projects to it, had it search for extra details and pasted relevant inflection into the session before starting the voice chat.
I do not fully understand the complexity behind achieving full-duplex but I hope this sets the bar for Anthropic to follow. Turn-based simplex is yesterday.
What I’m missing from this announcement is the capability to use connectors and tools. I don’t really get it - NONE of the frontier assistants can use tools / connectors while in voice mode - Claude, ChatGPT, Gemini, Grok. It seems so obvious: I want to be able to research stuff, pull up documents, jot down notes and do productive work while I’m talking to it, and not end voice mode whenever I need to connect to an app or service.
It’s weird. The old Claude voice mode WAS able to use tools but when they revamped it, it lost that capability and is now pinned to Haiku :(
So, yay for finally a voice mode that’s powered by a frontier model and hopefully as good as Grok voice, but sad to still not see tool use while in voice mode.
(I haven’t tried it yet, only read the announcement)
I could see it relating to tools having unpredictable latency but if they already do background hand off to 5.5 then it seems like they could just enable it within that context.
I’ve been using gpt-realtime-1 for my personal assistant that runs my company and plans my day. And it works pretty well, even makes tool calls and all that.
But the multi modal stuff has resulted in a lot of debugging with weird events and message and audio sequences having race conditions, but overall it is pretty awesome.
Looking forward to moving to this model later today and will chime back in with results.
Because they take too long to run, and have an unpredictable latency and success rate. Seeing loading spinners and error messages in a visual interface is fine, but it would firmly put a natural language conversation in uncanny valley territory.
Regular chat already supports voice input, so might as well use that.
A big part of this announcement does seem to be _delegation_ in the background; they give the example of web search but that could be any tool. I haven't tried it yet either but sounds like they've found a reasonable UX that mixes that sort of high-and-variable latency tool calling (potentially with agentic loops) with a continuously speaking live voice.
A timer tool with a callback feature would take like a couple of hours to implement, a model which has a native internal ability to know the time would take ages to make
I don't think it needs an 'internal clock'. It needs two things. (1) the timer tool, (2) to know not to lie about having a tool or using a tool. I think (2) is the hard thing.
One of the videos on the announcement page shows someone making coffee and it _appears_ that the agent counts 30 seconds in real-time. Curious to know how they made it do that without tool support.
> I don’t really get it - NONE of the frontier assistants can use tools / connectors while in voice mode - Claude, ChatGPT, Gemini, Grok.
Is that true? I have a friend who often brainstorms with Gemini (I think just on an Android phone), and he has it actually do stuff related to the conversation (including adding content to notes).
In any case, you can always vibe code one with pi!
Using gpt-realtime-2 you can achieve very similar results with low latency streaming which gives you this feeling of an assistant that is there in the room with you. I made a tweak to so you can try out the gpt-realtime-2 model with tooling that gives you this: https://sippet.ai/?enabler=hn
Exactly. I told the assistant to create world cup google calendar entries and it said it can't connect to my calendar, whereas I can have it do that with no issues on the normal gpt client.
Funny enough I have only encountered two voice modes that were okay to use tools and they are so far out there that you wouldn't even think to try. Bixby and Perplexity (in android phone digital assistant mode) both seem happy to use whatever the account is connected to. I mainly use it for managing my local phone's calendar. Claude chat can interact with it but voice can't which is frustrating.
This is something I built for myself, and to experiment with inference stacks. You can obviously just transcribe audio and hand it off to frontier models, so all you really need is a good voice stack and a “driver” for the interaction (like a phone call, place to see their work).
There are two big problems with this space IMO. One isn’t that you can’t get this to work but that people generally aren’t willing to pay for it for themselves, rather as a way to screen or automate stuff to be used by other people. Did you know Claude Code has a voice mode and that openai launched whisper a year ago, both of which have positive sentiment and adoption in heavy ai tool users? Yet it’s a blip in their marketing or why people use their products, meanwhile outside of coding, most of the biggest and highest earning AI product companies so far are voice agents targeting customer service, sales, business processes, etc.
The second is related: voice is genuinely a low-bandwidth medium, so as a primary interface for interacting with AI there is not a lot you can get out of it compared to eg complex technical work or visualizations or interactive applications. It is physically and mentally demanding to speak-aloud a highly detailed prompt fast enough that VAD won’t cut you off and you have something with comparable information density or specificity vs text. But to keep up a shorter and more natural cadence you’ll not be able to wait on a lot of thinking/tool unless you play UI tricks (ums and fillers, two models in a trench coat), break the illusion of a single coherent conversation, or take a lot of long pauses.
That’s why for the supplementary coding use case it’s mostly used for remote steering, and for general use marketed towards the large and very not-online group of people for whom typing is not a natural or common thing for them to spend their time on. Now that so much spend goes through heavily used token subscriptions and they’ve proven that kind of product, they’re not marketing “tool to get the most tokens per $ running your subscription 24/7” anymore lol.
What I’m most interested in is true “ambient” tool use against my own data or work, and for-later (or pushed live via your phone) visualizations or “five models in a trench coat but still coherent” UX, which you probably are too. But I think unless you work a lot with AI tools already it’s hard to understand how that’s any different from asking Alexa to set a timer, and either way something you’re not so desperate to have that you go looking for it, or pay smaller vendors/set up yourself.
Huh? You can expose tool functions to Gemini's live API, and it will call them as part of thinking about responses. I've used this to expose e.g. current GPS location for a smartwatch voice assistant.
watched the live translation video very impressive
Seems like a shift from previous voice models where it sequentially processes voice to text then feeds it to LLM and then back which cant escape the clunky lag
not sure how pipecat stands now, gpt live seems like it takes audio tokens and does inference on it directly
I've wondered if this would happen, although doing inference directly on speech tokens would seem to imply an entirely different model (trained on lots and lots of actual speech).
GPT-Live-1 is the first version of a new generation of models, and we believe the full-duplex architecture + delegation enables entirely new ways of human-AI interaction.
Can I connect it to my skills/tools? Example case, I have a knowledge base and event log in my company. I need a brainstorm companion, which will have full access to this knowledge, can converse about it and can invoke skills/tools available in the repo.
In ChatGPT, Voice doesn't yet support connectors, but we're hoping to add support soon! Once GPT-Live launches in the API, you can also build custom integrations yourself.
+1 to the op. This is one of the first things I thought about after using it for a few minutes. I have a large collection of local knowledge bases that I use through codex, having this real time thing be able to plug into it would be the killer feature, otherwise the utility is severely limited. I don't want to talk to it about the weather, I want to brainstorm about the things I am working on, which it currently knows nothing about.
Hey! Bit of an unusual question maybe: if this stuff further exarcerbates the loneliness epidemic and atomization of society, will you be able to live with yourself you think? If you hear about teenagers only spending time with your chatbot in 5 years, will you feel some amount of personal responsibility or not? Always curious to hear you guys' perspective on that kind of stuff!
Oh no, now I'd better blame Hitler and Stalin's ancestors for their misdeeds. Of course, the SS soldier bears no responsibility, however, as he was just doing his job.
> if this stuff further exarcerbates the loneliness epidemic and atomization of society, will you be able to live with yourself you think?
Looking at the 30,000-foot view of how society is set up: laws, economic system, employee incentives, etc, do you suppose it matters what the individual contributors think? I say this not to absolve anyone of responsibility, but to point out the obvious outcomes of our incentives across the strata (polity -> shareholders -> boards -> C-suite -> employees)
I will bet you dollars to donuts, somewhere inside OpenAI is a frequently-used revenue dashboard, but not for loneliness - if anything, OpenAI will make horny models and tout itself as a solution to loneliness, a la character.ai - if that earns them more money.
Of course they don’t believe in or care about any of this lol. To them it’s a minor inconvenience that some of the walking future cash flows have bleeding hearts, so they’ll make placating noises sometimes, but they obviously don’t care. They’re winners, everyone else is a loser, get over it, etc.
People who do this kind of stuff are very irritating. You clearly have some problem with the work they do. Instead of saying and approaching that outright, you pass it in some passive aggressive fake bullshit. Makes you sound like the kind of person I would much rather not be speaking to, which is kind of ironic given your comment.
Technology is what you make of it. If you want to make it your best friend and your only friend, that's your choice. But I would guess more people would use it as a personal tutor. If it works as well as the demo, this already crushes most language learning apps. Actually the only apps for learning language that it would fail to replace would still be Tandem and HelloTalk because it still can't replicate the nuance of real human interaction.
"Blame the individual" doesn't really make sense when we're talking about things that have a society-wide impact. As an analogue, personally I'm not on social media, but I am affected by social media because everyone around me is on social media and their happiness and wellbeing also impacts me. AI companies need to stop pretending they can just outsource responsibility for their products.
Go read "Careless People". The social media people knew exactly how damaging their product was early on, as evidenced by the fact that they wouldn't even let their own children use it. I think AI people understand how damaging their product is too; and so I will never respect them unless they take responsibility for the things they create. Going "not my problem bruh, I got my bag!" is a bad ethical position.
That's not really a fair question, especially given the audience here on HN. Most here have blood on their hands from the previous 25 years of tech-eats-the-world.
Well this is a new brand of whataboutism I haven’t seen before!
Did something bad? Better ablate yourself of the responsibility of holding people to account for making it _worse_! Acting this way just makes it seem more like you regret the blood on your hands because it has dirtied your shirt, and not because you’ve done something actually bad, otherwise you’d have at least some degree of guilt and reticence to see things get worse.
I would say no if the question was asked of me. It’s like asking if you’d feel bad about marriage if you ended up divorced with a child. Or if you’d feel bad about commenting if it made someone feel like they lost brain cells.
The morality of an action isn’t based on actual consequences because the future isn’t known in advanced. All we can do is act on the perceived consequences of our actions, and if we think those are good, pursue them.
You don't go into a marriage assuming you'll get divorced. You assume you won't, hope for the best, work not to, and then it happens. You work through those problems with the child so that it hopefully doesn't impact them as much as it otherwise could.
The loneliness epidemic is driven by companies maximizing keeping their customers engaged with their screens, something OpenAI is wont to do. Knowing that the company wants customers engaged and that this will do that, and also knowing that that plays into the loneliness epidemic by substituting human interaction, makes it far different than getting married and then maybe or maybe not getting divorced.
Its weird how years of relentless doom trolling and threatening peoples livelihoods, mixed with absolutely real, measurable, AWFUL societal impacts have soured people!
Was this directed at the voice technology in the article, or at AI in general - as they are two very different conversations?
If you're talking about ChatGPT in general (as I guess you are) I think the Jury's still out on whether this will have a net positive or negative on society.
Right now, it's leaning towards negative, but there are optimistic futures to be had at the same time.
As someone who likes to think that they would turn down a lot of tech jobs due to moral dilemmas - for me, currently working on AI wouldn't be one.
For me at-least it's a tool to learn quicker, and reduce friction on projects I otherwise may not have dived into. As with all new technology, we're still in this grace period / lack the bigger picture we now have on other, now known to be destructive technologies.
> GPT-Live-1 is the first version of a new generation of models, and we believe the full-duplex architecture + delegation enables entirely new ways of human-AI interaction.
Intuitively, decoupling semantic and audio-timeslice-space generations with coupled but distinct histories is right model architecture, not just for these sorts of assistants, but for domains like robotics too.
I'm currently on the 20 $/mo subscription and using codex meaningfully, and i'm loving this.
I am considering bumping my subscription to the 100 $/month and this might be the reason i switch, BUT: i really envision me using this also through other means as well (eg: agents like openclaw/hermes) in agentic ways.
Will this be supported?
I can make OpenAI stuff the center of my agentic AI life, but I need it to be interoperable.
We're adding support to the API soon, which will let you integrate with any agent in the background. Would love to see the community go wild with it. You can sign up to be notified here: https://openai.com/form/gpt-live-1-in-the-api/
If it's not able to connect to my other apps like gcal, it should suggest that I use the standard gpt client, instead of telling me 'I don't have the capability to do that' which is misleading as some might presume the normal client doesn't have that capability either.
I like it! I was watching a YouTube video of someone driving in a big city. I saw an interesting skyscraper and asked Chatgpt what it was. After Chatgpt answered, I asked for a photo of the building to confirm, and Chatgpt helpfully showed it in-chat. It was the correct building!
And because the voice is so frictionless to talk to, I asked about what company owns the building, then that company's industry, then how that industry works in this particular country etc. I probably wouldn't have bothered going down a rabbit hole like this if I'd had to type. Voice is much easier than typing.
Super nice to be able to finally have voice mode search stuff in the background. The more conservational vibe is also a nice touch. The real game-changer we're all waiting on, though, is being able to vibecode/deeply interact with an OS through voice mode. Until then voice mode will continue to be a niche, interesting product. Once that's implemented, though, it will be one of the most important technologies available. There's a lot of people I know that don't use a lot of the frontier capabilities of AI right now because of the required typing interface. Once that's changed and you can interact with these capabilities through voice mode (and ideally on your phone) I definitely see them and lots of people like them becoming big users of AI.
I'm so mad that this might make me re-subscribe to ChatGPT. I wouldn't have believed how much I use the voice feature before LLMs and ChatGPT currently has the best voice interface. I think Grok's interface is the next best, then Claude.
I for one am greatly looking forward to the day these kind of voice models can be run locally. It seems like the gap between open-weight and frontier is way larger for voice models than coding/language models.
Probably not _as_ good, but you can run gemma 4 for the ears/brain (accepts audio input), and kokoro TTS for the mouth. You want something like silero VAD sitting in front of the LLM so you aren't passing wasteful audio to gemma 4, so you send only voice activity segments. I type all this because I tried it out recently as a Zork experience, seeing how creative Gemma 4 12B could be. It was surprisingly good!
Yeah, I mostly mean for the TTS, which in my testing is flat and has no emotional register, or no nuanced variation. OpenAI speech in my experience is very good at things like communicating understanding or questions non-verbally using intonation, and I'm yet to see a local TTS model that will do contextual intonation well at all.
There doesn’t seem to be any indication whether this is available in the chat-got app nor is there any indication in the app that anything has changed. Anyone know how to actually try this?
GPT‑Live is rolling out now to ChatGPT users globally across iOS, Android, and ChatGPT.com. GPT‑Live‑1 will become the default model powering ChatGPT Voice for Go, Plus, and Pro users, and GPT‑Live‑1 mini will become the default for Free users.
We're beginning the rollout now, and will roll out in the next few days to ChatGPT users globally. Make sure to update to the latest version of the app!
In the voice settings, there were 2 options - Standard and Advanced, as well as a help link saying they were in the process of rolling out the newest model -- I think the new model is named Live (and is not Standard or Advanced). The help link explained it all.
So at least for me in the app there did seem to be a way to check.
447 comments
[ 0.22 ms ] story [ 101 ms ] threadThis time is the most natural version that exists and it is a natural as a conversation.
It's exciting for the future, though!
My dream would be open source full duplex with function calling or some kind of rudimentary text output. PersonaPlex is still interesting although it was looking like we would need to fine tune it to handle outgoing or avoid going off the rails easily.
https://si.inc/posts/hertz-dev/
It's only 8.5B and doesn't sound like it's quite conversational.
Awesome that they've improved that aspect of voice chat, though.
The concern is though as these get better will people struggle to distinguish these with real human connections?
(see https://www.reddit.com/r/MyBoyfriendIsAI/)
It's less that you're convinced it's real and more that you no longer care if it is. "Feels real enough" is good enough.
I'm a technical user first, so I'm not sure if models have improved for RP the way they improved for applied STEM tasks and technical brainstorming. But if there is an improvement curve there, I wouldn't be surprised if this only grows in popularity.
One thing I noticed is that we lost vision feature for some reason on the live chat?
This was an extremely useful feature. Not sure if it’s a regional thing or that they just removed that from the current live chat.
I imagine it will be even more useful with this new version.
GPT-Live does not support video at this point, but we're working hard to introduce it soon. In the meantime, our previous Advanced Voice Mode will continue to be available and supports video.
For some reason, I can't use the camera while in Live mode. The only option I see is the plus item, which does show the camera, but when I open it up and ask "Are you seeing my camera?" it will always say no and recommend me to open it.
Feels like the official camera icon does not show up for me? iPhone 13, ChatGPT Pro subscription.
You can choose among 9 voices in the app, all newly refreshed for GPT-Live. If you meant whether it can detect multiple people, it can (like in the livestream), but not always perfect. Would love to hear your feedback once you try it.
Almost felt a bit *uncanny valley* for what "natural" conversation is supposed to be like. If the "uh huh" isn't timed correctly, it'll feel like a zoom call with lag.
The best feature is that it can delegate questions out to GPT-5.5 in the background, so you're no longer restricted to a voice model that's several years behind the frontier.
I did report a fun bug with it though: it was interrupting me and laughing at my (not really intended as) jokes while I was still talking! They seem to have clamped that behavior down thankfully, it felt a bit rude and condescending.
That said a huge pickup truck is about as far as you can get from a Camaro... Then again I'm not exactly David Hasslehoff myself either... Meh if it talks that's close enough!
Europeans are notorious for their good taste.
Wow. That's exactly what I hoped they would do.
This issue has held me back from using ChatGPT's voice mode as much as I otherwise would have, because I also use it for brainstorming while commuting, exercising, etc., and don't want it to feel stuck in the past.
Can't claim originality though - it was inspired by Sesame - where their models will invoke a search, or check the weather etc, and make a vocalisation to keep you engaged.
Turn taking is one of the hardest things to get right for the exact reasons mentioned - but does seem to be the way that Claude.ai's voice works - in a very obvious way.
Anthropic + OpenAI both rug-pulled voices I liked and got used to and OpenAI really dumbed down their voices at the same time - Arbor went from Estuary English and almost "jack the lad" to some generic English accent. Claude had a Birmingham accent and said things like "shit", ending sentences like "So you're telling me that they asked for a 90% discount yeah?" - then it changed overnight to a mock Derbyshire accent with a dull tone.
ChatGPT's voice also gaslights me for conventional opinions - "my Eastern European neighbour helped me lift a wardrobe upstairs - something you just can't ask your typical neighbour neighbour"... then you get a full on left-leaning lecture from the safety layers rather than a head nod or "what luck!"
Claude + Sesame are nowhere near as overbearing.
In both cases - from edgy and engaging to something that just didn't gel.
The point of making my own assistant is that I can talk for as long as I want, episodic memory is personal and private, there's no "trust me bro, we're a big corporation" vibes.
This was not my first attempt - when I had a bunch of Opus credit around Jan/Feb - I tried really hard and created something that was not good enough. What I have now, is working, and each session is training Claude/Codex on what to tune, and to fix.
"Just had a convo - can you look into what happened?" And if it's one I don't mind sharing with the model - I'll say, "and what did you think of the questions I asked?" Sometimes it'll give a lovely commentary on how the model did.
af_heart is probably the smoothest voice - but yes it's more like another commented - more "StarTrek" than "telesales assistant that pauses and laughs at your jokes".
If you're on a similar path and want something full duplex - the go to solution is PersonaPlex from Nvidia based upon Moshi.
Before this model, the voice models were pretty dumb and annoying to work with. We'll see if this changes that.
we know pretty well at this point that this is world warping technology (either for good or bad), not a small matter of taste.
And what are you doing about it?
As a progressive person, the constructive thing is to try to see how to harness it for good or how to fight it, it's not to pretend that it's weak. It's not weak.
The thing I don't get is...no one would say this about listening to a podcast or audiobook on a walk.
I'm not sure why people choose to demonize this specific use of time during walks.
Just for the record, I would say the same thing in that situation.
Recently I witnessed a CTO mention in a public channel that with Claude remote control people can now work while getting a coffee or other breaks during the day.
Tech is actively moving in a direction of destroying all the gains from the labour movement in service of enriching capital out of a combination of FOMO or fear of being replaced.
So yeah, when folks here "hey look now I can even work during leisure activities!" yeah, the reaction is negative.
I'm far more surprised that this surprises you.
Workers in 2026, even non-tech workers have an easier life than workers alive at any other point in human history.
Yes ofc there are problems - collectivist land use laws that ban construction near homeowners continue to drive housing costs higher for example. But if you asked any worker today if they would have preferred to be alive 40 or 400 years ago I would be shocked if any said yes.
Is that what I should take away from your comment?
But since I’ve been using an LLM, it’s been bright green.
I always check in code manually. I don’t let the LLM do it.
Sure, but that's not everyone. I want this because it's hard to carve out time for my fun side projects. :)
Also, not everyone feels burnt out or overworked, and may find their actual work enjoyable. I've definitely had many showers where I've been thinking about interesting work problems and never felt particularly burdened by it (quite the opposite, in fact).
If it's 'walking time', I probably don't want to consume that with work. But if it's 'working time', it could be great to have a nice walk during it.
Some times I'm still amazed that AI "gets" humour much more effectively than the character of Data did before his emotion chip, e.g.: https://www.youtube.com/watch?v=G4VZ5kQIdV0
Other times I'm amazed in the opposite way, that the script writers cover basically the same talking points about the character as we have today about LLMs, e.g.: https://www.youtube.com/watch?v=GBJCYHwyZhw
I have friends who have brainstormed with an LLM (voice chat) for 10-30 minutes, and reported very positive experiences.
When I speak to one - while I'm impressed at how far they've progressed - the LLM just doesn't talk like someone I'd want to discuss a technical problem with (the way I would with a human).
(And my friends aren't even using a custom prompt - some of them are just talking to the default Gemini on their phone!)
Gemini’s being particularly egregious (always ending in some deranged question, can not reliably be prompted away) prompted me to build my own client for my real harness that simply does STT -> model -> TTS (both being independently useful).
I guess I see some value in a model responding quickly and with more nuance, but it’s not much. I can wait for it to finish. I’d much rather have it be actually useful. I’m not looking for a digital friend.
The delegation feature lets me see some value in a voice model for orchestration type features. But in either case, I don’t really like (or understand why others would like) talking to a model with different features and quirks just because I’m using a different medium to communicate over.
But the Deepseek v4 flash model I am using through OpenRouter is killing me on latency. Any suggestions to improve that?
In other words, if you’re building your own voice inference tooling you’re just about the polar opposite user demographic than the one that truly needs and will value this. You’re using voice as a medium of convenience doing what existing models are technically and practically “shaped” to be able to do, knowing how they work well enough that conversation is more like typing/prompting with your voice than a natural interface. I’m guilty of this myself but have you ever even paid for a voice/audio model or hardware?
Compare that to the millions of people with an Alexa device in their home who buy products through it, or who prefer calling support to get a human over poring over technical documentation. They’re actually very close to finally getting a version of “Alexa” that lives up to its promise and I’m happy for them
Ahh, this makes sense. I was wondering when they would start doing this. I stopped using voice mode all together because it was frustrating talking to a dumb AI, when most of the time I discuss things with Opus 4.8 or gpt 5.5.
I was working on a phone call agent recently, and thought about doing this. It makes sense
The full duplex is awesome, and the feedback that it is getting what you're saying is ok, but in some of the demos was a little overkill.
I'll agree that using the "Golden Girls" was at least more entertaining than the usual pitch.
Why do you care? You can select other voices. Why do you need to control others?
What is at the root of your need for domination?
It’s weird. The old Claude voice mode WAS able to use tools but when they revamped it, it lost that capability and is now pinned to Haiku :(
So, yay for finally a voice mode that’s powered by a frontier model and hopefully as good as Grok voice, but sad to still not see tool use while in voice mode.
(I haven’t tried it yet, only read the announcement)
But the multi modal stuff has resulted in a lot of debugging with weird events and message and audio sequences having race conditions, but overall it is pretty awesome.
Looking forward to moving to this model later today and will chime back in with results.
Regular chat already supports voice input, so might as well use that.
Sam's reaction was "Yea, it doesn't have access to tools like a timer. It's a known issue. Should be coming in about a year"
Edit: here's the clip: https://www.youtube.com/shorts/Py2YgJe8fqQ
Is that true? I have a friend who often brainstorms with Gemini (I think just on an Android phone), and he has it actually do stuff related to the conversation (including adding content to notes).
In any case, you can always vibe code one with pi!
Just super difficult.
See more here:
https://github.com/sibblegp/ODAI/blob/main/routers/app_voice...
I usually do a very simple test and tell it to verbatim tell me all tools that are connected
This is something I built for myself, and to experiment with inference stacks. You can obviously just transcribe audio and hand it off to frontier models, so all you really need is a good voice stack and a “driver” for the interaction (like a phone call, place to see their work).
There are two big problems with this space IMO. One isn’t that you can’t get this to work but that people generally aren’t willing to pay for it for themselves, rather as a way to screen or automate stuff to be used by other people. Did you know Claude Code has a voice mode and that openai launched whisper a year ago, both of which have positive sentiment and adoption in heavy ai tool users? Yet it’s a blip in their marketing or why people use their products, meanwhile outside of coding, most of the biggest and highest earning AI product companies so far are voice agents targeting customer service, sales, business processes, etc.
The second is related: voice is genuinely a low-bandwidth medium, so as a primary interface for interacting with AI there is not a lot you can get out of it compared to eg complex technical work or visualizations or interactive applications. It is physically and mentally demanding to speak-aloud a highly detailed prompt fast enough that VAD won’t cut you off and you have something with comparable information density or specificity vs text. But to keep up a shorter and more natural cadence you’ll not be able to wait on a lot of thinking/tool unless you play UI tricks (ums and fillers, two models in a trench coat), break the illusion of a single coherent conversation, or take a lot of long pauses.
That’s why for the supplementary coding use case it’s mostly used for remote steering, and for general use marketed towards the large and very not-online group of people for whom typing is not a natural or common thing for them to spend their time on. Now that so much spend goes through heavily used token subscriptions and they’ve proven that kind of product, they’re not marketing “tool to get the most tokens per $ running your subscription 24/7” anymore lol.
What I’m most interested in is true “ambient” tool use against my own data or work, and for-later (or pushed live via your phone) visualizations or “five models in a trench coat but still coherent” UX, which you probably are too. But I think unless you work a lot with AI tools already it’s hard to understand how that’s any different from asking Alexa to set a timer, and either way something you’re not so desperate to have that you go looking for it, or pay smaller vendors/set up yourself.
Seems like a shift from previous voice models where it sequentially processes voice to text then feeds it to LLM and then back which cant escape the clunky lag
not sure how pipecat stands now, gpt live seems like it takes audio tokens and does inference on it directly
GPT-Live-1 is the first version of a new generation of models, and we believe the full-duplex architecture + delegation enables entirely new ways of human-AI interaction.
Would love to hear your feedback!
1. The voice model delegates to one agent.
2. The voice model delegates to multiple agents, and keeps track of tasks.
3. The voice model delegates to an orchestrator agent, which then delegates to sub-agents and keeps track of tasks.
YMMV depending on the exact product experience you care about, because there is a tradeoff between latency and layers of delegation.
Our current implementation is backed by one model, but you can imagine this getting much better with time.
Looking at the 30,000-foot view of how society is set up: laws, economic system, employee incentives, etc, do you suppose it matters what the individual contributors think? I say this not to absolve anyone of responsibility, but to point out the obvious outcomes of our incentives across the strata (polity -> shareholders -> boards -> C-suite -> employees)
I will bet you dollars to donuts, somewhere inside OpenAI is a frequently-used revenue dashboard, but not for loneliness - if anything, OpenAI will make horny models and tout itself as a solution to loneliness, a la character.ai - if that earns them more money.
I think there's a logic leap here that ignores most of what big companies know about how to gey people addicted to things
Did something bad? Better ablate yourself of the responsibility of holding people to account for making it _worse_! Acting this way just makes it seem more like you regret the blood on your hands because it has dirtied your shirt, and not because you’ve done something actually bad, otherwise you’d have at least some degree of guilt and reticence to see things get worse.
The morality of an action isn’t based on actual consequences because the future isn’t known in advanced. All we can do is act on the perceived consequences of our actions, and if we think those are good, pursue them.
The loneliness epidemic is driven by companies maximizing keeping their customers engaged with their screens, something OpenAI is wont to do. Knowing that the company wants customers engaged and that this will do that, and also knowing that that plays into the loneliness epidemic by substituting human interaction, makes it far different than getting married and then maybe or maybe not getting divorced.
If you're talking about ChatGPT in general (as I guess you are) I think the Jury's still out on whether this will have a net positive or negative on society.
Right now, it's leaning towards negative, but there are optimistic futures to be had at the same time.
As someone who likes to think that they would turn down a lot of tech jobs due to moral dilemmas - for me, currently working on AI wouldn't be one.
For me at-least it's a tool to learn quicker, and reduce friction on projects I otherwise may not have dived into. As with all new technology, we're still in this grace period / lack the bigger picture we now have on other, now known to be destructive technologies.
Awesome. Are you guys able to share anything about the model architecture? I've been interested lately in split-transformer RVQ-based conversational agents, e.g. via stuff like https://arxiv.org/abs/2412.10208 (ResGen) and https://arxiv.org/abs/2603.18090 (MOSS-ITT) and of course Moshi (https://arxiv.org/pdf/2410.00037).
Intuitively, decoupling semantic and audio-timeslice-space generations with coupled but distinct histories is right model architecture, not just for these sorts of assistants, but for domains like robotics too.
- The videos felt scripted and dishonest
I'm currently on the 20 $/mo subscription and using codex meaningfully, and i'm loving this.
I am considering bumping my subscription to the 100 $/month and this might be the reason i switch, BUT: i really envision me using this also through other means as well (eg: agents like openclaw/hermes) in agentic ways.
Will this be supported?
I can make OpenAI stuff the center of my agentic AI life, but I need it to be interoperable.
i.e. how will full duplex & delegation enable/enhance desktop flows w/o corresponding leaps in UI.
One group's expectation of interruption for pleasant conversational flow can be just as off-putting as another's expectation of patient silence.
And because the voice is so frictionless to talk to, I asked about what company owns the building, then that company's industry, then how that industry works in this particular country etc. I probably wouldn't have bothered going down a rabbit hole like this if I'd had to type. Voice is much easier than typing.
Anyhow it's fun! Thanks for making it!
Are we seeing any conversational layer integrated with codex soon?
We're beginning the rollout now, and will roll out in the next few days to ChatGPT users globally. Make sure to update to the latest version of the app!
In the voice settings, there were 2 options - Standard and Advanced, as well as a help link saying they were in the process of rolling out the newest model -- I think the new model is named Live (and is not Standard or Advanced). The help link explained it all.
So at least for me in the app there did seem to be a way to check.