Locally hosted, no cloud, no tokens to pay. Reads a book using a full cast of characters. Quotation attribution detection (for my novel) is at 97.2% accuracy (485/499 quotes identified and assigned correctly). The autofill of character voice descriptions uses the prose to determine how the character sounds.
> annotate your text with extra 'stage directions'
Good idea, not something I've considered yet. Wouldn't take much to add it since there's already a feature for selecting a quotation and assigning it an intonation. Same infrastructure could be reused to select arbitrary text and assign stage directions.
Nice! One suggestion: don't just make this apply to text, but also to a specific cursor position (you could say a zero-length text selection where start=end). That's natural if you eg want to say that between one sentence and the next, or one word and the next, you want to insert the sound of a train arriving or a sigh.
I've been working on a similar project all year and as a tip, you should try Fish Audio or Higgs as a replacement for Qwen3. Both yield much better prosody and are much easier to listen to for long runs.
I wasn't able to find a version of these that can create voice samples based on voice designs. Do you mean to use Qwen3 TTS Voice Design to create samples followed by Higgs or Fish Audio to clone the sample voices and narrate the novel?
MOSS-TTS 2.0 will apparently have voice design, as well, on par with ElevenLabs quality.
For the voice design, these don’t support it, but for the final render, they’re much better. So your pipeline could for example generate voices with one tool and render with another.
> Voice replication: Recreate consistent vocal profiles from just a 30-second audio sample of your voice or a voice you have the rights to use, backed by built-in consent verification, SynthID watermarking, and C2PA credentials to protect both developers and their vocal talent.
I guess voice cloning is widely enough available now from other providers that Google are no longer hesitant to ship it.
I mean yeah? If google offers a cloud nmap tool, should everyone get in a tizzy about how google is "evil", even though it saves baddies maybe 5 minutes of work?
I don't know why this argument is brought up all the time anyway, it literally means nothing. They can name themselves "Don't Be Evil Inc" and continue to do evil stuff cause evil isn't an objective measure. If squeezing juice of puppies made money, any business can just say it's "not evil."
And if you think they're evil why would you trust them to follow their own guideline of not doing evil? An evil corp would be more likely to just hide behind that phrase, not quietly remove it as some subtle hint that they want to be openly and proudly evil all of a sudden.
Yesterday night I was doing a project with QwenTTS 1.7B.
After some debugging, making a clean dataset with clean recordings, and experimenting with a good fine tune recipe (much props to the new GPT models yesterday being cheaper).
I was able to make a robo-me that sounds absurdly good, family was shocked, all in a matter of a few hours.
so one of my interests is reducing the payload size for video games
the vast majority of the image sizes have audio recordings in different qualities for the last two decades, and this is still the case, as more varied and comprehensive audio is pursued by studios at unfathomable expense and still failing to cross a bar of realism
good voice models are just a few gigabytes in comparison and can supplant all of that, and be run locally at this point. Future ubiquitous hardware configurations in consumer devices will make inference dedicated and computationally cheaper and faster
although AAA studios are hamstrung and will be deeply unpopular if they stopped booking voice actors
everyone else who would have never had the capital for voice actors will just use this and have richer experiences until they themselves are AAA studios from the market buying their rich experiences
this videos vastly supplants the assumed and uninspired “tricking humans” use case from that video. once it crosses a threshold of ease, the applications will expand
This has been possible for quite a long time (probably 1-2 years). There are multiple open models that can do this quite well. Recent example from my YT feed:
"Super tinny monotone robotic voice" does not sound neither tinny nor monotone. Compared to what TTS from 90s sounded like. Or even how actors impersonated robots in movies. Has the model been eating too much hype DJs?
None of these examples are really what the prompt asked for. It's just like image models, once you get over how unbelievable it is that a computer produced this you realize the result isn't actually what you want.
I direct my own extended daydream Star Trek fanfic (okay, I'm on season 2 episode 17) and recently I looked to see if I could have each scene file be read aloud a la an audiobook or radio drama.
Getting GPT-Live to have unique enough voices and to be expressive with how I imagine the voices going in my head is hard to direct, there's not enough control there.
So this Gemini 3.8 specific large voice library and ability to tightly control (if you are willing to write a script) is nice to find, and while I'm not sure which of the 5,286 Gemini products this is, nor how to onboard and get started feeding this my own text files, nor what training will happen to my data if I did somehow use it, I love that the state of the industry is such that Google can do this and release it publicly, because that means eventually an equivalent product can come from someone else and be used locally / confidently that the generated audio or inputs won't be retained and misused.
I'm personally using this locally: https://github.com/mateogon/pdf-narrator (it's a Python frontend for Kokoro) on my M1 Macbook Air (from 2020, with 8GB RAM) and it's incredible. I make my own audiobooks now - for free!
Sorry if this is in that article, but I am on my phone and can't see it. How much would this cost to batch generate an audiobook? Right now I just listen to things in the 11 labs app which is free, but I would rather just generate audio files.
That is the biggest difference here for me. I have not looked at every solution, but many. They are either much more expensive or garbage quality. For example my next target audiobook is a monster 250k words, 1.5m characters, so about $15 here or $75 using elevenlabs (0.05 per 1k characters). For me that is the difference between I will or I won't use it.
Seems like voice actors are safe for now. This is technologically incredible, but the results are really not very good, and usually not particularly close to the prompt. In basically all of these examples some core part of the prompt is completely ignored.
I listened to some of the voices. The male voices are believable while all the female voices sound the same and artificial. For some reason, it also reminds me of the voices in Toy Story movies.
It's giving me an error when I try to generate a voice with Voice Design in AI Studio. It also says voice replication isn't available in my region.
Also weird that there are no "neutral gender" voices in the English language. There's also limited "use cases," like the "Gaming" use case is empty?
And there's no pricing listed anywhere.
I don't know, I guess their roll out is a bit sloppy. It's a bit of a shame, though, since the voices which are available all sound like generic Gemini voices to me. Nothing stands out is being particularly interesting or impressive about this.
Yes, I find nearly every "SOTA" voice model I try intolerable to listen to because of the fake exaggerated expression/emotion. It's actively distracting because it pulls focus to emphasize randomly. ChatGPT Voice models are so insufferable to put up with for a conversation longer than 45 seconds.
All I want is a clear, technically flawless, even/restrained "computer voice" for pretty much every use case (except audiobooks). But that doesn't make for splashy demos/score well for RLHF raters.
Yes, I wish there was more focus on correct pronunciation over emotion. You need a mechanism to control/guide the voice which gets under balance right between not having to specify everything and still letting you fix certain cases (where you know a certain sense of a word is meant)
They trained the female voice like it will be used for sex chats. Real life females don't talk like they are flirting with you, at least in my own experience.
I vibed a wikipodcast app that has a wikipedia dump, gemma llm for search and summarization and a tts model for audio gen so I could just ask about topics while on the go, and the app would just go off and tell me stuff out of the Wikipedia database.
It was especially nice during a bike trip along the Rhine, I listened to a lot of the history of the industrial area and its cities.
I've also see comments like "Microsoft Edge's read-aloud feature is amazing for TTS" but I haven't tried it myself.
Personally I use Kokoro with a python front-end on my Macbook, I linked to it in this comment: https://news.ycombinator.com/item?id=49818923 it outputs MP3, so I just copy those to my phone and listen as audiobooks.
Probably $$$ and limit to internal compute. Rumour is they even internal google employees have trouble getting compute for AI stuff. Guessing they're pretty slammed infrastructure wise with adding AI answers to default Google searches
I vibe coded a playground UI for trying this out. The conversation mode is neat, and it's very expensive - most of my experiments have cost less than a cent.
Nice but in general I wouldn't put my API key into some third party website, no matter if it claims to not store it. It's not something personal, you are a bit of a celeb here so I wouldn't think you'd save the keys, it's just good general data hygiene. Especially not if a leaked API key can rack up thousands of dollars in fees quickly.
131 comments
[ 2.2 ms ] story [ 71.1 ms ] threadhttps://www.youtube.com/watch?v=WAeHgE94rVo
Locally hosted, no cloud, no tokens to pay. Reads a book using a full cast of characters. Quotation attribution detection (for my novel) is at 97.2% accuracy (485/499 quotes identified and assigned correctly). The autofill of character voice descriptions uses the prose to determine how the character sounds.
Is it possible to annotate your text with extra 'stage directions' that influence how the book is read out?
Good idea, not something I've considered yet. Wouldn't take much to add it since there's already a feature for selecting a quotation and assigning it an intonation. Same infrastructure could be reused to select arbitrary text and assign stage directions.
I wasn't able to find a version of these that can create voice samples based on voice designs. Do you mean to use Qwen3 TTS Voice Design to create samples followed by Higgs or Fish Audio to clone the sample voices and narrate the novel?
MOSS-TTS 2.0 will apparently have voice design, as well, on par with ElevenLabs quality.
I guess voice cloning is widely enough available now from other providers that Google are no longer hesitant to ship it.
https://abc.xyz/investor/board-and-governance/google-code-of...
ctrl/cmd+f "evil"
I don't know why this argument is brought up all the time anyway, it literally means nothing. They can name themselves "Don't Be Evil Inc" and continue to do evil stuff cause evil isn't an objective measure. If squeezing juice of puppies made money, any business can just say it's "not evil."
And if you think they're evil why would you trust them to follow their own guideline of not doing evil? An evil corp would be more likely to just hide behind that phrase, not quietly remove it as some subtle hint that they want to be openly and proudly evil all of a sudden.
After some debugging, making a clean dataset with clean recordings, and experimenting with a good fine tune recipe (much props to the new GPT models yesterday being cheaper).
I was able to make a robo-me that sounds absurdly good, family was shocked, all in a matter of a few hours.
So yeah the cat is out of the bag for sure.
https://youtu.be/M-IVVJkZnuo?t=236
so one of my interests is reducing the payload size for video games
the vast majority of the image sizes have audio recordings in different qualities for the last two decades, and this is still the case, as more varied and comprehensive audio is pursued by studios at unfathomable expense and still failing to cross a bar of realism
good voice models are just a few gigabytes in comparison and can supplant all of that, and be run locally at this point. Future ubiquitous hardware configurations in consumer devices will make inference dedicated and computationally cheaper and faster
although AAA studios are hamstrung and will be deeply unpopular if they stopped booking voice actors
everyone else who would have never had the capital for voice actors will just use this and have richer experiences until they themselves are AAA studios from the market buying their rich experiences
this videos vastly supplants the assumed and uninspired “tricking humans” use case from that video. once it crosses a threshold of ease, the applications will expand
No que por los dos?
Don’t worry, we will be able to destroy careers and scam people at the same time. We don’t have to pick and choose!
but let’s not pretend solo developers were ever going to have a voice actor suite and hire that talent en masse
the transactions were never going to happen
and now the outcome will be better than the studios that are making those transactions
But I think you’re dancing on the grave of an entire industry, and every person that’s going to lose their life savings due to this.
the market doesn’t want 15 year lead times and overly expensive and delayed games that don't experiment on anything
every friction plaguing the industry is solved by distributed indie developers being able to make richer experiences faster and cheaper
It took like 1min - capture something from a youtube or video and put in your own text. It worked also really good for a german test.
Made a voice message for my wife from one of our favorite actors, telling here how nice it would be to make some breakfast :D
https://m.youtube.com/watch?v=WENMgQE9tws
Getting GPT-Live to have unique enough voices and to be expressive with how I imagine the voices going in my head is hard to direct, there's not enough control there.
So this Gemini 3.8 specific large voice library and ability to tightly control (if you are willing to write a script) is nice to find, and while I'm not sure which of the 5,286 Gemini products this is, nor how to onboard and get started feeding this my own text files, nor what training will happen to my data if I did somehow use it, I love that the state of the industry is such that Google can do this and release it publicly, because that means eventually an equivalent product can come from someone else and be used locally / confidently that the generated audio or inputs won't be retained and misused.
Also the Qwen3-TTS demo is cool, you can describe the voice you want: https://huggingface.co/spaces/Qwen/Qwen3-TTS
I came across both on this subreddit, it's very active: https://www.reddit.com/r/TextToSpeech/
I'm personally using this locally: https://github.com/mateogon/pdf-narrator (it's a Python frontend for Kokoro) on my M1 Macbook Air (from 2020, with 8GB RAM) and it's incredible. I make my own audiobooks now - for free!
My favorite voice is am_michael and here's a sample: https://voicerankings.com/voice/kokoro-82M/male/am_michael/s...
https://m.youtube.com/watch?v=jDudeaWppSE
Price per hour:
- 3.8 Flash TTS, standard: $0.81
- 3.8 Flash TTS, batch: $0.41
- 3.8 Flash‑Lite TTS, standard: $0.54
- 3.8 Flash‑Lite TTS, batch: $0.27
Official pricing can be seen here: https://ai.google.dev/gemini-api/docs/pricing#gemini-3.8-fla...
They all sound like Americans putting in their best fake British accent.
Having a voice under 1Mo is crazy, even if it sounds robotic.
https://tts.ampixa.com/sanoTTS/
I listened to some of the voices. The male voices are believable while all the female voices sound the same and artificial. For some reason, it also reminds me of the voices in Toy Story movies.
Bias in the training data?
Also weird that there are no "neutral gender" voices in the English language. There's also limited "use cases," like the "Gaming" use case is empty?
And there's no pricing listed anywhere.
I don't know, I guess their roll out is a bit sloppy. It's a bit of a shame, though, since the voices which are available all sound like generic Gemini voices to me. Nothing stands out is being particularly interesting or impressive about this.
How long is this stored? What could go wrong? :P
All I want is a clear, technically flawless, even/restrained "computer voice" for pretty much every use case (except audiobooks). But that doesn't make for splashy demos/score well for RLHF raters.
Is there a good browser extension that does this with a flexible TTS backend? I know Qwen, Kokoro, and VibeVoice all have decent quality..
It was especially nice during a bike trip along the Rhine, I listened to a lot of the history of the industrial area and its cities.
I found all these on this subreddit: https://www.reddit.com/r/TextToSpeech/
I've also see comments like "Microsoft Edge's read-aloud feature is amazing for TTS" but I haven't tried it myself.
Personally I use Kokoro with a python front-end on my Macbook, I linked to it in this comment: https://news.ycombinator.com/item?id=49818923 it outputs MP3, so I just copy those to my phone and listen as audiobooks.
Please let me know if you have any questions or feedback.
It sends it to your Spotify playlist so you can listen in the same place as the rest of your podcasts etc
https://storage.googleapis.com/gweb-uniblog-publish-prod/ori...
$0.50 per hour pricing could last a long time with back and forth conversation use.
https://tools.simonwillison.net/gemini-tts-playground#compos...
https://aistudio.google.com/generate-speech?model=gemini-3.8...