kokoro is surprisingly great at nuance but it's tough to improve that last ~2% or so. kokoro + rvc is really great too; i use that for ELEMENT47, the LLM-centric comedy podcast i do that i wish more people would listen to. (e47.net , feel free to subscribe!)
I'm using Kokoro for a fun little side-project browser-based game I'm working on. It's legitimately super good for being only 85mb (for the wasm version) or 300mb (for the webgpu version).
I used to keep a version of whisperx around, because I think it's important to have not just transcription, but also timing and speaker identification (e.g. for subtitles)... It depends on pyannote, though, which has some wierd licensing (and is tougher to script the installs because of it), so I wanted to look at something that both had better transcription, and supported diarization (the speaker and timing). I decided on parakeet for the transcription with softformer (the diarization), but most of the available engines for it don't include softformer.
I coded up an OpenAI compatible server for parakeet-rs ( https://github.com/altunenes/parakeet-rs ) (which does support softformer) and I've been using it with OpenWhispr (a desktop app for transcription that handles all sorts of neat thing).
I'm doing CPU-only transcription (because I use my GPUs for other stuff and haven't gotten around to adding in the GPU-path), but it's incredibly empowering to be able to have local transcriptions at will.
I have used Kokoro fairly extensively for an accessibility product. I have loved working with it (especially because I don't have an NVidia GPU like many TTS of similar quality require).
I particularly appreciate the fact that it lets you manually add IPA pronunciation guides. There have been some cases where an important word is a homograph and Kokoro assumed the wrong pronunciation.
The place where it falls a little short is in saying just a single word or two. Try having it say simply "six" and it almost always says something like "ah-six-ah". I found a way around that though. If you give it a longer sentence to say (eg "The word is: six") it will say it fine. The trick is that the Kokoro API gives you the timestamp of each word in the sentence. So you can have a Python script crop out just the word you care about. The intonation is a little flat this way, but is very reliable.
I asked about this on the discord, and was told that it is a limitation of the small parameter size. But in fairness to Kokoro, even eleven-labs' voices suffer from this occasionally.
Love Kokoro tts. I wrote https://github.com/Jud/kokoro-coreml to try pushing the limits a bit on speed & size. Such great quality at a given size. As others have mentioned short utterances are problematic, but solvable.
Both Text-to-Speech and Speech-to-Text now have local models that are good enough to get the job done. Kokoro for TTS, Parakeet for STT and Fluid-1 for text formatting (I use it with FluidVoice). I hope this is a trend that continues for other applications.
A couple months back I wrote a chrome extension that does this on any webpage, with simultaneous highlighting of the sentence being read. Skips both the container launching step and the copy pasting website contents step. Might be useful to anyone trying to use kokoro ergonomically.
It's interesting that the male voices are all so much worse than the female voices (several are quite good). There is bias in machine learning, but I wonder whether there is also systematically more training data of female speech?
Love this model. I’m GPU poor and have had FOMO that I haven’t played with local models at all. About a month ago I setup Kokoro on my GTX1650 to do TTS for an article reader. A simple WebUI lets me paste a URL or a chunk of copy pasted text. Python cleans it up and sends to Kokoro for TTS and it’s then served via RSS for Apple Podcasts. Then for my morning drive I’ll catch up on articles or blog posts I’ve gathered.
At some point I’d like to play with separate voices and see if I could build something like NotebookLM for kind of like a radio morning show of news items I’ve gathered.
I just hooked it up to my personal AI Japanese Teacher app, pretty good quality / natural sounding speech in mixed English / Japanese while running fast on CPU so I don't waste VRAM.
For Japanese TTS, AivisSpeech-Engine[1] works really well with mixed Japanese/English text in my experience. They also provide container images on ghcr.io for both CPU and GPU inference.
58 comments
[ 2.2 ms ] story [ 65.6 ms ] thread> AMD Ryzen 7 8745HS: 1.5 seconds
These two can probably do it much faster on their iGPUs.
the onnx version of pocket-tts does perform better. https://huggingface.co/KevinAHM/pocket-tts-onnx
I used to keep a version of whisperx around, because I think it's important to have not just transcription, but also timing and speaker identification (e.g. for subtitles)... It depends on pyannote, though, which has some wierd licensing (and is tougher to script the installs because of it), so I wanted to look at something that both had better transcription, and supported diarization (the speaker and timing). I decided on parakeet for the transcription with softformer (the diarization), but most of the available engines for it don't include softformer.
I coded up an OpenAI compatible server for parakeet-rs ( https://github.com/altunenes/parakeet-rs ) (which does support softformer) and I've been using it with OpenWhispr (a desktop app for transcription that handles all sorts of neat thing).
I'm doing CPU-only transcription (because I use my GPUs for other stuff and haven't gotten around to adding in the GPU-path), but it's incredibly empowering to be able to have local transcriptions at will.
I particularly appreciate the fact that it lets you manually add IPA pronunciation guides. There have been some cases where an important word is a homograph and Kokoro assumed the wrong pronunciation.
The place where it falls a little short is in saying just a single word or two. Try having it say simply "six" and it almost always says something like "ah-six-ah". I found a way around that though. If you give it a longer sentence to say (eg "The word is: six") it will say it fine. The trick is that the Kokoro API gives you the timestamp of each word in the sentence. So you can have a Python script crop out just the word you care about. The intonation is a little flat this way, but is very reliable.
I asked about this on the discord, and was told that it is a limitation of the small parameter size. But in fairness to Kokoro, even eleven-labs' voices suffer from this occasionally.
Quality is very close.
Will vary in your setup, but here is my script: https://github.com/DavidVentura/translator-rs/blob/master/sc...
https://chromewebstore.google.com/detail/local-reader-ai-on-...
I speak over sonos speakers when certain events happen. And use it as my voice assistant.
At some point I’d like to play with separate voices and see if I could build something like NotebookLM for kind of like a radio morning show of news items I’ve gathered.
[1]: https://github.com/aivis-project/AivisSpeech-Engine