HN Explorer

Show HN: Babytalk: Offline speech to text and text to speech on ESP32 (github.com)

2 points by tlack ↗ HN
I have been playing with a lot of $20 devices with no keyboards or screens. I wanted to use speech to interact with them, but without it having to run through a cloud service or use a fixed command vocabulary.

I worked with Claude to implement high performance 4bit int kernels for the ESP32S3 and P4 and quantize some larger STT and TTS models. Surprisingly, it's quite usable on these tiny boards.

Also includes a finetuned model that can better cope with my diesel drones, hacking coughs, and mumbling. Works with Micropython (RAM scarcity issues) and AtomVM (better).

6 comments

[ 0.21 ms ] story [ 215 ms ] thread
atmanactive
I was hoping this could be a hardware STT-to-HID, but, even after looking at the Git ReadMe, I still don't understand how is the text supposed to exit the device?
tamilini92
I had the same question. I built an on-device dictation app for the Mac, and getting the text out turned out to be harder than the model. I ended up typing the transcript into whatever field is focused, using the accessibility permission.

On an ESP32-S3 with native USB, showing up as a HID keyboard seems like the cleanest way out, since it would then work with any computer without a driver.

vivzkestrel
- need a real time original voice to clone voice open source offline library

- need it for recording gaming videos while talking into mic with my voice but output is clone voice

tlack
Great ideas! Right now live transcription is hampered by the amount of memory available. I'll research whether or not this can be improved.
vivzkestrel
- thank you, the only reason i came up with this is because i have a not so good voice for recording and its noisy in my area

- i mean i can audacity my way out of it but its too much work and i would like to cut my effort down

- i turn obs studio on, start recording with my actual voice and when i stop recording, i get a cloned voice as output with a preset that i can select

- i have not seen a single extension do this, it would be absolutely wild if someone can pull it off