Hi, I used the [WhisperSpeech](https://github.com/collabora/WhisperSpeech) model for the TTS part after I did some serious torch.compile optimizations to bring the latency down. The Whisper speech recognition and the LLM were optimized through TensorRT-LLM by Marcus and Vineet.
It's not perfect but I am still extremely proud of how it came out. :)
Tried this on 4090 and the responsiveness and real-time communication it offers are truly impressive. It has significantly improved my overall experience, especially in scenarios where minimal delay is crucial.
Compared to WhisperFusion, Rabbit R1 feels like it's stuck in the past, they could maybe use the OpenSource WhisperFusion.
I wish this was available in perplexity labs, I wouldn't even use it for long things or waffling, just for those commands I use over and over again (like "do it again but for X without Y") and know exactly how to say because I'm sick of typing them.
4 comments
[ 3.1 ms ] story [ 22.2 ms ] threadIt's not perfect but I am still extremely proud of how it came out. :)
Compared to WhisperFusion, Rabbit R1 feels like it's stuck in the past, they could maybe use the OpenSource WhisperFusion.