HN Explorer

Show HN: OWhisper – Ollama for realtime speech-to-text (docs.hyprnote.com)

289 points by yujonglee ↗ HN
Hello everyone. This is Yujong from the Hyprnote team (https://github.com/fastrepl/hyprnote).

We built OWhisper for 2 reasons: (Also outlined in https://docs.hyprnote.com/owhisper/what-is-this)

(1). While working with on-device, realtime speech-to-text, we found there isn't tooling that exists to download / run the model in a practical way.

(2). Also, we got frequent requests to provide a way to plug in custom STT endpoints to the Hyprnote desktop app, just like doing it with OpenAI-compatible LLM endpoints.

The (2) part is still kind of WIP, but we spent some time writing docs so you'll get a good idea of what it will look like if you skim through them.

For (1) - You can try it now. (https://docs.hyprnote.com/owhisper/cli/get-started)

  bash
  brew tap fastrepl/hyprnote && brew install owhisper
  owhisper pull whisper-cpp-base-q8-en
  owhisper run whisper-cpp-base-q8-en
If you're tired of Whisper, we also support Moonshine :) Give it a shot (owhisper pull moonshine-onnx-base-q8)

We're here and looking forward to your comments!

28 comments

[ 4.0 ms ] story [ 54.7 ms ] thread
yujonglee
Happy to answer any questions!

These are list of local models it supports:

- whisper-cpp-base-q8

- whisper-cpp-base-q8-en

- whisper-cpp-tiny-q8

- whisper-cpp-tiny-q8-en

- whisper-cpp-small-q8

- whisper-cpp-small-q8-en

- whisper-cpp-large-turbo-q8

- moonshine-onnx-tiny

- moonshine-onnx-tiny-q4

- moonshine-onnx-tiny-q8

- moonshine-onnx-base

- moonshine-onnx-base-q4

- moonshine-onnx-base-q8

JP_Watts
I’d like to use this to transcribe meeting minutes with multiple people. How could this program work for that use case?
solarkraft
Wait, this is cool.

I just spent last week researching the options (especially for my M1!) and was left wishing for a standard, full-service (live) transcription server for Whisper like OLlama has been for LLMs.

I’m excited to try this out and see your API (there seems to be a standard vaccuum here due to openai not having a real time transcription service, which I find to be a bummer)!

Edit: They seem to emulate the Deepgram API (https://developers.deepgram.com/reference/speech-to-text-api...), which seems like a solid choice. I’d definitely like to see a standard emerging here.

clickety_clack
Please find a way to add speaker diarization, with a way to remember the speakers. You can do it with pyannote, and get a vector embedding of each speaker that can be compared between audio samples, but that’s a year old now so I’m sure there’s better options now!
mijoharas
Ok, cool! I was actually one of the people on the hyprnote HN thread asking for a headless mode!

I was actually integrating some whisper tools yesterday. I was wondering if there was a way to get a streaming response, and was thinking it'd be nice if you can.

I'm on linux, so don't think I can test out owhisper right now, but is that a thing that's possible?

Also, it looks like the `owhisper run` command gives it's output as a tui. Is there an option for a plain text response so that we can just pipe it to other programs? (maybe just `kill`/`CTRL+C` to stop the recording and finalize the words).

Same question for streaming, is there a way to get a streaming text output from owhisper? (it looks like you said you create a deepgram compatible api, I had a quick look at the api docs, but I don't know how easy it is to hook into it and get some nice streaming text while speaking).

Oh yeah, and diarisation (available with a flag?) would be awesome, one of the things that's missing from most of the easiest to run things I can find.

DiabloD3
I suggest you don't brand this "Ollama for X". They've become a commercial operation that is trying to FOSS-wash their actions through using llama.cpp's code and then throwing their users under the bus when they can't support them.

I see that you are also using llama.cpp's code? That's cool, but make sure you become a member of that community, not an abuser.

elektor
Cool tool! Are you guys releasing Hyprnote for Windows this month?
bartleeanderson
Very cool. I was reading through the various threads here. I am working on adding stt and tts to an AI DungeonMaster. Just a personal fun project, am working on the adventure part of it now. This will come in handy. I had dungeon navigation via commands working but started over and left it at the point where I am ready to merge the navigation back in again once I was happy with a slimmer version with one file. It will be fun to be able to talk to the DM and have it respond with voice and actions. The diarization will be very helpful if I can create a stream where it can hear all of us conversing at once. But baby steps. Still working on getting the whole campaign working after I get characters created and put in a party :)
rshemet
THIS IS THE BOMB!!! So excited for this one. Thanks for putting cool tech out there.
notthetup
Is there a way to list all the models that are available to be pulled?
neom
Just wanna give a shout out to the hyprnote team - I've been running it for about a month now and I love how simple and no gimmicks it is. It's a good app, def recommend! (Team seem like a lovely group of youngins' also) :)
replwoacause
I’m looking for something that is aware of what is being discussed realtime, so if I zone out for a few minutes, I can ask it what I missed or to clarify something. Can this do that? If not, anybody know of something that can?
tinodb
Zoom does it quite ok
dcreater
Why cant it just use any OpenAI API endpoint?
tempodox
It seems to use https://api.deepgram.com (and other web endpoints) and apparently needs an API key, so it's not actually local. Why is it being compared to ollama, which does run fully locally?
theanonymousone
Given how sentiments towards ollama has become, I'm not sure this is a clever marketing line :D
vinni2
Very neat project! Congratulations to the founders. I was wondering why there is no one working on such a tool.

But I was hoping couple of features would be supported: 1. Multilingual support. It seems like even if I use a multilingual model like whisper-cpp-large-turbo-q8, the application seems to assume I am speaking English. 2. Translate feature. Probably already supported but I didnt see the option.

net_rando
Is a container version in your roadmap?
milchek
For people on osx looking for a no fuss open source pure whisper based local transcription to any input field in the OS you should also try OpenSuperWhisper (can easily be installed with brew)
pmarreck
Does it do speaker diarization? That's the one thing that I wish Whisper did out of the box. (I know WhisperX exists, but I haven't had a chance to try it yet.)

EDIT: Ah, I see this was already answered.

jp1016
Really neat work! I’ve been experimenting with something similar running a local Whisper model for quick transcriptions, then organizing the notes in a tabbed interface so I can keep different topics separate without switching windows. Vertical tabs have been surprisingly nice for keeping ongoing transcription sessions alongside reference material (I use beavergrow.com for this, but anything with a good tab system would work).
matcha-video
Question for folks who work a lot with STT models - What is your favorite model that supports word-level timestamps, has good dysfluency detection (whisper isn't great), and is also supported by transformers.js?