Show HN: Port of OpenAI's Whisper model in C/C++ (github.com)

399 points by ggerganov ↗ HN
Hi HN,

OpenAI recently released a model for automatic speech recognition called Whisper [0]. I decided to reimplement the inference of the model from scratch using C/C++. To achieve this I implemented a minimalistic tensor library in C and ported the high-level architecture of the model in C++. The entire code is less than 8000 lines of code and is contained in just 2 source files without any third-party dependencies. The Github project is here:

https://github.com/ggerganov/whisper.cpp

With this implementation I can very easily build and run the model - “make base.en”. It also allows me to run it on a wide range of devices. For example, I have provided examples of running the model on an iPhone, Raspberry Pi 4 and even in a web page via WebAssembly!

The implementation runs fully on the CPU and utilizes FP16, AVX intrinsics on x86 architectures and NEON + Accelerate framework on Apple Silicon. The latter is especially efficient and I observe that the inference is about 2-3 times faster compared to the current PyTorch implementation provided by OpenAI when running it on my MacBook M1 Pro. The WASM port utilizes SIMD 128-bit intrinsics - a feature supported in some modern web browsers [1].

I am very happy with the performance that I observe on Apple Silicon devices. I didn’t expect that the Accelerate framework [2] (i.e. CBLAS) offers such a dramatic performance boost for matrix multiplications so I was very pleasantly surprised! To enable the framework in your C/C++ projects, all you have to do is add `-framework Accelerate` to your clang command-line flags.

This entire exercise of implementing the Whisper model was very interesting to me and helped me understand a lot about how the transformer architecture works. I also got a lot of positive feedback from people finding and using my project. We brainstormed on a lot of interesting tools that can potentially be created with this library (such as speech-to-text plugin for Vim, RPi4 voice assistant, WASM chat bot, etc). If interested, checkout the “Examples” section and the “Show and tell” discussions for some ideas!

Would love to know what you think about this project and about your experience with using the Accelerate framework in any of your projects. Cheers!

[0] https://github.com/openai/whisper

[1] https://chromestatus.com/feature/6533147810332672

[2] https://developer.apple.com/documentation/accelerate

92 comments

[ 2.9 ms ] story [ 172 ms ] thread
(comment deleted)
10/10 you're doing god's work my friend, can't wait to spend some time this weekend to try and understand what's going on here. I can't overstate how much I value small libraries. I can't think of a faster way to learn about a concept than to step through someone else's barebones implementation.
Thanks! Indeed, I agree that the project has an educational aspect and value. For me, it helped me get a better understanding of the neural network layers involved in the transformer model. Also, it was a good playground to practice my low-level optimization techniques. I guess another cool thing was that with the help of the community, we came up with a faster way to evaluate the Encoder (at the cost of some accuracy), which ultimately enabled the WASM and RPi4 examples (see #137 if interested in the discussion).
I liked reading the different implementations of the low-level tensor ops (simple C/AVX/AVX2/WASM128bit/ARM-NEON) -- it will help me learn about how to use x86 ASM. Thank you for writing this! Do you have any other recommendations/examples on how numerical code can be optimized via SIMD routines?
I don't have other recommendations as I am a novice myself when it comes to SIMD. I think the multiplication routines in `whisper.cpp` are relatively basic - dot product and fused multiply-add. With a few trial and errors I came up with these implementations - not sure if they are optimal.
I don't fully understand what's going on behind the scenes, but I tried to use the repo some days ago (guess there's even a new model now) and everything seemed very simple to build to try so thank you for your amazing work.
Awesome work. Curious whether the whisper model could be ported to tinygrad and how the performance would compare to your implementation.
Thanks! I'm also interested in seeing a CPU comparison against tinygrad. From what I've seen, tinygrad already utilizes the AMX coprocessor, so I expect to have comparable performance between the 2 on Apple Silicon.
Why not Rust? C++ is obsolete
I wonder how ChatGPT3 would respond to this?

If they crawled /r/programming and hackernews, it should be annoyed and come up with something along the lines of "fork it and write in in Rust yourself", "C++ has lots of inertia", "his main stack is C/C++", etc

While there is no pure rust model yet, there is a crate with bindings to the cpp model (probably not to this one).
This kind of comment is exactly what Rust skeptics are talking about when they are referring to the toxic Rust community.
Offline models, especially speech recognition, is a game changer for many apps.

A fully CPU based implementation, simple enough with minimal dependencies is also something that helps tremendously reduce the initial friction and enable potential low-cost applications.

Excellent and impressive work, can’t wait to try this thing at home.

> about 2-3 times faster compared to the current PyTorch implementation

This is surprising to me. Is this about CPU only? Then it would make sense.

Also, is there a particular reason why the whole code is basically 2 massive files (3k and 8k lines respectively)?

I am not very good with Python, so there is some chance I am doing something wrong. But my explanation of the results that I get is that PyTorch currently does not fully utilize FP16 + Metal or AMX when running on Apple Silicon. In contrast, my implementation stores the weights in 16-bit floating point precision (FP16) and also utilizes the AMX coprocessor through the Accelerate framework. As I mentioned in OP, the latter is very efficient for doing the matrix multiplications. According to my experiments, it is comparable in performance to running them on the Apple GPU via Metal.
This is awesome. I'm sure the fact that most ML models require an insane mess of Python packages is holding applications back.

Hook this up to ChatGPT and you've got something better than Google Assistant with almost no work.

(You can tell ChatGPT an API, and ask it to generate a script in response to a voice assistant query.)

I've been watching this repo pretty much since the beginning, and the amount of work you've achieved is incredible.

I've started tinkering with the code about last week and despite knowing nothing about C/C++, I was able to make some edits to fit my use case, and connect it to a custom Python front end (I initially tried to use Qt in C++ but struggled so much to get to it to compile that I've switched to Python instead). This probably means your code is very clean and well documented.

It's a game changer in terms of accessibility: it can caption almost anything in live!

I'm very grateful for the effort that you've lead. Thank you ggerganov, and thanks to everyone who contributed.

This is very cool. Excellent work!

Perhaps in combination with https://www.npmjs.com/package/peertube-plugin-transcription your port of Whisper could be used for generating subtitles for videos in PeerTube?

I just recently set up a PeerTube instance of my own and uploaded my first video on it ("No Brain Required - ChatGPT solves Advent of Code in Rust, episode 1", https://video.nstr.no/w/6z7PxB4J92H3NHhgMmfYVw)

I want to try and make use of your port of Whisper on my PeerTube instance, so that I can have subtitles generated for my videos on it :D

I love this! That talk.wasm package has a lot of potential…
This is my favorite example! I recorded a few video demonstrations where I talk with the AI, but they all sounded very cringe. So, the flagship video demo of talk.wasm is currently of 2 browsers talking with each other, which I think is not as impressive. If somebody gets this running and manages to record a conversation - would be happy to hear it!
This is really outstanding work, thank you so much for doing this and open sourcing it! I'm sure this will enable many applications in the future!
(comment deleted)
Great work. It's a real bowl of fresh air coming from huge framework that use cuda.

So many different cuda version, with each framework using its own, that all rely on a different driver, and everything needs a new version every 3 months and takes ~10G, (and don't even talk about cudnn needing some manual logged-in install).

Here everything is just two files. For embedded system that don't have a GPU it's perfect.

Here the parallelization and vectorization has been done by hand, but there is a glimmer of hope coming from the side of various compiler projects :

Here is an interesting intel project that does the parallelization and vectorization automatically for different architecture that's definitely worth a look : https://ispc.github.io/ispc.html

For the auto-differentiation when I need performance or memory, I currently use tapenade ( http://tapenade.inria.fr:8080/tapenade/index.jsp ) and/or manually written gradient when I need to fuse some kernel, but Enzyme ( https://enzyme.mit.edu/ ) is also very promising.

MPI for parallelization across machines.

100%.

So much more practical to hack around and or build small apps.

> MPI for parallelization across machines.

Some things never change.

Ditto about the CUDA and cuDNN part. My project that was running fine for the past 4 years just "died" after a colleague's oversight on upgrading the GPU(1080Ti -> 3090) which isn't compatible with the new cuDNN. It is just too much of a hassle maintaining that *expletive* jargon so I did the wise decision to kill it.
It's very cool, I love the talk.wasm example.

It looks like it will open a new application scenario for low-performance hardware products.

This is really cool, and I have been meaning to get my hands dirty with Whisper

I looks like you’ve definitely maximized the parallelization on the CPU/AMX here, but have you tried getting it to run on the GPU or the Neural Engine? I love the portability, but I feel like you would get a massive parallelization boost while dramatically cutting energy consumption.

I did some experiments with adding Metal Performance Shaders support, but the performance that I achieved was only marginally better compared to the one I get when using just the Accelerate framework (there is an unmerged PR with the tests).

Honestly, I am bit confused with all the different types of processing units available on Apple Silicon. If I understand correctly, we have: CPU, GPU, AMX coprocessor and Neural Engine on a single chip. I don't fully understand how these interact with each other. Can we use them all at the same time, or would there be some penalties? I'm interested in finding some resources/information on the topic.

OpenCL would be appreciated much... Opens the door to use this on many more low powered devices but, it could be very difficult as you have already mentioned.
You are correct, in that those are the four

My understanding is that the AMX is more tightly wound with the CPU, ultimately being accessible via an instruction set (https://github.com/corsix/amx), and it is useful if you need to do matrix multiplications interleaved with other CPU tasks. A common example would be a VIO loop or something where you want that data in the CPU caches.

The GPU and Neural Engine are not that – they take some time to set up and initialize. They also can parallelize tasks to a much higher degree. The GPU is more generalizable, because you can write compute shaders to do anything in parallel, but it uses a lot of resources. I'll have to check out the PR to see how exactly the MPS implementation matches up with the task at hand, because you could also consider writing Metal compute shaders by hand. Even if the performance is not much better, the CPU is free to do other things.

I know the least about the ANE, but it has specific hardware for running ML models, and you have to process the weights ahead of time to make sure they are in the right format. It can run ML models very efficiently and is the most battery friendly.

I suggest looking into Halide as it will make trying different paths much easier (https://halide-lang.org/).

I haven't looked at your code closely so can't say with certainty it would be the right fit but worth a look.

I've been watching this repo evolve since it was created. Seriously exemplary work Georgi.
Awesome work.

Honest question, do you think with this marked improvement it could be worth making a wrapped library for this c/c++ version in python - for example like numpy?

I think PyTorch will very soon catch up in performance once Apple Silicon gets properly supported. On x86, I don't observe very big performance improvement compared to current PyTorch. So overall, not sure if it is very worth it, although it's not hard to wrap it thanks to the C-style API.
I vouch for this. Pretty solid and keeps improving. The OP is in the class of Magic Wizards of programming like Fabrice Bellard!

There are frequent updates and performance improvements. There is also a small community of active users around this.

All most all feedbacks get implemented and the OP is very responsive.

The OP made it possible to do state of the art voice recognition without the PyTorch baggage and in C/C++, pretty incredible! Its one of those rare high value projects.

Very grateful for this project and respect to the OP!

Some day if a ChatGPT open version becomes available, this could mean voice assistants that speak sense and understand the human - as long you have a beefy machine.

The current efficiency is pretty surprising, even on a low spec device it performs faster than real time.

I don't know what to say. But I'm blown away.

I expect to see more magic from the OP in future.

He has even a project for a cool sound modem that works over ultrasonic! Not new stuff, but the implementation is the most robust I have seen.

I recommend hackers here to check out his other project too and maybe contribute with testing and patches and stuff!

Yup this is so magical. I've always felt there was something off about requiring setting up what is essentially a pytorch/ml dev environment everytime end users "just" want to run inference.

A single binary that does this all w/o the python stack is just incredible!

edit: Got it going in 1 min!

I grabbed the prebuilt artifacts (windows)

- https://github.com/ggerganov/whisper.cpp/actions/runs/363552...

Then downloaded ggml-base.bin (148mb) and put it in models/ggml-base.en.bin

- https://huggingface.co/datasets/ggerganov/whisper.cpp/blob/m...

Ran it and everything worked! Amazing. Note that only the large (3GB) whisper-v2 is available at the moment, but haven't seen any errors yet from the older small ones. Wild.

Can you expand on your steps a bit more? I've never used Github Actions which seems like step 1. Not sure how to get an installer.
Ok nevermind, figured it out, it requires login. Then the archive is at the very bottom of the page.
Hi Georgi,

I am experimenting with your code now. Is there a way to force Whisper to only consider a limited vocabulary and then respond with confidence levels? I am working on an app where it is important to restrict answers and I would like to know how confident that a response is one of a set of words. If the answer could be word A with a confidence level of 95% and word B with a level of 50%, I would want to know that so that I could perform context verification.

Thanks!

Bill

Hi, it's not obvious how to achieve this, but it feels it could be done. I think all the "tools" are available in the existing interface in `whisper.h` - for example, `whisper_get_probs()` gives you the probability for each token.
A follow up on this - I came up with an interesting strategy to achieve this. Still a prototype, but I think it looks very promising:

https://github.com/ggerganov/whisper.cpp/pull/271

The source code is in the `command.cpp` and I will soon write some more details how it works. If you give it a try, definitely let me know if it worked for you.

Amazing, even more so that you rolled your own inference and didn’t use ONNX runtime. Thanks so much for sharing.
There are tons of open source STT models, what makes whisper so valuable? I especially don't get it on mobile, where the native STT built into the OS is now real-time and includes punctuation (at least for iOS). I love the open-source approach to the model, but it didn't strike me as particularly better than other open-source or built-in models.
In my limited testing, I've found Whisper to be much better (accuracy-wise) than other STT models.
For some reason, I recalled Whisper to be on-par or slightly worse than the other open source ones, but much better across languages. I appear to be wrong.
Just curious on What open source ones did you compare them with ? do you mean WER was high or ?