So, this project consists of a ~175 line README and a ~500 line Python program that glues yt-dlp and Kroko together. Neat.
I guess if it encourages you to install and figure out how to use ffmpeg, yt-dlp, kroko, numpy, and onnx that's a good thing. Sometimes just knowing a thing is possible is a huge benefit.
Had Claude test it out on 3 videos. Worked at 5-8x realtime. The beauty of it is that it works on all videos, not just the one with transcripts. Combine it with YouTube search and LLM takeaways from transcripts, and you have super-efficient content consumption. There are SaaS products that charge 1 cent per video for those with transcripts. There is a viable product in here somewhere, methinks.
Wouldn't it still be more efficient to do GPU transcriptions anyways? is this something we could actually put the effectively useless NPUs to use in modern laptops?
If someone is interested, this is my supershort zsh/bash scripts that I keep in .zshrc for doing the same thing using plain whisper.cpp, ffmpeg and yt-dlp (`brew install whisper-cpp yt-dlp` for Mac); I output it in vtt format (subtitles) though, but it's easy enough to change it to txt.
yt_to_srt() {
local url="$1"
local output_base="$2"
local language="${3:-en}"
yt-dlp -x --audio-format wav --postprocessor-args "-ar 16000" -o "$output_base.wav" "$url"
whisper-cli --language "$language" --model "$WHISPER_MODEL" --split-on-word --max-len 65 --output-vtt --output-file "$output_base" --file "$output_base.wav"
rm "$output_base.wav"
}
file_to_srt() {
local filepath="$1"
local language="${2:-en}"
local filename=$(basename "$filepath")
local filename_no_ext="${filename%.*}"
local output_base="$filename_no_ext"
local temp_wav="$output_base.wav"
ffmpeg -i "$filepath" -vn -acodec pcm_s16le -ar 16000 -ac 1 "$temp_wav"
whisper-cli --language "$language" --model "$WHISPER_MODEL" --split-on-word --max-len 65 --output-vtt --output-file "$output_base" --file "$temp_wav"
rm "$temp_wav"
}
plus additional bootstrap script for large-v3-turbo model from my chez-moi dotfiles:
#!/bin/bash
# Download whisper.cpp models from Hugging Face (runs once per machine).
set -euo pipefail
MODELS_DIR="$HOME/whisper-models"
BASE_URL="https://huggingface.co/ggerganov/whisper.cpp/resolve/main"
MODELS=("ggml-large-v3-turbo.bin" "ggml-tiny.bin")
mkdir -p "$MODELS_DIR"
for model in "${MODELS[@]}"; do
if [ ! -f "$MODELS_DIR/$model" ]; then
echo "Downloading $model..."
curl -L --progress-bar -o "$MODELS_DIR/$model" "$BASE_URL/$model"
else
echo "$model already exists, skipping."
fi
done
echo "Whisper models ready at $MODELS_DIR"
Am I a bit thick, but first we created this amazing way to transfer any text very cheaply and fast over network, then we (well, I think it was Meta and Google) decided that no, everything must be a video, then we added subtitles and AI-transcriptions to those videos and now we just dowload transcriptions of those videos presumably to feel LLM to make summaries of them in order to… Read. Them.
Good point! I haven't found a faster way to consume info than reading. But depends on the type of learner you are (visual, auditory, hands-on/interactive, etc)
Tis is very simple and very cool! Just installed it on my Hetzner box where I run a remote controlled local agent so now I can basically chat/email a video link to get a summary and/or ask questions. The only issue was YouTube's PO Token requirement (web/mweb clients refuse to serve formats from datacenter IPs without a valid Proof-of-Origin token.) So I had to find a client that still work without PO Token first. Thanks for sharing!
Very cool, I'm also working on a captioning/subtitling project for the lecture recordings for the university I work at.
My biggest challenge is finding a proper language model that is fast enough and accurate enough since I have to caption about 600 hours of video per week and I preferably want to run all of this on a tiny server (2 cores 4 GB memory). This tool could easily do that with the kroko model but I'll have to test if the accuracy is good enough.
Also in my own scripts I'm using ffmpeg to download just the audio of the videos that I want to caption, which saves a lot of bandwith and speeds up the whole process. As far as I can see this tool doesn't do that, that would be a nice functionality to add, plus an option to turn the output into a working .srt file.
Added Diarization / Speaker Separation that is fast and CPU only. Thank you all for the great feedback and support. PRs welcome!
yapsnap "https://www.youtube.com/watch?v=NzKJ-xO-VhE" --diarize
SPEAKER_00 [00:00]: Welcome to the show.
SPEAKER_01 [00:03]: Glad to be here, thanks for having me.
SPEAKER_00 [00:08]: Let's get started.
So just tested this on a bunch of videos. It's crazy accurate (not 100% obviously), fast and resource efficient.
Not only was I testing the stt part, but also, timestamps and speaker identifications. Tested on 3 different videos, local and online both.
Timestamps were precise down to sub 500-ms level even on longer 20+ minute videos. Speaker identifications worked equally well. My old M1 Air didn't hang a single bit while the transcription was going on.
31 comments
[ 0.17 ms ] story [ 71.5 ms ] threadI guess if it encourages you to install and figure out how to use ffmpeg, yt-dlp, kroko, numpy, and onnx that's a good thing. Sometimes just knowing a thing is possible is a huge benefit.
sudo apt update && sudo apt install -y ffmpeg python3-pip python3-venv && git clone https://github.com/kouhxp/yapsnap.git && cd yapsnap && python3 -m venv ~/yapsnap-venv && source ~/yapsnap-venv/bin/activate && pip install --upgrade pip && pip install .
On a 32GB ThinkPad X13, a 21 minute YouTube video was processed by yapsnap under 2 minutes.
Very well done!
I think I’m gonna go read a book.
My biggest challenge is finding a proper language model that is fast enough and accurate enough since I have to caption about 600 hours of video per week and I preferably want to run all of this on a tiny server (2 cores 4 GB memory). This tool could easily do that with the kroko model but I'll have to test if the accuracy is good enough.
Also in my own scripts I'm using ffmpeg to download just the audio of the videos that I want to caption, which saves a lot of bandwith and speeds up the whole process. As far as I can see this tool doesn't do that, that would be a nice functionality to add, plus an option to turn the output into a working .srt file.
Not only was I testing the stt part, but also, timestamps and speaker identifications. Tested on 3 different videos, local and online both.
Timestamps were precise down to sub 500-ms level even on longer 20+ minute videos. Speaker identifications worked equally well. My old M1 Air didn't hang a single bit while the transcription was going on.
---
1. Here's one from a single speaker video (https://www.youtube.com/watch?v=-X6YzlY_8tM): https://pastebin.com/vPVPNnne
2. A shorter with up to 4 different speakers and mixed, complex scene/narration changes (https://www.youtube.com/watch?v=4tASl0auPOg): https://pastebin.com/iHZZD8Qe
--
My zsh shorthand: `alias transcribe="yapsnap --timestamps --diarize "`