I've actually done that same concept a while back when whisper.cpp came out. A significant challenge is sane paragraph segmentation, as even humans don't often agree on the best place for a line break. I wonder what approach you've used.
I've adopted a very simple approach: 80 words per "paragraph". I am now experimenting with computing the embeddings of each sentence and try to detect topic segments. But the simple approach yields pleasant segments AFAIK.
This is absolutely amazing, the fact that you can click on text and it takes you immediately to the part of the audio where that text is being said is great.
I wish the video was shown as well, but other than that, excellent work!
13 comments
[ 2.6 ms ] story [ 44.8 ms ] threadYou can share the link as it contains the list of timestamps corresponding to your highlights.
So others or yourself can see them again.
I wish the video was shown as well, but other than that, excellent work!
Perhaps split in 2 columns on wide displays (video left, transcript right), and portrait (e.g. mobile phone), video on top and transcript below it?
Demo in my tweet https://twitter.com/ldenoue/status/1788231159140196804
Example: https://www.youtube.com/watch?v=0AaNT7XO41I https://www.appblit.com/watch?v=0AaNT7XO41I