I indexed 669 GB of my GoPro videos using my M1 Max computer and local ML models

439 points by iliashad ↗ HN
TLDR: I had 2,207 GoPro videos, and I need to rewatch them to find interesting moments from my cycling journey. I built a project to index them locally on my M1 Max using open-source ML models, search for those moments, and send the best clips straight to my DaVinci Resolve timeline. I indexed 628 videos (668.68 GB, 15h 13m 18s of footage duration), more details in the metrics table in the last section of this article.

Full article: https://iliashaddad.com/blog/i-indexed-669-gb-of-my-gopro-videos-using-my-m1-max-computer

50 comments

[ 2.9 ms ] story [ 59.9 ms ] thread
This would fit most best as a “Show HN:” post :)
Does it work for porn collections too?
Why it’s always the same question? Hahah. I posted my project over Reddit and I got the same one hahah
(comment deleted)
I would love your feedback and suggestions for new improvements or features you wanna have, either in the source available version, the desktop app or blog post itself?
I was surprised to learn that the

    M1 Max CPU is an ARM/SoC, comparable to an 11th gen Intel i9
Do I have it right? Would Windows ARM performance be similar for those cpu?

ref: https://www.cpubenchmark.net/compare/4585vs4245/Apple-M1-Max...

“Comparable” is maybe true if we are talking about single core performance, but for memory bandwidth, the M1 Max is about 8 times faster. Wider bus, lower latency, not even close.
To your question, I can’t deny or confirm that because I didn’t tried it this project over a Windows machine yet or a machine with this config
No comparison. M1 Max has 400GB/s RAM bandwidth while Snapdragon X2 Elite, the latest and greatest , has 228GB/s RAM bandwidth.
it is possible to use apple gpu with containers. either with podman + runkit + recent mesa or with recent vllm-metal from docker https://www.docker.com/blog/docker-model-runner-vllm-metal-m...
I was looking for a solution for this issue of running docker containers over MPS and utilizing their GPU power. I think this project will be the solution for it, I’ll try it very soon and add support for it. Thank you, much appreciated
Grab frames, lower res, classify, combine meta data. Write to sql
Not really. Grab frames, lower res, classify, combine metadata, transcribe the audio, convert those data (text, visual and audio) to embedding, save them over a vector DB and SQL DB. Which helped me to do semantic search, RAG, search using a screenshot of the video to find the exact the moment in the video plus search using an audio file as well. And other features unlocked with vector DB
DaVinci 21 has indexing built-in (AI IntelliSearch). Not to diminish the work you did, but this is now available to many users (probably only Studio users since it has AI in the name)
(comment deleted)
I’d like to see embedding of actual video clips become practical in this type of workflow.

Frame level embedding it covering a lot, but can miss out on a lot of action related searches.

Now this ^^ is an awesome use case!
if anyone is interested in searching large video collections local and offline I suggest taking a look at Jumper https://docs.getjumper.io

comes with some nifty features like NLE- integrations, people search, MCP, API etc

Disclaimer: one of the co-founders

I have an RTX 5090 card but it only has 32 GB RAM, can something like this work on my machine?
Something I've enjoyed more than I expected is Google and Apple photos sending me photo memories and compilations of various things in my life and my kids lives over the last decade.

I'm really bullish on taking more video of my kids, with the thought that it will become easier and easier for AI to put them into little compilations I can enjoy later.

I think the Apple stuff is done 100% on device.

Google loves scanning stuff on in the cloud though.

> Then, run the frame analysis pipeline, which will divide the video into separate video scenes (1s each, or 1fps) > (…) > Frames analyzed 57,537

Aha, it makes total sense. This number sounds much more reasonable than “669 GB”, since the actual total size of processed frames would be like 10-30 GB.

(Not downplaying anything. Doing-at-home always requires some math on practicality)

> Total compute time 67h 40m 42s

I’m just curious tho — is there any paying options that can accelerate this kind of process? Just spin up GPU instances?

this is really cool. was looking to do something similar on mbp 64gb
Cool build but the example videos you provide at the end are . . . not what I would hope for when thinking about the highlights of 2000+ videos of biking? For example the dog barking video only has one scene repeated two or three times and it's five seconds long?
> Many of the videos I captured amazing moments, and sometimes it's kind of hard to watch the full videos to get those moments.

Yep. I had the same problem.

> Then, run the frame analysis pipeline [...] I have a face recognition plugin using my custom faces data, object detection, on-screen text, shot type, and scene description [...] we will have three vector DB collections that have all the information about our videos, like video location metadata, camera name, faces recognized, objects detected, on-screen text, transcription, description of each scene, and many more [...] we can get better indexed data if you use the advanced mode indexing to use the Qwen2.5-VL-7B-Instruct model to understand and describe your video much better, but at a slower indexing speed

Yeah, uhm... ok :)

If anyone else has a similar problem, the real solution is as follows:

1. When recording, if you witness an interesting moment worth saving later, press the power button — this will mark the current moment in the video as a chapter.

2. Find the chapters later when editing and cut them into clips.

3. You're done :)

This has two main benefits over the insanity above:

1. It's trivially simple instead of insanely complex and inefficient.

2. It will reliably catch all the stuff you find interesting, since you're the one doing the marking.

The downsides:

1. Doesn't work retroactively.

2. It may miss interesting stuff if you miss it at the time as well.

3. Only works for this use case.

4. Nerds won't salivate over your usage of cutting edge tech.

can vlm be used instead or it's too heavy and slow