Hi HN! I've been building Dayflow, a macOS app that automatically tracks what you're actually working on (not just which apps you have open).

Here's what it does:

- It creates a semantic timeline of your day;

- It does it by understanding the content on your screen (with local or cloud VLMs);

- This allows you to see exactly where your time went without any manual logging.

Traditional time trackers tell you "3 hours in Chrome" which is not very helpful. Dayflow actually understands if you're reading documentation, debugging code, or scrolling HN. Instead of "Chrome: 3 hours", you get "Reviewed PR comments: 45min", "Read HN thread about Rust: 20min", "Debugged auth flow: 1.5hr".

I was an early Rewind user but rarely used the retrieval feature. I built Dayflow because I saw other interesting uses for screen data. I find that it helps me stay on track while working - I check it every few hours and make sure I’m spending my time the way I intended - if I’m not, I try to course correct.

Here’s what you need to know about privacy:

- Run 100% locally using qwen2.5-vl-3b (~4GB model)

- No cloud uploads, no account

- Full source available under MIT license (https://github.com/JerryZLiu/Dayflow)

- Optional: BYO Gemini API key for better quality (stored in Keychain, with free-tier workaround to prevent training on your data)

The tech stack is pretty simple, SwiftUI with a local sqlite DB. Uses native macOS apis for efficient screen captures. Since most people who run LLMs locally already have their tool of choice (Ollama, LLMStudio, etc.), I decided to not embed an LLM into Dayflow.

By far the biggest challenge was adapting from SOTA vision models like Gemini 2.5 Pro to small, local models. My constraints were that it had to take up <4GB of ram and have vision capabilities. I had to do a lot of evals to figure out that Qwen2.5VL-3B was the best balance of size and quality, but there was still a sizable tradeoff in quality that I had to accept. I also got creative with sampling rates and prompt chunking to deal with the 100x smaller context window. Processing a 15 minute segment takes ~32 local LLM calls vs 2 Gemini calls!

Here’s what I’m working on next:

Distillation: Using Gemini's high-quality outputs as training data to teach a local model the patterns it needs, hopefully closing the quality gap.

Custom dashboards where you can track answers to any question like "How long did I spend on HN?" or "Hours until my first deep work session of the day

I'd love to hear your thoughts, especially if you've struggled with productivity tracking or have ideas for what you'd want from a tool like this.