Show HN: I built a local Elixir/Python pipeline to curate 14,000 RAW photos (qwelian.com)

5 points by qweliantanner 2mo ago ↗ HN

I had 14,000 photos sitting on a drive and wanted an excuse to play with local vision models and Elixir/Phoenix. I originally tried to get LLaVA to tell me if a photo was 'good' or matched my style, but quickly learned that LLMs have terrible taste. I ended up demoting the LLM to just extract metadata, and built a custom CLIP/Ridge Regression pipeline to actually learn my preferences based on how I rate things.

The stack is Phoenix/Oban on the orchestrator side, and Python/FastAPI/Instructor for the AI workers. Happy to answer any questions about the architecture, fighting with local RAW file ingestion, or the pains of Pydantic validation with open weights.

2 comments

[ 3.0 ms ] story [ 11.9 ms ] thread

aleksuix 2mo ago ↗

what was the biggest I/O bottleneck? I would assume RAW files are really heavy?

qweliantanner 2mo ago ↗

Locally this isn’t an issue but over a network call it would introduce some latency. It happens pretty fast tho. I’d say it’s a bottleneck because it happens too fast. I don’t have enough RAM to perform the next step which is metadata extraction leading to a lot of files being constantly processed.

It would be nice to be able to convert RAW files natively in elixir in tho