looks nice, congrats on open sourcing it cztomsik! Good to have another reference implementation, excited to dig through and check out how you're integrating llama.cpp and SwiftUI. I went with an architecture based on hitting examples/server on localhost.
Would you be interested in adding with otherbrain [0] to help build an open human feedback data set? Happy to help with PRs if so.
If you're interested in supporting open model training, we'd love to have you!
Makes sense, thanks for considering (and fwiw I'm also building with a private AI / offline philosophy). In my implementation, the compromise is 1) being super loud with an alert warning the user when they're sharing feedback and 2) providing a setting where the user can turn off feedback buttons. I had gotten user feedback around wanting to tell the AI when it was doing a good/bad job and this seemed like a privacy preserving compromise that could actually improve models. Anyway, good luck and congrats again!
Kudos for saying no. I can't believe we're living in an age where open source developers are being asked openly to leak the private conversations of their users to the public.
It should work fine if you're on Apple silicon, everything else is unfortunately second-class citizen, because I don't have any other devices and it's currently just me working on it. But if you'd like to maintain the windows, linux or intel-based mac releases, ping me on discord and I'd be happy to discuss and to accept PRs.
ah, got it. yeah the llama.cpp speeds I've seen are like 5-10 tokens/second on intel but I've been impressed with the relatively ancient hardware llama.cpp works on (2017 iMac with 8GB RAM at 5.5 t/s with a 7B model!).
LM Studio is great :) Also Faraday is great. But neither of them is open-source and this is going in a little different direction too (QuickTools, Workflows, there will be RAG and Vision support)
We changed the URL from https://tomsik.cz/posts/ava-oss/ to the project page, because it doesn't look like this project has been discussed on HN before. Readers might want to look at both links!
Yes, it's at localhost:3002/api/ (and api/*.zig files respectively)
The Workflows screen is feature-flagged for now and only visible in the DEV build.
Yes, that's the whole idea behind QuickTools - to make repetitive tasks available quickly. It's also supposed to be in the main menu (drag-drop, reorderable, etc.) but I didn't have time to work on that yet.
27 comments
[ 0.20 ms ] story [ 69.3 ms ] threadhttps://github.com/cztomsik/ava/blob/main/LICENSE.md
Would you be interested in adding with otherbrain [0] to help build an open human feedback data set? Happy to help with PRs if so.
If you're interested in supporting open model training, we'd love to have you!
[0]: https://www.otherbrain.world/human-feedback
yes, I know it definitely works, so it has to be some stupid mistake but it's impossible to reproduce it without actual intel mac :-/
I had enough trouble with supporting monterey, which also does not work 100% AFAIK
The big news here is that it's now open-source: https://github.com/cztomsik/ava
In the video @ https://github.com/cztomsik/ava, I see "Workflows" but not in the app. What are these, and are they coming to the app?
Are quick tools basically the same as saving a playground prompt with variables?
The Workflows screen is feature-flagged for now and only visible in the DEV build.
Yes, that's the whole idea behind QuickTools - to make repetitive tasks available quickly. It's also supposed to be in the main menu (drag-drop, reorderable, etc.) but I didn't have time to work on that yet.