Is it implied anywhere? That's a feature I'd love and also why I haven't bothered delving into LLMs very much; I didn't know there were any that could locally index your library and train on that data. I'd love to ask it a question and have it reference my local ebook library.
It probably doesn't. The only one that read PDFs for me was the Nvidia ChatRTX.
It would be easy to add modules from pip that do this but you'd have to code up the input pipeline. It's not terribly difficult but it is definitely not point and click.
For AI projects, afaict, 12k stars or forks is more akin to downloads than contributors & downstreams. GitHub is the app distribution, not just source distribution. I've been curious how to model this better..
It seems like that until you actually try to use them. Not many are actually polished, support formatting, history, and multiple endpoints. There's lots of trivial apps abandoned after a few days, but what are the actually functional, good quality alternatives to this one? (That don't pass your query/answer through a third-party for data collection)
I use https://www.typingmind.com/. It is paid, but I've found it to be a reliable front end to OpenAI/Claude/Google, supporting everything you mention. I haven't done any hyper detailed security audit but after watching network requests I'm pretty confident it's not sending my chats anywhere except to the relevant provider endpoints.
Considering how much I use it, I've found it to be well worth the cost. The creator is pretty on top of model/API changes.
i’ll second that recommendation… i use it through the SetApp store and i’ve been very pleasantly surprised by its documentation and ability to work with most services.
I got say I’ve been using LLM studio as it exposes the models in the ui as well as through a local open ai compatible server so I can test different models against my workflows locally.
Still hoping we'll eventually stop using Fibonacci to show off recursion, because that's one of those examples where the maths might be expressed as recursive relation, but the implementation should never be =)
Good AI would go "you don't want that, that's horribly inefficient. Here's an actually performant implementation based on the closed-form expression".
GPT4All makes it annoyingly difficult to run any other than their "approved" models. I'd like to kick the tires on a whole host of random GGUF quantizations on Hugging Face, please.
I've poked around the doc, not sure if Jan can do that better.
In the mean time, I use text-gen-ui (Oobabooga) as a back-end and have it run with `--api` to use the front end of my choice.
Unless I just can't find it, there seems to be no setting for customizing the prompt format for local models. You can edit the prompt itself, but not the format of the prompt or the subsequent messages. This would make using many models difficult, or give poor results, since they don't all use the same format.
I try some LLM on my notes and well... They was unable to give me insights that are hard to spot, like follow the flaw of notes identifying patterns, find similar notes from the past and so on. In ALL cases classic tags/riprgrep full-text search was far quicker and equally or more effective.
Long story short: LLMs might be useful on hyper big mass of information, like a new kind of search engine that try do achieve a semantic goal mimicking it. But not more than that IMVHO. Marginally LLMs might help computer-illiterate to manage their files, seen https://www.theverge.com/22684730/students-file-folder-direc... but I doubt they can go any further for the next 5+ years at least.
They've been very useful in quickly answering common questions using a too-large-to-manually-scan knowledge base in my experience at my job, and I don't consider myself or my colleagues "computer-illiterate".
That's follow the "might be useful as a new kind of search engine", though it might be a sign of an a bit messy KB. The issue of potential hallucinations however is still there so even such usage, a different search engine, demand extra attention.
It's not a free critic to those who have designed, implemented and trained LLMs, it's just the observation that practical usage is far less than the advertised one and it's still not much good. It's still an advancement, a good thing to have, the start of a revolution, but still far from being what many dreams.
It does make it easier for the end user who doesn't want to fiddle around with python dependencies, command lines, building C++ projects, etc.
Just install it, point it to a model, and go. Now you have a local LLM.
If you want something more, click the "start server" button and you have a local OpenAI compatible API which you can point more advanced front-ends to.
This looks awesome. Trying it out. Suggestion, can we please change the "Download Jan for PC" to perhaps just "Download" or "Download for Desktop" or whichever that makes sense but not "PC". I almost move away thinking this is Windows, thus not for us.
I recently stumbled on https://mindmac.app which is a non-subscription app that uses multiple AI tools (not just OpneAI). Looks Promising.
These are some of the really good ones. I'm tending more towards trying out the likes of MindMac just for the fact that I can plug and switch between multiple tools.
This looks great relative to others (very similar to MacGPT), and I particularly like how advanced settings are available but tucked away behind discoverable affordances.
It's interesting that you have team pricing.
Can the Team leverage shared system prompts and/or assistants from a OneDrive-for-Business (SharePoint) folder or GitHub repo?
If not, what makes it "Team" instead of just individual?
Hi. Actually I don’t have a pricing plan for teams yet. It’s still under (heavy) development. I changed my headline to reflect the direction I want to take this year (focus on teams)
And yes, some of my customers wanted team and collaborative features like shared prompt, internal plugins and integrations, RAG on internal documents…
But I haven’t launched these team features yet.
Are you interested in this? Would love to talk to you if it’s something you’re looking for.
From the screenshots it looks like there is an activation limit, with a maximum of four devices. Reading the license, I could not confirm this. Is there a limit, and if, what is the maximum?
Sorry for the confusion. I need to improve my pricing page.
The license is per user, and can be used on maximum 3 devices. I figured this is enough for most users. If you have more devices or need a custom license, please send me an email (my email is in bio)
What's the value proposition for TypingMind as a commercial product ($3500 to run locally for 5 seats)?
But let me contrast that last "native app" with Machato and MacGPT:
== Machato ==
Machato is feature-full for system prompts and transcripts, connecting to to OpenAI, Claude, and any "server" endpoint that's OpenAPI API compatible, and surfacing parameter and token settings per conversation right on your text entry bar. You can also point a given conversation to a local ollama endpoint such as Mixtral 8x7B and it works as well.
The best feature is the selective forking and suppression of exchanges within conversation threads.
MacGPT is highly integrated throughout MacOS, and works with either OpenAI key or a ChatGTP Pro login. It's quite similar to BoltAI mentioned elsewhere in this thread, but in addition to the OpenAI key based mode, also works with a ChatGPT Pro subscription in a ChatGPT web UI pop-up.
69 comments
[ 5.0 ms ] story [ 156 ms ] thread01 Run local AI or connect to remote APIs
02 Browse and download models
Considering how much I use it, I've found it to be well worth the cost. The creator is pretty on top of model/API changes.
honestly feels like site this was launched a couple of days too soon.
Good AI would go "you don't want that, that's horribly inefficient. Here's an actually performant implementation based on the closed-form expression".
https://www.youtube.com/watch?v=2akt3P8ltLM
https://gpt4all.io/
https://llm.mlc.ai/
I've poked around the doc, not sure if Jan can do that better.
In the mean time, I use text-gen-ui (Oobabooga) as a back-end and have it run with `--api` to use the front end of my choice.
Long story short: LLMs might be useful on hyper big mass of information, like a new kind of search engine that try do achieve a semantic goal mimicking it. But not more than that IMVHO. Marginally LLMs might help computer-illiterate to manage their files, seen https://www.theverge.com/22684730/students-file-folder-direc... but I doubt they can go any further for the next 5+ years at least.
It's not a free critic to those who have designed, implemented and trained LLMs, it's just the observation that practical usage is far less than the advertised one and it's still not much good. It's still an advancement, a good thing to have, the start of a revolution, but still far from being what many dreams.
Seems to be Llama.cpp via 'Nitro', which was discussed here before [2].
[0] https://github.com/vllm-project/vllm
[1] https://github.com/ggerganov/llama.cpp
[2] https://news.ycombinator.com/item?id=38887531
Just install it, point it to a model, and go. Now you have a local LLM.
If you want something more, click the "start server" button and you have a local OpenAI compatible API which you can point more advanced front-ends to.
I recently stumbled on https://mindmac.app which is a non-subscription app that uses multiple AI tools (not just OpneAI). Looks Promising.
Like the others in the comments, I've tried https://www.typingmind.com (via SetApp).
Sindre Sorhus have a pretty stable Native App https://sindresorhus.gumroad.com/l/quickgpt
These are some of the really good ones. I'm tending more towards trying out the likes of MindMac just for the fact that I can plug and switch between multiple tools.
Give it a try if UI & UX is important to you.
[0]: https://boltai.com
My use case is especially for my daughter so I can just plug in my OpenAI API and let her ask away.
It's interesting that you have team pricing.
Can the Team leverage shared system prompts and/or assistants from a OneDrive-for-Business (SharePoint) folder or GitHub repo?
If not, what makes it "Team" instead of just individual?
And yes, some of my customers wanted team and collaborative features like shared prompt, internal plugins and integrations, RAG on internal documents…
But I haven’t launched these team features yet.
Are you interested in this? Would love to talk to you if it’s something you’re looking for.
The license is per user, and can be used on maximum 3 devices. I figured this is enough for most users. If you have more devices or need a custom license, please send me an email (my email is in bio)
But let me contrast that last "native app" with Machato and MacGPT:
== Machato ==
Machato is feature-full for system prompts and transcripts, connecting to to OpenAI, Claude, and any "server" endpoint that's OpenAPI API compatible, and surfacing parameter and token settings per conversation right on your text entry bar. You can also point a given conversation to a local ollama endpoint such as Mixtral 8x7B and it works as well.
The best feature is the selective forking and suppression of exchanges within conversation threads.
https://untimelyunicorn.gumroad.com/l/machato
== MacGPT ==
MacGPT is highly integrated throughout MacOS, and works with either OpenAI key or a ChatGTP Pro login. It's quite similar to BoltAI mentioned elsewhere in this thread, but in addition to the OpenAI key based mode, also works with a ChatGPT Pro subscription in a ChatGPT web UI pop-up.
https://www.macgpt.com/
HN got any good LLM suggestions to run with this that are equivalent or better than GPT-3.5 / claude?
I'm looking to use its api with LLama Index