HN Explorer

6 comments

[ 4.6 ms ] story [ 31.5 ms ] thread
littlestymaar
> To use aiq label, you'll also need an OpenAI key. The other commands can be used on their own with no API key, as they run locally on your computer. Set the key as an environment variable

There are plenty of providers and self-hosted solutions that expose an OpenAI-compatible API so instead of just asking for on OpenAI key you could ask for an API end-point URL (optional with OpenAI as the default if you want to keep the current UX) and a authentication key (optional as well since self-hosted stuff don't need it).

coolspot
They provide that with base URL parameter:

--api_base_url: You can use this flag (or the OPENAI_BASE_URL environment variable) to pass a different API URL from the default "https://api.openai.com/v1". This allows you to use other OpenAI-compatible LLM providers like TogetherAI, self-hosted vLLM or Ollama, etc.

andersonbcdefg
Hello! Yes, the other person who replied to you beat me to it, but this already works with any LLM provider that is OpenAI-compatible, via providing a custom API base URL. For hosted providers, this includes Together AI, Mistral, and I think even Google has an OpenAI-compatible endpoint for Gemini now. This should also work with self-hosted options like vLLM and Ollama. I led with OpenAI in the quickstart as this is the provider that most people are likely familiar with, and easiest to get started.
svg7
It would be nice if you could report some accuracy metrics of this approach on well known text datasets, after hiding the label
andersonbcdefg
Great suggestion, I'll look into that. My expectation is that this library would not be state-of-the-art compared to training on labeled data (the intended purpose is building models where labels aren't available, if you have labels, it's obviously good to use the labels, ha). But it would be interesting to see how much of the performance is retained relative to training on the gold labels.
bschmidt1
`embeddings.json` this makes me happy