HN Explorer

Ask HN: Is anybody selling API access to HuggingFace models?

3 points by JuanPosadas ↗ HN
I've used the OpenAI and GCP chat-bison APIs with great success, but I'd like to put my money into more ethical companies with free-as-in-freedom-er models.

I don't want to fickle around with buying or renting GPUs or installing random LLM scripts that don't work 90% of the time.

I also don't want to pay $XX/month subscriptions, I want clean API billing like OpenAI and GCP.

What exists? Is anybody doing anything?

7 comments

[ 2.7 ms ] story [ 28.8 ms ] thread
smoldesu
Many of the "free as in freedom" models are not trained or licensed for commercial use. For example, selling access to a LLaMA API would be a direct violation of the license the model was provided with.
version_five
I don't think you mean free as in freedom. Maybe freemium, I can't think of a closer short description. They don't cost anything but you're not allowed to use them for most stuff either.

Otoh, there are plenty of Apache and MIT licensed models. For example Open LLaMA and Falcon, Dolly, etc. HF lets you filter by license.

smoldesu
You're right. In fairness, the actual code for training LLaMA is "FOSS" technically. It's just that the model license (and weights afaik) are noncommercially licensed.

I guess it would be cool to see a paid Falcon/Dolly API, but in my experience those models get smoked by a good LLaMA 7b tuning. You would really have to hate OpenAI to seek out a non-GPT3 solution here...

version_five
Sort of related, I'm surprised there are no "edge" GPUs available in the style of cloudflare workers or AWS lambda. I heard that cloudflare was going to do something like this but it never materialized. It would be great (as an alternative to API access) to be able to pay for a worker GPU that you only used when doing inference. I know there are technical challenges to that, I wonder of it's still too hard to make something responsive enough to be useful?
ilaksh
What about replicate.com and RunPod?
version_five
Thanks! I was looking for something like this last year and couldn't find anything I'm glad to see progress in this space!
brucethemoose2
This is not really an answer, but I really like the "AI horde" API setup.

https://aihorde.net/

Namely, you have a pool of gen AI volunteer workers constantly chewing through API requests. Host requests are prioritized, but the "downtime" of your AI worker instance is spent fulfilling other requests, which earns karma that can be used to prioritize requests from other hosts should you need more throughput.

Economically, this is way more efficient than self hosting or "renting" an instance that will be idle between requests, and you aren't paying a massive tax for a proprietary API.