HN Explorer

Show HN: LlaMaKey – One master key for all cloud LLM/GenAI APIs (github.com)

6 points by forrestbao ↗ HN
Key management for multiple users and multiple cloud LLM/GenAI APIs is difficult to be both safe and convenient. Sharing keys among users risks leaking the key and makes it difficult to curb the leakage without interruptions. But assigning one key per user per cloud API results in too many keys to keep track of.

Meet LlaMa(ster)Key, the secure and easy solution for API key management: * For each user, one master key for multiple APIs. * The master key is unique to each user. Granting and revoking a user's access won't impact other users. * The actual API keys to authenticate with cloud APIs are never known to users. Never leak a key due to a careless intern. * No code change! Keep using the official Python SDK of the cloud AI APIs, including OpenAI, Cohere, AnyScale, Perplexity, HuggingFace (free-tier inference so far), and Vectara.

LlaMaKey works by taking advantage of the BASE_URL and API_KEY/TOKEN environment variables that the official Python SDKs of major LLM/GenAI APIs rely on. Your users only need to set them via the OS environment variables or a dotenv file, or many other ways.

Yes, LlaMaKey is a proxy deployable locally or on-premises. It's totally free under the MIT license.

The Github repo is at: http://LlaMaKey.ai (will redirect to Github).

Feel free to try it out and give us feedback. You can always reach us at hello@LlaMaKey.ai or simply open an issue at Github.

2 comments

[ 4.8 ms ] story [ 13.9 ms ] thread
xy-li
Amazing project!

I really appreciate this simple and good idea to manage API keys/tokens in the era of LLM. Personally, I'm often anxious about the safety of my own keys/tokens across several platforms. Recent news about some open-source projects that abuse the secret keys also recall me of the leakage concern. More details can be referenced in this post https://gist.github.com/win3zz/0a1c70589fcbea64dba4588b93095... and a relevant paper: https://arxiv.org/abs/2309.02926 Many risks are avoidable if the developer team deploys their service properly. I believe a good tool like this project is also an in-demand solution to reduce the concern and burden during their R&D stage.

forrestbao
Thank you very much! Appreciate your comment and encouragement.

I conceived this idea while working on a project with multiple people to evaluate multiple LLMs. Too many API keys floating here and there in the code. So I said, what if we could use only one master key for all the APIs and hide the real key from users. Then, LlaMakey was born.