This project provides high-level functionality on top of the popular LLMs, allowing you to decouple the models and switch between them based on your use case.
The available functionality:
- Chatbot: define the provider as chatGPT or llama chat.
- Evaluation: Run evaluation across multiple models by sending a query and array of target answers. This will call the models in the background, generate the vector, and compare the distances. This supports cohere, openai, replicate, and sage maker models.
- Semantic Search: Apply search beyond the keywords using vectors.
- Direct models: directly access the model providers like Sage maker llama or hugging face.
- Offline model loader: under development.
The micro service published to docker hub:
```
docker pull intellinode/intelliserver:latest
docker run -p 80:80 -e API_KEY=$API_KEY -e ADMIN_KEY=$ADMIN_KEY intellinode/intelliserver:latest
```
I am working to add more features to the micro-service, Let me know which functions you think going to be useful.
1 comment
[ 3.4 ms ] story [ 9.3 ms ] threadThe available functionality:
- Chatbot: define the provider as chatGPT or llama chat.
- Evaluation: Run evaluation across multiple models by sending a query and array of target answers. This will call the models in the background, generate the vector, and compare the distances. This supports cohere, openai, replicate, and sage maker models.
- Semantic Search: Apply search beyond the keywords using vectors.
- Direct models: directly access the model providers like Sage maker llama or hugging face.
- Offline model loader: under development.
The micro service published to docker hub:
```
docker pull intellinode/intelliserver:latest
docker run -p 80:80 -e API_KEY=$API_KEY -e ADMIN_KEY=$ADMIN_KEY intellinode/intelliserver:latest
```
I am working to add more features to the micro-service, Let me know which functions you think going to be useful.