This is an open source GPT Code Interpreter with a special focus on data and privacy (by using local LLM models).
It’s built on gradio with guidance as the llm wrapper, and executes code against a local Jupyter kernel. Bridging between these 3 tools is the concept of the agent, which essentially runs the logical flow of the bot and is easily extendible for new strategies (eg. attempting to execute the code that’s written, if it errors, feeding it back in as a prompt for correction, etc.).
Right now the backend for full-local-mode is the `StarChat-alpha` model (License: BigCode Open RAIL-M v1). Unfortunately to use this model you must run this on a machine with a CUDA device with at least 24 GB of RAM. I am hoping to add GGML based models (CPU-local-mode) within the next week, so please stay on the lookout for that!
Also, if you’re looking to try it out with the best available 3rd party models, I also included the ability to use gpt-3.5-turbo and gpt-4 (reduces the privacy: the conversation and any data that shows up in the conversation is sent to openai) but at least the full data files and execution stays fully local.
You can try it out immediately by running the docker image, or by running it in colab notebook (examples in repo’s readme)
I’m open to any contributions, and am looking forward to seeing what people do with it!
1 comment
[ 4.4 ms ] story [ 15.7 ms ] threadI’m excited to share dataDM!
This is an open source GPT Code Interpreter with a special focus on data and privacy (by using local LLM models).
It’s built on gradio with guidance as the llm wrapper, and executes code against a local Jupyter kernel. Bridging between these 3 tools is the concept of the agent, which essentially runs the logical flow of the bot and is easily extendible for new strategies (eg. attempting to execute the code that’s written, if it errors, feeding it back in as a prompt for correction, etc.).
Right now the backend for full-local-mode is the `StarChat-alpha` model (License: BigCode Open RAIL-M v1). Unfortunately to use this model you must run this on a machine with a CUDA device with at least 24 GB of RAM. I am hoping to add GGML based models (CPU-local-mode) within the next week, so please stay on the lookout for that!
Also, if you’re looking to try it out with the best available 3rd party models, I also included the ability to use gpt-3.5-turbo and gpt-4 (reduces the privacy: the conversation and any data that shows up in the conversation is sent to openai) but at least the full data files and execution stays fully local.
You can try it out immediately by running the docker image, or by running it in colab notebook (examples in repo’s readme)
I’m open to any contributions, and am looking forward to seeing what people do with it!