The "All Tools" omni-prompt takes a whopping 2,756 tokens, but it's also using the GPT-4 32k model, with a 32,767 token context window.
It also reduces default DALL•E generations to one 'caption' (with a conflicting instruction later to generate 2 images).
Custom Instructions are going to need some work, since all this stuff comes before CIs.
The other thing that I found interesting: the built-in RAG accepts a limited number of file types, and uses the same `tools` middleware that the `browser` tool uses. It seems that it renders those files directly in their headless browser container, and captures text from it exactly the way it does when using a the Browse with Bing.
It also stores those files in the `/mnt/data` sandbox used by the `python` tool, making the uploads automatically ephemeral.
1 comment
[ 2.8 ms ] story [ 15.0 ms ] threadIt also reduces default DALL•E generations to one 'caption' (with a conflicting instruction later to generate 2 images).
Custom Instructions are going to need some work, since all this stuff comes before CIs.
The other thing that I found interesting: the built-in RAG accepts a limited number of file types, and uses the same `tools` middleware that the `browser` tool uses. It seems that it renders those files directly in their headless browser container, and captures text from it exactly the way it does when using a the Browse with Bing.
It also stores those files in the `/mnt/data` sandbox used by the `python` tool, making the uploads automatically ephemeral.
File types supported:
There are three middlewares (for lack of a better term) used by ChatGPT to talk to external tools.- `tools` (browser)
- `tools2` (python/jupyter)
- `tools3` (plugins)
It's weird that DALL•E clearly uses plugins, but the model metadata endpoint [0] doesn't declare that.
[0]: https://github.com/spdustin/ChatGPT-AutoExpert/blob/main/_sy...