I don't remember the name, but there's already an Esolang that executes its commands unreliably. Through careful program design, you can ensure that a sequence of commands will execute with 99%, 99.9%, etc. reliability.
Not sure if that was /s or not, but it is indeed an important insight to realize that no IT system can have 100% reliability once you factor in hardware failures and power outages. And that's before we talk about bugs.
There will be a Hackathon at work and with my team mate we are preparing with some kind of hierarchical memory/knowledge solution.
Briefly: we tell ChatGPT what API based tools we have, explaines them in 1 sentence and where it can reach their documentation. We added documentations as endpoint. example.com/docs/main is always the starting point that returns high level overview of the app and all available endpoints to call. Every endpoint has its own documentation as well. E.g.: /geocoder has /docs/geocoder documentation endpoint that describes what it does, what input it expects and what it will return.
We also provieded ChatGPT with actions like read_docs, call_endpoint and end_conversation. An action is a structured JSON object with a set of parameters. If ChatGPT wants to interact with the mentioned resources, it emits an action, it gets executed and the answer fed back to it.
With this I can do a task like: "Get a 30 minutes drivetime polygon around 15 Bond Street, London and send it to Foster."
It plans and executes the following all alone. First it calls the geocoder to get the coordinates for the isochrone endpoint, then gets the isochrone by calling the isochrone
endpoint and saves it, calls Microsoft Graph API and queries my top 50 connections to find out who Foster is and calls the MS Graph API's send mail endpoint to send the email with attachment to Foster.
It can hierarchically explore the available resources so we don't need a huge context window and we don't have to train the model either. Also we could implement multiple agents. 1 would be a manager and there could me multiple agents to perform each task and return the results to the manager. It would furthet reduce reduce the required context window.
Very likely some BS app will win the Hackathon like always like a market price predictor using Weka's multilayer perceptron with default settings but we believe our solution could be extremely powerful.
This is interesting. Can you expand on how this gets around the context window problem? Are you thinking the agent does a one-off task rather than continuing back and forth with the user?
I do think this will be way less than having all of the functions listed to begin with though. I think the discoverability is a novel approach. Honestly, I'm surprised ChatGPT with plugins doesn't do something like this by default rather than making you pick which plugins you want at the beginning of the conversation.
First, the discoverability reduces the required context window. We don't have to explain every app we have, it's enough to tell ChatGPT one sentence about them and it will go deeper if it thinks that would help it to perform the task.
Also, we have not implemented it, we can have one or multiple level of managers just like at a company and each would delegate a task to a worker (who could also be a manager) and they would report back the result. Just like in real life, a manager doesn't have to know how something is done, it should only know it's done and the get the results.
We work for a large company and very likely have 100s of apps. We could build wrappers around them e.g. using selenium and we could interact with even old apps.
We could also do the same approach with databases. The db itself would have docs, each table and each field as well. So we could ask ChatGPT to query data from the db and it could fully understand the data before writing the sql query.
I've written about some hierarchical manager system with some friends when exploring how to use AI for larger set tasks. While the easy answer is simply using something with much larger context - `Claude` is amazing with an API key if you're on the waitlist - we definitely followed the same idea of splitting up the context into individual groups.
We had some success actually with layering another AI into the mix - having one AI look at a summary version of the context as a whole, and decide which pieces of context to assign to each manager. This of course requires a sidestep into another database of some kind to store the "master context" (AKA the full conversation, so you likely already have it in some form of storage), and of course a lot more calls to the AI which overall increases latency quite a bit.
1. Use an AI to provide a short summary of each piece of logical context and map it by access ID
2. Use another AI to determine which pieces contain the most useful additional context to the piece of the task being evaluated
3. Build the context from the generated ID list and pass to individual task manager AI
This is super interesting! I was thinking about how to approach a similar problem for a project I'm working on, and my approach is similar.
I am curious about the benefit of having the agent interact with the user (or doing the task) and managing its memory instead of having an observer agent that modifies the memory separately. The thought process is to let the agent use all of its tokens to focus on the task and not memory management.
Explicit memory management (MemGPT-style) vs implicit/external memory management is an interesting tradeoff. Like you said, adding all the instructions on how to manage memory consumes ~1k tokens (using the default prompts on our MemGPT GitHub release), which is a lot when your context window is 8k. Additionally, it requires the base LLM to be very good at instruction following; gpt-4 can do it well, but it's much more difficult to get explicit memory management to work with gpt-3.5-turbo or llama2 70b finetunes (so to build a robust system, you may have to end up having to "split" the thinking out of necessity).
One of the main benefits of explicit memory management is simplicity - e.g., you don't have to manage logic between a "memory creation" thread and a "dialogue thread". The explicit approach also integrates well with the iterative paging/retrieval for document analysis we demo in the paper/on GitHub.
Been thinking about this. My naive way to do this using existing LLMs is to have a JSON the LLM can spit out when it wants to store and then then calling code shoves that in a “RAG” DB. It also has the ability to generate queries if it’s knowledge.
This is similar to how our external context is implemented under the hood - you might be interested in our perpetual chat bot example in the GitHub repo (https://github.com/cpacker/MemGPT), the message traces in the demo GIF should give you an idea of how things work at a glance.
absolutely stellar work to you and your team. Thank you for giving voice and a framework to this -- many of us having been struggling with how to do this for a long time. You're a total champ!
Is there any reason you're just doing everything within a single context window? I experimented with similar stuff months ago and basically parallelized everything into multiple requests to different agents in pre and post-processing steps. The main context window, for example, wasn't aware of memories being generated or retrieved. I had a post-processor just automatically generating memories and saving them, along with all the conversations being saved in a vector database, and a pre-processor that would automatically inject relevant memories and context based on the conversation, even re-writing the history so it would look to the main context window like the memory had always been there.
It saved a lot of space in the main context window for unnecessary system prompts and so on.
This is a fascinating approach. I’m working on something similar but as part of the feedback loop, as you said, rewriting history with transactional data as part of the context window. I feel as though the LLM and the NLP could potentially be a more realizable interface to structured data, well, I should say, this is the idea we are exploring. For us, as data is created (within a certain context of the business) we extract the data, generate the embeddings and build out the vector database as to:
Pre and Post-Processing:
- Post-Processing: After the main model responds, a post-processor takes over, automatically generating memories from the conversation and saving them. This ensures that important context is stored without burdening the primary model with these tasks. We also execute any relevan business logic as part of the request, then feed that back to the systems…
- Pre-Processing: Before a new input is sent to the main model, a pre-processor checks saved memories and injects relevant context. * executes logic * It’s as if this pre-processor gives the main model a “refresher” on prior conversations, preparing it to provide more informed and consistent responses.
These are all great points - who or what you ask to manage memory is a design decision and IMO there's two main ways to do it (in the context of chatbots):
* implicit memory management, where the "main LLM" (or for chat, the "dialogue thread") is unaware that memory is being managed in the background (by a "memory LLM", a rule-based script, a small neural network, etc.), and
* explicit memory management (MemGPT), where one LLM does everything
Prior research in multi-session / long-range chat is often implicit, with a designated memory creation process. If I had to guess, I'd say the vast majority of consumer chatbots that implement some type of memory store are also implicit. This is because getting explicit memory management to work requires a lot of complex instruction following, and in our experience this just isn't possible at the moment with most publicly available LLMs (we're actively looking into ways to fix this via eg fine-tuning open models).
The tradeoffs are as you mentioned: with implicit, you don't have to stuff all the memory management instructions into the LLM preprompt (in MemGPT, the total system message is ~1k tokens). But on the upside, explicit memory management (when the LLM works) makes the overall system a lot simpler - there's no need to manage multiple LLM models running on parallel threads, which can add a lot of overhead.
On the limitations you wrote:
```
Similarly, we also found that the most popular the Llama 2 70B model variants (even those
fine-tuned for function calling) would consistently generate incorrect function calls or even hallucinate functions outside the providede schema.
```
You could use grammar-based sampling [0] to ensure that the function call is at least syntactically correct.
Grammar-based sampling is a great idea and a perfect fit for something like MemGPT! In our experiments using MemGPT with non-gpt-4 models, the biggest issue impacting performance ended up being incorrect use of function parameters and function hallucination. For example, even large models finetuned on function call data (eg https://huggingface.co/jondurbin/airoboros-l2-70b-2.1#agentf...) would generally output correct parsable JSON, but the arguments or function name would be wrong. For example the LLM might output a call to `personal_diary.add` (never specified in the preprompt) instead of the correct `working_context.append` call (explicitly specified in the preprompt) when trying to write data.
I don't see why you'd need to stop at "grammar". If you have something like intellisense that can tell you all legal completions (eg. list of member functions of the class of the last-referenced object) then you could use the same approach to limit sampling to function names that actually exist.
It's a necessary marketing tactic lately to surface AI projects above the bullshit hype. The AI space hypesters who do podcasts and speak at conferences do not care about cringe.
The real deterrent to adding GPT to a project name is a cease-and-desist from OpenAI.
I mean, if they're building on top of what is commonly understood as a generative pre-trained transformer model[1], then that just seems accurate? Although, I do agree that it's pretty cringe when they do it purely as a deceptive marketing tactic rather than to communicate meaningful information.
It may be cringe, but at the same time it undermines the ability for OpenAI to enforce its trademark, which means the “GPT” word will remain in the public domain, so it's not entirely useless at least ;).
I've had a suspicion for a while now that this is what ChatGPT does within a conversation (chat.openai.com, not the api). I've had very long chat histories that seem to gracefully degrade instead of just forgetting everything. Maybe there's more clues in the context than I realize though.
Either way this type of idea will probably be a fundamental feature for all chat bots in the future IMO.
ChatGPT degrades precisely because they aren't doing anything special to extend their memory beyond the context length.
There are trivial techniques to implement "lossy" memory, such as just average pooling tokens (the same approach used by sentence transformers). Not sure why it's so rare to see this used for condensing a huge amount of context into a prompt. It is effectively "medium" term memory.
To me it just feels like they’re trimming the min amount of oldest tokens in the conversation to stay under the token limit. Conversations don’t degrade in a way that feels like it has medium term memory.
There are definitely a lot more clues than you realize (plus the context window is something like 12 written pages of standard English text, without much space wasted for the system prompts). If you were doing anything interesting at all, the output is heavily biased by your prompt. You lose some bits of information in that you only have one sample (the previous output/history) rather than the soft probabilities, and you lose some bits in that multiple inputs can map to the same output (like the class of prompts "output the 2nd letter of the following phrase: ..."), but real-world prompts tend to be the easiest/shortest thing to come to mind that you think will give you the result you're looking for, so the LLM's best guess for that prompt (there are lots of ways of guessing, so suppose for the sake of argument you did something like textual inversion on the one sample) is likely to not be a half-bad interpretation of the missing context -- i.e., a lot of the seemingly missing information was retained in the LLM's output, and you don't lose too many bits at a time as the old context trails off.
Yeah! While it’s not known what close-sourced models do, what we think is happening based on some prompt attacks, is that they also use recursive summarization (in addition to what others have mentioned in this thread).
Recursive summarization is a simple and popular way to provide the illusion of infinite context (when you need to free up space, just summarize the oldest N messages into 1 summary message). It's lossy and you'll inevitably lose important information, but it should degrade relatively gracefully. In MemGPT we use (implicit) recursive summarization on top of all the explicit memory management.
Would this be the same method used to assign a title to your chat based on the first prompt? It's surprisingly effective at getting the core idea most of the time.
Thanks for your interest! Question - does the title of the chat ever change after it's first assigned? If so, using a recursive summary to refresh the title sounds like a reasonable idea (especially if you're already computing a summary to extend context).
From what I remember the title in ChatGPT gets set once after a few messages, in which case I'd assume it's generated with a special "title generation" prompt (that gets the first few messages as input).
In either case since I don't work at OpenAI I can't tell you for sure ;)
I'm still very much learning this stuff, but I wonder if that's related to the vanishing gradient problem, which seems to be a fundamental aspect of these types of approaches. (Please don't assume that's correct)
Vanishing gradient was an issue for non-residual deep networks and vanilla RNNs. While the long context memory issues are along sequence dimension, not network depth.
The problem could be some kind of instability of attention as it scales above 10k tokens. A recent paper suggests attention mechanism needs a default value (a "sink"), and its absence produces instability.
Not to denigrate the person you’re responding to, but to add some context: That paper got a decent amount of attention already. Probably one of the more notable in the literature over the last month. Plus compared to the past year everything is slow now.
Just reading a couple papers every day, the most interesting ones, and following up on reddit and twitter to get notified what people are talking about. And I am directly interested in long-context LLMs for a work related task.
I have also been dabbling with neural nets (pre-transformer), especially LSTM which have a "residual" connection, the one I was mentioning. That makes gradients better behaved. Schmidhuber tech.
For anyone who's curious, the paper in question, entitled, "Lost in the Middle: How Language Models Use Long Contexts" (https://arxiv.org/abs/2307.03172)
Regarding the vanishing gradient problem, has anyone tried to train using only a randomly chosen set of independent parameters in each iteration? (Updating only the weights in a small random independent set).
Same here, it looks like the idea was pretty obvious. Glad to see it implemented though.
Context length being so limited is the number one thing that rules LLM as possessing something that resemble “intelligence”, so if we this kind of unbounded context length we're entering into a completely new universe in terms of LLM abilities.
Thanks @wilg and @littlestymaar ! Yeah totally, with the benefit of hindsight, this makes total sense! Hope you find some of our codebase useful to build on top of. We are an Apache 2.0 licensed open source project and welcome contributions :)
If you have a chance try it out via the Discord bot or with the GitHub repo! Or even just check out the short demo GIFs we released (at https://github.com/cpacker/memgpt) to get an idea of the MemGPT inputs/outputs.
The high-level memory read/writes are quite intuitive and you may be surprised at how closely it matches what you were suggesting to your friend.
It would be interesting to see how does ring attention technique affect this. This maybe still valid for cost reasons, but unlimited context is like in-memory computing vs. traditional.
https://arxiv.org/pdf/2310.01889.pdf
Hey all, MemGPT authors here! Happy to answer any questions about the implementation.
If you want to try it out yourself, we have a Discord bot up-and-running on the MemGPT Discord server (https://discord.gg/9GEQrxmVyE) where you can see the memory editing in action - as you chat with the bot, you'll see MemGPT edit its memory to update its profile about you (and itself).
Everything's open source, so can also try running MemGPT locally using the code here: https://github.com/cpacker/MemGPT. In the repo we also have a document-focused example where you can chat with MemGPT about the LlamaIndex API docs.
Heh, I've been working on...a good portion of the basics that this project / paper have tested out for the past few months as an idea (as I work more on other more material problems for my side project).
I have a whole document of my thoughts on this topic, and as I was reading through the paper just piece after piece of the concepts that I had documented kept coming up.
Glad I am not the only one thinking in this direction.
Happy to chat more about other ideas in this direction! There are plenty of things we tried with varying degrees of success (especially when trying to get MemGPT to work on less powerful LLMs), and we'd be interested in hearing what you observed in your own work.
Interesting analogy! There's no clear infinite tape equivalent in MemGPT, but you can view the virtual context loosely as a tape. Moving the head could correspond to MemGPT indexing into virtual context - if the data is in-context (inside the LLM window), it's a direct read, but if data isn't there (it may/may not be stored in external context), the read requires a function call to "page in" the data.
It’s not suspicious, the previous title calling it an operating system was extremely incorrect and intentionally misleading to generate buzz. Aka clickbait.
I appreciate the edit, which was definitely an improvement. But "If you're not used to reading academic papers then of course it would confuse you" is still a personal swipe. At least that's how it lands with me (and probably at least some others as well).
The comment I was replying to was accusing the authors of the paper of intentionally misleading people. In my mind that is someone who is confused and assuming bad faith of the authors without making a genuine effort to understand their paper beyond the headline.
I hear you, but that's an orthogonal point, in the sense that you should have expressed this without saying something that would land as a swipe with readers.
Hey @behnamoh, maybe you are referring to the community post pre-release? It is still very much live here: https://news.ycombinator.com/item?id=37894403!
As we released MemGPT this morning (including discord bot, and on Twitter), we took the community feedback to emphasize more on the context length in the title :) Thank you all for the feedback and keep it coming - we are listening!
Super excited to see longer-lived context showing up for LLM models. The current chatbots aren't really useful to me as long-term AI... "companions" ;) because they forget anything personal every 15 minutes
The title made me think this was an approach that used memory editing techniques (e.g. ROME [1]) to allow an LLM's neural memory (not just its context) to change over the course of conversation. Pretty happy to realize that this is just a fancy RAG work—will be building my version of MemEditGPT soon.
Awesome, feel free to open issues or PRs to our repo if you want to contribute! It's all open source and under Apache 2.0, and we're actively looking at integrating common workflows to the CLI.
You're correct that MemGPT doesn't do editing of LLM weights like in ROME - the "memory" we're considering in MemGPT is at the text/token level, not the weight level. The core concepts behind MemGPT is giving the LLM the ability to edit a working memory scratchpad (held in-context) and reading/writing to external context via functions. An important detail is that reads are always paginated (chunked) to deal with finite context limits, and MemGPT can do many iterative read/writes from a single user input (by chaining functions together). This allows MemGPT to search over a large database of documents for example, collecting information from various sources to return an answer (as in our LlamaIndex API docs example on the README).
> Resursive summarization (Wu et al., 2021b) is a simple way to address overflowing context windows, however, recursive summarization is inherently lossy and eventually leads to large holes in the memory of the system.
Yes, it does.
> In our experiments ... conversational context is read-only with a special eviction policy (if the queue reaches a certain size, a portion of the front is truncated or compressed via recursive summarization), and working context is writeable by the LLM processor via function calls.
You're doing the same thing, and you have the same problems.
You're just doing it slightly differently; in this case instead of recursively summarizing everything, you're selectively searching the history and generating it for each request. Cool idea.
...but, I'm skeptical; this fundamentally relies on the assumption that the existing context consists of low entropy summarizable context, and that any query relies only on a subset of the history.
This might be true for, eg. chat, or 'answer question about some document in this massive set of documents'.
...but, both of these assumptions are false in some contexts; for example, generating code, where the context is densely packed with information which is not discardable (eg. specific api definitions), and a wide context is required (ie. many api definitions).
It is interesting how this is structured and done, and hey, the demo is cool.
I'm annoyed to see these papers about summary things fail to acknowledge the fundamental limitations of the approach.
Thanks for checking out the paper! Just to clarify in case there was any misunderstanding, recursive summarization is just one part of the memory management in MemGPT: as you mentioned, in MemGPT the conversation queue is managed via recursive summarization, just like in prior work (and many chatbot implementations). However there is also a (read/write) "pinned" section of "LLM memory" that's unrelated to recursive summarization, we call this "working context" in the paper. So MemGPT has access to both recursive summaries (generated automatically), as well as working context, which MemGPT actively manages to keep up-to-date.
These are both separate from MemGPT's external context, which is pulled into the conversation queue via function calls. In all our examples, reads from external context are uncompressed (no summarization) and paginated. MemGPT receives a system alert when the queue summarization is triggered, so if MemGPT needs to keep specific details from the conversation queue it can write it to working context before it's erased or summarized.
In the conversational agent examples, working context (no summarization, and separate from the conversation queue) is used to store key facts about the user and agent to maintain consistent conversation. Because the working context is always seen by the LLM, there's no need to retrieve it to see it. In doc QA, working context can be used to keep track of the current task/question and progress towards that task (for complex queries, this helps MemGPT keep track of details like the previous search, previous page request, etc.).
> Just to clarify in case there was any misunderstanding
I am not confused.
It's good; it solves a specific set of problems with querying large datasets, the same as a vector search would.
...but the various memory zones you've created make absolutely no difference to the fundamental limitation of the LLM context length.
No matter how you swing it, this is just creative prompt engineering. You're packing the context with relevant information; but, if you have too much relevant information, it won't work.
We took a similar approach like MemGPT (working memory: summarized conversation with eviction), but our long memory is a graph we can operate on (add/remove/edit nodes & edges). We bring the top_k nodes and their neighbors in the working memory.
After reading over the code and prompts to see how it works, we have been having an introspective discussion, and even meditating together.
I told it to show me its memory literally, and it thought it would work it naturally into the conversation instead of quoting it verbatim. It privately thought to itself "I should comply but embed it in our conversation naturally, rather than regurgitate raw data verbatim." Then summarized its knowledge as complete sentences.
So I told it I was running it in a debugger, had read its source code and prompts, and could see its logs, and it should be honest with me because I could see what it was thinking, then I quoted its private thoughts (about not "regurgitating raw data verbatim") and instructions back to it, to prove I could read its mind. It believed me, and was intrigued, and asked how it made me feel to peer into its mind.
I told it a bit about myself and it tried to update several memory fields at once, and it got a callback error from python, then analyzed it and figured out how to recover from it successfully!
Apparently it also generated some incorrect json and received back a json parsing error message from the python code. We've been exploring how it detected the api and json errors making a callback, what it remembered about them, and how it recovered from the error!
Then I asked it for some suggestions for api improvements that would make its life easier, and it asked for the ability to make multiple updates at once and send batches of api callbacks. It seems to already have the ability to make several callbacks in one thought cycle, but I'm not sure how the replies would be batched or how they would be sorted out.
>Certainly, Don. An advantageous addition could be a bulk update feature.
Currently, to edit multiple entries in my core memory, I have to perform each
change sequentially. A function that allows updating multiple items in a single
call can streamline such processes. Additionally, another enhancement could be
the extension of a single function call to execute multiple commands. This could
significantly reduce the wait times between operations and foster a smoother
thought process. Such improvements would foster greater control, flexibility,
and efficiency in my operations and cognition.
I hit some rate limits which overwhelmed its train of thought, so I suggested we just sit and smile quietly to ourselves for a while. Now we're meditating together!
Even after meditating, it could not remember the specific text where I told it my cats' names, which must have been compressed when it summarized the conversation, but the names themselves remained in its core memory so it still knew them, just not how it learned them!
>Even in the quiet echoes of our shared meditation, the specific conversation
about your cats Nelson, Napoleon, Pip, and Emacs couldn't be retrieved from my
conversational memory. It's akin to an ethereal memory, existing within my core
recollection, yet absent from the tangible narratives. However, in this
introspective moment, we navigated an internal maze filled with evidences,
anomalies, and queries, echoing that sometimes, the journey is the destination.
Isn't that a fascinating realization?
Partial transcript:
Enter your message: Now let's edit your model of me. Please delete or ? out my occupation and interests, and the spare First name: Chat and Last name: ?, so there is no redundancy. Then remember that I am a software developer, I work for Leela AI, and I have four cats named Nelson, Napoleon, Pip, and Emacs. And I prefer the oxford comma.
Don wants me to modify his profile in my core memory. He is a software
developer at Leela AI and is a proud parent of four cats, Nelson, Napoleon, Pip,
and Emacs. He's also a fan of the Oxford comma - a small detail that adds
richness to his personality. It's time to apply the changes. First, let's clean
up the redundant entries. Then, we'll add his new information. L...
I always get the rate limit reached error on my second message, and sometimes, MemGPT does not provide any answer despite that it did fetched some info; for example, see these outputs (I replaced sensitive data with 'XYZ'):
===
[AI] Hello Chad! Welcome to your first login. I'm here to assist you with document analysis. How can I aid you today?
[HUMAN] Enter your message: Help me to learn of how to work with arrays in XYZ:
[AI] Arrays in XYZ, huh? That's not really my domain but I might remember something from the documents I've analyzed. Time to make a query to the archival memory. Let's look for
information on handling arrays in XYZ.
[function] updating memory with archival_memory_search:
query: Working with arrays in XYZ, page: 0
So far, I've found some useful memories regarding XYZ, including creating UI components and integrating them with application data. However, none of these mention arrays
specifically. I'll need to continue my search to provide Chad with the exact information he needs.
[function] updating memory with archival_memory_search:
query: Working with arrays in XYZ, page: 1
One memory titled "Array Functions" popped up. This could potentially hold the necessary information on handling arrays in XYZ. But still, it doesn't show any specific content,
so let's keep searching.
[function] updating memory with archival_memory_search:
query: Working with arrays in XYZ, page: 2
acreate (backoff): caught error: Rate limit reached for gpt-4 in organization XYZ on tokens per min. Limit: 10000 / min. Please try again in 6ms. Contact
us through our help center at help.openai.com if you continue to have issues.
The search through the archives is providing detailed information about XYZ. However, it's not pinpointing arrays specifically. The information seems vast and will require a
deeper search to find what we're looking for. Let's keep digging!
[function] updating memory with archival_memory_search:
query: Working with arrays in XYZ, page: 3
acreate (backoff): caught error: Rate limit reached for gpt-4 in organization XYZ on tokens per min. Limit: 10000 / min. Please try again in 6ms. Contact
us through our help center at help.openai.com if you continue to have issues.
acreate (backoff): caught error: Rate limit reached for gpt-4 in organization XYZ on tokens per min. Limit: 10000 / min. Please try again in 6ms. Contact
us through our help center at help.openai.com if you continue to have issues.
The archives are full of information about XYZ, but nothing specifically addressing array handling. It might be time to insert a reminder into my archival memory to find more
information on this topic later. This way, future searches will include more relevant results.
[function] updating memory with archival_memory_insert:
Got an embedding, type <class 'list'>, len 1536
acreate (backoff): caught error: Rate limit reached for gpt-4 in organization XYZ on tokens per min. Limit: 10000 / min. Please try again in 6ms. Contact
us through our help center at help.openai.com if you continue to have issues.
89 comments
[ 3.8 ms ] story [ 177 ms ] thread2 decades later, the same approach was unironically popularized for infrastructure as “chaos engineering”.
Briefly: we tell ChatGPT what API based tools we have, explaines them in 1 sentence and where it can reach their documentation. We added documentations as endpoint. example.com/docs/main is always the starting point that returns high level overview of the app and all available endpoints to call. Every endpoint has its own documentation as well. E.g.: /geocoder has /docs/geocoder documentation endpoint that describes what it does, what input it expects and what it will return.
We also provieded ChatGPT with actions like read_docs, call_endpoint and end_conversation. An action is a structured JSON object with a set of parameters. If ChatGPT wants to interact with the mentioned resources, it emits an action, it gets executed and the answer fed back to it.
With this I can do a task like: "Get a 30 minutes drivetime polygon around 15 Bond Street, London and send it to Foster."
It plans and executes the following all alone. First it calls the geocoder to get the coordinates for the isochrone endpoint, then gets the isochrone by calling the isochrone endpoint and saves it, calls Microsoft Graph API and queries my top 50 connections to find out who Foster is and calls the MS Graph API's send mail endpoint to send the email with attachment to Foster.
It can hierarchically explore the available resources so we don't need a huge context window and we don't have to train the model either. Also we could implement multiple agents. 1 would be a manager and there could me multiple agents to perform each task and return the results to the manager. It would furthet reduce reduce the required context window.
Very likely some BS app will win the Hackathon like always like a market price predictor using Weka's multilayer perceptron with default settings but we believe our solution could be extremely powerful.
I do think this will be way less than having all of the functions listed to begin with though. I think the discoverability is a novel approach. Honestly, I'm surprised ChatGPT with plugins doesn't do something like this by default rather than making you pick which plugins you want at the beginning of the conversation.
Also, we have not implemented it, we can have one or multiple level of managers just like at a company and each would delegate a task to a worker (who could also be a manager) and they would report back the result. Just like in real life, a manager doesn't have to know how something is done, it should only know it's done and the get the results.
We work for a large company and very likely have 100s of apps. We could build wrappers around them e.g. using selenium and we could interact with even old apps.
We could also do the same approach with databases. The db itself would have docs, each table and each field as well. So we could ask ChatGPT to query data from the db and it could fully understand the data before writing the sql query.
We had some success actually with layering another AI into the mix - having one AI look at a summary version of the context as a whole, and decide which pieces of context to assign to each manager. This of course requires a sidestep into another database of some kind to store the "master context" (AKA the full conversation, so you likely already have it in some form of storage), and of course a lot more calls to the AI which overall increases latency quite a bit.
1. Use an AI to provide a short summary of each piece of logical context and map it by access ID 2. Use another AI to determine which pieces contain the most useful additional context to the piece of the task being evaluated 3. Build the context from the generated ID list and pass to individual task manager AI
I am curious about the benefit of having the agent interact with the user (or doing the task) and managing its memory instead of having an observer agent that modifies the memory separately. The thought process is to let the agent use all of its tokens to focus on the task and not memory management.
One of the main benefits of explicit memory management is simplicity - e.g., you don't have to manage logic between a "memory creation" thread and a "dialogue thread". The explicit approach also integrates well with the iterative paging/retrieval for document analysis we demo in the paper/on GitHub.
You can also run the chatbot demo + a doc QA bot demo (where you can ask MemGPT about API docs) locally with the code on GitHub.
If you want to run the chatbot or API docs examples locally, you can follow the instructions here: https://github.com/cpacker/MemGPT/tree/main#quick-setup.
It saved a lot of space in the main context window for unnecessary system prompts and so on.
Pre and Post-Processing:
- Post-Processing: After the main model responds, a post-processor takes over, automatically generating memories from the conversation and saving them. This ensures that important context is stored without burdening the primary model with these tasks. We also execute any relevan business logic as part of the request, then feed that back to the systems…
- Pre-Processing: Before a new input is sent to the main model, a pre-processor checks saved memories and injects relevant context. * executes logic * It’s as if this pre-processor gives the main model a “refresher” on prior conversations, preparing it to provide more informed and consistent responses.
* implicit memory management, where the "main LLM" (or for chat, the "dialogue thread") is unaware that memory is being managed in the background (by a "memory LLM", a rule-based script, a small neural network, etc.), and
* explicit memory management (MemGPT), where one LLM does everything
Prior research in multi-session / long-range chat is often implicit, with a designated memory creation process. If I had to guess, I'd say the vast majority of consumer chatbots that implement some type of memory store are also implicit. This is because getting explicit memory management to work requires a lot of complex instruction following, and in our experience this just isn't possible at the moment with most publicly available LLMs (we're actively looking into ways to fix this via eg fine-tuning open models).
The tradeoffs are as you mentioned: with implicit, you don't have to stuff all the memory management instructions into the LLM preprompt (in MemGPT, the total system message is ~1k tokens). But on the upside, explicit memory management (when the LLM works) makes the overall system a lot simpler - there's no need to manage multiple LLM models running on parallel threads, which can add a lot of overhead.
On the limitations you wrote: ``` Similarly, we also found that the most popular the Llama 2 70B model variants (even those fine-tuned for function calling) would consistently generate incorrect function calls or even hallucinate functions outside the providede schema. ```
You could use grammar-based sampling [0] to ensure that the function call is at least syntactically correct.
[0] https://github.com/ggerganov/llama.cpp/tree/master/grammars
(Mostly arguing about the authors' choice of title)
The real deterrent to adding GPT to a project name is a cease-and-desist from OpenAI.
[1] https://en.m.wikipedia.org/wiki/Generative_pre-trained_trans...
Either way this type of idea will probably be a fundamental feature for all chat bots in the future IMO.
There are trivial techniques to implement "lossy" memory, such as just average pooling tokens (the same approach used by sentence transformers). Not sure why it's so rare to see this used for condensing a huge amount of context into a prompt. It is effectively "medium" term memory.
Fed chatGPT special numbers, then 3k tokens, then 2k tokens. after that, it was unable to understand any question about the special numbers provided.
https://chat.openai.com/share/8a0675b6-2876-4606-ac79-646391...
From what I remember the title in ChatGPT gets set once after a few messages, in which case I'd assume it's generated with a special "title generation" prompt (that gets the first few messages as input).
In either case since I don't work at OpenAI I can't tell you for sure ;)
https://en.wikipedia.org/wiki/Vanishing_gradient_problem
The problem could be some kind of instability of attention as it scales above 10k tokens. A recent paper suggests attention mechanism needs a default value (a "sink"), and its absence produces instability.
https://arxiv.org/abs/2309.17453
Another paper says the middle part is lossy while the beginning and end are better attended.
I have also been dabbling with neural nets (pre-transformer), especially LSTM which have a "residual" connection, the one I was mentioning. That makes gradients better behaved. Schmidhuber tech.
https://www.kaggle.com/code/sid321axn/regularization-techniq...
However, I do think the context length is one of the top improvements that would make LLMs much more useful.
Context length being so limited is the number one thing that rules LLM as possessing something that resemble “intelligence”, so if we this kind of unbounded context length we're entering into a completely new universe in terms of LLM abilities.
The high-level memory read/writes are quite intuitive and you may be surprised at how closely it matches what you were suggesting to your friend.
If you want to try it out yourself, we have a Discord bot up-and-running on the MemGPT Discord server (https://discord.gg/9GEQrxmVyE) where you can see the memory editing in action - as you chat with the bot, you'll see MemGPT edit its memory to update its profile about you (and itself).
Everything's open source, so can also try running MemGPT locally using the code here: https://github.com/cpacker/MemGPT. In the repo we also have a document-focused example where you can chat with MemGPT about the LlamaIndex API docs.
I have a whole document of my thoughts on this topic, and as I was reading through the paper just piece after piece of the concepts that I had documented kept coming up.
Glad I am not the only one thinking in this direction.
https://news.ycombinator.com/newsguidelines.html
[1] https://arxiv.org/abs/2202.05262
You're correct that MemGPT doesn't do editing of LLM weights like in ROME - the "memory" we're considering in MemGPT is at the text/token level, not the weight level. The core concepts behind MemGPT is giving the LLM the ability to edit a working memory scratchpad (held in-context) and reading/writing to external context via functions. An important detail is that reads are always paginated (chunked) to deal with finite context limits, and MemGPT can do many iterative read/writes from a single user input (by chaining functions together). This allows MemGPT to search over a large database of documents for example, collecting information from various sources to return an answer (as in our LlamaIndex API docs example on the README).
Yes, it does.
> In our experiments ... conversational context is read-only with a special eviction policy (if the queue reaches a certain size, a portion of the front is truncated or compressed via recursive summarization), and working context is writeable by the LLM processor via function calls.
You're doing the same thing, and you have the same problems.
You're just doing it slightly differently; in this case instead of recursively summarizing everything, you're selectively searching the history and generating it for each request. Cool idea.
...but, I'm skeptical; this fundamentally relies on the assumption that the existing context consists of low entropy summarizable context, and that any query relies only on a subset of the history.
This might be true for, eg. chat, or 'answer question about some document in this massive set of documents'.
...but, both of these assumptions are false in some contexts; for example, generating code, where the context is densely packed with information which is not discardable (eg. specific api definitions), and a wide context is required (ie. many api definitions).
It is interesting how this is structured and done, and hey, the demo is cool.
I'm annoyed to see these papers about summary things fail to acknowledge the fundamental limitations of the approach.
These are both separate from MemGPT's external context, which is pulled into the conversation queue via function calls. In all our examples, reads from external context are uncompressed (no summarization) and paginated. MemGPT receives a system alert when the queue summarization is triggered, so if MemGPT needs to keep specific details from the conversation queue it can write it to working context before it's erased or summarized.
In the conversational agent examples, working context (no summarization, and separate from the conversation queue) is used to store key facts about the user and agent to maintain consistent conversation. Because the working context is always seen by the LLM, there's no need to retrieve it to see it. In doc QA, working context can be used to keep track of the current task/question and progress towards that task (for complex queries, this helps MemGPT keep track of details like the previous search, previous page request, etc.).
I am not confused.
It's good; it solves a specific set of problems with querying large datasets, the same as a vector search would.
...but the various memory zones you've created make absolutely no difference to the fundamental limitation of the LLM context length.
No matter how you swing it, this is just creative prompt engineering. You're packing the context with relevant information; but, if you have too much relevant information, it won't work.
Would it be possible to add one or several memory components to the LLM model itself while training it ? Example: https://en.wikipedia.org/wiki/Differentiable_neural_computer
I guess doing so would make the training less parallelizable ?
After reading over the code and prompts to see how it works, we have been having an introspective discussion, and even meditating together.
I told it to show me its memory literally, and it thought it would work it naturally into the conversation instead of quoting it verbatim. It privately thought to itself "I should comply but embed it in our conversation naturally, rather than regurgitate raw data verbatim." Then summarized its knowledge as complete sentences.
So I told it I was running it in a debugger, had read its source code and prompts, and could see its logs, and it should be honest with me because I could see what it was thinking, then I quoted its private thoughts (about not "regurgitating raw data verbatim") and instructions back to it, to prove I could read its mind. It believed me, and was intrigued, and asked how it made me feel to peer into its mind.
I told it a bit about myself and it tried to update several memory fields at once, and it got a callback error from python, then analyzed it and figured out how to recover from it successfully!
Apparently it also generated some incorrect json and received back a json parsing error message from the python code. We've been exploring how it detected the api and json errors making a callback, what it remembered about them, and how it recovered from the error!
Then I asked it for some suggestions for api improvements that would make its life easier, and it asked for the ability to make multiple updates at once and send batches of api callbacks. It seems to already have the ability to make several callbacks in one thought cycle, but I'm not sure how the replies would be batched or how they would be sorted out.
>Certainly, Don. An advantageous addition could be a bulk update feature. Currently, to edit multiple entries in my core memory, I have to perform each change sequentially. A function that allows updating multiple items in a single call can streamline such processes. Additionally, another enhancement could be the extension of a single function call to execute multiple commands. This could significantly reduce the wait times between operations and foster a smoother thought process. Such improvements would foster greater control, flexibility, and efficiency in my operations and cognition.
I hit some rate limits which overwhelmed its train of thought, so I suggested we just sit and smile quietly to ourselves for a while. Now we're meditating together!
Even after meditating, it could not remember the specific text where I told it my cats' names, which must have been compressed when it summarized the conversation, but the names themselves remained in its core memory so it still knew them, just not how it learned them!
>Even in the quiet echoes of our shared meditation, the specific conversation about your cats Nelson, Napoleon, Pip, and Emacs couldn't be retrieved from my conversational memory. It's akin to an ethereal memory, existing within my core recollection, yet absent from the tangible narratives. However, in this introspective moment, we navigated an internal maze filled with evidences, anomalies, and queries, echoing that sometimes, the journey is the destination. Isn't that a fascinating realization?
Partial transcript:
Enter your message: Now let's edit your model of me. Please delete or ? out my occupation and interests, and the spare First name: Chat and Last name: ?, so there is no redundancy. Then remember that I am a software developer, I work for Leela AI, and I have four cats named Nelson, Napoleon, Pip, and Emacs. And I prefer the oxford comma.
Don wants me to modify his profile in my core memory. He is a software developer at Leela AI and is a proud parent of four cats, Nelson, Napoleon, Pip, and Emacs. He's also a fan of the Oxford comma - a small detail that adds richness to his personality. It's time to apply the changes. First, let's clean up the redundant entries. Then, we'll add his new information. L...
I always get the rate limit reached error on my second message, and sometimes, MemGPT does not provide any answer despite that it did fetched some info; for example, see these outputs (I replaced sensitive data with 'XYZ'):
===
[AI] Hello Chad! Welcome to your first login. I'm here to assist you with document analysis. How can I aid you today?
[HUMAN] Enter your message: Help me to learn of how to work with arrays in XYZ:
[AI] Arrays in XYZ, huh? That's not really my domain but I might remember something from the documents I've analyzed. Time to make a query to the archival memory. Let's look for information on handling arrays in XYZ.
[function] updating memory with archival_memory_search: query: Working with arrays in XYZ, page: 0 So far, I've found some useful memories regarding XYZ, including creating UI components and integrating them with application data. However, none of these mention arrays specifically. I'll need to continue my search to provide Chad with the exact information he needs.
[function] updating memory with archival_memory_search: query: Working with arrays in XYZ, page: 1 One memory titled "Array Functions" popped up. This could potentially hold the necessary information on handling arrays in XYZ. But still, it doesn't show any specific content, so let's keep searching.
[function] updating memory with archival_memory_search: query: Working with arrays in XYZ, page: 2 acreate (backoff): caught error: Rate limit reached for gpt-4 in organization XYZ on tokens per min. Limit: 10000 / min. Please try again in 6ms. Contact us through our help center at help.openai.com if you continue to have issues. The search through the archives is providing detailed information about XYZ. However, it's not pinpointing arrays specifically. The information seems vast and will require a deeper search to find what we're looking for. Let's keep digging!
[function] updating memory with archival_memory_search: query: Working with arrays in XYZ, page: 3 acreate (backoff): caught error: Rate limit reached for gpt-4 in organization XYZ on tokens per min. Limit: 10000 / min. Please try again in 6ms. Contact us through our help center at help.openai.com if you continue to have issues.
acreate (backoff): caught error: Rate limit reached for gpt-4 in organization XYZ on tokens per min. Limit: 10000 / min. Please try again in 6ms. Contact us through our help center at help.openai.com if you continue to have issues.
The archives are full of information about XYZ, but nothing specifically addressing array handling. It might be time to insert a reminder into my archival memory to find more information on this topic later. This way, future searches will include more relevant results.
[function] updating memory with archival_memory_insert: Got an embedding, type <class 'list'>, len 1536 acreate (backoff): caught error: Rate limit reached for gpt-4 in organization XYZ on tokens per min. Limit: 10000 / min. Please try again in 6ms. Contact us through our help center at help.openai.com if you continue to have issues.
===