The ReAct paradigm is one of the more powerful tools in the recent LangChain package which allows a more batteries-included approach to using it with models like GPT-3 and the ChatGPT API.
I'm a huge fan of langchain, so Yea if you want to just use this pattern try their agents, but if you want to better understand how it actually works, the blog post from simonw below includes a snippet that does it in a very small amount of simple code.
Clicked a lot faster reading his code than digging through langchain (though I'd still use langchain now that I understand how this works).
I wrote my own simplest-possible implementation of ReAct in Python here, which I think helps demonstrate quite how much you can get done with this pattern using only a very small amount of code:
Love this example! No offense to OP research paper but I appreciate the simplicity of your Python version instead
PS also thanks for this genuine LOL moment from the intro:
> A popular nightmare scenario for AI is giving it access to tools, so it can make API calls and execute its own code and generally break free of the constraints of its initial environment.
Cheers, Simon - been seeing your comments around and enjoying your blog and coverage of this stuff.
Is that prompt in your TIL really all it takes to inform it of these 3 actions? That's pretty impressive. I wonder how many actions it can scale to? I kind of expected some kind of classifier layer to predict if an action was necessary!
I've not tested its limits yet. The thing to consider is prompt length - depending on the model you get around 4,000 tokens, and the prompt itself is already 264 according to https://platform.openai.com/tokenizer - you need a bunch of space left for providing the output of your various actions, so tokens get used up pretty quickly.
The ReAct paper talks about fine-tuning to teach a model actions. I'd be interested to see an experiment that fine-tunes the LLaMA model to teach it actions - I have a hunch that might work really well, and save a bunch of token space in the actual execution phase.
How reliable has it been for you? After struggling with langchain & GPT4 I was planning to try your lib next before maybe writing my own - I plan to make a super generalized version so that my bot can code itself, so might have to pull out some tricks. (Before I got api access I got ChatGPT4 to do this - just not with a proper ReAct pattern - via a tampermonkey extension it wrote me…lol)
I spent about 30 minutes writing the code and 15 minutes writing it up - I haven't spent much time at all testing it and making sure it's robust and reliable. I just wanted to illustrate the concept.
It’s not particularly robust, but its so accessible that its easy to tweak and improve.
The code seems fine, even if clearly not structured the way a “production” product would be. The prompt can definitely use tweaking to improve the ability of ChatGPT to make best use of the actions – particularly, I’ve found it needs a firmer reminder to use valid Python expressions or its tends to pass math-in-English – e.g., with thousands separators, for instance – or other syntactically invalid things to the ‘calculate’ action, but once you’ve got the code and the prompt you’ve done as a starting point, adding actions, tweaking the prompt, etc., is pretty straightforward.
Thanks for writing this up. I read through it the other day and it was a lot simpler to get through than digging through Langchain for the first time.
I took your example and added a couple other "actions" like searching a searxng instance and returning the markdown version of a certain url. It's surprising how much more useful it can be when it has the ability to look stuff up on the internet.
Cheers Simon! I really appreciated your article on how LLMs and LLaMA are having their Stable Diffusion moment. It was well thought through. I might take your python and expand it to try to make an actually intelligent home assistant that can answer more helpful questions.
Where is "reason" in this model? A chain of semi-related thoughts isn't reason. LLMs need a set of axioms and formal logic to establish truthfulness of arbitrary statements.
If you can figure out how to do that - extend an LLM with formal logic to establish truth v.s. fiction - you'll be solving something that so far the big AI labs have all failed to do.
Language models can establish truthfulness in a formal logic system. There is a lot of literature that indicates this. Look at this for instance: https://openreview.net/forum?id=qFVVBzXxR2V
And you can test this quite easily with a prompt like this. gpt-4 answers this correctly without any tuning:
you have 2 boxes, box a and box b. Box a has 3 index cards, each with a giant letter printed in the middle
with a small caption below the letter. Based on the contents of Box A and Box B, infer what the blank
index card in Box B should most likely say
The contents of box a:
letter P, caption P:O
letter T, caption T:S
letter J, caption J:K
The contents of box b:
index card with O printed in the middle
index card with S printed in the middle
blank index card
I interpreted "establish truthfulness of arbitrary statements" as referring to the challenge of telling things like if a statement like "mars has a population of 12 million" is truthful or not, given the available training data for the LLM.
That's also "solved". Yes models exhibit hallucinations, but chain of thought prompting and ReAct prompting improve accuracy on factual tasks greatly, particuarly in a data augmented generation setting. We are finding new and better ways to prompt language models. Scaling up the models even further will also improve performance.
Anyone had luck getting this going in GPT-4 yet? I tried a couple of the chat-specific agents in langchain a couple days ago but it seems like the extra chat RLHF makes GPT-3.5/4 stubborn about not wanting to write messages in the needed format. I could get it working some of the time, but it was really unreliable. Next up I’ll try Simonw’s (the G.O.A.T.) micro-lib for this.
Also, man, what an annoying context to see “As a language model I cannot…”.
I have had luck with doing this on GPT4 with careful prompting, but GPT 3.5 is pretty reluctant to respond with anything other than straight up conversational answers.
29 comments
[ 2.9 ms ] story [ 64.8 ms ] threadhttps://langchain.readthedocs.io/en/latest/modules/agents/im...
https://langchain.readthedocs.io/en/latest/modules/agents/ex...
Clicked a lot faster reading his code than digging through langchain (though I'd still use langchain now that I understand how this works).
The LangChainHub repo seems dead (last commit 2 months ago).
Do you know of any alternative repos/marketplaces of chains/tools/prompts for LangChain?
https://til.simonwillison.net/llms/python-react-pattern
PS also thanks for this genuine LOL moment from the intro:
> A popular nightmare scenario for AI is giving it access to tools, so it can make API calls and execute its own code and generally break free of the constraints of its initial environment.
> Let's do that now!
Is that prompt in your TIL really all it takes to inform it of these 3 actions? That's pretty impressive. I wonder how many actions it can scale to? I kind of expected some kind of classifier layer to predict if an action was necessary!
The ReAct paper talks about fine-tuning to teach a model actions. I'd be interested to see an experiment that fine-tunes the LLaMA model to teach it actions - I have a hunch that might work really well, and save a bunch of token space in the actual execution phase.
The code seems fine, even if clearly not structured the way a “production” product would be. The prompt can definitely use tweaking to improve the ability of ChatGPT to make best use of the actions – particularly, I’ve found it needs a firmer reminder to use valid Python expressions or its tends to pass math-in-English – e.g., with thousands separators, for instance – or other syntactically invalid things to the ‘calculate’ action, but once you’ve got the code and the prompt you’ve done as a starting point, adding actions, tweaking the prompt, etc., is pretty straightforward.
I took your example and added a couple other "actions" like searching a searxng instance and returning the markdown version of a certain url. It's surprising how much more useful it can be when it has the ability to look stuff up on the internet.
Anyone who figures out a way to "establish truthfulness of arbitrary statements" in large language models will be making a HUGE breakthrough.
And you can test this quite easily with a prompt like this. gpt-4 answers this correctly without any tuning:
Also, man, what an annoying context to see “As a language model I cannot…”.