I have tried a similar idea before. I had two experts: one to propose a plan and another to critique it. It worked well, but often required 5 rounds of iteration to get to a great answer. The cost wasn't ideal and GPT4 still performed better on the first go, so I abandoned the idea.
I am curious how this approach compares to one-shot response from a larger model.
There are dozens of new prompting strategies that are less verbose that I would have tried before this.
CoT is considered legacy approach since at least self-discovery technique in feb. This pretend panels-of-experts in 1-shot just seems like it is making up a narrative to meet the expected word count of meeting minutes. If it works for you Quiet-STaR probably works better for you as 1-shot, apparently moreso with anthropic.
emergentmind.com - trending AI papers, i just look at the simple papers that pop up from time to time and maybe check their reference links for something it builds on, occasionally get another LLM to extract the actual prompt style to test. I wish there was a simple pre-loaded prompt test arena actually, probably too expensive though. Every time there is an advance in proompting techniques it gets negated by model improoving so it's a funny space.
You'll have to check research publications if you want the latest stuff. The approach indicated in the comment you replied to is from this paper from February: https://arxiv.org/pdf/2402.03620
If you want the semi-latest stuff, you can also try various blogs, but they vary wildly in covered topics and quality.
I may not articulate this perfectly, but I cannot be the only person frustrated with the state of affairs.
New features which have some random amount of error do not make me happier. They make me more afraid.
I have been grilled on single commas being missing in documents - for good reason! It was a financial model.
For me -
LLMs are
- English majors
- Decompression systems
They cannot conform to my expectation of code
- their output is inherently randomly distributed
- they are being pigeon holed into doing logic / Thinking.
Any use case that
- exposes an LLM to infinite interactions (open to users at internet scales)
Simply represents a unsolvable edge case issue to me.
Production solutions need to figure out
- a way to verify the output is correct (fast, scaled, cheap!),
- figure out use cases that accept uncertainty gracefully.
From what I am seeing, there is a bit of both happening.
1) Get the LLM output close enough to 99% correct,
2) send it to a human as a “co-pilot” to verify it is correct.
It is fatiguing to track every new variation on getting somewhat closer to some figure between 89 to 99.9999%.
I love the tech. I love what it actually achieves. I use it regularly to make my life better. This is not an attack on the tech.
My life would be MUCH easier, if someone could tell me their bot or solution is going to screw up massively 89% of the time, and that expert users can identify the 11% cases 100% of the time.
Also that my experts arent going to suddenly see their workload expand by infinity% if this tool goes live.
Code generation should be relatively straightforward with language model. Start with a problem description and let the LLM generate the code. It should be fairly simple to ask for a proof of correctness for the generated code, then verify that proof (for instance by creating property-based tests), and finally ask for a human-readable summary of the proof. Pass this to human code review and you have a pretty good code generation machine. Failure at any step can simply lead to another iteration.
The thing is: SE doesn't work like this anymore. People aren't used to formalized requirements and take 99% just for granted. (Which web developer really takes network errors into account nowadays?).
With "AI" we might have the tools we need to really solve the software crisis.
This is likely why Coding assistants are talked about in glowing terms. “35% increased coder productivity with co-pilot.”
This still hits the “expert required” problem. Use an LLM to code in a language or domain you have 0 experience in.
Contrast your experience with that of any explainer video where an expert in a domain uses an LLM to make an MVP.
Experts will get something up in the span of a video. They will avoid innumerable mis-steps and rabbit holes because they know code that gets to the result effectively on sight.
Non experts wont even know what part of the output is wrong.
——-
Generalize this problem away from code. Think of financial models and another version of the verification problem.
Your LLM works, it feeds signal into a financial model that predicts the future value of a firm.
The model output is dramatically out of sync with market estimates.
If an analyst showed this result, they would be told to fix their excel or tweak the assumptions. Thats the typical “verification” loop.
Let’s say hyou tune the model to get to 99.9%. I still have to run sampling /EVALS / QC all the time to make sure my output is within tolerances.
I still need to know that I can expect X% accuracy, that we have a plan for the 1-X% cases, and that this isn’t going to overwhelm the people with domain expertise in my team.
12 comments
[ 2.8 ms ] story [ 31.7 ms ] threadI am curious how this approach compares to one-shot response from a larger model.
CoT is considered legacy approach since at least self-discovery technique in feb. This pretend panels-of-experts in 1-shot just seems like it is making up a narrative to meet the expected word count of meeting minutes. If it works for you Quiet-STaR probably works better for you as 1-shot, apparently moreso with anthropic.
https://www.emergentmind.com/papers/2404.11018
If you want the semi-latest stuff, you can also try various blogs, but they vary wildly in covered topics and quality.
New features which have some random amount of error do not make me happier. They make me more afraid.
I have been grilled on single commas being missing in documents - for good reason! It was a financial model.
For me - LLMs are
- English majors - Decompression systems
They cannot conform to my expectation of code
- their output is inherently randomly distributed - they are being pigeon holed into doing logic / Thinking.
Any use case that - exposes an LLM to infinite interactions (open to users at internet scales)
Simply represents a unsolvable edge case issue to me.
Production solutions need to figure out - a way to verify the output is correct (fast, scaled, cheap!), - figure out use cases that accept uncertainty gracefully.
From what I am seeing, there is a bit of both happening.
1) Get the LLM output close enough to 99% correct, 2) send it to a human as a “co-pilot” to verify it is correct.
It is fatiguing to track every new variation on getting somewhat closer to some figure between 89 to 99.9999%.
I love the tech. I love what it actually achieves. I use it regularly to make my life better. This is not an attack on the tech.
My life would be MUCH easier, if someone could tell me their bot or solution is going to screw up massively 89% of the time, and that expert users can identify the 11% cases 100% of the time.
Also that my experts arent going to suddenly see their workload expand by infinity% if this tool goes live.
The thing is: SE doesn't work like this anymore. People aren't used to formalized requirements and take 99% just for granted. (Which web developer really takes network errors into account nowadays?).
With "AI" we might have the tools we need to really solve the software crisis.
This still hits the “expert required” problem. Use an LLM to code in a language or domain you have 0 experience in.
Contrast your experience with that of any explainer video where an expert in a domain uses an LLM to make an MVP.
Experts will get something up in the span of a video. They will avoid innumerable mis-steps and rabbit holes because they know code that gets to the result effectively on sight.
Non experts wont even know what part of the output is wrong.
——-
Generalize this problem away from code. Think of financial models and another version of the verification problem.
Your LLM works, it feeds signal into a financial model that predicts the future value of a firm.
The model output is dramatically out of sync with market estimates.
If an analyst showed this result, they would be told to fix their excel or tweak the assumptions. Thats the typical “verification” loop.
Let’s say hyou tune the model to get to 99.9%. I still have to run sampling /EVALS / QC all the time to make sure my output is within tolerances.
I still need to know that I can expect X% accuracy, that we have a plan for the 1-X% cases, and that this isn’t going to overwhelm the people with domain expertise in my team.