10 comments

[ 3.0 ms ] story [ 31.7 ms ] thread
> the information an AI system needs to produce accurate ... outputs

I would have stuck a qualifier in there

I feel like AI is going to be doing all the fun stuff and I will just left organizing the data and docs it needs to generate code.
Putting engineering after a term doesnt make it engineering.
I don’t really think this reflects the current era of challenges?

The “enforcement layer” is the hardest and most important part, and is barely addressed.

- is the answer structurally / syntactically valid?

- is it appropriately grounded and evidenced?

- is it accurate? In what ways does it fall short?

Each of these should be triggering an agent to rework and resubmit etc. or failing that a disclosure to the user about how the answer falls short and should be reviewed / remediated.

This feels like it’s from the era of trying to oneshot a good enough answer.

No numbers/measurements/benchmarks and you dare call it "a working" one? Any real proofs that this 'works'?
I like it, it's a good start.