This summer I’ve been investigating agentic coding with local LLMs, and while I’m far from an expert, one thought that has been on my mind is leveraging techniques from “old-school” AI such as heuristic search to guide agents when it comes to planning. The use of Datalog in this article resonates with me, since logic programming was a major part of old-fashioned symbolic AI. I’m very curious about this combination of “old-school” AI and LLMs.
It's an agent system that basically embeds the core idea of CodeQL (Datalog extraction from codebases) and then allows a model to pose questions and answer them.
Something of this capacity would be useful in investigating obscure hardware failures in the logs that I couldn't confirm because the problem was not being observed while the device was in my shop. the problem was surfacing in another location probably due to some set of circumstances in the software that I could recreate, or some particular peripherals that were attached.
I ran into the very same problem of the LLM forgetting that we ruled out a conclusion that was verified not to be the cause as it came up further in the conversation history while I was exploring possibilities.
I had to keep reminding we ruled out that conclusion prior.. I just carried on with having the LLM capture some of the supporting sources of other people experiencing the same problem and kept having to refine those sources because it was focused only on summaries, but eventually i got the sources to a point where they were good enough hypothesis that we could formulate a better conclusion on what the potential cause was.
Very cool. I recall an HN submission (which I can't find offhand unfortunately) that did something similar -- it used an LLM to decompose articles into a set of statements which were used to construct an entity-relationship graph of facts and events. It then queried that using conventional graph query methods, much like DataLog / Lemmalog is doing here. I remember it was particularly effective at answering timeline-based queries that LLMs (back then) sucked at.
I think approaches like this are going to be (or maybe already are?) the basis of effective grounding of LLM responses in authoritative data sources. It should be possible to pinpoint any error to an incorrect traversal or an incorrect "fact." This would work best for concrete, unambiguous facts, however; fuzzy, ambiguous or opinion-based information will probably remain the purview of LLMs.
My problem with memory is that it goes stale, and updates to facts are often not changing all locations of that fact. Such a system should make it easier to maintain a single source of truth, and versioning, no?
Oh, interesting, is this based on your experience, or is there a reference I could look up? My layman intuition is better grounding should result in more accurate results, is that not the same as better reasoning?
I reached a similar conclusion - that LLMs should only be used at the terminals for any request fulfilment. The two terminals in this case being:
1. User request understanding: converting natural language to a more rigorous form, in my case Datalog.
2. Query result interpretation: converting facts, derived facts back into natural language.
Between those terminals should be pure mechanical reasoning over an ontology of some sort.
That connects to another principle I've been thinking about, which I call weathering: useful reasoning should leave durable residue in the system. If an LLM had to infer a relation, mapping, rule, or abstraction once, the next similar request shouldn't require it to rediscover the same thing from scratch.
In fact with continuous use, the system should require less and less probabilistic intelligence over time.
Theoretically but practically any LLM generated infra/classification set is going to drift due to inaccuracy and harm IR/whatever logical process you're using. I am a big fan of using a loose taxonomy but it's not been revolutionary.
What you call "Weathering" has been a constant gripe of mine. We have LLM-driven softwares toward that almost seem to start from scratch every time a request comes in - there are mechanisms to learn or generalize, like writing out a memory, but they are not reliable or reliable in general. There is no convenient lever to be able to say "yes this is in the memory but the request seems like it needs a fresh scan of data, so ignore your memory", or the opposite "you can infer this from stuff in the memory - don't re-reason!". There is some work like Dynamic Cheatsheets [1] and Agentic Context Engineering [2] that have studied this aspect, but we are far from a generally reliable solution. And till we have that, I think the system variations for systems trying to solve this problem are going to be (a) LLM-leaning: create unstructured memory files, with human in the loop as a filter to reject inaccurate responses (b) LLM-as-a-layer: what you describe and the article kind of is doing.
I’ve been thinking about this.. the same idea behind Dynamic cheat sheets paper and the ACE paper. Thanks for those links! Nice to see those ideas academically studied.
My problem is the context of today’s models (that I’ve been using - Claude Opus 5 and GPT-Sol) are a black box to a user like me. I cannot tell what they already hold in their context over the duration of a coding session. I cannot tell influencers a little by using/not using MCP tools, AGENTS.md like MD files, but what they _actually_ use at inference time is still opaque.
For example, if I insist that GPT-Sol use a size bounded dynamic cheat sheet for my project, and even have those reflector&curator agents refine them concurrently, what is to say GPT-Sol doesn’t have an intrinsic cheat sheet like that already, and it is just reading/curating/managing this one in addition and only getting confused needlessly along the way?
Similarly, all those codegraph like tools and semantic search addons.. what is to say the model genuinely uses them instead of relying on its own inference techniques and just “perform” for the user audience as if they are using these fancy tools?
This is not to say we shouldn’t try more ways of managing context and just give up to our AI provider overlords. Just thinking through the logic of what we are working with.
> My problem is the context of today’s models (that I’ve been using - Claude Opus 5 and GPT-Sol) are a black box to a user like me.
Would it even help if it weren’t a blackbox? Maybe to the frontier labs with resources, but even trying to read through the thinking traces of an open model doing a nontrivial task easily takes me an entire day to do correctly (i.e. actually looking at the code it read in) and much more if I’m trying to tune a specific repeatable task.
Applying that to agentic coding seems like a losing proposition. The only way to manage it seems to be more agents and that’s its own rabbit hole of compounding errors (like Claude’s useless comments).
This really matches up to my experience on long research projects with Claude.
It’s very hard to remove information - Claude has a habit of recording things all over the place and will happily treat things as facts even after they’ve been disproved.
What is currently true can get easily contaminated with old “facts”.
It seems like you might be inventing a form of non-monotonic logic. Check out answer set programming, it actually does exactly what you want of "unlearning" facts that you've learned. Not sure if it helps in your particular instance, but it's very cool stuff and IIRC there is an implementation that extends datalog. https://en.wikipedia.org/wiki/Answer_set_programming
This is the fact I’ve been struggling with for quite some time. It’s not because it forgets the facts, it’s because the invalidation doesn’t propagate.
My way of handling that is a decision log. For every project since I started doing that it’s working great. My CLAUDE.md instruct the agent to store my every decision to the file with a metadata when I made this decision and what was the context. The agent is using this file as an index of decisions and rarely lose a track. It also helps team members to find out more about the development phases.
Does your system invalidate the parts of the memory if these are not valid or relevant anymore or just store/retrieve?
What I found out during malware analysis is that LLM agents have a couple of quirks that you can solve by:
- optimize for short lived agent workflows
- use symbols as function contracts
- maintain decision and discovery state
- give LLMs CLI linters
- give LLMs access to knowledge bases
The linter part is mindblowing. I built linters that validate HTML or markdown or docx files, for example, and output what kind of structure is expected (e.g. h4 inside h1? Must be h1 > h2 ...).
With linters the output quality of agents is just soo much better.
For program analysis, I'm currently exploring the idea of using an external ebpf daemon that programs can be observed with via a public API (which is the tool for the agent to use). Not sure if it'll do the trick yet, but I think it has lots of potential.
In general, what all the big LLM providers are doing is moving towards classical & neural (neuro-symbolic) AI - even though they dont publicly admit it because that would counter their claims for years of "scale is all you need" (which has vanished with diminishing returns, see $MS / altman's GPT-5 bet).
It seems like the two approaches compliment each other nicely. Human intelligence also relies on parallel information processing. LLMs are like a massive working memory, incredibly effective but with a similar set of limitations. What they lack is a symbolic model of reality, something that they can build and refine.
The various advances in LLM technology tend to rhyme with the advances in computer programming in general. For example, the stunts that involved getting LLMs to create compilers and browsers are really just extremely expensive[0] versions of genetic programming (none of it would have worked without using the test-suite as a fitness-function). The recent news of migrations from one test-framework to another (featuring Asana, I believe), was something that we could always do trivially in a language that was based on S-Expressions (Lisp, Scheme, etc).
In fact, both Cyc and the "AI" Labs have the _same basic thesis_: Intelligence is, primarily, a data entry problem. They just disagree about what kinds of heuristics should be run over that data (logic-programs, neural-nets).
Whenever I read about someone using LLMs to write code, it _very closely_ resembles how Lenat was using Eurisko/Cyc to solve problems: they let the system run continuously, and they "nudge" it in "interesting" directions, "when it gets stuck", or "runs out of steam". (Quotes indicate their phrasing, not mine)
Even Lee Spector noticed something analogous with his genetic programming system. When he tried to get it to discover optimal data structures (or maybe it was sorting algorithms, I forget), the system would quickly "run out of steam", without a solution. But when they added new verbs/opcodes to the system, that were a better fit for that domain (e.g. index-based memory loads + stores), it converged on a solution very quickly (even for GP, domain specific languages keep delivering unreasonable wins). You will note that this rhymes with the "micro-theories" of Cyc, which in turn rhyme with the SLMs of the AI labs.
In my personal experience, most of the "silver bullets" do not work (obviously), but some of them do nudge you towards being a better programmer (by refining your intuition about the problem specifically, and computers more generally).
[0]: I suspect that, this was mostly a triumph of enormous amounts of hardware, more than the actual LLM technology. I further suspect that a traditional GP approach, on the same quantity of hardware, could have gotten there faster (if not better as well).
I would not necessarily call it genetic programming. But you can discover the same pattern e.g. for human knowledge. We have a domain language for math and when we go deeper we will have domain language for algebra, discrete math etc.. and then breaking it to even for more specific areas simply to compact the information to deliver more information with less word or be able to fit the knowledge into the processing capacity.
So I would not compare it to the advances in programing languages, but more like to compacting information/representation. It's similar but not quite genetic programming.
I am not calling it GP. I am pointing out that AI-coding systems tend to resemble GP systems in terms of both broad architecture, and in terms of how the code/genome is generated. Also, I already mentioned domain specific languages, in the context of GP, and how (effectively) creating one (i.e. selecting only relevant genes/verbs) that was a better match for the problem domain, made the GP find a solution infinitely faster[0]. Clearly these two things are not contradictory, and are clearly related, to the point that they can be combined. I am not sure what about my comment gave you the impression that I thought one was contradictory with the other. Hope this follow-up clears things up.
[0]: Because, in effect, we are decreasing the size of the search-space. Every programming-language project is basically a set of bets, that the design choices will decrease the size of the search space for the users of that language (most Unix commands fit this pattern, as do most well designed APIs). Mathematics is similar, in that the notation is specialized and sparse (even the named constants and variables have symbolic names, instead of meaningful names). Most system dynamics models could have the names of entities replaced with id-numbers, and they would continue to function. The whole point of (for example) a library is to give a tiny parameterized notation/language, that the user can use to express a solution to their problem, in terms of the problem domain itself. This decreasing of the search-space, is an act of compression. I have to say, I am a little confused by your reply, because it has the tone of a disagreement, but it does not really disagree with anything that I have written. Also, I have given various examples to support my analogies, but you have offered zero examples, and I suspect that there would be less confusion (on my part) if you did.
It's strange to frame this as classical vs scale. Us humans have a powerful inference engine in our heads. We also use a calendar to avoid re-deriving everything from first principles before we've had our morning coffee. Businesses couple many creative (human) agents together. They also have processes and rules.
It isn't, and it is not what was described: its about unbounded imagination (neural / LLM) that needs reality constraints (symbols / rules) to produce useful output.
Think of it like human imagination may do anything (flying cars), but the real world has constraints and we use language/writing (symbols) and rules bound to them to simulate and reconcile our imagination with reality to actually flow our energy into something that may work in the real world.
> they dont publicly admit it because that would counter their claims for years of "scale is all you need" (which has vanished with diminishing returns, see $MS / altman's GPT-5 bet).
Is my reading comprehension just completely broken or something? The above certainly sounds like "the powers that be want us to believe that attention is all you need".
Great work! If anyone is looking for a way to integrate something like this into their own harness or the pi coding agent, then you might be interested in DeepClause [0]. It comes with a Prolog-like language implemented on top of SWI-Prolog (WASM Version). The purpose of the project is to allow for broad experimentation around the intersection of LLMs/Agents and GOFAI. So you could use it to build memory systems like OP did, create executable specs, define graphs and loops for agents and subagents... It also comes with a pi extension that greatly simplifies getting started with it.
Opposed to OP, DeepClause uses Prolog semantics, so running some more complex queries on knowledgebases might cause some issues (which is the use case where a Datalog might be more useful). For smaller scales it should be fine though.
I'm not sure if this is the right direction, but it's certainly momentarily helpful. I think the right direction would be to enable the model itself do dynamic program analysis, deterministically and dynamically via runtime-inference.
btw. your comment is grayed out, not sure what it means. However, thanks for sharing, I'll look into it.
Limitation of LLM for and toward reverse engineering; it's the LLM innate error of forgetting states thru agentic recursion by overflow of context or prior premises being optimized away due to not using ternary-state (uninit/written/read) memory state.
Once again, on LLM being: a digital librarian, at its finest; logic a logic analyst, not so much.
Interesting and potentially has applicability in deriving logical rules from regulation, contracts etc. Are there already formal languages that can be used to codify, that sort of information.
yooo.. this is exactly what I wanted to do... I'm so happy because (a) I hadn't figured out how to do it (b) it seemed kind of difficult in general and (c) now I don't have to, lol.
103 comments
[ 0.23 ms ] story [ 6.4 ms ] threadI ran into the very same problem of the LLM forgetting that we ruled out a conclusion that was verified not to be the cause as it came up further in the conversation history while I was exploring possibilities.
I had to keep reminding we ruled out that conclusion prior.. I just carried on with having the LLM capture some of the supporting sources of other people experiencing the same problem and kept having to refine those sources because it was focused only on summaries, but eventually i got the sources to a point where they were good enough hypothesis that we could formulate a better conclusion on what the potential cause was.
(See also Cyc: https://en.wikipedia.org/wiki/Cyc)
I think approaches like this are going to be (or maybe already are?) the basis of effective grounding of LLM responses in authoritative data sources. It should be possible to pinpoint any error to an incorrect traversal or an incorrect "fact." This would work best for concrete, unambiguous facts, however; fuzzy, ambiguous or opinion-based information will probably remain the purview of LLMs.
https://news.ycombinator.com/item?id=41445445
But IRL it's too vague. The exploit hunting is a better use case.
1. User request understanding: converting natural language to a more rigorous form, in my case Datalog.
2. Query result interpretation: converting facts, derived facts back into natural language.
Between those terminals should be pure mechanical reasoning over an ontology of some sort.
That connects to another principle I've been thinking about, which I call weathering: useful reasoning should leave durable residue in the system. If an LLM had to infer a relation, mapping, rule, or abstraction once, the next similar request shouldn't require it to rediscover the same thing from scratch.
In fact with continuous use, the system should require less and less probabilistic intelligence over time.
“He weathered the storm” vs “the stone weathered into an arch”.
You can endure as long as you want to, but we're all going to erode eventually.
[1] https://aclanthology.org/2026.eacl-long.333/ [2] https://openreview.net/pdf?id=eC4ygDs02R
My problem is the context of today’s models (that I’ve been using - Claude Opus 5 and GPT-Sol) are a black box to a user like me. I cannot tell what they already hold in their context over the duration of a coding session. I cannot tell influencers a little by using/not using MCP tools, AGENTS.md like MD files, but what they _actually_ use at inference time is still opaque.
For example, if I insist that GPT-Sol use a size bounded dynamic cheat sheet for my project, and even have those reflector&curator agents refine them concurrently, what is to say GPT-Sol doesn’t have an intrinsic cheat sheet like that already, and it is just reading/curating/managing this one in addition and only getting confused needlessly along the way?
Similarly, all those codegraph like tools and semantic search addons.. what is to say the model genuinely uses them instead of relying on its own inference techniques and just “perform” for the user audience as if they are using these fancy tools?
This is not to say we shouldn’t try more ways of managing context and just give up to our AI provider overlords. Just thinking through the logic of what we are working with.
Would it even help if it weren’t a blackbox? Maybe to the frontier labs with resources, but even trying to read through the thinking traces of an open model doing a nontrivial task easily takes me an entire day to do correctly (i.e. actually looking at the code it read in) and much more if I’m trying to tune a specific repeatable task.
Applying that to agentic coding seems like a losing proposition. The only way to manage it seems to be more agents and that’s its own rabbit hole of compounding errors (like Claude’s useless comments).
The weathering need is real. An llm equivalent of dbms analysing usage pattern to tune indexes.
And I guess for 2, stay out as long as possible.
I guess this is somewhat different; it admits that in some cases you do need the LLM doing more work somewhere in the middle.
But it also means sometimes you never enter the LLM to begin with, or never re-enter it for output.
It’s very hard to remove information - Claude has a habit of recording things all over the place and will happily treat things as facts even after they’ve been disproved.
What is currently true can get easily contaminated with old “facts”.
My way of handling that is a decision log. For every project since I started doing that it’s working great. My CLAUDE.md instruct the agent to store my every decision to the file with a metadata when I made this decision and what was the context. The agent is using this file as an index of decisions and rarely lose a track. It also helps team members to find out more about the development phases.
Does your system invalidate the parts of the memory if these are not valid or relevant anymore or just store/retrieve?
What I found out during malware analysis is that LLM agents have a couple of quirks that you can solve by:
- optimize for short lived agent workflows
- use symbols as function contracts
- maintain decision and discovery state
- give LLMs CLI linters
- give LLMs access to knowledge bases
The linter part is mindblowing. I built linters that validate HTML or markdown or docx files, for example, and output what kind of structure is expected (e.g. h4 inside h1? Must be h1 > h2 ...).
With linters the output quality of agents is just soo much better.
For program analysis, I'm currently exploring the idea of using an external ebpf daemon that programs can be observed with via a public API (which is the tool for the agent to use). Not sure if it'll do the trick yet, but I think it has lots of potential.
My stuff in case you're interested:
[1] https://github.com/cookiengineer/exocomp
[2] https://github.com/cookiengineer/gobayashi
[3] https://github.com/cookiengineer/gonano
[0] https://mirekrusin.com/cave
After googling for an hour, I gave up.
Soon, he'll discover that he needs quantifiers. Then that "for all" is too strong sometimes, and he needs "for most". That way lies Cyc.
It's not a bad idea. But it does have a history.
In fact, both Cyc and the "AI" Labs have the _same basic thesis_: Intelligence is, primarily, a data entry problem. They just disagree about what kinds of heuristics should be run over that data (logic-programs, neural-nets).
Whenever I read about someone using LLMs to write code, it _very closely_ resembles how Lenat was using Eurisko/Cyc to solve problems: they let the system run continuously, and they "nudge" it in "interesting" directions, "when it gets stuck", or "runs out of steam". (Quotes indicate their phrasing, not mine)
Even Lee Spector noticed something analogous with his genetic programming system. When he tried to get it to discover optimal data structures (or maybe it was sorting algorithms, I forget), the system would quickly "run out of steam", without a solution. But when they added new verbs/opcodes to the system, that were a better fit for that domain (e.g. index-based memory loads + stores), it converged on a solution very quickly (even for GP, domain specific languages keep delivering unreasonable wins). You will note that this rhymes with the "micro-theories" of Cyc, which in turn rhyme with the SLMs of the AI labs.
In my personal experience, most of the "silver bullets" do not work (obviously), but some of them do nudge you towards being a better programmer (by refining your intuition about the problem specifically, and computers more generally).
[0]: I suspect that, this was mostly a triumph of enormous amounts of hardware, more than the actual LLM technology. I further suspect that a traditional GP approach, on the same quantity of hardware, could have gotten there faster (if not better as well).
[0]: Because, in effect, we are decreasing the size of the search-space. Every programming-language project is basically a set of bets, that the design choices will decrease the size of the search space for the users of that language (most Unix commands fit this pattern, as do most well designed APIs). Mathematics is similar, in that the notation is specialized and sparse (even the named constants and variables have symbolic names, instead of meaningful names). Most system dynamics models could have the names of entities replaced with id-numbers, and they would continue to function. The whole point of (for example) a library is to give a tiny parameterized notation/language, that the user can use to express a solution to their problem, in terms of the problem domain itself. This decreasing of the search-space, is an act of compression. I have to say, I am a little confused by your reply, because it has the tone of a disagreement, but it does not really disagree with anything that I have written. Also, I have given various examples to support my analogies, but you have offered zero examples, and I suspect that there would be less confusion (on my part) if you did.
Think of it like human imagination may do anything (flying cars), but the real world has constraints and we use language/writing (symbols) and rules bound to them to simulate and reconcile our imagination with reality to actually flow our energy into something that may work in the real world.
Is my reading comprehension just completely broken or something? The above certainly sounds like "the powers that be want us to believe that attention is all you need".
Opposed to OP, DeepClause uses Prolog semantics, so running some more complex queries on knowledgebases might cause some issues (which is the use case where a Datalog might be more useful). For smaller scales it should be fine though.
[0] https://github.com/deepclause/deepclause-sdk [1] https://github.com/deepclause/deepclause-pi
btw. your comment is grayed out, not sure what it means. However, thanks for sharing, I'll look into it.
Once again, on LLM being: a digital librarian, at its finest; logic a logic analyst, not so much.