I love the idea! OKF 0.2 solves for the problem of the AIs generating massive amounts of documentation (far more than humans ever created) and giving it equal-weight over what a human actually approved and committed to. I've been using it to attribute my decisions with specific directions on how to ensure that it takes strong direction from my explicit decisions and clarifies implicit / AI-driven decisioning.
Adding on progressive disclosure to this is brilliant, and I love the idea of a fast, in-memory, single-binary tool. This is a great way to approach the solution to this problem.
One thing I will say though - I would never be able to use this in my enterprise. It would just be too much of an uphill battle to purchase something that is so niche in utility - this tool is not a ton different than just having the md files locally and having it use ripgrep to search over them, and telling CLAUDE to write the OKF files as well as an index when it makes changes, is it? is the index generated dynamically / is anything about the progressive disclosure different than just having the agent manage it while it documents?
If you're going for smaller teams that can buy tools without a ton of approval / procedural overhead, I think that might have some success. another possible solution would be to make the cross-repo search something that you can handle with OSS but you have to self-host, and then pay for support. if you got enough usage and penetration within an enterprise from the teams just using OSS and self-hosting, they might consider buying support after-the-fact.
Love seeing projects like this. The performance benchmarks are nice to see. Have you done any benchmarks against approaches like OpenAI's Symphony for things like token usage or task completion?
How well does the model adhere to using this in a harness like Codex where it may be directed to use the built in memory tooling? Maybe I'll need to try an experiment directing it to save to its native memory to use OKF instead
This is usually my main concern with tooling like this that isn’t a first party project. Anthropic can tune Opus, Fable, etc and their harness to use their memory format or preferred method of tool calling. I have had mixed results getting LLMs to consistently use third party tools.
I’m very much in favor of things like OKF wikis for memory or knowledge storage/retrieval. So I too would love to know how well this really integrates into one of the coding harnesses (Claude code or Codex mainly).
I want this, but also for cross-project memory. Save me from building my own, which I have planned but figure something would eventually pop up in HN...
"With Astra, we’re introducing a new way for Codex to preserve and retrieve context when the context window fills. Historically, models have used compaction to summarize work during long sessions, such as when debugging complex issues or tackling large refactors. Each compaction can leave out details about why a fix failed or how a component behaves. In Codex, Astra can keep notes across context windows, preserving accumulated details without repeatedly compressing them into a single summary. Earlier context windows remain searchable, so Astra can find requirements or test results from previous messages and tool outputs—even if that information wasn’t captured in its notes. You can enable this experimental feature in your Codex config.toml, (opens in a new window) and it will become the default for Astra in the coming weeks."
This clearly is Codex-specific, not as much a feature of the model (though obviously they probably have trained it to be great at working with their own tools). Sounds somewhat similar to pi-observational-memory I'm using with Pi.
I don't get it. Why benchmark the latency instead of recall/precision? Optimizing for millisecond-level latency is meaningless in the context of LLM calls. Accuracy is the tool's greatest value, yet there is no testing for it?
Has anyone else benchmarked all these tools for precision/recall? I too want to know if agent memory is something I should add. I only do session memory for now and that is quite useful.
Yes. See LongMemEval, LoCoMo. Tons of research here.
But precision/recall is relatively "solved". What nobody has gotten close to solving is maintenance and provenance - what goes into memory, what qualifies as truth, how stale memory gets invalidated/superseded.
We're now in the phase of re-discovering 30+ years of pain of knowledgebases.
Personally I've always seen AI 'memory' as a pain point for people in their experience using LLMs than a benefit from the agent remembering the last unrelated thing you were working on. It wastes context similarly to 'skills'. The most efficient workflow imo is having a few well written (not by ai) md files across a clean codebase.
To avoid losing context, I mainly conduct the planning session and the implementation session separately.
From the standpoint of building enterprise products, what worries me most is whether the agent we are implementing may not have understood a completely different context.
If okf_memory maintains domain knowledge very well, it is expected that implementation will be possible in unit functional units within a consistently smooth session.
However, there is a risk in applying this idea directly to practical work, so I’ll have to test it separately on a personal project.
Nice tool! If you want to publish your okf bundles for humans to read (like on a github pages or your intranet), i developped an open source solution to do that : https://github.com/oak-invest/kiso - It's like Hugo for OKF
47 comments
[ 0.25 ms ] story [ 2.8 ms ] threadAdding on progressive disclosure to this is brilliant, and I love the idea of a fast, in-memory, single-binary tool. This is a great way to approach the solution to this problem.
One thing I will say though - I would never be able to use this in my enterprise. It would just be too much of an uphill battle to purchase something that is so niche in utility - this tool is not a ton different than just having the md files locally and having it use ripgrep to search over them, and telling CLAUDE to write the OKF files as well as an index when it makes changes, is it? is the index generated dynamically / is anything about the progressive disclosure different than just having the agent manage it while it documents?
If you're going for smaller teams that can buy tools without a ton of approval / procedural overhead, I think that might have some success. another possible solution would be to make the cross-repo search something that you can handle with OSS but you have to self-host, and then pay for support. if you got enough usage and penetration within an enterprise from the teams just using OSS and self-hosting, they might consider buying support after-the-fact.
https://github.com/ucsandman/declick
I’m very much in favor of things like OKF wikis for memory or knowledge storage/retrieval. So I too would love to know how well this really integrates into one of the coding harnesses (Claude code or Codex mainly).
Maybe?
Give it a try, hope it will help you to solve your need without injecting anything in your context all the time.
https://openai.com/index/gpt-6-astra/
https://github.com/fellowgeek/mcp-memory
https://news.ycombinator.com/item?id=49286073
But precision/recall is relatively "solved". What nobody has gotten close to solving is maintenance and provenance - what goes into memory, what qualifies as truth, how stale memory gets invalidated/superseded.
We're now in the phase of re-discovering 30+ years of pain of knowledgebases.
On 3 months of my own sessions I’ve seen that BM25 search was finding the correct answer in ~61%, where semantic had shown only ~37% of success.
After that it was easy for me to make the decision.
Got all info on how I did evals in here, if interested: https://github.com/tenequm/pond/tree/main/docs/researches/26...
To avoid losing context, I mainly conduct the planning session and the implementation session separately.
From the standpoint of building enterprise products, what worries me most is whether the agent we are implementing may not have understood a completely different context.
If okf_memory maintains domain knowledge very well, it is expected that implementation will be possible in unit functional units within a consistently smooth session.
However, there is a risk in applying this idea directly to practical work, so I’ll have to test it separately on a personal project.