HN Explorer

Ask HN: Thinking about memory for AI coding agents

7 points by hoangnnguyen ↗ HN
I’ve been experimenting with AI coding agents in real day-to-day work and ran into a recurring problem, I keep repeating the same engineering principles over and over.

Things like validating input, being careful with new dependencies, or respecting certain product constraints. The usual solutions are prompts or rules.

After using both for a while, neither felt right. - Prompts disappear after each task. - Rules only trigger in narrow contexts, often tied to specific files or patterns. - Some principles are personal preferences, not something I want enforced at the project level. - Others aren’t really “rules” at all, but knowledge about product constraints and past tradeoffs.

That led me to experiment with a separate “memory” layer for AI agents. Not chat history, but small, atomic pieces of knowledge: decisions, constraints, and recurring principles that can be retrieved when relevant.

A few things became obvious once I started using it seriously: - vague memory leads to vague behavior - long memory pollutes context - duplicate entries make retrieval worse - many issues only show up when you actually depend on the agent daily

AI was great at executing once the context was right. But deciding what should be remembered, what should be rejected, and when predictability matters more than cleverness still required human judgment.

Curious how others are handling this. Are you relying mostly on prompts, rules, or some form of persistent knowledge when working with AI coding agents?

7 comments

[ 2.9 ms ] story [ 18.3 ms ] thread
7777777phil
Earlier this month I argued why LLMs need episodic memory (https://philippdubach.com/posts/beyond-vector-search-why-llm...), and this lines up closely with what you’re describing..

But not sure it's a prompts vs rules problem. It’s more about remembering past decisions as decisions. Things like 'we avoided this dependency because it caused trouble before' or 'this validation exists due to a past incident' have sequence and context. Flattening them into embeddings or rules loses that. I see even the best models making those errors over a longer context rn.

My current view is that humans still need to control what gets remembered. Models are good at executing once context is right, but bad at deciding what deserves to persist.

nasnasnasnasnas
Claude.MD /agents.MD can help with this ... You can update it for a project with the base I formation that you want to give it... I think you can also put these in different places in your home dir too (Claude code / open code)
dabaja
We hit something similar building document processing agents. What helped was treating memory as typed knowledge rather than flat text. Three buckets: (1) constraints -- hard rules that should always apply, (2) decisions -- past choices with context on why, (3) heuristics -- soft preferences that can be overridden. Retrieval then becomes: constraints always injected, decisions pulled by similarity to current task, heuristics only when ambiguity is high. Still experimenting with how to detect "ambiguity" reliably -- right now it's a cheap classifier on the task description. The deduplication problem is real. We ended up hashing on (topic, decision_type) and forcing manual review when collision detected. Brutal but necessary. TBH we are not home yet but that's the path we walking.
GOGOGOD
IMO CLIs already have strong constraint channels (CLAUDE.md / agents.md / rules), the real pain is manual upkeep + agents forgetting to read/update them. If docs can be auto-maintained, that’s actually useful “memory”. We’re shipping an OSS experiment in ~1 week: https://github.com/hakoniwaa/Squirrel