Agent Memory That Shows Its Work: Facts, Derivations, and Deterministic Retraction

Every long-running agent eventually contradicts its own history. Jordy Zomer, writing on pwning.systems in August 2026, names the failure precisely: models “suggest an approach that we had already ruled out, forget that an assumption turned out to be false, or confidently continue reasoning from an observation that was no longer valid.” He hit it while building vulnerability-research agents, where a stale assumption does not just waste tokens. It sends the whole investigation down a path that was already closed.

His response, a system called Lemmalog, is the most interesting agent-memory design we have read this year, and its benchmark scores are worse than the specialized baselines it compares against. Both halves of that sentence matter. Lemmalog stores knowledge as Datalog facts and rules instead of prose chunks or embeddings. Conclusions carry their derivation paths. Retracting an input deterministically retracts every conclusion that depended solely on it; a conclusion with alternative support survives. Facts carry validity intervals. The price is recall on questions that require inference across facts, and Zomer publishes that price with error bars.

We have been circling this property from the governance side for months. Our survey of the three memory patterns mapped how teams store agent knowledge. The governance question asked who controls what an agent remembers and found that current systems could not answer “why does the agent believe this” in any structural way. Lemmalog is the first design we have seen where that answer exists by construction rather than by logging discipline. Whether it is the first anywhere, a single-author blog post cannot establish. What it can establish is the mechanism, and the mechanism is worth understanding in detail.

Memory as facts and rules, not text

Conventional agent memory, whatever the storage engine underneath, holds statements. “The auth service validates tokens locally.” “We ruled out the race condition in the queue.” The statements sit next to each other with no structural relationship. When the agent later retrieves them, nothing in the store says which statements were premises, which were conclusions, and which conclusions silently assumed a premise that has since died.

Datalog changes the unit of storage. A fact is a structured assertion. A rule derives new facts from existing ones. When the engine derives a conclusion, the derivation itself, which inputs produced it and through which rule, is recorded as part of the store. Ask the memory why it believes something and the answer is a proof tree, not a similarity score.

The machinery itself predates agents. Zomer’s post title, “I accidentally turned LLM memory into program analysis”, suggests where the machinery came from: the logic engines static analysis uses to trace what depends on what. The novelty is pointing that machinery at the knowledge an LLM agent accumulates across a long investigation, where the store must survive the agent being wrong.

Retraction is where the design earns its keep

Being wrong is the normal case. An observation gets superseded. An assumption tests false. In a prose-based memory, the invalidated statement can be deleted, but its descendants cannot, because nothing links them to it. The model keeps retrieving conclusions whose foundations are gone. That is the resurrection failure Zomer described, restated as a data-structure problem.

Lemmalog’s answer: the engine tracks multiple derivation paths per fact. Retract an input and the system walks the dependency structure. A conclusion supported only by the retracted input is invalidated. A conclusion that has an alternative derivation, another independent line of support, survives. The recomputation is incremental, touching what changed rather than replaying the whole knowledge base.

Validity intervals extend the same discipline into time. A fact can be true for a window and expire, so a stored observation that a service was throwing errors does not linger as a present-tense belief long after the deploy that fixed it.

For anyone who has operated agents in production, the appeal is immediate. Retraction stops being a prompt-engineering plea (“please disregard earlier findings about X”) and becomes a database operation with defined semantics.

The numbers, including the losses

Zomer benchmarked the system himself, and the results are mixed in a way he does not soften.

On LongMemEval, Lemmalog scores F1 0.463 ± 0.010. The baselines beat it: PropMem at 0.550, SimpleMem at 0.480. The context budget tells the other half of the story. Lemmalog answered with roughly 2,700 tokens of context against roughly 104,000 for the baseline setup, about 38 times smaller.

On LoCoMo it scores F1 0.533 ± 0.001 against PropMem’s 0.605, at around 6x less context. One category stands out: adversarial questions score 0.707, the category where, in Zomer’s own gloss, refusing the bait is rewarded. Zomer’s gloss: “‘no’ turns out to be quite a useful answer.” A memory that verifies facts before asserting them is unusually good at refusing bait.

The weak categories are just as legible. Preference questions score 0.128. Inferential questions score 0.164. And the strict fact-verification that makes the store trustworthy caused 32 of 102 refusals on questions whose answers required reasoning across facts: the system declined to answer rather than assert something it could not derive. For a chat assistant recalling that you prefer window seats, that behavior is a liability. For an agent whose conclusions feed security decisions, refusing to guess is closer to the point.

Two caveats belong next to every number here. This is a single-author side project, and the LongMemEval and LoCoMo comparisons are the author’s own runs, not third-party evaluations. Treat the exact figures as a first report, and treat the shape of the trade-off, weaker recall, structural provenance, as the finding that will survive replication.

Auditability by construction

Read those results against the question our governance essay left open. Every memory system we surveyed treats auditability as an add-on: log the writes, timestamp the reads, hope the logs reconstruct why the agent believed what it believed. The logs record that a memory was stored and retrieved. They do not record what it justified.

A Datalog store inverts that. Provenance is not a log beside the data; it is the data. The derivation path exists because derivation is how the fact came to exist. Deleting an input cannot leave orphaned conclusions, because the dependency structure is what the engine executes. When we wrote about memory benchmarks and architectures, the systems on the leaderboard optimized recall. Lemmalog optimizes for a different property: answers carry their derivations, and when it cannot derive one it refuses.

That guarantee has a compliance texture to it, and the trade-off has a familiar shape. Databases gave up the flexibility of free-text files for schemas, and got integrity constraints in return. Lemmalog gives up recall on fuzzy inference and gets a memory whose beliefs can be audited, retracted, and time-bounded with database semantics. Zomer built it for vulnerability research. The same property is what an auditor will eventually ask of any agent whose remembered conclusions drive real decisions.

What to do with this now

Pick your highest-stakes agent workflow and ask one question of its memory system: if a stored assumption is invalidated today, what happens to the conclusions that were derived from it? Trace it concretely. Find one memory entry that depends on another, delete the premise, and observe whether the dependent entry survives. If it survives with no flag, you have the resurrection failure in production, whether or not it has bitten yet. You do not need to adopt Datalog this quarter. You need to know whether your agent’s memory can show its work, and our bet is that for your stack, today, the honest answer is no.

This analysis synthesizes I accidentally turned LLM memory into program analysis (Jordy Zomer, pwning.systems, August 2026).

Victorino Group helps engineering teams design agent memory and governance architectures where provenance is structural, not aspirational. Let us talk.