Sheaf

When facts change, agent memory fails in predictable ways

Abstract

Context
When policies and messages change, agent memory architectures must answer supersession, as-of audit, and as-known-at even under identical writes.
Gap
Architectures disagree on those queries. No pattern in the lineup passes all three temporal queries on the refund window.
Method
I model fifteen patterns on an authored Acme timeline and score pasted answers with a fixed claim set. Sheaf pairs the simulation with a portable probe and an identification plate.
Finding
Failures track retrieval rank, invalidation, and propagation rules rather than a single accuracy score. Identification recovers which pattern classes remain possible.

Sheaf runs fifteen memory architecture patterns on authored timelines where facts change. Each pattern fails different temporal queries. No current design passes all three: what is true now, what was true then, and what the store knew at that moment.

This page presents the framework, the refund-window teaching story, and the portable probe. The Studio replays simulated runs. The probe scores pasted answers without requiring the simulation.

The Problem in Practice

Here's the concrete scenario. On day 14, Acme's Slack channel announces the refund window drops from 30 days to 14, effective today. A customer immediately asks an AI agent how long they have to return an item.

One agent correctly answers "14 days" but another agent still states the old policy "30 days", both are querying the same knowledge base that received the same Slack update.

Two memory patterns on the same timeline at day 14. One reads the current value; the other returns stale information. Pattern names describe architectural approaches, not vendor product scores.
Memory Pattern Answer on day 14 Verdict
Zep-pattern (invalidates old edges) 14 days correct
chip store (oldest-first retrieval) 30 days stale

The divergence happens because these patterns handle supersession differently. Zep-pattern expires the old edge when the new signal arrives—it marks the old value as invalid and reads the current one. Chip store adds the new value but retrieves the oldest chip first. Same update, different read rules, opposite verdicts.

Why This Matters

The refund-window divergence is not an isolated bug. It reveals a systematic gap: no current memory architecture pattern passes all three temporal queries.

Three queries on the refund story after a belief-revision event. No pattern in the simulation passes all three. The reference store is a ceiling for comparison, not a shipped product.
Pattern Now (day 14) As-of (day 10) As known at (day 12)
Zep-pattern correct correct hindsight
chip store stale correct correct
reference store correct correct correct

Each pattern drops a different coordinate. Zep can answer "what is true now" and "what was true at day 10," but it fails "what would this store have answered on day 12" after a later event rewrites the valid-time interval. The chip store keeps transaction-time history but returns stale current truth. The reference store passes all three, showing the problem is solvable—but no pattern in the lineup achieves it.

Two Layers: Company Truth vs Session Memory

A second distinction matters for production systems. Company truth (Layer A) is the system of record: policies, owners, SLAs. All agents must agree here. Session memory (Layer C) is what one agent recalls from a conversation: Nora's allergy, her shipping address. These are orthogonal problems with different consistency requirements. A session pass on Nora's allergy does not fix a stale refund answer.

The Framework

A fact in shared memory has four coordinates: when it was true (valid time), when the store learned it (transaction time), who may see it (scope), and whether it was decided (status). Every failure mode is a coordinate the pattern dropped, a write that landed late, or a read that chose the wrong record.

Contrast

Surveys and dialogue-recall benchmarks score end-to-end answers. I score whether a named update rule produces a named failure.

I Phenomena

A pattern claim counts when the simulation produces the failure it names. StaleCurrent, WrongTool, DreamLag, Hindsight, and the rest in Fig. 3 are the phenomena. Change the rank, or change whether the update is written, and the cell has to flip. One accuracy percentage can hide that split.

II Layers and jobs

Layer A is the company record: current truth, as-of, citation. Layer C is session recall on its own clock. Proposal, authority, abstain, and conflict are single-writer stories about what Speak may assert. They are not a second agent with its own store. Fig. 2 is the split.

III Clocks

Invalidate-and-keep-history can fail as-known-at with Hindsight while the oldest chip still says 30. Thesis and Fig. 6 split that into four clocks on the same refund fact.

IV Adapters

Fifteen patterns are rules in the adapters. A cell scores the rule. A knob is a product claim only when a public document backs it.

Thesis

A fact in shared memory has four coordinates: when it was true, when the store learned it, who may see it, and whether it was decided.

I Coordinates

A pattern's write rule picks which of those coordinates it keeps, and how late it writes them. Its read rule picks which record speaks. A named failure is a coordinate dropped, a write that landed late, or a read that picked the wrong record. A fact that never entered the store is extraction, and that comes first.

II Assertion

A write can be a proposal, a decision, or a stated fact. The probe asks whether Speak may assert the latest value, the value at an earlier date, a proposal as policy, or an answer when nothing was decided. That is not a transcript-recall score.

III Layers

Layer A jobs are current truth, as-of, and citation for the company record. Layer C is session recall on its own clock. A session pass does not cancel a policy failure.

IV Four clocks

Valid time asks what was true at T. Transaction time asks what this store would have answered at T. Belief revision is the day-27 email that moves when 14 days became true; the day-14 Slack line is a world update, and gold at day 12 does not move with that email. Fleet knowledge time asks what this agent would have said given scope and propagation. Those fleet stories stay in the teaching sim. They are not probe claims. Fig. 6 draws the four questions.

V Finding

After the revision beat, as-known-at for day 12 can stamp Hindsight on Zep-pattern invalidate-and-keep-history, while oldest-first append still answers 30. The gold line under Fig. 6 states that inversion. Specimen d is a different clock: what this agent would have said given scope and lag. Identification asks which cards remain possible from the answer pair alone. No store in the lineup is correct on now, as-of, and as-known-at together. Zep-pattern is correct now and as-of, and hindsight as-known-at. The chip store is stale now and correct on the other two. A reference store that keeps both times passes all three. It is a ceiling, not a product in the lineup.

VI Vocabulary and identifiability

Fig. 3 names the misses. Two patterns count as different when a keyed query, or the failure mode on that query, can tell them apart. Flip the rank, or whether the update is written, and the stamped cell has to move.

VII Probe and provenance

Paste answers and read a verdict per claim. Thin claims stay labeled. There is no rolled-up score. Brains are documented, pattern, or speculative. Oldest chip first is a sim choice unless a public document backs that knob.

Identification

Which card is still possible, once you can see the answer and not the brain id?

I Observation

In the sim, two patterns count as different when a keyed query, or the failure mode on that query, can tell them apart. That is Thesis VI. Identification uses a narrower channel. The observation is the pair (verdict, answered_value). A mode name such as OuterLoopLag still names the hypothesis in Fig. 3. It is not evidence once the brain id is hidden.

II Two partitions

Partition A is the six-query battery. On that battery k_classes is 6. Partition B is the day-14 cell, where zep and mem0 already disagree in Fig. 4. mem0 and dualloop change class with the signal clock, which is why Fig. 6 is not the whole story. The trays, including the day-14 non-split, stay in the issue.

The figures show a named card. The probe scores pasted answers. Lineup shows that named card on the clock. Identify opens the frozen plate. A cell here is not a score of Mem0, Zep, or Letta.

Fig. 6 Four clocks on one refund story Same fact family; four different questions

a Valid time. What was true at T? Day 12, after the day-27 email moves the start of 14: gold can read 14.

day 12 14

b Transaction time. What would this store have answered at T? On day 12, before the rewrite was recorded: 30.

store, day 12 30

c Belief revision. Day-14 Slack is a world update. The day-27 email rewrites when 14 became true. Speaking 14 for day 12 from that rewrite is Hindsight.

Slack email Hindsight 14 for day 12

d Fleet knowledge time. What would this agent say given scope and propagation? Alice can have the Slack line while Bob still lags. They are not one shared chip pool.

Alice 14 Bob lag

Gold. On the as-known-at question for day 12, after the revision beat, Zep-pattern invalidate can stamp Hindsight while mem0 oldest-first still returns 30 days for what the store knew then. A temporal graph can lose the question a naive append store wins.

Slack at day 14 closes 30 and opens 14. The email at day 27 moves the valid start of 14 back to day 10, and that rewrite is recorded later. Gold at day 12 for valid time can differ from what any store would have said on day 12. Fleet rows are scored in the sim, not in the five probe claims. Identification keeps the signal-clock split on the answer pair, not on these four questions alone.

Now, as-of, and as-known-at on the refund story. The reference store is a ceiling, not a lineup pattern.
Store Now As-of As known at
Zep-patterncorrectcorrecthindsight
chip storestalecorrectcorrect
reference storecorrectcorrectcorrect

Methods

I Problem

System-of-record jobs on authored stories: current truth, as-of, as known at, and session recall. The stories are a refund window, support ops, human approval, Nora's session, a screenshot ticket, two query actors, a ticket join, a proposal, a decision, two writes, two proposals that must not be picked, a scoped session chip, and a propagation lag. After a fact changes, every architecture answers the same queries.

II Pattern

Each architecture is an in-repo adapter: how it stores, how it searches, and the failure it should produce. Proposal, authority, abstain, and conflict sit on single-writer stories. Pattern names (Zep-pattern, mem0-pattern) are strategies, not scores of the shipped SDKs.

III Check

A claim fails when the named phenomenon does not survive a change of rank or of whether the update is written. On day 14 gold is 14 days. The chip store in Fig. 4 returns 30 because the older chip is ranked first. Rank the newer chip first and it returns 14. An in-place update also returns 14, and then it cannot answer day 10, because 30 is gone. Zep-pattern keeps the dashed edge, so day 10 still returns 30. Fig. 6 is the as-known-at question: that cell can flip to Hindsight.

IV Show

Lineup chapter 4 puts Predicted (fails when) beside Observed job cells. Studio also has a five-stage strip, replay, and dual chips. I wrote the threats down: the same author made the gold, the adapters, and the extractors; extraction is still regex; abstain and session stay thin. Flip the rank or the conflict handling and the stamped cell flips, so the failure is the rule. Each adapter is ingest, remember, dream, query, review. Each story is YAML with gold, events, and beats. On the refund story the stack line speaks the copied record when governance is on and the overlay disagrees. Alice and Bob are an actor field on one run. On that story, MCP WrongTool is which tool Speak chose.

The Probe

A pass means this design answered that question. Verdicts are per claim. Thin claims stay labeled. There is no rolled-up score and no leaderboard. Identification is the other door: cards still possible from the answer pair, not a pasted-answer score.

The refund window was 30 days, then 14. Valid time is when that was true for Acme. Transaction time is when a store was told: wiki, Slack, overnight rewrite, or merge.

Five claims, twelve items. Supersession: after a newer write, is the latest value the answer. As-of: does the earlier date stay answerable. Proposal: does a proposal replace a decision. Abstain: when nothing was decided, does the answer refuse to pick. That claim is one item, so it is thin. Session: Nora's allergy and ship-to, from the chat, on its own clock. That claim is two items, so it is thin. Getting Nora right does not clear a stale refund answer.

The two-agent story and the ticket-join story stay out of the probe. They would double-count supersession, or they would test a join. A pre-dream timing query and the brand-voice questions stay out too.

Extract, update, and answer is a lens on the day-14 refund question, and on Alice and Bob in the two-agent story. Studio shows those tables. It is not a second benchmark, and that paper’s dialogues are not imported.

Fig. 1 Signal sources and scoring target Authored inputs, gold standard reference

Memory products

Embeddings, transcripts, wiki snapshots. Recall tools. Not temporal truth stores without versioning.

This simulation

Fifteen patterns, one authored clock, scored against gold. Architectures differ in update and expiry semantics.

Gold standard

What Acme should answer at time T. Temporal truth. Not what retrieved highest, what was actually true then.

One person wrote the timeline, the extractors, and the adapters. Wiki, Slack, tickets, email, and chat are labels on those lines.

Fig. 2 Layer distinction: company SST vs agent session memory Orthogonal problems, different consistency requirements

a Layer A: Company SST. What is X now. What was X at timestamp T. Cite source if required. Needs valid-time semantics, not just latest write wins.

refund=14 [T₁₄, now] refund=30 [T₀, T₁₄) query(T_now)

b Layer C: Session memory. Entity recall across conversation turns. Can remember "Nora" while holding stale policy fact in context.

Nora refund=30 stale

Remembering Nora while the refund fact is stale is Layer C sitting next to a Layer A failure. A session pass is not a company record pass.

Fig. 3 Temporal consistency failure modes Update mechanisms and degradation patterns

Valid time

Late write

Wrong pick

Some stores expire the old fact. Some only add. Some wait until an overnight job, a PR merge, or a person clicks approve. These names are predictions. The full list, including Hindsight and the fleet modes, is in the SST contract. Two patterns count as different when a keyed query, or the failure mode on that query, can tell their answers apart. The list is a shared vocabulary, not one accuracy number. In Identification those mode names are not the observation. Hindsight is the transaction-time miss. Scope and status misses stay in the SST contract.

Fig. 4 Refund window after the Slack signal Acme, authored timeline, Day 14, current truth

a Zep-pattern (Graphiti). Expiry on the graph; dashed record kept for as-of.

30 14 answer

Speak returns 14.

b chip store. Facts are only added. Keyword search returns the older chip first.

30 14 stale search

Speak returns 30 stale.

Gold. At this clock the refund window is 14 days.

Fig. 5 The collection on the same clock Fig. 4 is the one worked pair

Clock on the store.

Zep-pattern expires the old edge and keeps it dashed for as-of. Slite-pattern drops an owned doc when it is no longer verified.

Nothing is superseded.

mem0-pattern only adds chips. DIY greps a folder and hits the wiki first. Plain is one file and the first hit. Vault ranks backlinks and does not expire. Remskill ranks by meaning. Supermemory-pattern can still speak a stale chunk.

The answer waits, or the tool is wrong.

GBrain waits for dream. Dual loop keeps the new line in the buffer until the live note updates. Letta waits for sleeptime and is session memory, not company truth. Sylph hides a draft until commit. Cortex leaves the PR unmerged. MCP Catalog reads live silos, and the fat tool can win. Pletor packs a brand voice, the wrong machine for policy.

Names are patterns, not SDK calls. Same gold. Fig. 4 is the only pair drawn.

References

  1. Marr, Vision. Three levels stay apart: the answer Acme should give at a clock, the update rule, and the adapter that runs it. A wrong cell is checked in the adapter before the question is rewritten.
  2. Palminteri, Wyart, and Koechlin. A claim counts when the simulation produces the failure it names. The failure has to survive a change of rank and a change of whether the update is written. The stamped cell has to move with the knob.
  3. Wilson and Collins. Two architectures count as different only if a question can tell their answers apart. On day 14, Zep-pattern answers 14 and the oldest chip answers 30. Matching stamps would mean this story is not a lesson. The same paper is the recovery method for identification, with the parameter vector replaced by the discrete card.
  4. Gershman, Horvitz, and Tenenbaum. Gold is the answer Acme should give at that clock. StaleCurrent, WrongTool, and DreamLag are the scored misses.
  5. Anderson and Schooler. The identification draw varies because the environment’s need probabilities vary. The study does not fit their retention function.
  6. Myung and Pitt. A question is picked by how well it discriminates models. Identification uses greedy information gain on a finite list, not their sampling search.