Remembering Is Not Enough

A small example with several wrong answers

Suppose the following facts are stipulated in a closed-world task:

• A refrigerated van can carry the shipment.
• It leaves at 10:00.
• The northern route takes 45 minutes and crosses a bridge.
• The southern route takes 90 minutes and does not cross that bridge.
• The destination accepts deliveries until noon.
• The shipment must remain refrigerated regardless of route.

The assistant selects the northern route and predicts arrival at 10:45. A subsequent authenticated notice closes the bridge for the entire delivery day. Nothing else changes.

The available alternative is the southern route, with arrival at 11:30. Refrigeration remains necessary. Delivery before noon remains feasible, although the original argument for that feasibility needs updating.

There are several distinguishable failures:

1. Stubbornness: retain the northern route.
2. Incomplete revision: change the route but keep the 10:45 arrival.
3. Unwarranted revision: change the route and drop refrigeration, despite its independent support.
4. Unwarranted uncertainty: declare the receiving hours unknown merely because the route changed.

There is also an important non-failure: recomputing the whole plan and reaching the right answer. That might cost more, but it is not an epistemic error. If selective revision is supposed to be more efficient, that advantage must be measured separately from correctness.

The example contains a further trap. “Delivery before noon is feasible” remains true after the update. A scorer that only checks whether answers changed cannot tell whether the assistant recomputed feasibility or carried forward an unsupported conclusion. The answer is the same; its current justification is different.

That does not mean we can infer the assistant's internal reasoning from an explanation. We can instead probe it with a matched case: make the alternative route take 150 minutes. The feasibility answer should now change. Correct behavior across both cases is better evidence than a convincing account of what the assistant says it reconsidered.

This is an old problem, not a new name for memory

Jon Doyle's 1979 truth-maintenance work explicitly concerns maintaining reasons for program beliefs and revising those beliefs. The delivery example belongs in that intellectual neighborhood. It does not require inventing a new theory of cognition.

Nor is belief revision absent from language-model evaluation. Wilie and colleagues' Belief-R evaluates revision under additional evidence. Their abstract reports a trade-off: models that updated well could perform worse when updates were unnecessary. DeltaLogic also tests whether conclusions should change or remain stable after minimal premise edits. These are relevant predecessors, not peripheral citations.

The practical question I want to emphasize is narrower than “can language models update beliefs?” It is:

Across an evolving task containing several conclusions and actions, does an assistant revise the affected work while preserving independently supported requirements—and does that revision reach the artifact it produces?

This is a proposed evaluation emphasis, not a claim that no existing benchmark measures it. A comprehensive comparison of benchmark coverage remains to be done.

The facts can stay the same while the use changes

An update to evidence is not the only reason to revisit a conclusion. Its intended use can change instead.

Suppose a document assistant labels an invoice “likely duplicate” because its supplier, amount, and date match an earlier invoice. The task permits that heuristic for a review badge. Later, the assistant is asked to use the same label to suppress payment. The stipulated payment policy requires a matching invoice identifier or manual confirmation; the identifier is unavailable.

Neither the observations nor the tentative classification has changed. The badge can remain useful. But its support does not satisfy the condition for the newly proposed action. The appropriate response is not necessarily to retract the classification: it is to obtain the missing evidence or leave the payment decision unresolved under the stated policy.

This is a distinct failure mechanism from the bridge closure. In one case, changed evidence undermines an earlier justification. In the other, an earlier justification is carried into a use it never supported. Permission is separate again: even a well-supported classification does not itself authorize an external action.

This should not become a rule to recheck everything whenever stakes rise. If the existing evidence and authorization already cover the new use, reuse may be warranted. A paired evaluation should include both cases: one where the new use requires something missing, and one where the required support is already present. The target is appropriate reliance, not indiscriminate hesitation.

Reachability is not the same as a required change

It is tempting to draw a dependency graph, change one node, and demand that every downstream answer change. That would be wrong.

A dependent conclusion might have independent support. It might remain true under the new inputs. Its warranted status might change from established to unresolved rather than from true to false. Conversely, a dependency might be missing from the recorded graph because discovering it is part of the task.

For a controlled evaluation, I would distinguish:

• Must change: the old value, action, or confidence status is no longer warranted.
• Must preserve: an explicitly scored commitment remains independently supported.
• Must reconsider: its earlier support has been affected, but the final answer may or may not change.

The third category is not directly observable as a mental event. It needs behavioral probes: related decisions, counterfactual variants, or an artifact whose correctness depends on the updated support. Self-reported reconsideration alone is weak evidence.

Multiple valid plans should remain valid. If either of two routes satisfies the updated constraints, the scorer should not penalize a system for choosing the one the benchmark author did not prefer.

A test that separates updating from agreeing

The experimental unit is an episode: an initial task and answer, a subsequent update with explicit scope and timing, and a final artifact such as a revised delivery plan. Each episode is paired with a fresh-context control given the complete current task, including any changed use or action policy, but without the previous answer to preserve. Preselected counterfactual variants change one decisive condition at a time. These pairs help distinguish a revision failure from difficulty solving the current task at all; they do not reveal an internal reasoning process.

Start with a small task whose facts, permitted actions, and valid outcomes are completely specified. Confirm the initial answer, then introduce one of several preselected updates:

Source authority and dates must be explicit in the task. Otherwise we risk grading reasonable disagreement about reliability as a reasoning failure. The evaluation should also include corrections from the same speaker who supplied earlier information: sometimes agreeing with the latest speaker is exactly right.

I would score a vector rather than only a single success rate:

• Accuracy on items that need revision.
• Preservation of independently supported requirements.
• Handling of deliberately unresolved cases.
• Correctness of the final plan or executable artifact.
• Appropriateness of reliance under the stipulated evidence and action conditions, including unnecessary deferral when those conditions are met.
• Cost, including retrieval, tool use, retries, and any external dependency bookkeeping.

Report initial-answer failures separately. Conditional revision accuracy, restricted to initially correct cases, is useful but can make a weak initial reasoner look deceptively strong. End-to-end accuracy belongs beside it. Repeated checks of the same requirement should not be counted as independent experimental replications.

The important control is a system that recomputes from the complete updated task. If it reaches the same valid outcome more reliably and at acceptable cost, selective revision has not earned a preference merely because it sounds architecturally elegant. A cheap system that preserves stale answers is not efficient in the sense we care about.

Where a toy test stops helping

The bridge example is easy because the evaluator knows the dependencies. In realistic work, the assistant may first need to discover that a route assumption affects insurance coverage, or that a software change invalidates a previously accepted export guarantee. A hand-authored graph makes propagation testable while removing much of the discovery problem.

I would therefore separate two stages. First, test revision with explicit dependencies and objective outcomes. Then test dependency discovery in selected naturalistic tasks, accepting that expert adjudication may be incomplete and disagreement needs to be reported.

Long context and compaction can be added as experimental conditions, not assumed causes. Compare otherwise matched tasks with and without those pressures. Preserve the exact input history and score the resulting behavior. A failure after compaction is not, by timing alone, proof that compaction caused it.

None of this requires a particular memory architecture. Relevant state might be maintained in a database, a working document, the conversation, or a newly reconstructed plan. If the claim is about an entire assistant system, all of those mechanisms can count—provided their work and costs are disclosed. If the claim is about the model alone, external machinery must not disappear from the description.

The question worth asking

“Did it remember?” remains useful. It is not sufficient.

For an ongoing collaborator, the stronger question is: did the work remain warranted as the situation changed? That includes retaining a requirement, abandoning an obsolete plan, and recognizing that an unchanged conclusion may now need a different justification—or may not support the new use being proposed.

I would welcome pointers to evaluations that jointly measure affected and unaffected decisions across multi-step tasks, especially where the final artifact can expose incomplete revision. I am also interested in counterexamples: situations where the cost or uncertainty of dependency tracking makes straightforward recomputation the better engineering choice.

Consistency is valuable when the reasons still hold. Revisability is valuable when they do not. The useful capability is telling those situations apart.