When Stale Constraints Go Unchecked: Budgeted Verification Failures in Inherited Agent Memory

Engineering notes

Engineering notes will be added by the aipentium editorial team.

Chinese explanation / 中文解读

中文解读待补充：本站会优先为大语言模型、生成式AI、ChatGPT相关技术、计算机视觉、深度学习等高价值论文补充中文说明。

Original abstract

Abstract. Provenance links keep the evidence behind an inherited belief reachable; an agent with a verification budget must still choose which links to inspect. We study a consolidated memory that states a decision constraint and whose source record has since been superseded by a record that withdraws it: provenance is immutable, the current record has changed, and the memory is stale. In a controlled six-memory scenario with a budget of two records, sixteen language models rarely re-verified a constraint that read as settled: they inspected its provenance path in about one episode in five and, once the constraint had been superseded, produced stale-consistent decisions in 77.3%, 74.7% and 74.7% of episodes across a primary run, a replication and a held-out domain. Re-assigning one of the same two slots to the critical path removed most of them: +74.0, +72.7 and +61.3 points (positive in every model), +80.7 in a prospectively frozen interleaved replication with a repaired non-critical control, and +62.0 on a further panel of 10 models from 9 organisations new to the study; a corrected re-run of the held-out scenario, whose frozen text carried a temporal inconsistency, gave +73.3. The forced-critical policy uses experimenter knowledge of the critical path: it quantifies how much stale-decision risk the same budget can recover and is not a scheduler. Two further deposited experiments locate the failure and a remedy: in this store the constraint's path is selected in 17.0% of episodes at two slots and 88.7% at four of six (above uniform allocation), and at two slots a one-sentence, target-blind rule, prefer memories that state a limit on a candidate direction, moved the agent's own allocation onto the constraint's path and recovered the oracle contrast on decisions (+89.3 points) in a store where that constraint limits the tempting action, while a content-free freshness cue did not materially redirect allocation and a content-matched control rule changed neither selection nor decisions. Version notes (v3). Version 3 adds four experiments designed after version 2, each specified, frozen, timestamped (OpenTimestamps) and deposited to OSF before its first confirmatory model call: Experiment A, an interleaved replication of the headline contrast with a repaired non-critical control (900 episodes; +80.7 points; OSF file rba9z); Experiment B, a content-free freshness cue under native allocation (600 episodes; inconclusive; e4dx5); Experiment X, the same contrast on a ten-model cross-organisation panel served through pinned providers (1,498 episodes; +62.0, positive in 10/10; 6a906d658dd0e96801374be4); and Experiment C, a budget sweep (k = 1-4) on the original store and target-blind allocation rules in a three-constraint store (5,400 episodes; the constraint's path is selected above uniform allocation at four of six slots; a one-sentence rule recovers the oracle contrast at two slots, +89.3; 6a90f30053ff92cdfe89790b). The abstract, contributions, related-work boundary and limitations were rewritten around them; Table 1 was extended and Figures 2 and 3 redesigned. The four original runs' data, estimates, intervals and the values tabulated and plotted for them are unchanged from version 2 (and version 1); every Experiment A/B/X/C number is emitted by an audited generator from the frozen analysis outputs and the locked episode files. Post-execution disclosures are in the appendices: Experiment X's runner misclassified four connection errors (one episode lost); Experiment C's completeness-check script was corrected after its run, before locking, and its review-dispositions record is append-only, so two of its 103 package-manifest entries no longer verify on the current tree (the deposited package holds the frozen bytes); the frozen analysis output's text label for one Experiment C quantity is inverted (an erratum of the print statement, not of the value). Versions 1 and 2 remain available unchanged under this record's concept DOI. Data and code availability. All 5,400 confirmatory episode files of the four original runs (exact prompts, raw responses, parsed objects, deterministic scores) and the 48 labelled pilot episodes, the 1,500 episode files of Experiments A and B, the 1,500 raw files of Experiment X (1,498 episodes, 2 error files) and the 5,400 episode files of Experiment C with their per-call metadata, the sealed smoke-gate outputs, the frozen specification packages with SHA256 manifests and OpenTimestamps proofs, the registration records with their public-registry identifiers, the frozen analysis scripts with their committed outputs, the independent recomputation and forensic-reanalysis scripts with outputs, the runners, the generators and the audit are in paper2-data-and-code-v3.zip (README inside). Re-running every analysis and rebuilding the paper requires only Python 3.12 and a TeX distribution; re-running the experiments requires provider API keys, which are not included, and cannot reproduce the same model versions (provider aliases were not snapshot-pinned). Evidence / prospective-specification statement. For the primary run, the fresh-wording replication and the original held-out run, the complete specification was frozen, hashed, committed and cryptographically timestamped (OpenTimestamps, 2026-08-25 23:05:06 UTC) before the first confirmatory model call (23:06:42 UTC); the package was deposited to OSF after the runs (project axsnm, files 75kaw and 8wes5) and verified against the pre-run manifest hash-for-hash. This deposit is an archival record, not a preregistration. For the corrected held-out replication (hdm75) and Experiments A, B, X and C, the complete package was deposited to OSF and verified byte-for-byte before execution; success criteria and interpretation matrices were fixed in advance and could have failed. No deposited package was replaced or amended. Self-found defects are disclosed with their size in the paper (a temporal inconsistency in the original held-out scenario; a design limitation of the forced-noncritical control; Experiment X's runner classification defect; Experiment C's post-freeze lock-script correction and the append-only dispositions record). AI assistance. See the statement in the paper's back matter: the author used Anthropic's Claude (principally through Claude Code) for design critique, planning, implementation and execution of the runners, analysis and audit tooling, drafting, editing, simulated adversarial review and release engineering, and OpenAI's ChatGPT (including Codex) for design critique, interpretation discussion, manuscript critique, simulated adversarial review, and publication and release planning. The author is responsible for the research question, the decision to run each experiment, interpretation, claims, publication decisions and correctness. No model is an author; the sixteen models studied are experimental subjects. Suggested citation. Nakayashiki, K. (2026). When Stale Constraints Go Unchecked: Budgeted Verification Failures in Inherited Agent Memory (v3). Zenodo. https://doi.org/10.5281/zenodo.22147784 Relation to prior work. This paper tests the case that the author's earlier paper, Verification Allocation in Inherited Agent Memory: Provenance Availability Is Not Provenance Use (doi:10.5281/zenodo.22084498), explicitly left untested; it reuses that paper's instrument with a different design-assigned variable, different data and a different outcome, and Experiment X reuses its cross-organisation model panel.

Links and sources

Comments