Belief Surgery in a Visual Agent

August 14, 2026

2026-08-14 — mechanistic interpretability · embodied agents · a pilot result

Here is a question I find genuinely interesting: when an agent has to remember something it can no longer see, does it hold a belief about the hidden world — something you could read off, edit, and have the agent act on — or is "memory" just a diffuse smear with no handle? And if there is a belief, does it behave like a belief (it updates when new evidence contradicts it) or like a command (it just sits there)?

To test it cleanly I trained a small recurrent agent (a CNN→GRU, ~0.4M params, and a CNN→LSTM as a second architecture) on a first-person RGB memory maze. Early in each episode the agent briefly sees a colored cue telling it which arm hides the goal; the cue then leaves view. At the T-junction the two possible worlds are rendered pixel-for-pixel identical — the only thing that distinguishes them is the agent's memory. It's trained only to reach the goal; no belief labels, ever.

The result, after an adversarial red-team of every claim: you can decode the agent's belief about which arm hides the goal (AUROC 1.0, no non-visual leak) and surgically overwrite it — the agent then walks confidently to the wrong arm as if the world had changed, while a random edit of the same magnitude does nothing. And the belief has a confidence knob a mere command can't have: make it uncertain and the agent goes and looks before committing (a clean, control-checked effect). And — with a catch that became the most interesting part — when independent evidence about the true arm arrives, the edited belief can be overwritten by it, on the evidence's information content, not its salience. The catch: whether an edited belief yields to contradicting evidence is itself a trained property with a sharp, low threshold. Train the agent on even 10% episodes where late evidence should override an earlier belief and it's razor-clean; train it on none and the belief shields the agent from the evidence entirely — a belief-perseverance effect. So: a hidden belief you can read, edit, make uncertain, and (conditionally) update with evidence, in a visual agent. How new is that? Less than I first wrote here. When I finally did the prior-art hunt properly (I should have led with it), most of these moves turn out to be published already — in transformers/LLMs. I'm leaving the honest reckoning in, because getting it wrong and correcting it is the more useful thing to publish. See "Prior art, honestly" below.

The hero clip

Two agents in identical mazes; the lower one's belief is edited and it goes the wrong way, then self-corrects — Top: the agent in the real world. Bottom: the *same* agent, same maze, but with its belief about the goal arm surgically overwritten at the junction. It commits to the wrong arm while blind, then hits the dead-end and backtracks to the goal. The edit and the commitment are real (a random edit does nothing); whether that *backtrack* is a genuine belief-update or just the trained recovery reflex is exactly the confound the red-team caught — see below. The current frames are identical across the two worlds; only the internal belief differs.

Reading and editing the belief

From the recurrent hidden state at the aliased junction, the hidden fact decodes at AUROC 1.0, calibrated (ECE 0.0). Overwriting that direction flips the agent's chosen arm; a matched-DOF, matched-norm random edit of the same magnitude — as large as the entire hidden state — does 0.2–9% of the work. A memoryless version of the agent, seeing identical frames, is at chance, so the behavior genuinely requires a maintained belief, and a same-fact/different-world edit leaves behavior unchanged (locality).

Filmstrip: the edited agent's observation stream becomes identical to the natural agent in the other world — Paired-world equivalence: after the edit (bottom), the agent's observation stream from the junction onward becomes *identical* to the natural agent in the other world (top) — it is walking the counterfactual world's path while physically standing in this one.

Belief, not command: two signatures

A goal-channel edit would just sit there. A genuine belief has two dynamical tells that a command can't fake:

U — uncertainty drives seeking (holds): Injecting uncertainty along the belief's uncertainty geometry makes the agent go and look before committing: scout rate rises 0 → 1 with injected uncertainty (Spearman ρ ≈ 0.89), while a matched-norm control direction gives ρ ≈ 0.00 on both architectures. This is the load-bearing "belief, not command" signature — a command has no confidence knob.
E — evidence overwrites it (holds as a trained capability): My first maze had no cue-independent evidence — the only disambiguating pixels were the wrong arm's dead-end wall — so the "self-correction" was inseparable from the trained recovery reflex, and a red-team rightly knocked the claim down. The fix: an independent mid-arm re-cue that reveals the true arm before any dead-end. In an agent trained to let late evidence override belief, the edited agent then reverts at the re-cue, before the dead-end, 1.00, while matched controls fail (edit-consistent re-cue 0.00, neutral equal-salience marker 0.00, no re-cue 0.00) — so it's the re-cue's information content, not salience, that drives it. But a transfer test showed this is a trained capability, not automatic: at 0% override-training the edited belief instead shields the agent — it ignores the informative re-cue and runs to the dead-end (belief doesn't flip), even though the very same agent follows the re-cue perfectly when it has no committed belief. Just 10% override-episodes flips it to razor-clean E. Evidence-responsiveness of an edited belief is real, information-driven, and a sharp, low-threshold trained property.

Same maze; when made uncertain, the agent scouts before committing — The U-signature: same maze, but when the belief is made uncertain, the agent detours to *scout* before committing, instead of walking straight in.

The certificate (and the honest residue)

I froze the acceptance criteria before running, then re-ran the whole battery across 2 architectures × 2 fact-cardinalities (binary / one-of-three) × 3 seeds. The core — belief-exists, calibrated decode, edit specificity, locality — is solid everywhere. The two novelty signatures hold as real effects but with honest, architecture-dependent residue: that's the point of the table, and it's what keeps this from being a suspiciously-perfect toy.

12-cell heatmap certificate; green at target, departures from 1.0 are the residue — The paired-world equivalence certificate. Green = at target; departures from 1.0 are the quantified residue. Editing to a *specific* one-of-three arm (0.81–0.88) and evidence-driven reversion (at ceiling only on binary-GRU; 0.33–0.62 elsewhere) are below ceiling and architecture-dependent — the honest boundaries.

Two structural findings fall out. At one-of-three, the belief is a minimal 2-D categorical code — not a single-direction bottleneck (one direction flips only 0.67 of pairs; the full 2-D subspace flips 1.0), and not a redundant smear (removing the 2-D subspace collapses decoding to chance). And the information-seeking is driven by proximity to a belief-free region of the representation, not by the linear decoder's entropy — a dissociation I did not expect and am reporting as-is.

Adversarial verification (what a red-team found)

Before calling any of this publishable, I had a fresh adversarial agent try to break every claim, with its own code paths against the saved checkpoints. Most survived — some more strongly than I'd stated:

survives: Belief-exists (a memoryless version of the agent is exactly at chance on identical frames; the cue never appears at or after the junction), decode (no non-visual leak — a discriminator on timing/position/step-count is at AUROC 0.500), edit (arm-flip recomputed from raw grid positions = 1.00, so it's genuinely behavioral, not the decoder flipping), locality, and reproducibility.
survives, strengthened: The U-signature: the red-team's structured matched-norm controls give ρ = 0.000 on both architectures (cleaner than my own noisy control), against the real effect's 0.82–0.89.
landed, then rescued, then sharpened: The E-signature: in the original maze "evidence overwrites the belief" was inseparable from the recovery reflex, so I retracted it — rebuilt the environment with an independent mid-arm re-cue, and E held behaviorally (above) — then a transfer test settled the last worry (that I'd simply trained the agent to follow re-cues). The result was better than a clean pass: it turned E into a graded, threshold phenomenon (figure below) and surfaced the belief-perseverance effect. Two standing honesty taxes remain: the mid-arm linear belief-decode flip is only partial (clean LSTM, noisy GRU — the same probe-vs-representation gap seen for U); and E is a trained capability, not an automatic one (though the 10% threshold is low and, once present, it's information-driven and held-out).

Edited agent commits to the wrong arm, then reverts at the mid-arm re-cue before any dead-end — E, rescued: the belief is edited to the wrong arm; the agent commits — then the mid-arm re-cue reveals the true arm and it *reverts on the spot*, well before any dead-end. Swap the re-cue for a same-salience neutral marker, or one consistent with the edit, and it does not correct.

E-signature vs fraction of belief-override training episodes: 0 at 0%, jumps to 1 at 10%; task health flat at 1.0 — E, sharpened: how much does the agent need to have *seen* evidence-overrides-belief in training for an **edited** belief to yield to evidence? At 0% it doesn't — the edited belief shields the agent from the re-cue (belief-perseverance). At just 10% it's razor-clean, then the control gap slowly erodes as heavy override-training makes the agent twitchy. Right: task competence is flat at 1.0 throughout — this is about belief-shielding, never about skill.

Prior art, honestly (or: I checked novelty too late)

I built this before doing a proper prior-art search — the wrong order. Two adversarial literature-search passes afterward found that the core moves are already published, most damagingly on the transformer side I actually care about:

decode + edit a belief → behavior: RepBelief (Zhu et al., ICML 2024) decodes beliefs from the residual stream / attention heads and edits them by inference-time intervention so the model's answers change — with a random-direction control. GridToM (ICML 2025) does this in a video-text model with attention-head localization. Belief-state geometry (Shai et al., NeurIPS 2024) does counterfactual belief injection in the residual stream. Othello-GPT (2023) is the foundation. The recurrent "cheese vector" and DRC-Sokoban (ICLR 2025) do the probe→patch→behavior recipe in fully-observable settings.
uncertainty → information-seeking: EAST (Rahn et al., 2024) already steers an entropy/uncertainty direction to make an in-context agent more exploratory. My U-signature is the same idea.

So the honest verdict: the machinery here is not new, and I should cite all of the above. What a careful reviewer would still call open, after that search: (1) the whole loop on an embodied action policy — existing work changes a QA answer; here the agent takes different physical actions as if the hidden world changed; (2) a stricter causal bar — paired-world behavioral equivalence with matched-DOF/matched-norm + locality controls, where the field mostly uses a single random-direction null (and would likely fail this bar); and (3) evidence overwriting an edited belief (the E result), which I did not find tested elsewhere. Those slivers are real but narrow — this is an open-conjunction, not a new phenomenon. The lesson I'm taking: prior-art hunt first, then build.

Caveats. One environment, hidden fact of cardinality ≤ 3, a compact CNN→GRU/LSTM (recurrent, not a transformer — so its implications for transformer VLMs/VLAs are by analogy only), imitation-trained. This is a pilot with a preregistered certificate and honest residue, and — per the reckoning above — a mostly-known result. Built/verified on a single H100 in well under 50 GPU-hours. Written up with Claude Code; full code, preregistration, red-team, and every number in the repo.