Back to Search View Original Cite This Article

Abstract

<title>Abstract</title> <p>Chain of Continuous Thought (Coconut) performs computation in a continuous latent space, which makes its intermediate steps difficult to monitor directly. Work seeking to interpret it relies on projecting latent states into vocabulary space, and treats a change in that readout as evidence that the underlying computation has changed — an inference that has not itself been characterized. We measure it. Using the J-lens readout as a training signal, we plant bridge entities in latent states to obtain a testbed in which the readout is controlled, then apply a dose-response protocol that fixes the magnitude of each intervention. Bridges are planted without loss of accuracy (100% answer accuracy, 0.999 readout probability). We decompose the causal intervention delta into the component the lens detects and the component it does not, and control with a random direction matched on both norm and lens-invisibility. Across the two relation types used for intervention, three seeds and a 2B model, the readout and the computation have different thresholds along the same direction. The readout flips at relative perturbation magnitudes of 0.13-0.16, but the answer does not move there; the answer threshold is 2.03x higher on one relation and is never reached before the model breaks on the other. Conversely, along a direction to which the readout does not respond once in 1,800 trials, 96-97% of answers are transplanted. The random direction matched on magnitude and invisibility produced no readout change, no answer change and no breakage in the same 1,800 trials. This implies that minimum-norm readout-targeted intervention stops at the readout threshold and therefore measures below the causal threshold — at half of it on the one relation where that threshold is reached at all. Our own initial experiment produced a false null of 0/90 in this trap, with a stopping magnitude of 0.119 against a readout threshold of 0.125. Readout-based interpretability claims should be reported as dose-response curves rather than at a single magnitude.</p>

Show More

Keywords

readout threshold magnitude intervention answer

Related Articles

PORE

About

Connect