When Recursive Models Finish Computing

Trajectory-conditioned anisotropic stability as a dynamical signature of completion in Tiny Recursive Models

Hare Krishna; Shubham Singh; Stephen Ebert; Hao-Yu Sun· 2026· DOI 10.48550/arXiv.2609.26487

The core problem

Recursive models can continue updating their latent states beyond their nominal inference budget, so an incorrect output at that budget does not show whether computation is unfinished or has entered a persistently unsuccessful regime. This ambiguity motivates a dynamical study of completion in attention- and MLP-based Tiny Recursive Models (TRMs) on 1,000 hard Sudoku puzzles. The central question is whether nominal-budget failure can be distinguished from completed computation by analyzing the evolution of latent states under extended recurrence.

Extending recurrence from the nominal 16 steps to 512 steps increases cumulative exact-solve accuracy from 59.2% to 87.5% for the attention model and from 74.4% to 91.9% for the MLP model, solving more than two-thirds of the puzzles unsolved in the nominal budget. Across both architectures, latent-state motion drops sharply after the first exact solution. Completed states are typically locally contractive along the trajectory direction, even though the same local Jacobian retains strongly expanding directions. We characterize this phenomenon as trajectory-conditioned anisotropic stability. Perturbation experiments confirm this directional stability across both models. The multi-step fate of the maximally expanding direction differs: it is absorbed within 16 steps in the attention model but persists longer in the MLP model. The anisotropic-stability pattern also holds for a second attention checkpoint.

Recursive models can continue updating their latent states beyond their nominal inference budget, so an incorrect output at that budget does not show whether computation is unfinished or has entered a persistently unsuccessful regime. This ambiguity motivates a dynamical study of completion in attention- and MLP-based Tiny Recursive Models (TRMs) on 1,000 hard Sudoku puzzles. The central question is whether nominal-budget failure can be distinguished from completed computation by analyzing the evolution of latent states under extended recurrence.

We study two TRM architectures: an attention-based model and an MLP-based model. Both are evaluated on 1,000 hard Sudoku puzzles. The nominal inference budget is 16 recurrent steps. We extend recurrence to 512 steps and measure cumulative exact-solve accuracy. To characterize the dynamics, we analyze latent-state motion, local Jacobians, and perturbation responses. The local Jacobian \( J = \frac{\partial f}{\partial h} \) at a latent state \( h \) describes the linearized dynamics. We compute the trajectory direction \( v_t = h_{t+1} - h_t \) and examine the directional derivative \( J v_t \). Perturbation experiments test the stability of the trajectory direction across both models. A second attention checkpoint is used to verify the generality of the anisotropic-stability pattern.

The sharp drop in latent-state motion after the first exact solution indicates that the model has reached a completed state. The local contractivity along the trajectory direction, despite expanding directions in the full Jacobian, suggests that completion is characterized by a stable trajectory-conditioned subspace. This anisotropic stability provides a dynamical signature that distinguishes nominal-budget failure from completed computation. The difference in the fate of the maximally expanding direction between attention and MLP models highlights architectural influences on the completion dynamics. These findings have implications for understanding recurrent computation and for designing inference budgets that adapt to the model's internal state. Future work could explore whether similar signatures appear in other recurrent architectures and tasks.

CS practitioners and researchers

Opening member content…