Looped Transformers under the Jacobian Lens: Does the Global Workspace Survive Recurrence?
cs.AI
Quick verdict
Claims looped transformers keep workspace function under recurrence but access becomes loop/window-shaped; multi-estimator causal study, one checkpoint per family limits generality.
Summary
This paper asks whether the 'global workspace' band previously identified in standard feedforward transformers also forms when depth is implemented through weight-tied recurrence. The authors extend the Jacobian lens — per-layer linear transport maps fitted from mean input–output Jacobians — to looped and depth-recurrent models via a virtual-unrolling adapter, and run the full workspace suite (lens fitting, readout, eleven causal experiment families) on Ouro-2.6B (48 layers looped ×4), Huginn-0125 (4-layer core recurred ×16), and Qwen3.6-27B as untied baseline. Both looped models show the workspace signature — verbalisable, causally steerable, task-selective content — but recurrence reshapes access. Ouro's supervised checkpoints re-derive content each loop, so writes must span every remaining loop (38% vs 21% for loop 2 alone), and linear transport dies at loop boundaries. Huginn carries content across all sixteen recurrences, yet reads, writes, and ablations act only within a roughly two-recurrence window, sustained by re-injected input rather than memory. Verbalising newly injected content tracks explicit per-iteration supervision (Ouro 31% top-1; Huginn 0/97), while steering existing content does not (Huginn 98% verbal report). Limitations: one checkpoint per architecture confounds architecture with training recipe; measured capacity (0.7–2.4 words) falls far below the original study's ~25; several families rest on small prompt sets or aggregate-only records.
Review complete
Assessment at a glance
3
risk flags identified
● high 0
● medium 1
● low 2
5 strengths · 5 weaknesses · 5 author questions
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Both looped/depth-recurrent models exhibit the full global-workspace signature (verbalisable, causally steerable, selectively engaged content), so depth-as-recurrence supports workspace function.
- Ouro's supervised checkpoints decode and re-encode workspace content each loop, destroying it in transit: linear transport cannot cross loop boundaries and writes/ablations must span every remaining loop (38% all-loops vs 21% loop-2-only).
- Huginn carries workspace content across all sixteen recurrences, but reads, writes, and ablations act only within a roughly two-recurrence window of a lens's own target, sustained by continuous recomputation from re-injected input rather than memory.
- Verbalising newly injected content requires explicit per-iteration deep supervision (Ouro top-1 31%; Huginn 0/97; 27B Qwen null), whereas steering already-computed content into words does not (Huginn 98% verbal report).
- Weight tying makes the fitted mean transports repeat across iterations (cosine 0.9991 on Huginn), so one iteration's lens transfers to all, legitimising loop-relative comparisons.
Strengths
Triangulated measurement across estimators with independent failure modes
Convergence by loop 3 on Ouro is supported three ways (§5.1): per-loop J-lens readouts, checkpoint logit-lens KL (1.7/0.5/0.1 nats, up to 92% top-1), and the trained early-exit gate [0.03, 0.14, 0.34, 0.49] — the third involves no readout at all, so the picture cannot be a single-instrument artefact.
Causal grounding with concept-specific controls
Eleven intervention families (Table 5) run identically on all three models; the unrelated-direction ablation control keeps 100% of answers on every model, and the ten-direction random-atom control keeps 87–97%, isolating the lens direction itself as the operative variable.
Instrument self-audit rare in interpretability work
Exceptional reproducibility posture
Robustness to downstream training
Weaknesses
One checkpoint per family confounds architecture with training
Small stimulus sets underpin headline geometry
Capacity results sit far below the original study
Four families are only aggregate-verified
Huginn tested at a single operating point with weak baselines
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
This paper extends the Jacobian-lens account of a transformer global workspace (Gurnee et al., 2026) to weight-tied architectures, under one shared protocol. Three models: Ouro-2.6B (48 unique layers looped ×4, virtual depth 192, deep supervision at every loop end), Huginn-0125 (2-layer prelude + 4-layer core recurred ×16 + 2-layer coda, virtual depth 68), and Qwen3.6-27B (64 untied layers) as reproduced baseline. Claims: (1) the workspace signature — verbalisable, causally steerable, task-selective content — forms in the iterated part of both looped models; (2) recurrence reshapes access interfaces: Ouro's supervised checkpoints decode and re-encode content so writes and ablations must span every remaining loop, while Huginn carries content across all sixteen recurrences but supports reads/writes only within a ~2-recurrence window; (3) verbalising injected content requires explicit per-iteration supervision, steering existing content does not. Consciousness claims are explicitly disclaimed (§2.3).
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 3 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is the Jacobian lens used in this paper?
It is a per-layer linear map equal to the mean input–output Jacobian of the computation between a source and a target layer, estimated over about 1000 prompts. It transports a mid-depth state linearly toward the output for readout, and its pullback directions support causal edits such as injection, ablation, and clamping. This paper extends it to weight-tied (looped) models by treating each firing of a shared block as its own virtual layer.
Do looped transformers have a global workspace?
According to this study, functionally yes: both a 48-layer model looped 4 times (Ouro-2.6B) and a 4-layer core recurred 16 times (Huginn-0125) show verbalisable, causally steerable, task-selective content in their iterated computation. What changes is access: on Ouro an intervention must span every remaining loop, while on Huginn reads and writes act only within about a two-recurrence window.
Is this paper claiming that AI has consciousness?
No. The authors explicitly state they make no claim about consciousness; they use the global-workspace experimental suite only as a functional characterisation of representations and ask how its outcomes depend on architectural recurrence.
What is the practical difference between Ouro and Huginn's workspaces?
Ouro (trained with deep supervision) decodes content into token-aligned checkpoints at each loop end and re-derives it next loop, so interventions must persist across all remaining loops. Huginn (trained for latent reasoning) keeps a continuous latent state whose content persists across all 16 recurrences, but lenses read and write it only within about two recurrences of their target, and the persistence is continuous recomputation from re-injected input rather than memory.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figure 2's in-figure text (axis labels and tick marks) is jumbled by PDF extraction; its interpretation here follows the §5.2 body text and caption, which are intact. The extraction otherwise runs complete through Appendix H with references and all tables legible.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.