State of Thought Enables Endogenous Reasoning
Zhiren Gong, Yikun Hou, Zihao Zeng, Ming Xiao, Chau Yuen, Wei Yang Bryan Lim
cs.CL · cs.AI
Quick verdict
SoT: a 582-parameter hidden-state controller gates evidence and stopping; gains are broad and efficient but baselines look weakened and stats untested.
Summary
The paper introduces State of Thought (SoT), a test-time reasoning framework in which the LLM's internal state, rather than an external prompt script or search policy, governs reasoning. At each sentence-level step, SoT extracts a four-dimensional state (within-sentence dispersion, displacement, directional consistency, predictive entropy) from final-layer hidden states and applies a 582-parameter controller that (i) selects which historical sentences remain active as evidence and (ii) decides when to stop and answer. Across Llama-3.1-8B, Qwen2.5-14B, and Mixtral-8x7B on 16 benchmarks, SoT reports the best domain average in all four reasoning domains (e.g., Llama long-context 31.9 vs. 21.8 for the best baseline), with aggregate token reductions of 62.6% and latency reductions of 44.6%; transfer to Qwen2.5-VL-7B/32B yields +3.8 points over reasoning baselines. Training-free and embedding-only variants retain reported gains, and a trajectory-based judge reaches 84.1% agreement on three API models. Main concerns: several baselines collapse implausibly (CoT scores 1.7 F1 on DROP; GRPO-SP is near zero), efficiency wins are aggregate while SoT is sometimes slower than CoT (Mixtral), several Mixtral ablation rows are numerically identical, no significance tests or confidence intervals are reported, and code availability is signaled but not verifiable from the text.
Review complete
Assessment at a glance
6
risk flags identified
● high 2
● medium 3
● low 1
4 strengths · 4 weaknesses · 5 author questions
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- A 582-parameter controller reading a 4-dimensional hidden-state summary can replace externally scripted reasoning by governing evidence retention and stopping on frozen LLMs.
- Across 3 LLMs and 16 benchmarks, SoT attains the best domain average in all four reasoning domains while cutting generated tokens by 62.6% and end-to-end latency by 44.6% in aggregate.
- The mechanism transfers to two Qwen2.5-VL scales (+3.8 points mean accuracy) and retains reported gains in training-free and embedding-only variants.
- A trajectory-only predictor (SoT-Judge) estimates answer correctness from sentence-embedding statistics, reportedly reaching 84.1% agreement across three API models, concentrated in long-context settings.
Strengths
Transparent, extremely lightweight controller on frozen backbones
582 parameters total (16×32 relational selection projection + 4→1 stopping head); selector AUC 0.779±0.005 across 5 problem-held-out refits (Sec. 2.3, Appendix C); deployment hyperparameters fully disclosed (Table 10).
Mechanism-level validation beyond end metrics
Coordinate interventions (Table 8: zeroing ct reroutes 56.2% of top-1 evidence selections; zeroing δt costs 6.7–22.5 accuracy points on GSM8K/DROP/MATH; zeroing vt lengthens trajectories 4–5×) plus a state–unit permutation test (GSM8K 79.0→62.4) that rules out recency/surface-similarity shortcuts (Appendix B.2).
Broad evaluation with disclosed protocol
Boundary probing of its own ingredients
Weaknesses
Degenerate baselines inflate relative gains
Efficiency claims are aggregate and contradicted per-model
Ablation table integrity problems
No statistical uncertainty; abstract aggregates do not reconcile with tables
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Preliminary Review: State of Thought Enables Endogenous Reasoning
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 4 weaknesses, analyzed in detail
- 6 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is 'State of Thought' (SoT)?
SoT is a test-time reasoning method for large language models in which a tiny 582-parameter controller reads a 4-number summary of the model's internal hidden-state dynamics (dispersion, movement, directional consistency, uncertainty) and uses it to decide which past reasoning sentences stay in context and when to stop and answer — instead of imposing an external reasoning script like chain-of-thought or running extra search.
How much faster and cheaper is SoT than alternatives?
The paper reports on average 62.6% fewer generated tokens and 44.6% lower end-to-end latency across three LLMs and 16 benchmarks versus a broad baseline suite. These are pooled figures: per-model results vary, and on Mixtral SoT is sometimes slower than plain chain-of-thought, while on some long-context tasks it also uses more tokens than CoT but scores much higher accuracy.
Does SoT require fine-tuning the LLM or access to its internal states?
No fine-tuning is needed — the backbone stays frozen and only the 582-parameter controller is trained on offline trajectories. The full version needs white-box access to hidden states; for API-only models the paper offers an embedding-only variant (using sentence embeddings instead of hidden states) and a post-hoc 'SoT-Judge' that predicts answer correctness from the finished text trajectory alone.
Is code or data available for this paper?
The paper's header references a code release and project page, but the extracted text does not include working URLs, so availability could not be verified in this review. All 19 evaluation benchmarks are public, and the method's hyperparameters are disclosed in the appendix.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- VLM results tables (Tables 12–13), the Llama main ablation table (Table 3) and the scale-transfer table (Table 4) were absent from the extracted text; VLM and main-ablation claims rest on abstract/intro numbers.
- All figures (state-space plots, Pareto plots, case studies) were lost in text extraction and could not be assessed.
- Code/project-page links appear only as placeholder words ('Code Project Page Tutorial') in the header; actual availability is unverified.
- Two mid-document spans were elided by extraction, cutting parts of the theory appendices and references.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.