NeoHorse-1: Towards Recursive Self-Improvement via Agentic Post-Training with Routing Harness
NeoHorse Team, Guoliang Cao, Guohao Dai, Tianyu Guo, Kai Han, Hailin Hu, Zihan Jiang, Xiang Kuang, et al.
cs.CL
Quick verdict
Claims routing-harness data + curriculum/OPD lifts 4B/9B macro-avg to 64.87/69.04; good evals, but key curriculum claim lacks a random-order ablation.
Summary
NeoHorse-1 is a family of 4B and 9B agent-native models built by post-training on trajectories logged by a deployed routing harness that assigns each user turn to one of four capability tiers (C0–C3) within a heterogeneous model pool. The pipeline converts harness interactions into user-turn training examples preserving interleaved reasoning, tool calls, and harness context, filtered by rule-based structural validation, a six-dimension semantic judge, and subscene-level Scene/Goal/Outcome labeling. Routing-derived capability-demand scores organize supervised fine-tuning into a three-stage curriculum and schedule routing-guided on-policy distillation, where a fixed teacher scores student-generated responses with a top-K reverse-KL objective; a capability-guided allocation step is described to feed evaluation deficits into the next data mixture. Across ten benchmarks (agentic, tool use, coding, instruction following), the macro-average rises from 58.94 to 64.87 for the 4B model and from 65.60 to 69.04 for the 9B model over their Qwen3.5 bases, with the post-trained 4B nearly matching the 9B base. A controlled comparison shows routing-harness data beating public Toucan data by +6.26 average under matched training. Key limitations: only a single evaluation–selection–update pass is demonstrated (no recursive iteration), the central routing-curriculum is never ablated against random ordering, SFT vs OPD contributions are not decomposed, and corpus scale, hyperparameters, and the OPD teacher are unspecified.
Review complete
Assessment at a glance
5
risk flags identified
● high 1
● medium 3
● low 1
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Trajectories from a deployed routing harness, admitted through structural validation, six-dimensional semantic evaluation, and subscene labeling, provide stronger agentic SFT supervision than public tool-agent data (+6.26 average over Toucan under a matched recipe, Table 3).
- Routing-derived capability-demand scores organizing SFT and on-policy distillation into a three-stage progression raise the ten-benchmark macro-average from 58.94 to 64.87 at 4B and from 65.60 to 69.04 at 9B.
- The post-trained 4B model (64.87) substantially narrows the aggregate gap to the untrained 9B base (65.60).
- A capability-guided allocation step converts evaluation feedback into the next training mixture, establishing an operational (but single-pass, not yet iterated) evaluation–selection–update loop toward harness-mediated RSI.
Strengths
Exceptionally transparent evaluation protocol
Shared benchmark harnesses, per-model official chat templates, exact sampling params (temp 1.0, top-p 0.95, top-k 20, presence penalty 1.5, SGLang v0.5.17), per-benchmark run counts (3 runs for QwenClaw/WorkBuddy/τ²-Bench), ∗ flags for third-party-reported numbers, and disclosed substitution of the VitaBench simulator/judge with DeepSeek-V4-Flash (§5).
Fully specified training mathematics
Masked token-level SFT normalized by supervised token count (Eq. 1), hard/soft tier ordering scores from normalized C0–C3 vectors (Eq. 2), and coarsened top-K+1 reverse-KL OPD with response-level averaging (Eqs. 3–4); the context and loss-mask policy is spelled out with Figure 4.
Data QC designed for auditability
Controlled data-source ablation
Honest self-assessment
Weaknesses
The central scheduling mechanism is never ablated
SFT vs OPD contribution never decomposed
The allocation half of the RSI loop is design-only
Statistical reporting too thin for the margins claimed
Reproducibility currently low
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
NeoHorse-1 (arXiv 2609.08183, cs.CL) is a technical report from TokenRhythm and collaborators presenting two agent-native models (4B/9B, initialized from Qwen3.5-4B/9B) plus the training system behind them. The central claim: a deployed routing harness — a heterogeneous model pool plus a router assigning each user turn to one of four capability tiers (C0–C3) — already logs what recursive self-improvement (RSI) needs: trajectories, capability-demand estimates, and outcomes. Three sub-claims follow: (1) harness trajectories admitted through structural validation, six-dimensional semantic evaluation, and subscene labeling beat public agent data (+6.26 macro over Toucan, Table 3); (2) routing scores organize SFT into a three-stage curriculum and schedule routing-guided on-policy distillation (R-OPD), lifting ten-benchmark macro-averages from 58.94→64.87 (4B) and 65.60→69.04 (9B); (3) capability-guided allocation closes an evaluation–selection–update loop — demonstrated for exactly one pass.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is NeoHorse-1?
NeoHorse-1 is a family of 4B and 9B agent models from TokenRhythm and collaborators, post-trained on interaction trajectories logged by a deployed LLM routing harness. The harness's routing records (predicted capability tier C0–C3, served model, outcome) become training data, a three-stage curriculum signal, and a data-allocation feedback mechanism; the models are evaluated on ten agentic, tool-use, coding, and instruction-following benchmarks.
How much better is NeoHorse-1 than Qwen3.5?
Compared with its same-size Qwen3.5 bases, the macro-average across ten benchmarks rises from 58.94 to 64.87 for the 4B model and from 65.60 to 69.04 for the 9B model; the post-trained 4B nearly matches the untrained 9B base (65.60). Gains concentrate on harness-based agent and tool-use benchmarks, while some 9B instruction-following and coding scores stay flat.
Does NeoHorse-1 actually achieve recursive self-improvement?
Not yet. The paper demonstrates one pass of an evaluation–selection–update loop; it does not show gains compounding across successive model generations. The authors explicitly call it an initial prototype and list multi-iteration validation as future work, so 'towards RSI' in the title is aspirational framing.
Is NeoHorse-1's code and data available?
The report links a Hugging Face model collection and a GitHub repository (TokenRhythm/NeoHorse), but the text does not itemize which artifacts (weights, training code, trajectories, manifests) are actually released. The core corpus — proprietary deployment trajectories — is described only at 10^5–10^6 scale, and no hyperparameters are given, so independent retraining is not currently possible from the paper alone.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figure 1's per-bar values run together in the extraction; all numbers cited here were cross-checked against Tables 1–2, which are internally consistent with Figure 1 wherever both report the same configuration.
- The supplied metadata abstract says 'eleven benchmarks' while the paper's own abstract, conclusion, and both result tables consistently say ten; this review follows the full text.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.