ZGCM-1: A Fully Open and Extremely Efficient Foundation Model for Math and Agentic Search
Jiyan He, Guang Liang, Hao Liu, Haoxiang Guan, Jinbo Sun, Junyi Guo, Wenjun Feng, Yantai Xie, et al.
cs.AI
Quick verdict
Fully open 7B model claims frontier-level math/agentic results; highly transparent, but 4.2x speedup is a composed estimate and baselines mix harnesses.
Summary
ZGCM-1 is a 7.39B-parameter dense decoder-only model trained from scratch for mathematical reasoning and agentic search over a 256K-token context. Its architecture interleaves 27 gated 128-token sliding-window attention layers with 5 global layers at a 5:1 ratio, cutting per-token KV cache from 128 KiB to 20 KiB (a 6.4x reduction at 256K) and yielding a 3.94x throughput speedup at 256K. Pre-training on 0.99T plus 3.20T tokens uses FP8 with delayed scaling, TWEO regularization, and the Muon optimizer, sustaining roughly 585 TFLOP/s/GPU (~60% BF16 MFU); the headline ~4.2x 16K time-to-loss speedup versus a BF16/AdamW baseline is, however, a composed estimate from four separately measured factors rather than one end-to-end run. Mid-training scales context 16K to 256K over 600B tokens while recasting agent traces as MDP state-action prediction. After aggressive SFT quality filtering of 4.92M examples and GRPO-based RL, the model reports, in think mode (mean pass@1 over 32 runs), 97.1% on MATH-500, 75.0% on AIME 2026, 63.1% on WebWalkerQA, and 19.4% on BrowseComp, competitive with far larger frontier models. Yet SWE-bench Verified Mini50 (4.0%) and Terminal-Bench 2.0 (2.25%) sit near floor, and baseline figures are taken from prior releases rather than a unified rerun. The release of weights, checkpoints, code, data recipes, and W&B logs is unusually complete, and negative results are candidly reported.
Review complete
Assessment at a glance
4
risk flags identified
● high 1
● medium 2
● low 1
4 strengths · 5 weaknesses · 5 author questions
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- A 7.39B dense model with 5:1 gated-SWA/global hybrid attention ranks first on average across 14 reasoning benchmarks at the 7B-8B scale and is competitive with much larger frontier models on selected math and agentic-search suites.
- Combining SWA 5:1, FP8 with delayed scaling and TWEO, Muon, and Pre-LN yields approximately 4.2x improvement in 16K pre-training time-to-loss over a BF16/AdamW baseline.
- MDP-style state-action mid-training over 600B tokens with staged context scaling (16K to 256K) establishes long-context agentic capability such that moderate-length SFT suffices at 256K (Finding 7).
- Pruning roughly half of the SFT candidate pool improves the six-benchmark mean from 67.78 to 68.83, and jointly interleaving general and agentic data beats sequential scheduling (Findings 4 and 6).
- AI-native R&D autonomy is task-dependent: operational tasks such as experimentation/monitoring and deployment reach L4, while architecture and learning-algorithm design stay at L2 (Finding 8).
Strengths
Exceptional open-science release
Weights from pre-training, mid-training, and post-training stages, intermediate checkpoints, training code and configurations, per-stage data and data recipes, W&B logs, and evaluation harnesses are all released (Section 1, Conclusion); Appendix F further binds results to model hash, dataset revision, seeds, and harness internals.
Candid reporting of negative results
SWE-bench Verified Mini50 at 4.0% (2/50, Appendix C.3) and Terminal-Bench 2.0 at 2.25% (2/89, Appendix C.4) are disclosed with full harness detail rather than hidden; ACE self-diagnoses instruction following (49.74%) and truthfulness (68.91%) as weakest categories (Figure 17); capability curves are explicitly called descriptive, not monotonic (Appendix A.2).
Measured architecture and data ablations
Controlled SFT data-quality evidence
Weaknesses
The 4.2x headline speedup is a composed estimate
Baseline comparability is weakened by mixed provenance
Agentic framing outpaces audited hard-suite results
Think-mode inference budget asymmetry
Curriculum effect size is not matched
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
ZGCM-1 (arXiv 2609.13356v2) is a technical report for a fully open 7.39B dense decoder-only foundation model trained from scratch for mathematical reasoning and agentic search over a 256K context. Central claims: (1) a hybrid gated-SWA/global attention design (5:1) delivers a 6.4x KV-cache reduction and 3.94x throughput at 256K; (2) FP8 + TWEO + Muon + Pre-LN yields ~4.2x better 16K pre-training time-to-loss than BF16/AdamW; (3) MDP-formulated mid-training over 600B tokens (16K to 256K) builds long-context agentic capability that moderate-length SFT can elicit (Finding 7); (4) the model ranks first on average among 7B-8B peers across 14 reasoning benchmarks and 'remains competitive with frontier models orders of magnitude larger' on selected math and agentic suites. A complete artifact release (weights at all stages, checkpoints, code, data recipes, W&B logs) is claimed.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 5 weaknesses, analyzed in detail
- 4 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is ZGCM-1?
ZGCM-1 is a fully open-source 7.39B-parameter dense language model trained from scratch for mathematical reasoning and agentic search with a 256K-token context. It combines a hybrid attention architecture (27 gated sliding-window layers plus 5 global layers at a 5:1 ratio), FP8 training with the Muon optimizer and TWEO regularization, and an MDP-style mid-training phase. The team releases weights, intermediate checkpoints, training code, per-stage data recipes, W&B logs, and evaluation harnesses on GitHub and Hugging Face (zgcagi/ZGCM-1).
How can a 7B model compete with models like Qwen3-235B or GLM-5.1?
The paper's thesis is that compact models can offset limited parametric knowledge by combining long chain-of-thought reasoning with active tool use (web search, terminals, binary analysis). In think mode with up to about 258K generated tokens per problem, ZGCM-1-7B reports 97.1% on MATH-500, 75.0% on AIME 2026, and 63.1% on WebWalkerQA, which is competitive with much larger models on those suites. The advantage does not extend to all agentic work: it solves only 4.0% of a 50-task SWE-bench Verified subset and 2.25% of Terminal-Bench 2.0.
Is ZGCM-1 really fully open source?
According to the paper, yes: model weights from the pre-training, mid-training, and post-training stages, intermediate checkpoints, training code and configurations, per-stage data and data recipes, W&B training logs, and evaluation harnesses are all released (GitHub: zgcagi/ZGCM-1; Hugging Face: zgcagi/ZGCM-1-7B). Independent verification that the artifacts are complete and runnable is the natural next step for the community.
What is the claimed 4.2x training speedup and how was it measured?
The authors report roughly 4.2x better 16K pre-training time-to-loss than a comparable BF16/AdamW baseline. It is a composed estimate: 1.4x from hybrid sliding-window attention, about 1.5x from the FP8 systems configuration, about 1.8x from the Muon optimizer, and an estimated 1.1x from Pre-LN, multiplied together. It is not a single end-to-end training run against the baseline, which is the main caveat a reader should keep in mind.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- The provided text is a sampled excerpt with explicit '[...sampling gap...]' markers; the full Section 5 result tables, parts of Section 6.1, Appendix B.2.1 tool-schema details, and the middle of the reference list were skipped by sampling, so table-level cross-checks beyond quoted figures and captions were not possible.
- Headline benchmark values cited in this review come from the Figure 1/Figure 2 caption text and Section 5.1 prose present in the excerpt; the corresponding full tables in Section 5 were not part of the provided text.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.