Hunyuan-A13B Technical Report
Tencent Hunyuan Team, Ao Liu, Botong Zhou, Can Xu, Chayse Zhou, ChenChen Zhang, Chengcheng Xu, Chenhao Wang, et al.
cs.AI
Quick verdict
Tencent's 80B/13B-active MoE claims near-frontier scores; broad tables but no variance, hardware specs or ablations—statistically unverified but promising.
Summary
Tencent Hunyuan Team's technical report presents Hunyuan-A13B, an open-source Mixture-of-Experts language model with 80 billion total parameters, of which 13 billion activate per token. The architecture uses one shared plus 64 fine-grained routed experts (8 active), GQA attention, and a 128K vocabulary. Pretraining runs 20T tokens through the Hunyuan-TurboS curation pipeline with an added 250B-token high-quality STEM corpus, followed by annealing and NTK-aware long-context stages reaching a 256K window. Post-training is a four-stage pipeline: reasoning SFT; GRPO-based reasoning RL with outcome rewards and a 36-language code sandbox (150K prompts at a 2:2:1:1 math:code:logic:science ratio); all-scenarios SFT; and generative-reward-model RL across 16 subtopics. A dual-mode Chain-of-Thought lets users pick fast (empty think block) or slow (full traces, default) reasoning via control tags. Reported results: the base model beats Hunyuan-Large on 12/14 benchmarks and Qwen3-A22B on 7/12 (Table 2); in slow-thinking mode it tops AIME2024 (87.3), BBH (89.1), ZebraLogic (84.7) and all four agent benchmarks (Table 3); RULER QA averages 76.7 with 73.9 at 64K–128K. Main limitations: no variance estimates or sampling protocols for small-N exams like AIME, no comparative throughput evidence or hardware details despite 'superior throughput' claims, no ablations, and reliance on unreleased internal benchmarks.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- An 80B-total/13B-active MoE matches or beats much larger models: 12/14 benchmarks versus Hunyuan-Large (52B active) and 7/12 versus Qwen3-A22B (§4.1.2, Table 2).
- In slow-thinking mode the model tops AIME2024 (87.3), BBH (89.1), ZebraLogic (84.7) and all four agent benchmarks against OpenAI-o1-1217, DeepSeek-R1 and Qwen3-A22B (Table 3).
- The dual-mode CoT scheme lets one model serve latency-sensitive (fast) and complex-reasoning (slow, default) workloads via /no_think and /think control tags (§3.3).
- The 256K context degrades gracefully: RULER QA averages 76.7 with 73.9 in the 64K–128K bucket, above DeepSeek-R1 (65.6) and Qwen3-A22B (66.6) (Table 6).
- Absolute throughput (190.84 tok/s at batch 1; 1981.99 at batch 32, Table 7) makes the model suitable for latency-sensitive deployment.
Strengths
Strong parameter-efficiency with internally consistent tables
Table 2: the 13B-active model beats 52B-active Hunyuan-Large on 12/14 benchmarks; the §4.1.2 claim of '7 of 12' versus Qwen3-A22B verifies row-by-row (wins: MMLU, MMLU-Redux, EvalPlus, MultiPL-E, MBPP, MATH, GPQA).
Unusually concrete post-training pipeline
§3.1.2 specifies GRPO with the KL term removed, 150K RL prompts at a 2:2:1:1 math:code:logic:science ratio, 10% SFT overlap, 24K→32K context phases, and a 36-language sandbox with 1000+ concurrent executions; §2.3 gives exact LR schedules and NTK alphas (50/1000).
Deployable dual-mode CoT mechanism
Long-context claim backed by degradation data
Breadth of evaluation
Weaknesses
Throughput superiority claimed without any comparison
No statistical reporting anywhere
No ablations for any design claim
Headline wins ride on unreleased internal benchmarks
Unexplained fast>slow inversion and citation sloppiness
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
This is an industry technical report for Hunyuan-A13B, an open-source Mixture-of-Experts LLM with 80B total / 13B activated parameters (Table 1: 32 layers, GQA 32/8 heads, 1 shared + 64 routed experts with 8 active, SwiGLU, 128K vocabulary). The paper makes five central claims: (1) a 20T-token, STEM-enhanced pretraining plus a four-stage post-training pipeline yields performance comparable to substantially larger models — the base model beats Hunyuan-Large (52B active) on 12/14 benchmarks and Qwen3-A22B (22B active, 235B total) on 7/12 (§4.1.2, Table 2); (2) in slow-thinking mode it tops AIME2024 (87.3), BBH (89.1), ZebraLogic (84.7) and all four agent benchmarks versus OpenAI-o1-1217, DeepSeek-R1 and Qwen3-A22B (Table 3); (3) dual-mode CoT (§3.3) serves latency-sensitive (fast, empty think block) and complex (slow, full traces, default) workloads in one model via /think and /no_think tags; (4) the 256K context degrades gracefully (RULER QA average 76.7, 73.9 at 64K–128K, Table 6); (5) inference throughput is 'superior' (abstract, §4.4).
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is Hunyuan-A13B?
An open-source large language model from Tencent's Hunyuan team built on a Mixture-of-Experts architecture with 80 billion total parameters but only 13 billion activated per token. It was pretrained on 20 trillion tokens, supports a 256K-token context window, and offers two reasoning modes — fast and slow 'thinking' — selected with /no_think and /think control tags, with slow mode as the default.
How does Hunyuan-A13B compare to Qwen3 and DeepSeek-R1?
Per the paper's own tables: the base model beats Qwen3-A22B on 7 of 12 comparable benchmarks; in slow-thinking mode it leads on AIME2024 (87.3), BBH (89.1), ZebraLogic (84.7) and four agent benchmarks, but trails Qwen3-A22B on AIME2025 (76.8 vs 81.5), LiveCodeBench (63.9 vs 70.7) and OlympiadBench (82.7 vs 85.7). These are self-reported, single-run numbers without variance estimates, so treat them as indicative rather than definitive.
Is Hunyuan-A13B open source and how can I run it?
The paper states the model is released to the open-source community and integrates with vLLM, SGLang and TensorRT-LLM for one-click W16A16 inference. Weight-only INT8, W8A8 and FP8 KV-cache quantization are supported, together with Tensor Parallel, Expert Parallel and FusedMoE acceleration.
What is the dual-mode (fast/slow) chain-of-thought?
A training and inference scheme in which the same model emits either an empty <think> block (fast mode, low latency, /no_think) or a full reasoning trace (slow mode, default, /think) depending on a control tag. It trades reasoning depth for speed without needing a separate router model.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Table formatting (bold best / underlined second-best) was lost in extraction; rankings in Tables 2–4 were re-derived manually from the raw numbers.
- Figure 1 (post-training flow) is not analyzable from the extracted text, and any appendix content is absent.
- Date metadata is inconsistent: the title page reads 2025-06-27 while the arXiv stamp embedded in the text reads 23 Sep 2026 — confirm versioning against the arXiv landing page.
- Evaluation decoding configurations, hardware and compute budget are absent from the text; a fuller manuscript version might close some gaps.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.