Miles v0.1: Production-Level Post-Training
RadixArk, :, Tom Chen, Mao Cheng, Shi Dong, Kangrui Du, Yanbin Jiang, Jiajun Li, et al.
cs.LG · cs.CL
Quick verdict
Credible full-stack RL post-training system report with unusually honest self-reporting, but evidence rests on a single self-run case study without baselines.
Summary
This is a system report for Miles v0.1, an open-source, full-stack framework for reinforcement-learning post-training of large language models, built on the slime design. Rollout runs on SGLang engines with cache-preserving session affinity (96% prefix-cache hit rate in the reference run); a session server records exact token IDs and expert routing so the trainer sees precisely what the policy sampled, with optional rollout-routing replay for MoE models. Fully asynchronous training overlaps generation and updates through a bounded data buffer with staleness checks. Trainers come in Megatron-LM and FSDP backends, with FP8/MXFP8/NVFP4 recipes (the latter two Beta) and three weight transports; peer-to-peer transfer cuts a GLM-5 744B weight update from 58.30 s to 8.48 s on 16-node fleets. The end-to-end case study trains GLM-5.2 744B-A40B on terminal-use coding tasks across 64 NVIDIA GB300 GPUs at a median 263 s per step, with mean log-probability divergence 0.0369 and a single-run reward rise (0.438 to 0.556) the authors explicitly report as an observation, not a proven improvement. Main limitations: no comparison against alternative frameworks, single-run evidence, thin step-time reporting (median over the first 30 steps), and several features marked Beta or living in unmerged pull requests.
Review complete
Assessment at a glance
4
risk flags identified
● high 0
● medium 2
● low 2
4 strengths · 4 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Miles v0.1 is a production-ready full-stack post-training system in which rollout (SGLang), training (Megatron-LM or FSDP), and weight synchronization (broadcast, P2P RDMA, disk-delta) are verified, clean, and customizable components.
- Token-exact TITO sessions plus optional rollout routing replay keep the trainer's log-probabilities faithful to what the policy actually sampled, even across multi-turn agentic episodes and MoE expert routing.
- Fully asynchronous RL over a bounded, staleness-checked data buffer keeps separate rollout and training GPU pools concurrently busy; a GLM-5.2 744B-A40B model trains on 64 GB300 GPUs at a median 263 s per step (first 30 steps).
- Peer-to-peer weight transfer cuts per-update time by up to 86.4% on multi-node fleets (Kimi K2 1T-A32B: 53.28 s to 7.23 s at 32 nodes/side) while being slower than broadcast on single nodes.
- The same architecture supports LoRA RL, on-policy distillation, true-on-policy bit-exact alignment, and diffusion post-training, with explicit per-recipe evidence levels.
Strengths
End-to-end numerical fidelity chain with verification, not assertion
TITO session server (Sec 2.4) checkpoints exact token IDs, logprobs and routed experts per turn and enforces an append-only invariant via two checks (CPU render check plus live GPU check) per model registration; a random-fill weight-verification check (Sec 4.4) catches silent transport omissions; R3 (Sec 2.5) replays expert assignments to close the MoE routing gap that Ma et al. showed can cause collapse.
Unusually disciplined honest reporting
The OPD result (Sec 5.2) explicitly refuses to claim accuracy improvement because the 1.2-point move (84.0% to 85.2%) lies inside the ~1.6-point standard error; the case-study reward rise is labeled an observation, not a measured improvement (Sec 9.2); diffusion recipes carry explicit evidence levels from fully-gated to not-verified (Sec 6); single-node P2P being up to ~70% slower is stated in Sec 4.2.
Concrete, configuration-anchored performance numbers
Frontier-scale case study is fully specified
Weaknesses
No comparative evaluation whatsoever
Single-run evidence with a narrow reporting window
Many advertised capabilities are Beta, experimental, or unmerged
Fidelity and precision verification is spot-check coverage
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
This is a systems/infrastructure report for Miles v0.1, an open-source full-stack RL post-training framework built on slime's design (Sec 1). Its claims are architectural and operational rather than algorithmic: (1) each loop component — SGLang rollout, Megatron-LM/FSDP training, three weight transports — is verified, clean, and customizable; (2) numerical fidelity is preserved end to end via TITO token-exact sessions (Sec 2.4), rollout routing replay for MoE (Sec 2.5), and precision contracts (Sec 3.1); (3) fully asynchronous RL over a bounded staleness-checked buffer (Sec 2.2) sustains frontier-scale throughput, demonstrated by a GLM-5.2 744B-A40B run on 64 GB300 GPUs at a median 263 s/step (Sec 9). The paper repeatedly scopes its own claims — a refreshing and rare practice.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 4 weaknesses, analyzed in detail
- 4 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is Miles?
Miles v0.1 is an open-source, full-stack system for reinforcement-learning post-training of large language models, built by RadixArk on the design of the slime framework. It combines SGLang rollout engines, a trainer with Megatron-LM or PyTorch FSDP backends, and three weight-synchronization transports, and also supports LoRA RL, on-policy distillation, supervised fine-tuning, true-on-policy alignment, and diffusion models. Code is at github.com/radixark/miles.
What results does the GLM-5.2 case study report?
On 64 NVIDIA GB300 GPUs (32 for rollout, 32 for training), Miles trains GLM-5.2 744B-A40B on terminal-use coding tasks with fully asynchronous RL. The median training step takes 263 seconds over the first 30 measured steps, the mean train-inference log-probability divergence is 0.0369 over 100 steps, and the raw task reward rises from 0.438 to 0.556 — which the authors explicitly call an observation from a single run, not a proven improvement.
How much faster is P2P weight transfer than NCCL broadcast in Miles?
On multi-node H100 fleets, peer-to-peer RDMA transfer cuts per-update time substantially: GLM-5 744B drops from 58.30 s to 8.48 s at 16 nodes per side (−85.5%) and Kimi K2 1T-A32B from 53.28 s to 7.23 s at 32 nodes per side (−86.4%). On single-node setups the paper reports P2P is up to about 70% slower than broadcast, so broadcast remains the default.
Is Miles the same as slime?
No. Miles builds on slime's clean design but adds, among other things, token-in-token-out (TITO) sessions that preserve exact sampled token IDs across multi-turn episodes, a fully asynchronous training mode with a staleness-checked data buffer, peer-to-peer and disk-delta weight transports, low-precision FP8/MXFP8/NVFP4 end-to-end recipes, and a diffusion-model extension.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1-5 appear in the extraction only as captions and axis labels; visual trends (e.g., the shape of the reward curve or KL trajectory) cannot be assessed beyond the numeric values quoted in the text and captions.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.