Phase-Decoupled, Model-Calibrated Power Control for Disaggregated LLM Serving
Jae Gon Kim, Donghoon Yoo, Hanyul Ryu, Sungho Ha, Juyeon Lee, Soojung Ryu
cs.LG · cs.DC
Quick verdict
Claims per-lane calibrated power control beats NVIDIA Max-Q 1.5-3.4x on tokens/J at lower latency on B200 MoE serving; credible and honest, but small-N.
Summary
The paper presents a power-control system for prefill/decode (PD) disaggregated LLM serving on an 8×B200 node, motivated by modest realized gains from NVIDIA's shipped Max-Q profile (+8.6% tokens/J at +5.2% mean e2e latency on a 480B agentic workload). The controller assigns each lane its own actuator: the compute-bound prefill lane runs under an SM-clock window whose floor is a latency guarantee by construction, while the memory-bound decode lane runs under a power cap placed by automatic calibration just above a measured throughput/latency cliff, leaving frequency choice to the GPU's power manager. Setpoints come from a roughly one-hour calibration pass per (model, quantization, engine, hardware) combination, accepted only if they preserve both throughput and ITL-p99 tail latency, with an engine fingerprint triggering recalibration. On Qwen3-Coder-480B (FP8) under agentic load, balanced mode delivers +20.4% tokens/J at +3.5% mean e2e versus Max-Q's +8.6% at +5.2%; on 235B NVFP4 every mode holds the ITL-p99 SLO in every repetition, where both vendor profiles miss it in a fraction of agentic repetitions. A decode actuator A/B shows the calibrated cap beats static clock locks (+24.1% vs +17.2%/+15.5% on 480B agentic), and a three-day run saves 32.3% of a lane pair's electricity. Main limitations: the headline 480B result uses N=2 repetitions from one session; the proprietary in-window clock policy is not ablated; adaptive DVFS controllers were not compared; scope is MoE-only and single-node.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Per-(model, quantization, engine, hardware) calibrated, phase-decoupled control achieves 1.5-3.4× the shipped NVIDIA Max-Q profile's efficiency gain at equal or lower latency, Pareto-dominating it on agentic workloads.
- On disaggregated decode lanes, a calibrated power cap outperforms static clock locking (+24.1% vs +17.2%/+15.5% tokens/J, 480B agentic A/B), reversing POLCA's frequency-first recommendation for this topology.
- Every latency-gated operating mode holds the ITL-p99 SLO in every repetition, whereas both vendor profiles violate it in a fraction of agentic repetitions (100% vs 96% compliance in the 235B paired study).
- Throughput-only setpoint acceptance admits SLO-violating configurations; tail-latency gating at calibration time is necessary, not optional.
- The reclaimable efficiency is MoE-specific: a comparable-scale dense model recovered roughly 5× less from cap-only control (+5.7±8.0% vs +26.0±6.4% tokens/J, Appendix A).
Strengths
Unusually candid self-reporting
§5.1 flags the N=2 single-session headline; §5.6 concedes a 6.0-point session-to-session disagreement and warns its ±1σ figures are within-session only; §6 lists eight limitations; the guard-never-fired fact is stated repeatedly rather than buried.
In-flight vendor baselines with a hygiene experiment
Max-P (profile 0,6) and Max-Q (1,6) are the shipped profiles, and §4 shows a uniform clock-lock reconstruction would save only 4.0-4.1% of baseline power versus the profile's 12.5-12.9% — a threefold baseline understatement avoided by construction.
Latency-gated acceptance gate as a portable contribution
Sustained-operation validation rare in this literature
Valuable negative results
Weaknesses
Headline result is statistically weak
Prefill attribution is confounded
No adaptive-DVFS comparison
Runtime guard unvalidated
Narrow external validity
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
This systems paper targets steady-state energy efficiency of prefill/decode (PD) disaggregated LLM serving on a single 8×B200 node, explicitly scoped to MoE models (Qwen3-Coder-480B FP8 TP4/TP4; Qwen3-235B-A22B NVFP4 TP1) on one engine stack (Dynamo + sglang-runtime 1.2.0). Three hypotheses drive it: H1 — optimal power settings are properties of the deployed (model, quantization, engine, hardware) combination, not the GPU class; H2 — under disaggregation each lane warrants its own profile and indeed its own actuator mechanism; H3 — converting SLO headroom into energy requires latency-gated calibration plus a runtime guard. Central claims: the calibrated ladder achieves 1.5–3.4× the shipped NVIDIA Max-Q profile's efficiency gain at equal or lower latency (+20.4% tokens/J at +3.5% mean e2e vs +8.6% at +5.2% on 480B agentic); a calibrated decode power cap beats static clock locks (+24.1% vs +17.2%/+15.5% tokens/J, 480B agentic A/B, §5.3); and every latency-gated mode holds the ITL-p99 SLO in every repetition where both vendor profiles miss it in a fraction of agentic repetitions (100% vs 96%, §5.1 table).
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is PD disaggregation in LLM serving and why does it matter for power management?
Prefill (processing the prompt) is compute-bound and power-hungry, while decode (generating tokens) is memory-bound and draws flat, low power. PD-disaggregated serving places the two phases on separate GPU pools, so each GPU's hardware regime is known statically. This paper exploits that: instead of one phase-blind power profile for all GPUs, it applies an SM-clock window to prefill GPUs and a calibrated power cap to decode GPUs.
How much energy does this controller save compared to NVIDIA's Max-Q profile?
On an 8×B200 node serving Qwen3-Coder-480B (FP8) under agentic load, the balanced mode delivered +20.4% tokens/J at +3.5% mean end-to-end latency versus Max-Q's +8.6% at +5.2%. In a three-day sustained run on a 235B model, the controller saved 32.3% of a decode/prefill lane pair's electricity versus no control, where Max-Q saved 15.8% on the same setup.
Does power capping hurt LLM decoding performance?
Not on a disaggregated decode lane when the cap is calibrated. The paper's A/B found a calibrated cap beats both throughput-optimal and aggressive static clock locks on tokens/J (+24.1% vs +17.2%/+15.5% on 480B agentic), because decode draws flat memory-bound power so the cap binds continuously while the GPU's own power manager adjusts clocks underneath it. A cap at or near TDP never binds and saves nothing — placement is what matters.
Does this power-saving method work on dense (non-MoE) models?
The authors scope their claims to Mixture-of-Experts models. An exploratory dense-model comparison (405B dense vs 480B MoE at comparable scale, Appendix A) found cap-only control recovered roughly 5× less efficiency gain (+5.7±8.0% vs +26.0±6.4% tokens/J), because dense models saturate tensor pipes and draw near the device power ceiling, leaving little slack for a cap to reclaim.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figure-internal plot geometry was not extracted as data; all cited numbers come from captions, tables, and axis-label text present in the extracted prose.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.