How Do Prompt Variations Affect Energy Consumption in On-Device LLMs?
Wei Hu, Xiaolong Tu, Dawei Chen, Yitao Chen, Kyungtae Han, Haoxin Wang
cs.CL
Quick verdict
Solid measurement study: prompt design shifts on-device LLM energy via token usage and per-token cost; findings plausible but length-confounded, stats thin.
Summary
The paper presents what the authors describe as the first large-scale empirical study of how prompt design affects the energy consumption of on-device LLM inference. Two prompt properties are studied: phrasing pattern (seven sub-properties — aggressive, conversational, chain-of-thought, formatting, persona, polite, technical — from the CLEF 2025 ELOQUENT dataset) and cognitive load (intrinsic, extraneous, germane; a new dataset generated with Gemini-2.5-Pro from SVAMP, BoolQ, and AI2-ARC base prompts, validated by rubric scores of at least 8 on target and at most 3 on non-target properties plus embedding cosine similarity of at least 0.6). Five quantized models (SmolLM2-360M to Gemma-2-2B) run via MLC-LLM on Pixel 8 Pro and Pixel 7 with locked CPU/GPU frequencies; 404 prompts with three runs each yield 7,620 inference runs, with phase-level (prefill/decode) energy integrated from ADB-sampled power traces. Key findings: per-token energy is dominated by the model; prefill per-token cost exceeds decode by 3-6x on most models; cognitive load mainly raises decode per-token energy (up to 1.29x on Qwen-2.5-1.5B on BoolQ), while phrasing pattern shifts energy mainly through token usage (CoT and aggressive tone generate more tokens). Energy-quality Pareto frontiers are model-dependent, and some variants (format) beat the base prompt on both axes. Limitations: prompt length is uncontrolled and confounds cognitive-load effects; a single LLM judge (Gemini) generates and evaluates; no statistical significance testing; cognitive-load experiments use only one device.
Review complete
Assessment at a glance
4
risk flags identified
● high 0
● medium 2
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Per-token energy of on-device LLM inference is primarily determined by the executed model rather than the prompt sub-property (Sec. 4.1, Obs. 2).
- Cognitive load mainly modulates per-token decode energy (up to 1.29x Base on Qwen-2.5-1.5B for extraneous load on BoolQ, Fig. 3b), while phrasing pattern affects energy mainly through token usage (Sec. 4.2).
- On most tested models, per-token prefill energy exceeds per-token decode energy by 3-6x, reflecting on-device runtime limits and fixed prefill-window overheads (Sec. 4.1, Obs. 1).
- Prompt choice shifts the energy-quality frontier in a model-dependent way, and some phrasings (e.g., format for Gemma-2-2B) Pareto-dominate the base prompt under fixed weights and decoding (Sec. 4.3, Fig. 6).
Strengths
Unusually rigorous measurement setup
7,620 inference runs (2 prompt properties, 10 sub-properties, 4 datasets, 404 prompts, 5 models, 2 phones, 3 runs each; Appendix D) with locked CPU/GPU frequencies, minimum brightness, fixed decoding (T=0.6, top-p 0.8), and an explicitly defined extended prefill window (request to first token) separating prefill from decode energy via Eq. 2 power integration.
Principled, inspectable prompt construction
Cognitive-load variants are generated under single-property isolation rules with the full template printed (Appendix B), then filtered by rubric scores (target >=8, non-target <=3) and embedding cosine similarity >=0.6 (Sec. 3.2); evaluator template also disclosed (Appendix C).
Clean mechanistic decomposition
Rare transparency about measurement conventions
Actionable model-aware guidance
Weaknesses
Prompt-length confound undermines the causal story
No statistical inference
Single-judge circularity around Gemini-2.5-Pro
Cross-model comparisons conflate precision with architecture
Asymmetric device coverage and unreported profiling parameters
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper asks whether prompt design — specifically phrasing pattern (7 sub-properties from CLEF 2025 ELOQUENT) and cognitive load (intrinsic/extraneous/germane) — changes the energy of on-device LLM inference, and whether prompt choice can improve the energy-quality trade-off. Headline claims (Sec. 1, 4, 6): (1) per-token energy is model-dominated; (2) cognitive load mainly modulates per-token decode energy while phrasing pattern shifts token usage; (3) prefill per-token cost exceeds decode 3-6x on most models; (4) energy-quality frontiers are model-dependent and some prompts Pareto-dominate the base. The evaluation space is 2 properties x 10 sub-properties x 4 datasets x 404 prompts x 5 quantized models x 2 phones, 3 runs each = 7,620 inference runs (Appendix D).
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 4 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
Do prompt styles actually change how much battery on-device LLMs use?
Yes, but through different routes. In this study of five small LLMs on Pixel phones (7,620 runs), surface phrasing changes energy mainly by changing how many tokens are generated (chain-of-thought and aggressive prompts generate more tokens), while cognitive-load prompts mainly change the energy cost of each generated token (up to 1.29x baseline on Qwen-2.5-1.5B). The model itself, not the prompt, is the biggest driver of per-token cost.
Which prompt style is most energy-efficient without losing answer quality?
There is no universal winner; it is model-dependent. On Gemma-2-2B, a structured format prompt used roughly 12% less decode energy than the base prompt while scoring slightly higher on quality; on Llama-3.2-1B, Qwen-2.5-1.5B, and SmolLM2-360M at least one style achieved both lower energy and higher quality than the base, but the best style differs per model, so the authors recommend model-aware prompt selection.
What are the main weaknesses of this study?
Four stand out: prompt length was not controlled, so cognitive-load variants (which are longer) confound per-token energy effects with input length; a single LLM judge (Gemini-2.5-Pro) both created and evaluated the prompts; no statistical significance tests were run (three-run means only); and cognitive-load experiments used only one phone, the Pixel 8 Pro.
Is the code and data available?
The paper states that code, datasets, and scripts are available at the project page amai-gsu.github.io/PromptProperty, and the prompt generation/validation templates are printed in the appendices. Note that reproducing the energy measurements requires physical Pixel 7/8 Pro hardware with ADB power sampling, which cannot be done purely in simulation.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figure-internal data (Figs. 1-6) is extracted as jumbled glyph streams; figure-based observations were cross-checked against captions and the intact appendix tables (Tables 8-12) wherever possible.
- Shaded color encodings in Tables 8-12 carry no information in text form; better/worse judgments rely on the numeric values, which are intact.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.