Paging the Experts: A Reproducible Characterization of Flash-Backed MoE Inference on iPhone
cs.PF · cs.LG
Quick verdict
An unusually honest systems characterization of flash-paged MoE inference on one iPhone: valuable negative results, but tiny scope and no performance baseline.
Summary
This systems paper presents Routide, a Swift/MLX runtime that executes the text path of a pinned 4-bit quantized Qwen3.6-35B-A3B mixture-of-experts checkpoint (18.1 GB of routed-expert payload) on an iPhone 17 Pro Max, streaming expert weights from flash storage into a byte-budgeted memory cache (512–1,024 MiB). Offline replay of five recorded 128-token route traces shows the apparent 512 MiB capacity cliff is policy-dependent: LRU scores 0.00% demand hits because the minimum expert reuse distance (312) exceeds the 303 experts that fit, while seeded random eviction reaches 18.80% and pinned farthest-next-use 54.49% (38.58% for LRU at 576 MiB). Same-runtime Mac controls establish exact output invariance under eviction and prefetch across 2,560 token comparisons, but resident-Python versus phone sequences diverge on all five prompts — traced to an MLX library version difference — so cross-runtime equivalence is explicitly not claimed. The paper further documents cache pollution caused by prefetch lookups refreshing LRU recency without any speculative I/O, sampled process-footprint peaks of 1.87–2.73 GiB, a retained thermal stopping event, negative timing pairs, and one heavily qualified whole-device power estimate. Key limitations: a single device and checkpoint, small designed workloads, serial single-token prefill, and no throughput or quality comparison against smaller resident models. It is a rigorously scoped feasibility characterization, not an efficiency claim.
Review complete
Assessment at a glance
0
risk flags identified
● high 0
● medium 0
● low 0
5 strengths · 5 weaknesses · 0 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- The 512 MiB capacity cliff for expert caching is a cache-policy/workload interaction (LRU 0.00% vs. random 18.80% vs. pinned-FNU 54.49% demand hits), not a universal memory requirement.
- Same-runtime outputs are exactly invariant under eviction and asynchronous prefetch, established by 2,560 exact token comparisons on a Mac.
- Cross-runtime sequence equivalence between resident Python MLX and the phone is not established: all five recorded sequences diverge, with the mismatch traced to an MLX library version difference.
- Confidence-gated prefetch can pollute the expert cache through LRU recency side effects alone, adding 1,120 demand misses at 576 MiB without any speculative I/O.
- Sampled process-footprint peaks of 1.87–2.32 GiB on short prompts and 2.39–2.73 GiB on one 340-token prompt bound memory feasibility for this checkpoint on iPhone 17 Pro Max.
Strengths
Mechanistic explanation of the capacity cliff
§5.1 shows the minimum distinct-expert reuse distance is 312 on every trace while only 303 expert payloads fit at 512 MiB, and an independently computed reuse histogram predicts every recorded LRU hit count — turning an empirical anomaly into a predictive, checkable model.
Exceptional claim hygiene
Appendix A enumerates supported vs. unsupported statements; negative evidence is retained (the fair-thermal stop row §5.5, the +7.18% slower 576 MiB pair §5.4), and cross-runtime equivalence is explicitly refused despite 97.5% conditional next-token agreement (§5.3).
Novel cache-pollution finding
Metric separation discipline
Reproducibility scaffolding
Weaknesses
Very small, designed workload base
No performance or quality baseline
Serial single-token prefill
Prefetch benefit never demonstrated
Cross-runtime divergence left open
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
Routide is a Swift/MLX runtime that executes the text path of a pinned public 4-bit Qwen3.6-35B-A3B checkpoint (§3.1: 20.40 GB tensor payload, 18.12 GB routed-expert payload, 256 experts across 40 layers, top-8 routing) on an iPhone 17 Pro Max, keeping expert weights in flash storage and a byte-budgeted subset (512–1,024 MiB) in memory. The paper claims exactly three things: an inspectable storage-to-execution path, a cache-policy characterization with negative results, and separated correctness/measurement claims. Appendix A is an explicit claim-boundary table that disclaims first-phone-runtime status, state-of-the-art throughput, quality parity, prefetch energy savings, and universal feasibility. This scope discipline is central to the paper's identity: it sells a reproducible characterization, not a product.
Central claims: (1) the 512 MiB LRU capacity cliff is a policy/workload interaction, not a memory requirement; (2) same-runtime outputs are exactly invariant under eviction and prefetch; (3) cross-runtime (Python vs. phone) equivalence is not established and is explicitly not claimed; (4) prefetch lookups can pollute cache recency without performing any speculative I/O; (5) sampled process footprints of 1.87–2.73 GiB bound feasibility for these workloads.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 0 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 0 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
Can a 35B-parameter MoE model really run on an iPhone?
For this paper's setup, yes: the text path of a 4-bit quantized Qwen3.6-35B-A3B checkpoint (18.1 GB of routed-expert weights) runs on an iPhone 17 Pro Max with expert weights kept in flash storage and only a 512–1,024 MiB memory cache. Sampled process memory peaked at 1.87–2.73 GiB and decode ran at roughly 1.8–3.2 tokens per second. It is feasible but slow, and only demonstrated on one device, one checkpoint, and small test workloads.
Why does an LRU cache fail at 512 MiB for MoE expert paging?
Each forward pass requests 8 experts from each of 40 layers (320 experts, about 540 MiB of payload). On these traces an expert is reused only after 312 other distinct experts are touched, but only 303 fit in 512 MiB, so LRU always evicts an expert before it returns, giving 0.00% hit rate. Seeded random eviction still achieves 18.80% and an offline future-aware policy 54.49%, so the cliff is a cache-policy artifact, not a hard memory requirement.
Are the iPhone's outputs identical to a Mac's?
With the same Swift runtime, yes: outputs matched exactly across cache-eviction and prefetch conditions (2,560 token comparisons on a Mac). Against a resident Python MLX reference, no: all five recorded 128-token sequences diverged, traced to an arithmetic difference between MLX library versions 0.31.2 and 0.32.2. Cross-framework numerical equivalence therefore remains unproven.
Is the code and model available so others can reproduce this?
Substantially: the runtime and analysis code are in the authors' public Routide GitHub repository, the checkpoint is a pinned public mlx-community release under Apache-2.0, and evidence hashes plus per-request measurements are published in the paper's appendix. Full reproduction additionally requires an iPhone 17 Pro Max, a high-memory Apple-silicon Mac, and the licensed weights; one power-profiling archive was withheld for privacy reasons.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1–2 appear only as captions and axis labels in the extracted text; curve shapes and bar values beyond the tabulated numbers could not be visually verified.
- Review is based solely on the provided extraction; the GitHub repository, research archive, and pinned checkpoint revision could not be verified (no network access).
- The study spans iOS 26.5.2–27.0 and multiple executable digests; some cohort-level instrumentation details (e.g., schema-6 vs. schema-12 exports) may be under-described in the extracted text.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.