Partition-Aware Scheduling for Mobile Heterogeneous Inference Co-Execution
Zhuojin Li, Marco Paolieri, Leana Golubchik
cs.DC · cs.LG · cs.PF
Quick verdict
Joint partitioning + DAG scheduling for mobile CPU-GPU inference; online search reaches 1.00-1.04x of offline Gurobi on 4 phones; no code, no variance stats.
Summary
This paper tackles mobile DNN inference on heterogeneous SoCs combining a mobile GPU with performance/efficiency CPU clusters. Existing approaches exploit either inter-operator parallelism (whole-operator DAG scheduling such as HEFT) or intra-operator parallelism (splitting each operator across devices, as in CoDL). Using ResNet16, Inception-v3, and HRNet timelines, the authors show the best strategy is topology-dependent: scheduling leaves CPUs idle on chain-like graphs (ResNet16: 6.8 ms vs 4.9 ms), while partitioning harms branch-heavy graphs (HRNet: 7.6 ms vs 6.7 ms HEFT, 6.0 ms joint). They formulate partition-aware DAG scheduling covering partition strategy, workload division, device assignment, and ordering, solved offline by Gurobi (300 s/stage) as reference. An online iterative search — staging at global join points (≤20 operators), slack-based criticality sampling, and GBDT latency predictors (conv MAPE <8.2%) — reaches 1.01×–1.04× average normalized latency in simulation and 1.00×–1.04× measured, across 18 CNNs and 4 phones, with overhead averaging 37.4% of TFLite initialization. Limitations: CNN-only, batch-1, static-shape workloads; no run-to-run variance or optimality-gap reporting; prior systems compared only via abstracted objectives; no code release stated.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- The best execution strategy for mobile heterogeneous inference is topology-dependent: schedule-only wins where inter-operator parallelism abounds, partition-only on chain-like graphs (Section 3).
- Joint partition-aware scheduling (Section 4.1) generalizes both paradigms and can be solved near-optimally online by the proposed iterative search with staging, criticality-aware sampling, and GBDT latency prediction.
- The iterative search achieves 1.01×–1.04× average normalized latency versus offline Gurobi joint optimization in simulation and 1.00×–1.04× in real measurements across 18 models and 4 phones.
- Total deployment-time overhead averages 37.4% of TFLite model-initialization time, making platform-specific scheduling practical at deployment.
- Cross-platform schedule reuse is safe only between devices with similar CPU-GPU relative performance; otherwise latency degrades up to 1.41× (Table 6).
Strengths
Unified formulation generalizes both prior paradigms
Section 4.1 models partition strategy (none/cout/cin/spatial), integer workload division, device assignment and execution order under full-barrier dependencies; Sections 4.2–4.3 show HEFT-style scheduling and CoDL-style co-execution as restricted cases, and the Gurobi encoding gives an offline reference point.
Compelling topology-dependent motivation
Section 3 timelines quantify the failure modes: on ResNet16, HEFT leaves CPU utilization below 4% (6.8 ms) vs 4.9 ms with partitioning (>85% utilization); on HRNet, partition-only (7.6 ms) is worse than HEFT (6.7 ms) while the joint method reaches 6.0 ms on OnePlus 11.
Unusually thorough evaluation matrix
Deployment-time practicality is quantified, not asserted
Honest scope and threats discussion
Weaknesses
No repeated-trial statistics for on-device results
Optimality of the normalization reference is unproven
Prior systems replaced by abstractions
CNN-only evidence behind a broader abstract
No artifact release; training protocol underspecified
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
This preprint (to appear in Performance Evaluation) targets latency-oriented, batch-1 inference of static-DAG models on mobile SoCs exposed as three logical devices — a mobile GPU plus large- and medium-core CPU clusters (Table 1: OnePlus 11/SD 8 Gen 2, Motorola Edge+ 2022, Pixel 5, Pixel 4). The claims are: (1) the best execution strategy is topology-dependent, since schedule-only (HEFT) leaves CPUs idle on chain-like graphs (ResNet16: 6.8 ms vs 4.9 ms partitioned) while partition-only hurts branch-heavy graphs (HRNet: 7.6 ms vs 6.7 ms HEFT and 6.0 ms joint, Section 3); (2) a joint formulation over partition strategy (none/output-channel/input-channel/spatial), workload division, device assignment and ordering (Section 4.1) is solvable near-optimally at deployment time; (3) the iterative search reaches 1.01×–1.04× (simulation, Table 4) and 1.00×–1.04× (measured, Table 5) average normalized latency versus offline Gurobi across 18 CNNs and 4 phones; (4) overhead averages 37.4% of TFLite initialization (Figure 5, Table C.11).
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is partition-aware DAG scheduling for mobile inference?
A scheduling formulation for running a neural network on a phone's GPU plus CPU core clusters in which the scheduler jointly decides, for each operator: whether to split it across devices (and how — by output channels, input channels, or spatial tiles), which device runs each piece, and in what order the pieces execute. It generalizes schedule-only methods (whole operators assigned to devices, like HEFT) and partition-only methods (split every operator, like CoDL).
How much faster is it than just using the GPU or HEFT?
In the paper's simulation across 18 CNN models, GPU-only execution averaged 1.61×–2.13× slower than the offline joint optimum, and the best schedule-only methods (HEFT family) averaged 1.28×–1.51× slower. The proposed online iterative search averaged within 1.01×–1.04× in simulation and 1.00×–1.04× in real measurements on four Android phones.
Does this work for LLMs or vision transformers?
Not as evaluated. The method requires a static operator DAG with fixed tensor shapes and batch size 1, which fits CNNs (and in principle fixed-shape ViTs, if suitable attention partition strategies and latency predictors existed). Autoregressive LLMs and dynamic Mixture-of-Experts routing violate the reusable fixed-plan assumption, and no transformer was evaluated in this paper.
What does the scheduler cost at deployment time?
On OnePlus 11 it averaged 37.4% of the TFLite model-initialization time (which runs once per model): 51–185 ms for latency prediction plus 24–1,339 ms for the iterative search, depending on model size (up to about 51% for the largest HRNet). The schedule is computed once and reused for every subsequent inference.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Timeline and box-plot figures (Figs 1–6) survive only as text annotations; visual utilization claims (e.g., 'below 4%' CPU utilization under HEFT) cannot be independently verified from the extraction.
- Runtime internals (OpenCL SVM coordination, predictor hyperparameters) rely on the authors' prior work [9], which is not fully contained in this text.
- Journal preprint marked '(to appear)' in Performance Evaluation; volume and pages absent, so citation details may change.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.