TASTE: Throughput-Aware Batch Size Tuning for On-Device Edge Learning
Avik Bhatnagar, Federico Nicolas Peccia, Oliver Bringmann
cs.LG · cs.AI · cs.AR · cs.PF
Quick verdict
TASTE shows edge-CPU training throughput peaks below max batch size and BO finds it, giving ~1.5-2x speedup without accuracy loss; convincing but narrow.
Summary
This paper proposes TASTE, a technique for on-device training of ResNet-18 CNNs on resource-constrained edge CPUs that combines an ONNX/TVM-based training pipeline with Bayesian optimization (Tree-structured Parzen Estimator) of the training batch size to maximize measured on-device throughput. Across four devices (Banana Pi BPi-F3 RISC-V, Raspberry Pi 4B, Jetson Xavier, ZCU102, all CPU-only) and CIFAR-10/100 plus TinyImageNet, the authors show training throughput peaks at a batch size well below the memory-imposed maximum (bsmax 128 or 512), a ceiling they attribute to activation storage for backpropagation, while inference throughput keeps rising. Tuning to the optimal batch size yields at least 1.5x — up to roughly 2x on the Raspberry Pi — higher training throughput than bsmax. Accuracy is preserved by pairing the optimal micro-batch with gradient accumulation (SL, effective batch 128, constant LR 0.1, 200 epochs) or linear learning-rate scaling (online continual learning with Experience Replay in mammoth, base LR 0.03, five seeds); the throughput-optimal batches match or beat the baseline batch of 20, while batches of 128/512 degrade continual-learning accuracy by up to 28.54% (TIL, CIFAR-10). Limitations: a single model and domain, SGD only, a buffer-size change (300 vs 100/200) confounding the 512-batch CL comparison, no significance testing, and no code release.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 2
● low 3
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Edge-CPU training throughput peaks at a batch size well below the memory-imposed maximum due to activation storage for backpropagation, while inference throughput continues rising with batch size (Figs. 2-3).
- A TPE-based Bayesian optimization identifies the throughput-optimal batch size using half the iterations of exhaustive grid search (Sec. II-B).
- Training at the optimal batch size with gradient accumulation and linear LR scaling increases training throughput at least 1.5x (up to ~2x on Raspberry Pi 4) over the maximum batch size without compromising accuracy (Sec. IV).
- In online continual learning with Experience Replay, throughput-optimal batches preserve the stability-plasticity balance, whereas batches of 128/512 degrade accuracy by up to 28.54%±3.81% (TIL, CIFAR-10) (Sec. IV-B).
Strengths
Clean dual-paradigm evaluation on heterogeneous ISAs
The same ResNet-18 backbone is evaluated for offline SL and online CL (Experience Replay) across four edge CPUs spanning ARM and RISC-V (Table I), with execution restricted to CPU everywhere (even the Jetson Xavier) to isolate general-purpose floating-point throughput; accuracy claims use five randomized seeds.
Strong, mechanistically coherent throughput characterization
Figs. 2-3 report mean training-throughput gains over bsmax of 59.9% (Banana Pi), 85.7% (ZCU102), 67.4% (Jetson Xavier), 98.4% (Raspberry Pi) versus only 0.6-3.7% for inference — the asymmetry cleanly supports the activation-storage explanation for the training throughput ceiling.
Accuracy preservation is quantified, not just asserted
Practical, literature-grounded training-regime adaptations
Discloses failure modes and setup deviations
Weaknesses
Narrow empirical scope vs. transferability claims
Buffer-size confound in the headline CL degradation
Optimization contribution is thin and unbenchmarked
Mechanism asserted, not measured
Minimal statistical rigor and no artifact release
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
TASTE (arXiv:2609.07444, ©2026 IEEE) addresses on-device fine-tuning of CNNs on resource-constrained edge CPUs. Two coupled claims: (1) training throughput exhibits a ceiling at a batch size well below the memory-imposed maximum (bsmax of 128 or 512) because backpropagation must retain intermediate activations, whereas inference throughput keeps rising with batch size (Figs. 2-3); (2) a Tree-structured Parzen Estimator (TPE) Bayesian search over even batch sizes identifies this optimum (bsopt) in half the grid-search budget, and training at bsopt — with gradient accumulation in supervised learning (SL) and linear learning-rate scaling (Eq. 5) in continual learning (CL) — yields at least 1.5x (up to ~2x on Raspberry Pi 4) training throughput over bsmax without accuracy loss. In online CL with Experience Replay, batches of 128/512 lose up to 28.54%±3.81% accuracy (TIL, CIFAR-10), while bsopt configurations stay within close margins of the batch-20 baseline. The paper claims novelty as 'the first to present the feasibility to maximize training throughput on resource-constrained edge-CPU devices.'
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is the optimal training batch size for a Raspberry Pi 4?
For ResNet-18 training on CIFAR-scale datasets, this paper found that training throughput on the Raspberry Pi 4B peaks at small batch sizes — the exact optimum varied by dataset and device (quoted values between 4 and 32), well below the memory maximum of 128 — and training at that optimum approximately doubles throughput compared with the maximum batch size, with accuracy preserved via gradient accumulation.
Why does increasing batch size stop improving training throughput?
Because training must store intermediate layer activations for the backward pass; memory traffic grows with batch size and creates a throughput ceiling. Inference retains no activations, so its throughput keeps rising with batch size — the paper measures only 0.6-3.7% average inference-throughput gains at the optimum versus large training gains.
Does a large batch size hurt continual learning?
In this paper's online class- and task-incremental experiments with Experience Replay, batches of 128 and 512 degraded accuracy by up to about 21% (class-incremental) and 28.5% (task-incremental) on CIFAR-10 relative to the baseline batch of 20, while small throughput-optimal batches with linearly scaled learning rates matched the baseline.
Is the TASTE code publicly available?
The reviewed paper text does not mention a public code release. It describes the method via standard tools — ONNX training graph generation, Apache TVM compilation, hyperopt's TPE, and the mammoth continual-learning framework — which a reader could reimplement, but results are hardware-bound to the four evaluated boards.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figure-internal numeric labels in Figs. 2, 4, and 6 are jumbled by PDF extraction; per-device batch-size optima were therefore only cited where stated in prose (e.g., bsopt=4/10/6 for CIL).
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.