Hidden in Rounds: Predicting the Time Cost of 802.11 Contention in Federated Learning
Satwat Bashir, Tasos Dagiuklas
cs.LG · cs.DC
Quick verdict
ns-3+FedAvg study: 802.11 contention barely moves rounds-to-target but scales constructed comm-time ~100x; honest, but timing is validated only against itself.
Summary
This simulation study asks how IEEE 802.11 CSMA/CA contention affects the cost of federated learning (FL) training. The design is decoupled: ns-3 measures per-station frame-delivery ratios and saturated throughput for six client densities (10–100 stations) and six offered loads (0.80–2.00× capacity), validated against the Bianchi model within 8.6% mean error; a separate PyTorch FedAvg trainer (LeNet-style CNN with ~178 KB updates; Fashion-MNIST and CIFAR-10; i.i.d. and Dirichlet splits; five seeds; 720 runs) uses the frame-delivery ratio as an update-admission probability and assigns each round a communication duration from a closed-form equation. Key results: all 720 runs reach their target within the round budget; rounds-to-target is nearly invariant to offered load (8.6 rounds at N=50 across all loads), while the constructed communication time-to-target grows 2.5× at N=50 and roughly 100× across density (6 to 640 seconds at saturated load). A two-stage Bianchi-anchored estimator predicts this constructed time with held-out MAPE of 2.3–10.2%, far outperforming a channel-free regression under load extrapolation. A persistent heterogeneous-participation arm finds no statistically distinguishable class-accuracy gap over five seeds, though confidence intervals are wide. The central limitation, candidly stated by the authors, is circularity: estimator error is measured against communication time built from the same round-duration equation, never against independently measured end-to-end completion time, and updates are never packetized. Code is available on GitHub.
Review complete
Assessment at a glance
5
risk flags identified
● high 1
● medium 3
● low 1
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Across 720 FedAvg runs (two datasets, two splits, six densities, six loads, five seeds), every run reaches its target accuracy within the round budget, and rounds-to-target is nearly invariant to offered load while constructed communication time-to-target grows 2.5× at N=50 and roughly 100× across density (6 to 640 s at saturated load).
- A two-stage Bianchi-anchored estimator predicts the constructed communication time-to-target with held-out MAPE of 2.3–10.2%, sharply outperforming a channel-free regression under load extrapolation (e.g., 4.1% vs 45.8% on Fashion-MNIST i.i.d.).
- Persistent heterogeneous participation, at matched mean admission probability, produces no statistically distinguishable worst-class or excluded-class accuracy gap over five seeds, though several confidence intervals are wide.
- The ns-3 contention model is validated against the Bianchi saturation-throughput analysis within 8.6% mean error (max 12%) before being used.
Strengths
Explicit measured-vs-constructed separation
Section III-A and the abstract state precisely which quantities come from ns-3 (frame counts, p_f, R_sat) and which are constructed by the trainer (admission, round duration) — unusually high epistemic hygiene for simulation studies in this area.
Contention model validated before use
Saturation throughput is compared against the analytical Bianchi model (Sec. III-C, Fig. 1) with 8.6% mean and at most 12% deviation across densities, anchoring the channel input to a known reference before it feeds the trainer.
Clean, well-quantified dissociation result
Extrapolation-grade estimator evaluation
Reproducibility provisions
Weaknesses
Circular validation of the headline estimator
Per-frame proxy conflated with per-update admission
Underpowered, mechanism-free bias analysis
No integrated-simulation spot check
Reproducibility gaps in numbers
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
This is a simulation-based performance study of FedAvg over an IEEE 802.11 CSMA/CA cell (Sections I, III). It makes four claims: (1) ns-3 measurements of frame-delivery ratio p_f(N,L) and saturation throughput R_sat(N) over six densities (10–100 stations) and six loads (0.80–2.00× ~15 Mbit/s capacity), validated against the Bianchi model within 8.6% mean error (Sec. III-C, Fig. 1); (2) a dissociation between rounds-to-target and constructed communication time-to-target across 720 FedAvg runs (Sec. IV-A, Fig. 3); (3) a two-stage Bianchi-anchored estimator with held-out MAPE 2.3–10.2% (Table III); (4) no detectable class-accuracy gap under persistent heterogeneous participation over five seeds (Sec. V, Table IV). Crucially — and unusually candidly for this area — the abstract and Sec. IV-B state that the estimator error is measured against communication time constructed from the same round-duration equation (Eq. 1), not against independently measured completion time. This is a study of an estimator within a constructed timing model, not a validation of that model.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
Does this paper measure real end-to-end federated learning training time over Wi-Fi?
No. It measures frame-delivery ratios and saturation throughput in the ns-3 simulator, then constructs communication time from a closed-form round-duration equation. The paper is explicit that the estimator's error is measured against this constructed metric, never against independently measured completion time, and complete model updates are never packetized or transmitted in simulation.
How accurate is the paper's communication-time estimator?
The Bianchi-anchored two-stage estimator achieves a held-out mean absolute percentage error (MAPE) of 2.3–10.2% across datasets, splits, and evaluation protocols (Table III). It greatly outperforms a channel-free regression under load extrapolation (e.g., 4.1% vs 45.8% on Fashion-MNIST i.i.d.), though this comparison uses different inputs and the ground truth is constructed from the same equation the estimator uses.
Did excluded or poorly connected clients hurt the global model's accuracy?
Not detectably in this setup. Comparing uniform participation against persistent heterogeneous participation at matched mean admission probability, the worst-class and excluded-class accuracy gaps had 95% confidence intervals that all included zero over five seeds. However, the intervals were wide (e.g., [−21,+0] percentage points), so a small bias effect cannot be ruled out.
Is the code and data for this paper available?
Yes. The paper states that the contention model, the FedAvg trainer, and the configurations needed to reproduce the experiments are available at github.com/satwatbashir/Contention-FL. The datasets used (Fashion-MNIST and CIFAR-10) are public.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures are extracted as scattered axis/tick text (e.g., Fig. 3's heatmaps appear as raw number grids); curve-level detail could not be verified visually, though every value cited in this review appears in the body text or tables.
- The Fashion-MNIST target accuracies are never stated numerically anywhere in the provided text (only CIFAR-10's 0.48/0.46 appear); this is an absence in the paper itself, not an extraction truncation — the text runs complete through reference [26].
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.