OCT-FedSIR: Toward Trustworthy Federated Ophthalmic Learning under Annotation Noise
Sina Gholami, Abdulmoneam Ali, Tania Haghighi, Rashadul H. Badhon, Behafarin Emam, Sally S. Y. Ong, Atalie C. Thompson, Theodore Leng, et al.
cs.LG · cs.AI · cs.DC
Quick verdict
OCT-FedSIR claims spectral identification and relabeling beat noisy-label FL on OCT under simulated label noise; results are broad and largely credible but simulation-only, with a suspicious perfect-identification outcome.
Summary
OCT-FedSIR targets federated OCT classification when some clients hold corrupted labels. The pipeline combines class-balanced spectral estimation, Stage-I logit adjustment, and three off-diagonal spectral descriptors clustered by a two-component Gaussian mixture to flag noisy clients, then relabels samples when representative- and residual-subspace predictions agree, and trains with knowledge distillation, logit adjustment, and distance-aware aggregation. Across 117 conditions on Kermany (4-class), UIC (DR severity), and Wake Forest (binary GA) data under symmetric (30-90%) and asymmetric noise and Dirichlet alpha in {2.0, 0.5, 0.1}, it averaged 86.73% accuracy versus 79.94% for RoFL and 78.75% for FedCorr, and improved on the original FedSIR identification, which dropped to 68.5-74.6 clean-client F1 under asymmetric noise. Relabeling recovered 77.2% of corrupted labels at 91.3% precision with a 3.5% false-correction rate; retention beat pruning by 9.30 points. Ablations attribute the largest gain to relabeling. All noise is simulated (acknowledged), WF uses only five clients, statistics are seed-level means and standard deviations without significance testing, and only Kermany is public, so real-world annotation-disagreement validation remains open.
Review complete
Assessment at a glance
4
risk flags identified
● high 0
● medium 2
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- OCT-FedSIR achieves the highest mean accuracy (86.73% across 117 conditions) among noisy-label federated methods on three ophthalmic tasks, beating RoFL by 6.79 and FedCorr by 7.98 points.
- The extended spectral identification perfectly separates simulated clean and corrupted clients (100% clean-client F1) in all evaluated conditions, whereas original FedSIR drops to 68.5-74.6 F1 under asymmetric noise.
- Spectral relabeling recovers a mean 77.2% of corrupted annotations at 91.3% precision with only a 3.5% false-correction rate.
- Retaining and correcting unreliable clients outperforms spectral pruning by 9.30 percentage points on average, showing annotation unreliability does not imply low data value.
- The relabeling component is the largest contributor to performance (-7.26 points when removed), ahead of KD (-1.82), LA (-1.05), and DaAgg (-0.52).
Strengths
Large, controlled evaluation across three distinct ophthalmic tasks
117 conditions spanning Kermany 4-class, UIC DR severity (445 scans/41 patients), and WF binary GA (11,513 B-scans), symmetric noise 30-90%, asymmetric 40-90%, Dirichlet alpha {2.0, 0.5, 0.1}, against 9 baselines with shared partitions, seeds, and initializations.
The FedSIR-vs-OCT-FedSIR identification comparison isolates a real transfer failure
Table 2: original FedSIR clean-client F1 drops to 68.5-74.6 under asymmetric noise, while the extended procedure (class-balanced spectra, Stage-I LA, smax descriptor) reaches 100.0 F1 everywhere - a concrete, ablation-backed motivation for the ophthalmic-specific redesign.
Sample-level correction metrics go beyond aggregate accuracy
Retention-vs-pruning comparison yields a practically meaningful finding
Honest, well-calibrated discussion of simulation limitations
Weaknesses
All annotation noise is synthetic label flipping
No inferential statistics despite 117 paired conditions
Modical scale of some federations and models
No hyperparameter sensitivity analysis
Reproducibility signals are mixed
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
OCT-FedSIR (arXiv:2609.14734) targets a real and underexplored gap in ophthalmic federated learning (FL): annotation reliability. Prior ophthalmic FL work (refs. 9-16) addresses statistical/domain heterogeneity but assumes local labels are trustworthy. The paper proposes a three-stage framework - spectral client identification, selective relabeling, and noise-aware optimization - and claims three headline results: (1) best mean accuracy across 117 conditions (86.73% vs 79.94% RoFL, 78.75% FedCorr, Table 1); (2) perfect separation of simulated clean/corrupted clients (Table 2) where the original FedSIR (CVPR 2026, ref. 31) falls to 68.5-74.6 clean-client F1 under asymmetric noise; (3) relabeling recovery of 77.2% of corrupted labels at 91.3% precision with 3.5% false corrections (Table 3), and retention beating pruning by 9.30 points. The scope is explicitly simulation: noise is injected by controlled transition matrices, and the authors state the asymmetric patterns 'should not be interpreted as estimates of real clinical error probabilities.'
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 4 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is OCT-FedSIR and what problem does it solve?
OCT-FedSIR is a federated learning framework for classifying OCT (retinal) images when some participating hospitals hold mislabeled data. It identifies which clients likely have corrupted annotations by examining the spectral structure (singular vectors) of learned class features, relabels individual samples using spectral references built from clean clients, and then trains with noise-aware techniques (knowledge distillation, logit adjustment, distance-aware aggregation). The key challenge it addresses is telling genuine clinical heterogeneity between hospitals apart from actual annotation errors.
How accurate is OCT-FedSIR compared to other federated learning methods?
Across 117 experimental conditions on three datasets (Kermany OCT, a UIC diabetic retinopathy cohort, and a Wake Forest geographic atrophy cohort), OCT-FedSIR achieved a mean accuracy of 86.73%, versus 79.94% for RoFL and 78.75% for FedCorr, the strongest baselines. It also identified corrupted clients perfectly (100% F1) in all simulated settings, where the earlier FedSIR method dropped to 68.5-74.6% F1 under asymmetric noise.
Does OCT-FedSIR work in real hospitals with real annotation mistakes?
Not yet demonstrated. All label noise in the study was synthetically injected by flipping labels according to predefined rules. The authors acknowledge that the perfect client separation partly reflects this controlled construction and explicitly call for validation on prospective multicenter data with naturally occurring grader disagreement before any clinical use.
Is it better to correct noisy clients' labels or simply remove them from federated training?
In this paper, correcting and retaining was much better: it beat pruning by 9.30 percentage points on average (86.73% vs 77.43%), because noisy clients still contain correctly labeled and clinically informative samples. However, the advantage was not universal - in the most extreme Kermany setting (severe asymmetric noise with the strongest heterogeneity), pruning performed better on aggregate, marking the boundary of dependable recovery.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1-7 (framework overview, spectral distributions, macro-F1 curves, relabeling schematic) are not analyzable from the extracted text; all judgments rest on tables and prose.
- The abstract supplied in metadata is truncated mid-sentence; the full abstract from the paper text was used instead.
- Supplementary tables S1-S6 were extracted fully, but small OCR-style artifacts in numeric rendering cannot be fully ruled out.
- The GitHub repository cited as [38] could not be inspected, so actual code quality/completeness is unverified.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.