Sharpness-Aware Minimization (SAM) Improves Classification Accuracy of Bacterial Raman Spectral Data Enabling Portable Diagnostics
Kaitlin Zareno, Jarett Dewbury, Siamak K. Sorooshyari, Hossein Mobahi, Loza F. Tadesse
cs.LG · q-bio.QM · stat.ML
Quick verdict
SAM modestly improves accuracy (+2.7% avg) on one public Raman bacteria dataset, but 'significant' claims lack tests and gains vanish on the worst split.
Summary
This ICLR 2024 paper applies Sharpness-Aware Minimization (SAM) to 1D ResNet classification of Raman spectra from 50 clinical bacterial isolates spanning five pathogen classes, using the public dataset and architecture of Ho et al. (2019). The authors compare Adam versus SAM under three activation functions (GELU, ReLU, SELU) across 10 subject-level random train/test splits with 5 trials each, holding out complete patients so no test-isolate spectra appear in training. SAM raises average accuracy by 2.7% (up to 3.0% per activation; best single-split gain 10.5%) and reports variance reductions of roughly 75-84%, with SELU+SAM reaching 93.1±4.8%. Gains persist in most splits but disappear in the worst-performing split (Figure 3b), and Table 1 contains counterexamples such as ReLU split 2 (Adam 83.6% vs SAM 83.1%). The paper claims statistical significance without reporting any test statistics, p-values, or confidence intervals; SAM's rho, learning rates, and other training hyperparameters are largely unspecified. Limitations acknowledged include a single dataset and architecture, demographic coverage, and instrument-to-instrument variance. Overall, the contribution is a useful, well-motivated empirical demonstration, but its evidentiary support is thinner than the abstract's framing suggests.
Review complete
Assessment at a glance
6
risk flags identified
● high 1
● medium 3
● low 2
4 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Training with SAM instead of Adam increases average test accuracy on clinical Raman spectral classification by 2.7% across all 10 subject-level splits, with a maximum single-split gain of 10.5% (Table 1, Figure 2).
- SAM reduces accuracy variance by 75.2-84.3% depending on activation function, about 79.5% on average, indicating more stable predictions (Section 4).
- The improvements hold across three activation functions and most splits, supporting SAM as a default optimizer for small, noisy clinical spectral datasets (Figures 2, A.2).
- SAM's generalization benefits position AI-powered Raman spectroscopy for single-spectrum, portable point-of-care pathogen diagnostics (Section 5).
Strengths
Leakage-resistant clinical evaluation design
Section 3.1 and Figure A.1 hold out complete patients (one per infection type per biological replicate) so no spectra from test isolates are seen during training; 10 randomized splits x 5 trials probe patient-composition variability rather than a single lucky split.
Transparent per-split reporting, including failures
Table 1 reports all 30 optimizer/activation/split cells with mean±std over n=5 trials, and Figure 3b explicitly shows the configuration where SAM gives no improvement over Adam — an unusual degree of negative-result disclosure.
Practical, low-cost, reproducible-in-principle intervention
Honest limitations section
Weaknesses
Significance claims with no statistical tests
Best-case framing and an overbroad universality claim
Missing baselines and mechanism ablations
Thin hyperparameter reporting undermines reproducibility
Variance-reduction claims rest on trial-level stds
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper asks a narrow, practical question: does swapping Adam for Sharpness-Aware Minimization (SAM; Foret et al., 2021) improve generalization when classifying clinical bacterial Raman spectra? It reuses the public dataset and 1D ResNet (6 residual blocks, 25 convolutional layers) of Ho et al. (2019): 50 patient-derived isolates across five pathogen classes (E. coli, E. faecalis, E. faecium, P. aeruginosa, S. aureus). Central claims: (1) SAM raises average test accuracy by 2.7% over Adam across 10 subject-level splits, up to 10.5% on a single split; (2) SAM cuts accuracy variance by 75.2-84.3% depending on activation (79.5% average); (3) this supports single-spectrum clinical classification toward portable diagnostics (Abstract, §1, §5).
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 5 weaknesses, analyzed in detail
- 6 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What accuracy does SAM achieve on bacterial Raman spectra classification in this paper?
Across 10 patient-level random splits, SAM with a SELU activation reaches 93.1±4.8% average test accuracy, about 2.7 percentage points higher than Adam on average; the largest gain on any single split was 10.5%. The data and network come from the public Ho et al. (2019) study of 50 clinical bacterial isolates across five species.
Is SAM's improvement over Adam statistically significant in this study?
The paper repeatedly uses the phrase 'statistically significant', especially for the best-performing split, but it never reports a statistical test, test statistic, or p-value. Reported uncertainty is mean±standard deviation over only 5 training runs per configuration, so the significance claims are not substantiated by the presented evidence.
Does SAM always beat Adam for Raman spectral data in this paper?
No. On the worst-performing split the authors report no meaningful improvement over Adam (Figure 3b), and in Table 1 there is at least one configuration where Adam scores higher (ReLU activation, split 2: 83.6% vs 83.1%). SAM wins in most, but not all, of the 30 reported configuration/split cells.
What dataset and model were used, and can the results be reproduced?
The study reuses the publicly available Raman spectral dataset and 1D ResNet (6 residual blocks, 25 conv layers) from Ho et al. (2019, Nature Communications), training with the open-source davda54/sam PyTorch implementation on a single NVIDIA T4 GPU. The authors state code is available at github.com/Tadesse-Lab/SAM-Raman-Diagnostics, though key hyperparameters like SAM's rho and the learning rate are not given in the paper itself.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1-3 and A.1-A.2 are available only as extracted text; visual cues such as the distribution overlap in Figure 2a cannot be verified from this extraction.
- Table 1 was extracted from flattened text, so individual mean±std values may contain transcription/OCR errors; the identical GELU+SAM and ReLU+SAM entries for splits 1-2 may be an extraction artifact.
- Review covers arXiv v1 (2609.19453); the linked GitHub repository and its code were not inspected.
- Section 3.1's patient-count arithmetic is ambiguous in the extracted text; the split composition question is flagged rather than resolved.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.