Empirical Evaluation of Data Poisoning Attacks in Supervised Learning
Toshif Khan, Muhammad Abusaqer
cs.CR · cs.LG
Quick verdict
Benchmark-style study showing label flipping degrades linear models while backdoors hit 0.9667-1.0 ASR; sound but single-seed, no stats, no defenses.
Summary
This MICS 2026 conference paper reports a compact benchmark of two training-time poisoning attacks — random label flipping and a BadNets-style backdoor (3×3 white square, bottom-right, target class 0) — on stratified 8,000-train/2,000-test subsets of MNIST and Fashion-MNIST (seed 42), using scikit-learn Logistic Regression (saga), Linear SVM, and Random Forest (100 trees) at 5%, 10%, and 20% poisoning rates. Clean baselines favor Random Forest (0.9475 MNIST / 0.8455 Fashion-MNIST accuracy); Linear SVM is weakest. Label flipping degrades the linear models most (Logistic Regression −0.0705 accuracy at 20% on MNIST; SVM −0.0660 at 10% on Fashion-MNIST), while Random Forest stays within 0.0075 of baseline. The backdoor reaches attack success rates of 0.9667–1.0000 across all 18 conditions, with clean accuracy typically near baseline (within 0.0115 on MNIST; worst loss 0.0330, SVM, Fashion-MNIST, 20%). The authors conclude that clean metrics miss targeted compromise and that accuracy does not imply security. Limitations: one seeded run per configuration with no variance or significance testing, a visible trigger with no detection/defense evaluation, classical models only, and no code release; a non-monotonic SVM result (Fashion-MNIST 10% vs 20%) is unexplained.
Review complete
Assessment at a glance
4
risk flags identified
● high 0
● medium 2
● low 2
4 strengths · 4 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Label flipping at 5-20% visibly degrades clean accuracy and macro-F1, most for Logistic Regression (−0.0705 accuracy at 20% on MNIST) and Linear SVM (−0.0660 at 10% on Fashion-MNIST), while Random Forest stays within 0.0075 of baseline.
- BadNets-style backdoor poisoning with a 3×3 white trigger reaches attack success rates of 0.9667-1.0000 across all 18 dataset-model-rate conditions while keeping clean-test accuracy near baseline (within 0.0115 on MNIST; worst 0.0330).
- Conventional clean-test metrics alone are insufficient to detect backdoor compromise; security-oriented metrics like attack success rate are necessary.
- Strong clean performance does not imply security: Random Forest had the best clean baselines yet achieved ASR 1.0000 on MNIST at all rates and on Fashion-MNIST at 10% and 20%.
Strengths
Complete factorial enumeration
2 attacks × 3 rates × 2 datasets × 3 models plus clean controls = 36 attacked configurations, all fully enumerated in Tables 2-3 with no cherry-picking; metric suite (accuracy, macro-P/R/F1, ASR per §3.7) is standard and the ASR definition (triggered non-target test samples only) is correct.
Clear, useful security takeaway
Table 3 shows ASR 0.9667-1.0000 in all 18 backdoor conditions while clean accuracy stays within 0.0115 on MNIST (worst 0.0330, SVM, Fashion-MNIST 20%); the Random Forest case — best clean baseline (0.9475/0.8455, Table 1) yet ASR 1.0000 at five settings — concretely demonstrates the accuracy≠security decoupling.
Honest scope discipline
Above-average protocol disclosure
Weaknesses
Single-seed design, zero inferential statistics
Unexplained non-monotonicity
Stealth claim lacks any detection/defense test
No code release, missing external comparison
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper positions itself explicitly as a compact, reproducible benchmark rather than a novel attack or defense (§1, §3.1). It evaluates two training-time poisoning families under one protocol: uniform random label flipping (§3.4) and BadNets-style backdoor poisoning via a visible 3×3 white square stamped bottom-right with target class 0 (§3.5). Settings: stratified 8,000/2,000 subsets of MNIST and Fashion-MNIST (seed 42, pixels [0,1], 784-d flattened, §3.2); three scikit-learn baselines — Logistic Regression (saga, 300 iterations), Linear SVM (5,000 iterations), Random Forest (100 trees) (§3.3); poisoning rates 5/10/20%. Three central claims: (i) label flipping visibly degrades clean-test metrics, most for the linear models; (ii) backdoor poisoning attains attack success rate (ASR) 0.9667-1.0000 in all 18 conditions while clean accuracy stays near baseline; (iii) conventional clean-test metrics are insufficient and clean accuracy does not imply security. Within its stated scope, all three claims are supported by Tables 1-3; the contribution is pedagogical consolidation in the tradition of Biggio [2], BadNets [3], Schwarzschild [4], and Yerlikaya & Bahtiyar [5].
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 4 weaknesses, analyzed in detail
- 4 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What poisoning rates and models were tested in this paper?
Label flipping and backdoor poisoning at 5%, 10%, and 20% of an 8,000-sample stratified training subset of MNIST and Fashion-MNIST, evaluated on Logistic Regression, Linear SVM, and Random Forest (scikit-learn), with a 2,000-sample clean test set and fixed seed 42.
How successful was the backdoor attack?
Attack success rate ranged from 0.9667 to 1.0000 across all 18 dataset-model-rate combinations, while clean-test accuracy stayed near baseline — within 0.0115 on MNIST and with a worst loss of 0.0330 (Linear SVM, Fashion-MNIST, 20% poisoning).
Is Random Forest robust to label flipping?
In this study it dropped at most 0.0075 accuracy on MNIST and 0.0070 on Fashion-MNIST across all poisoning rates, far less than the linear models. The authors explicitly caution this holds only for their setting (simple random corruption, these datasets), not general poisoning robustness.
Is code available for this paper?
No repository or code link is provided. The paper describes a seeded scikit-learn pipeline (8,000/2,000 stratified subsets, stated solver settings, a 3×3 white-square trigger) in enough detail for approximate reimplementation, but exact numbers cannot be verified without the notebook.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1-4 appear in the extraction as captions only; their plotted values could not be inspected, though the underlying numbers are present in Tables 1-3, so no figure-dependent claim is made in this review.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.