Adversarial Training for Tabular Credit Scoring: A Multi-Attack Robustness Evaluation in P2P Lending
Gijs A. F. Niewzwaag, Marijn G. S. Veth, Manuele Massei, Marcos R. Machado
cs.LG · q-fin.RM · stat.ML
Quick verdict
Multi-attack adversarial-training benchmark for P2P credit scoring, replicated on Prosper; but one ε and no significance tests temper the robustness claims.
Summary
The paper benchmarks adversarial training for tabular credit scoring on a curated Lending Club subset (396,030 loans, 28 features), asking whether defences generalise across attack types. Three model families — logistic regression, a feed-forward network, and an FT-Transformer — are trained on clean data plus five adversarial augmentations (FGSM, PGD, DeepFool, Salt-and-Pepper, mixed; 80% clean/20% adversarial mix) and evaluated on clean and four attacked test sets, with perturbations confined to applicant-mutable features (income, DTI, revolving balance/utilisation, purpose, zip code), a fixed ε=0.1 budget, and stratified 5-fold cross-validation. Clean-trained models degrade only moderately under attack (ROC AUC roughly 0.86–0.87 versus about 0.90 clean). Matched gradient-based training yields the largest gains — the transformer reaches ROC AUC 0.980 on FGSM/PGD tests — and FGSM and PGD robustness transfer mutually, but transfer weakly to S&P noise; mixed training is the most balanced option while preserving clean accuracy. Appendix B replicates the grid on Prosper, reproducing the qualitative ordering at much lower discrimination (clean AUC around 0.70). Main limitations: a single fixed attack budget, no significance tests or dispersion for the main results, and the authors' own caveat that matched-training gains may partly reflect recognition of a fixed perturbation pattern rather than genuine robustness. Code is linked in a public GitHub repository.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 4
● low 1
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Adversarial training sharply improves robustness against the matched attack family, most strikingly for gradient attacks, with the FT-Transformer reaching ROC AUC 0.980 on FGSM/PGD test sets versus 0.868 for the clean-trained model (Table 11).
- Robustness transfers almost perfectly between FGSM and PGD but weakly to Salt-and-Pepper corruption, so single-attack defences overstate real-world resilience.
- Mixed-attack training provides the most balanced robustness across heterogeneous attacks while preserving clean-test accuracy for the higher-capacity models.
- These qualitative patterns replicate on an independent Prosper dataset (Appendix B) despite much lower absolute discrimination (clean AUC 0.70-0.71).
Strengths
Full train-by-test robustness grid
Six training regimes (clean plus FGSM/PGD/S&P/DeepFool/mixed) crossed with five test conditions across three model families, reported as fold-averaged tables (Tables 9-11, appendices A.14-A.16); systematic cross-attack generalisation evidence of this breadth is rare in credit scoring.
Realistic, feature-constrained threat model
All perturbations restricted to six applicant-mutable features (annual_inc, dti, revol_bal, revol_util, purpose, zip code) via gradient masking and structurally valid one-hot swaps (Section 3.4), preventing implausible-applicant artefacts that plague tabular AML studies.
External replication on a second dataset
Candid interpretation of matched-training gains
Clean-performance trade-off quantified
Weaknesses
No inferential statistics for the main results
Single fixed attack budget
Matched gains entangle robustness with memorisation
Ambiguous categorical handling under gradient attacks
Weak literature benchmarking
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper asks whether adversarial training improves robustness of credit scoring models in P2P lending and, centrally, whether robustness transfers across attack types. Using a curated Lending Club subset (396,030 rows, 28 features; Section 3.2), it benchmarks three model families — logistic regression, a 3x150 feed-forward network, and an FT-Transformer — across a full train-test grid: clean training plus five adversarial augmentations (FGSM, PGD, S&P, DeepFool, mixed) evaluated on clean and four attacked test sets under stratified 5-fold cross-validation. Headline claims (Section 1): matched adversarial training sharply improves robustness (transformer ROC AUC 0.980 on FGSM/PGD tests versus 0.868 clean-trained, Table 11); transfer is strong within the gradient family (FGSM and PGD nearly interchangeable) but weak toward non-gradient S&P corruption; mixed training is the most balanced while preserving clean accuracy.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
Does adversarial training actually make credit scoring models more robust?
On this Lending Club benchmark (about 396,000 loans), adversarial training sharply improved robustness to the specific attack type used during training — for example the FT-Transformer reached ROC AUC 0.980 on FGSM/PGD test sets versus roughly 0.87 for clean-trained models. However, that protection did not extend to Salt-and-Pepper noise, and mixed-attack training was needed for balanced protection, while clean accuracy was preserved.
What data and models were used in this study?
A curated subset of the Lending Club P2P loan dataset (396,030 loans, 28 features, fully paid versus charged off) is the main dataset, with an external replication on the Prosper marketplace dataset in Appendix B. Three model families are compared: logistic regression, a feed-forward neural network, and an FT-Transformer for tabular data, all evaluated with stratified 5-fold cross-validation.
Which adversarial attacks were tested and how realistic are they?
Four attacks plus a mixed regime: FGSM and PGD (white-box gradient attacks), DeepFool (boundary-seeking), and Salt-and-Pepper min/max noise as non-gradient corruption. Perturbations were restricted to features an applicant could plausibly change — income, DTI, revolving balance and utilisation, purpose, and zip code — to mimic manipulation of self-reported inputs, with a fixed budget of ε=0.1.
What are the main limitations of the paper?
The main limitations are a single fixed attack budget (ε=0.1), no significance tests or dispersion measures for the main results, stylised attacks that may not capture real borrower manipulation, and the authors' own caveat that matched-training gains may partly reflect the model recognising the fixed perturbation pattern rather than genuine robustness.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1-3 are image-based; only their captions appear in the extraction, so visual details (e.g., the correlation panel of Figure 2b) were assessed solely through the describing text.
- Table 6 contains parenthetical cells such as (0.74) and (0.75)w whose footnote markers appear to have been lost in extraction, making the S&P column values at ε=0.1 ambiguous as printed.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.