Contrastive Learning for Authorship Verification
cs.CL · cs.LG
Quick verdict
Controlled PAN21 study: contrastive beats classification across 5 encoders; SOTA 98.4% but single-run, fanfiction-only, no long-context baseline.
Summary
This paper asks whether contrastive or classification-based fine-tuning serves transformer-based authorship verification better. On the PAN21 fanfiction benchmark (open-set: unseen authors and fandoms), five encoders from TinyBERT to ModernBERT-large are fine-tuned under both objectives, with batch size and learning rate tuned per approach and remaining conditions fixed, hyperparameters transferred from TinyBERT via a μTransfer-style width adjustment. Contrastive learning (InfoNCE/SupCon, temperature 0.01, mean pooling, cosine threshold calibrated on validation F1) wins across all five models at 512 tokens — e.g., ModernBERT-base reaches 0.900 versus 0.653 accuracy — and still wins at 256 tokens with less compute (Table 3). Ablations identify InfoNCE as best of six losses (Table 2), last-layer mean pooling over projection heads or first-last pooling (Table 4), and random text-span rotation as essential augmentation, without which performance falls to 0.713 (6 epochs) or 0.682 (40 epochs, overfitting). Scaling context to 4096 tokens with ModernBERT-large yields 0.984 accuracy/F1, above prior systems including the BigBird cross-encoder (0.960 overall). Limitations: single-run results without significance testing or seed variance; classification is never tested beyond 512 tokens, so the state-of-the-art result cannot be cleanly attributed to the training objective; and evaluation is confined to fanfiction, with other datasets left to future work. Code and data splits are released on GitHub.
Review complete
Assessment at a glance
4
risk flags identified
● high 0
● medium 3
● low 1
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Under controlled conditions (per-approach tuned batch size and learning rate, other conditions fixed), contrastive learning outperforms classification-based fine-tuning across all five tested transformer architectures at 512 tokens (Table 3).
- A ModernBERT-large bi-encoder with 4096-token context, batch 4096, and 40 epochs achieves 0.984 accuracy and F1 on PAN21 test, exceeding prior systems including the BigBird cross-encoder at 0.960 overall (Tables 5–6).
- InfoNCE is the best of six contrastive losses, while proxy-based losses (SoftTriple, Proxy Anchor) fail entirely with 238,815 author classes (Table 2).
- Random text-span rotation is essential augmentation: validation accuracy drops from 0.861 to 0.713 (6 epochs) without it and overfits to 0.682 at 40 epochs (Table 4).
- Contrastive training wins even at 256-token input with less computation than 512-token classification (Table 3, Half-Length column).
Strengths
Genuinely controlled objective comparison
Batch size and learning rate are grid-searched separately per approach while all other conditions are fixed (§2.4), directly addressing the confound left open by Li et al. [23] and the mixed prior evidence (Tyo [37] contrastive vs Peng [30] classification vs Nguyen [28] BigBird).
Systematic ablation coverage with concrete numbers
Six loss functions ranked (Table 2: InfoNCE 0.753 vs proxy-based collapse at 0.500 with 238,815 classes), pooling/projection-head variants (Table 4: 0.861 best), and context-length scaling 0.926→0.984 from 512→4096 tokens (Table 5).
State of the art on the official PAN21 metric suite
Compute-efficiency finding
Reproducibility artifacts released
Weaknesses
No statistical rigor anywhere
Missing long-context classification baseline undermines the headline attribution
Headline config drifts from the comparison protocol
Single-dataset scope
μTransfer only partially applied
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper (arXiv 2609.28471, single author, Georgia Tech) targets a precise question left open by Li et al. [23]: when the pre-trained model and input context are held fixed, does contrastive or classification-based fine-tuning better serve authorship verification? Evaluation uses PAN21 [18], the fanfiction open-set benchmark (unseen authors and fandoms). Three claims structure the work: (1) contrastive learning wins the controlled comparison across five encoders (Table 3); (2) six practical factors — loss function, batch size, training duration, model, context length, random-span augmentation — drive performance (Tables 2–5); (3) the resulting ModernBERT bi-encoder reaches 0.984 accuracy/F1, a new PAN21 state of the art (Table 6). The abstract's "98.4%" is supported by the reported tables, but §3.5 appropriately hedges attribution ("plausible that some of this can be attributed to the use of contrastive learning") — the paper knows its comparison explains only part of the headline number.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 4 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What accuracy does this paper achieve on PAN21 authorship verification?
98.4% accuracy and 0.984 F1 on the PAN21 test set, using a ModernBERT-large bi-encoder with 4096-token context, batch size 4096, and 40 training epochs. This exceeds prior PAN21 systems including the BigBird cross-encoder (0.960 overall) and Boenninghoff et al.'s O2D2 (0.955 overall), on the official metrics (AUC 0.995, c@1 0.982).
Is contrastive learning better than classification for authorship verification?
In this paper's controlled comparison — same data, epochs, and architectures, with batch size and learning rate tuned per approach — contrastive fine-tuning beat classification fine-tuning across all five tested models at 512 tokens (e.g., ModernBERT-base 0.900 vs 0.653 accuracy), and still won at 256 tokens with less compute. Caveat: classification was never tested at longer contexts, so the advantage at the 4096-token scale is unproven.
Is the code and dataset for this paper available?
Yes. The author released code and the dataset in Parquet format, including the exact validation split used, at github.com/petekirby/contrastive-av. PAN21 itself is a public CLEF/PAN benchmark for cross-domain authorship verification.
What data augmentation does the paper use and does it matter?
Random text rotation: each document is rotated to start at a random word boundary before tokenization truncation, creating thousands of distinct views per sample. The ablation (Table 4) shows TinyBERT validation accuracy falls from 0.861 to 0.713 at 6 epochs without it, and to 0.682 at 40 epochs due to overfitting — the augmentation is essential.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Review based on arXiv v1 full text (23 Sep 2026); Figure 1's architecture diagram was assessed from its caption only.
- Preprint with no peer-review status indicated; Table 6 prior-work numbers are taken as reported, not independently verified.
- PAN21 test-set size and per-pair statistics are not stated in the extracted text.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.