Form Over Content In Gradient-Based Data Attribution Methods
Sunwoo Kim, Seokwon Jung, Sohyung Kim, Seong Joon Oh, Alice Oh, Alice Oh
cs.CL · cs.AI
Quick verdict
5×7 factorial: LLM gradient attribution tracks answer format over task across training and scales; convincing, though format classes ad hoc, code unreleased.
Summary
The paper asks what gradient-similarity data attribution actually measures for supervised fine-tuning examples, where the literature disagrees between a semantic reading (task skills, per LESS) and a surface-form reading (Wang et al.). The authors decouple task and answer format by rendering the questions of five benchmarks (TriviaQA, SQuAD, GSM8K, ARC-Challenge, CRUXEval) in seven answer formats across three structure classes (binary judgment, four-option selection, JSON), yielding 35 datasets whose task and format vary independently. They compute 64 per-example answer-span gradients per dataset (transformer body only, excluding embeddings; seeded CountSketch to 16,384 dimensions), average cross-dataset cosines, and disattenuate by intra-dataset coherence. Same-format-class pairs align strongly (disattenuated cosine roughly 0.4–0.54) while the same benchmark rendered in a different class aligns near zero (~0.02, against a −0.001±0.008 null over 280 unrelated pairs). The ordering persists across OLMo pretraining checkpoints (1B–4T tokens), post-training stages, scales 1B–13B, and four model families. Analyzing LESS's released selections from a 270,679-example pool, each target over-represents its own answer format (MMLU 3.7×, BBH 2.4×, TydiQA 1.9×). Limitations: five English QA-style benchmarks only, an author-chosen format taxonomy with JSON the sole structured-generation format, possible answer-length confounds, an observational LESS analysis, and no code release yet.
Review complete
Assessment at a glance
4
risk flags identified
● high 0
● medium 2
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Gradient alignment between supervised fine-tuning examples follows answer format: same-format-class pairs reach disattenuated cosines of roughly 0.4–0.54, while the same benchmark rendered in a different format class aligns near zero (~0.02).
- This format dominance persists from the earliest OLMo pretraining checkpoints (1B tokens) through SFT, DPO and instruction-tuned stages, and across 1B–13B models from four families.
- LESS's released data selections over-represent each target's own answer format (3.7× for MMLU multiple choice, 2.4× for BBH chain-of-thought, 1.9× for TydiQA short answers), stable across the three released seeds.
- Consequently, influence measured by gradient-based attribution does not imply task relatedness, and such methods should be validated on data where task and answer format vary independently.
Strengths
Factorial design cleanly separates task from format
Rendering identical benchmark questions in 7 answer formats across 5 benchmarks (Section 3.2, Appendix C) makes task and format orthogonal by construction — something the observational prior debate (Xia et al. [4] vs Wang et al. [6]) could not achieve.
Measurement hygiene above average
Answer-span-masked gradients on the transformer body only, excluding embedding/unembedding tables to kill vocabulary-overlap shortcuts; seeded CountSketch to 16,384 dims preserving inner products; empirical null calibrated on 280 unrelated pairs (mean −0.001, SD 0.008, max 0.03) justifying the ±0.02 null band (Section 3.2).
Unusually broad robustness sweep
External validity via a deployed method
Candid limitations section
Weaknesses
Format taxonomy is ad hoc and the third class is an n-of-1
Answer-length/token-count confound not ablated
Narrow task axis under a broad claim
No code release
LESS analysis is observational
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper targets a genuine, well-documented disagreement in the data attribution literature: whether TracIn-style gradient similarity between supervised fine-tuning examples encodes task semantics (Xia et al.'s LESS reading, ref [4]) or surface form (Wang et al., ref [6]). The claim is sharply scoped — gradient alignment between SFT examples tracks answer format, not task content — and is supported at three levels: a controlled 5×7 factorial measurement (Section 3.2), robustness across training stages, scales and families (Sections 4.1–4.2), and an audit of a deployed method's released selections (Section 4.3). The abstract's framing that the authors resolve this debate is strong; the evidence supports it within the tested envelope of five English QA-style benchmarks, seven formats and three structure classes.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 4 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
Do gradient-based data selection methods like LESS actually find task-relevant training data?
According to this paper, largely no: when task and answer format are varied independently, gradient similarity tracks answer format, not task content. LESS's released selections over-represent each target's own answer format by 1.9–3.7×. However, format-matched data can still improve benchmark scores in practice — the paper challenges the semantic interpretation of why such methods work, not the measured gains themselves.
How did the authors separate answer format from task content?
They took the same questions from five benchmarks (TriviaQA, SQuAD, GSM8K, ARC-Challenge, CRUXEval) and rendered each into seven answer formats (True/False, Yes/No, A–D, 1–4, W–Z, a–d, JSON), creating 35 datasets. This makes dataset pairs that share a task but not a format, or a format but not a task, directly comparable — isolating each factor's effect on gradient similarity.
Does the result hold for small models or models early in training?
Yes, as measured: the format-over-task ordering appears at OLMo and Pythia checkpoints from as early as 1B pretraining tokens, persists through SFT, DPO and instruction-tuned stages, and holds across 1B, 7B and 13B models from four families (OLMo-2, Pythia, Llama 3.1, Qwen3).
Is the paper's code and data available?
The models (OLMo, Pythia, Llama, Qwen) and benchmarks used are all public, but at the time of writing the paper states its measurement code is not yet released, and exact rendering templates for the 35 rendered datasets are only sketched in the appendix.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figure 2a heatmap and Figure 4/5 enrichment values are embedded as flattened numeric text; the cell-to-format mapping was inferred from surrounding text and could be locally misaligned.
- Some appendix text (A, D, E) interleaves with figure content in the extraction, so minor ordering details may be off.
- ArXiv metadata beyond the embedded text (e.g., post-publication versions) was not verifiable offline.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.