Data-Centric Post-Training for Financial Reasoning: Mining, Distillation, and Verifiable Learning
Zhirayr Hayrapetyan, Andrei Kalmykov, Denis Kokosinskii, Dmitry Stanishevskii, Dmitry Zmitrovich
cs.CL
Quick verdict
Claims self-distilled SFT turns finance fine-tuning from a 3-4 pt loss into a 1-3 pt gain; credible pipeline, but single-run results on authors' own benchmark.
Summary
The authors present a data-centric pipeline for post-training LLMs on financial reasoning. Three corpora are built: 44,652 long reasoning traces mined from Glaive Reasoning v1 20M; 4,475 examples distilled from Finance-Instruct-500K using DeepSeek-R1-0528 traces; and 329,828 knowledge-graph-guided QA pairs synthesized with GraphGen from CFA textbooks and verified by a GPT-5.5 judge. Filtering combines semantic deduplication with three Qwen3-Embedding-0.6B classifiers for finance relevance (F1 0.97), missing context (F1 0.94), and answer verifiability (F1 0.95); the scored pools (645,232 and 785,149 examples) and classifier checkpoints are released. Experiments compare ordinary SFT, self-distilled SFT (0.8 distillation / 0.2 cross-entropy from a frozen start checkpoint), equal-weight model merging, and GRPO. On a 778-question FINESSE-Bench subset scored by GPT-5.2, ordinary SFT loses 3.2-4.0 points while self-distilled SFT gains 1.0-2.8; merging recovers 3.0 points over its SFT parent, and GRPO adds 0.4 on hard tasks and 3.0 on verifiable tasks (Qwen3-1.7B/4B only). Main limitations: single runs without variance estimates, evaluation on the authors' own benchmark via an LLM judge, an unexplained 72.0-vs-73.14 discrepancy for the same SFT checkpoint, no reported decontamination against the CFA evaluation questions, and no end-to-end combination of the strongest components.
Review complete
Assessment at a glance
5
risk flags identified
● high 2
● medium 3
● low 0
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Ordinary SFT on the curated finance mixture reduces FINESSE-Bench accuracy by 3.2 (Qwen3.5-4B) to 4.0 (Gemma-4-12B-IT) percentage points relative to the starting checkpoints (Table 3).
- Self-distilled SFT with a frozen starting-model teacher instead improves the same starting models by 1.0–2.8 points, demonstrating retention-aware domain adaptation.
- Equal-weight linear merging between an SFT checkpoint and its start recovers 3.0 points over the SFT parent and finishes 0.9 points above the original Qwen3.5-4B (76.09 vs 75.2, §5.4).
- GRPO adds 0.4 points on model-relative hard examples after self-distilled SFT (Gemma-4-12B-IT, §5.4) and 3.0 points when applied directly to classifier-selected verifiable tasks on Qwen3-1.7B/4B (Table 3).
- Three compact Qwen3-Embedding-0.6B binary classifiers (F1 0.94–0.97, Table 2) suffice to operationalize separate finance-relevance, self-containment, and verifiability decisions at pipeline scale.
Strengths
Fully auditable data funnels
Every filtering stage reports exact survivor counts — e.g., Glaive: ~700,000 → 645,232 → 220,707 (p(finance)>0.99) → 111,655 → 93,570 → 44,652 (§2.3); GraphGen: 785,149 → 706,699 → 440,183 → 335,158 → 331,458 → 32,000 verified (§2.5) — rare transparency for synthetic-data work.
Reusable released assets
Two pre-threshold scored pools (Fin-Glaive 645,232; Fin-CFA-GraphGen 785,149) plus three Hugging Face classifier checkpoints with F1/precision/recall model cards (Table 2) let others re-tune thresholds for their own applications (§2.1, §3).
Stage-matched filtering rationale
Retention-first honest reporting
Complete training hyperparameters
Weaknesses
No uncertainty quantification
Internal numeric inconsistency
Self-referential evaluation
Unaddressed contamination risk
Fragmented evidence chain
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
This technical report (arXiv 2609.10113, cs.CL) describes a production-oriented, data-centric pipeline for post-training LLMs on financial reasoning, together with adaptation experiments built on that data. Central claims: (1) three complementary corpora — 44,652 mined Glaive reasoning traces, 4,475 distilled Finance-Instruct examples, and 329,828 knowledge-graph-guided synthetic QA pairs (Table 1); (2) three Qwen3-Embedding-0.6B binary classifiers that operationalize stage-specific filtering (Table 2); (3) retention-aware adaptation (self-distilled SFT, WiSE-FT-style merging, targeted GRPO) improves FINESSE-Bench scores without the 3.2–4.0-point regressions caused by ordinary SFT (Abstract, Table 3).
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What does the paper 'Data-Centric Post-Training for Financial Reasoning' show?
It shows that naively fine-tuning (ordinary SFT) on finance data lowered accuracy on FINESSE-Bench by 3.2–4.0 points, while a self-distilled variant that keeps a frozen copy of the starting model as a teacher instead gained 1.0–2.8 points; equal-weight weight merging and targeted GRPO added further gains on a 778-question evaluation subset.
Which datasets and models were released with this paper?
Two scored candidate pools (Fin-Glaive, 645,232 examples; Fin-CFA-GraphGen, 785,149 examples) and three Hugging Face classifier checkpoints (fin-relevance, lack-of-context-detector, verifiability-classifier). The distilled Finance-Instruct subset is not redistributed, and no training code repository is mentioned.
How was the evaluation performed in this paper?
Models were scored by micro-accuracy on a fixed 778-question subset of FINESSE-Bench, covering CFA Level I/II, CMT Level II, VLigaBench-ru, technical-analysis trading, derivatives trading, and FinQA questions, with GPT-5.2 acting as a binary correctness judge.
Is the benchmark used in this paper independent of the authors?
No. The FINESSE-Bench authors (reference [15]) overlap fully with this paper's author list, so the evaluation is self-benchmarking. The paper also reports no decontamination between its CFA-book-derived training data and the benchmark's CFA questions.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figure 1's pipeline diagram survives in the extraction only as flattened text labels; the text itself ends with a complete reference list ([18], page 12), so no truncation is suspected.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.