FINESSE: An Agent-Based Simulator and Benchmark Dataset for Multimodal Financial Event Sequences
Tyler Farnan, Benjamin Eng, Adam Abate, Xirui Hou, Rizal Fathony, Nam H. Nguyen, Senthil Kumar
cs.LG
Quick verdict
Agent-based simulator for synthetic multimodal finance event data with 4-task benchmark; useful open infrastructure, but realism validation and rigor are thin.
Summary
Researchers at Capital One introduce FINESSE, an agent-based simulation framework that generates synthetic, structured datasets of interdependent financial event streams — transactions, loan payments, account state updates, and policy interventions — coupled through customers' latent, evolving states (merchant affinity and payment strategy policies, evolving fraud risk). From this simulator they derive FINESSE-Bench, a public benchmark supporting four tasks: balance forecasting (60-day input and output), transaction fraud detection (edge classification on a continuous-time dynamic graph), missed payment prediction (counts over 3/6/9/12-month horizons), and next event prediction (marked temporal point processes). Baselines span four methodology families: temporal graph networks (TGN reaches 0.7607 ROC-AUC and 0.0433 PR-AUC on fraud; DyRep 0.7561/0.0435), sequence models (MLP, GRU, Transformer, MAMBA, CoLES), time-series models (DLinear, PatchTST, Transformer, MambaSimple), and four temporal point processes (RMTPP, NHP, THP, IFTPP). Compared datasets (Berka, Data Fusion 2022, MBD, FinBench) lack payments, credit state, or policy-change streams (Table 1). Main limitations: no calibration of simulated dynamics against real-world data; results for two tasks appear only as figures without numeric tables; single runs without significance tests; fraud labels arise by construction from simulator rules; dataset scale parameters are not stated in the text. Code and dataset are released on GitHub and Hugging Face.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
4 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- FINESSE-ABM generates synthetic multimodal financial event sequences (transactions, payments, account state, policy changes) coupled through latent evolving agent states, independently of sensitive data.
- FINESSE-Bench uniquely combines payment, credit-state, and policy-change streams that existing open benchmarks lack (Table 1).
- Baselines from four methodology families establish initial reference results, e.g., TGN reaches 0.7607 ROC-AUC / 0.0433 PR-AUC on fraud and IFTPP attains the best next-event type error rates (0.4922 payment, 0.5785 transaction).
- The framework is extensible and publicly released (GitHub simulator, Hugging Face dataset) to accelerate research on structured multimodal event-sequence modeling.
Strengths
Unique stream coverage among open financial benchmarks
Table 1: only FINESSE provides payment events, credit state, and policy changes alongside transactions; compared sets (Berka [9], Data Fusion 2022 [13], MBD [16], FinBench [35]) lack at least two of these.
Embedded RCT-style policy interventions
Section 3.1: customers are assigned control/intervention cohorts with credit-limit, interest-rate, and minimum-payment changes — a data type essentially absent from open financial data, enabling policy-response modeling research.
Methodologically diverse baselines on standard tooling
Full public release and honest scoping
Weaknesses
No realism calibration
Missed-payment task framing is inconsistent
No statistical rigor
Fraud labels are simulator artifacts
Key quantities missing from the text
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
FINESSE (arXiv 2609.11993, Capital One) makes two coupled contributions: FINESSE-ABM, an agent-based simulator generating synthetic multimodal financial event streams — transactions, loan payments, account state updates, and policy interventions — coupled through latent, evolving customer states (§3); and FINESSE-Bench, a released dataset instantiating four tasks: balance forecasting, transaction fraud detection, missed payment prediction, and next event prediction (§4–5). Claims are appropriately modest — a framework, a benchmark, initial baselines — and the authors explicitly disclaim causal-inference support and real-world task fidelity (§1). The strongest substantive claim is the positioning in Table 1: among compared sources (Berka [9], Data Fusion 2022 [13], MBD [16], FinBench/FinPT [35]), only FINESSE includes payment events, credit state, and policy changes as first-class streams. That gap is real, and filling it is genuinely useful.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is FINESSE?
FINESSE is an open-source agent-based simulator from Capital One that generates synthetic multimodal financial event data (transactions, loan payments, account state updates, policy changes), plus FINESSE-Bench, a benchmark built from that simulator with four machine-learning tasks: balance forecasting, transaction fraud detection, missed payment prediction, and next event prediction.
Is the FINESSE dataset real customer data?
No. The dataset is fully synthetic, produced by rule-driven software agents rather than sampled from real records, specifically to avoid privacy problems. Policy interventions (credit limit, interest rate, minimum payment) are assigned through simulated randomized controlled trials inside the simulator.
What results do the FINESSE baselines achieve?
On fraud detection, temporal graph networks reach about 0.76 ROC-AUC but only about 0.043 PR-AUC (TGN 0.7607/0.0433, DyRep 0.7561/0.0435). For next-event prediction, IFTPP has the lowest type error rates (0.4922 for payments, 0.5785 for transaction categories). Balance-forecasting and missed-payment results are reported mainly as figures, with PatchTST and Transformer/GRU respectively among the better performers.
Where can I get the FINESSE code and data?
The paper points to GitHub at CapitalOne-Research/FINESSE for the simulator framework and to Hugging Face at capitalone/FINESSE for the benchmark dataset. Baselines build on standard public implementations such as Temporal Graph Benchmarks (TGB) and the EasyTPP package.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1-4 appear only as captions in the extraction; the plotted values behind the missed-payment (Fig 3) and balance-forecasting (Fig 4) results cannot be inspected, so those two tasks are assessed from the surrounding text's qualitative claims only.
- Table 2's repeated subscripts (the Type 2 row reuses μ1, σ1, λ1) appear as such in the extracted text; whether this is a paper typo or an extraction artifact is indeterminable from the text alone.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.