Financial Language Models as Applied Artificial Intelligence Systems for News-Based Trading under Market Frictions
cs.CL · cs.LG · q-fin.TR
Quick verdict
Proposes MFAST, a friction-aware news-to-trading LLM evaluation framework; strong contamination controls, but AUC 0.846 and net Sharpe 2.85 strain credibility.
Summary
The paper introduces MFAST, a Market-Friction-Aware Sentiment-to-Trading framework that evaluates financial language models as deployable trading systems rather than offline classifiers. It links 973,481 tradable Refinitiv news items (filtered from 3.13 million spanning 2010–2026) to 3,452 CRSP firms, fine-tunes six sentiment approaches (LLaMA-3-8B, OPT-1.3B, RoBERTa, BERT, FinBERT, Loughran-McDonald) on return-sign labels, and evaluates classification, next-day return prediction, and long-short portfolios under execution-timing rules, 5bp costs, liquidity screens, and 10% volume participation caps. The primary out-of-sample window (June 2024–January 2026) deliberately postdates foundation-model data-freshness cutoffs. LLaMA-3 achieves 0.787 accuracy, 0.846 AUC, and a net long-short Sharpe of 2.85 versus 0.503 dictionary accuracy; deflated Sharpe, White reality-check, and Hansen SPA tests remain significant. Gains concentrate in linguistically complex articles (negation, contrast, forward guidance). A public replication arm (GDELT, Financial PhraseBank) shows smaller but consistent magnitudes. Main limitations: headline effect sizes far exceed the news-trading literature; several promised baselines (XGBoost, LSTM, Longformer, FinGPT) and the placebo/falsification tests are described but never numerically reported; robustness grids appear only as unquantified prose; and the decoder-versus-encoder comparison confounds architecture with scale and pretraining recency. The framework is the strongest contribution; the headline Sharpe should be treated as an unverified upper bound.
Review complete
Assessment at a glance
5
risk flags identified
● high 2
● medium 2
● low 1
5 strengths · 5 weaknesses · 6 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- MFAST is a domain-specific deployment framework that jointly tests financial LLM outputs under event-time observability, calibration, execution timing, costs, liquidity, capacity, and statistical inference.
- Decoder-only LLMs (LLaMA-3, OPT) outperform encoder transformers and dictionary sentiment on classification, calibration, return prediction, and net portfolio performance.
- LLaMA-3 sentiment yields a net long-short Sharpe of 2.85 at 5bp costs that survives block bootstrap, White reality check, Hansen SPA, and deflated Sharpe corrections.
- Model gains concentrate in linguistically complex articles (negation, contrast, guidance), supporting a costly-information-processing mechanism amplified by market frictions.
- Credible evaluation of financial LLMs requires an end-to-end engineering view spanning operational diagnostics, ablations, and public-data replication.
Strengths
Contamination-aware evaluation design
Sections 4.3–4.4 and Table 4: the primary test window (June 2024–January 2026) postdates both the LLaMA-3 release (April 2024) and the disclosed family data-freshness cutoff (December 2023); January–May 2024 is quarantined as a validation buffer; retrieval/browsing disabled; a split-boundary duplicate audit removes test articles with cosine >0.80 to same-firm training articles within 20 trading days.
Complete threat-to-safeguard mapping
Table 4 and Algorithm 1 make every validity threat auditable: one-sided event-time novelty screen (cosine >0.80, 5-day window), execution-aligned observability rule (pre-open news may trade day 0, all else next day), delisting returns via (1+RET)(1+DLRET)−1, label windows truncated at split boundaries.
Unusually strong multiple-testing battery
Mechanism-level error analysis with human validation
Honest operational accounting and consistent arithmetic
Weaknesses
Implausibly large headline effect sizes
Promised baselines and tests never reported
Robustness asserted without a single number
Confounded decoder-vs-encoder claim
Label/portfolio circularity in evaluation
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
This paper (arXiv:2609.23703, revise-and-resubmit at Engineering Applications of Artificial Intelligence) proposes MFAST, a Market-Friction-Aware Sentiment-to-Trading framework that treats financial language models as deployable AI systems rather than offline classifiers. It links Refinitiv News Analytics to CRSP equities — 973,481 tradable news items on 3,452 firms (2010–January 2026), filtered from 3,129,924 raw items per Table 3 — fine-tunes six sentiment approaches (LLaMA-3-8B, OPT-1.3B, RoBERTa, BERT, FinBERT, Loughran-McDonald) on return-sign labels, and evaluates classification, next-day return prediction, and value-weighted quintile long-short portfolios under execution-timing rules, 5bp one-way costs, and 10% daily-volume participation caps. Central claims: (i) decoder-only LLMs dominate encoder and dictionary baselines; (ii) a net long-short Sharpe of 2.85 survives multiple-testing corrections; (iii) gains concentrate in linguistically complex news (Section 9, Table 13); (iv) credible financial-AI evaluation requires the joint deployment protocol the paper formalizes (Table 2).
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 6 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is MFAST?
MFAST (Market-Friction-Aware Sentiment-to-Trading) is a framework from this 2026 paper that converts timestamped financial news into sentiment scores using language models, then tests whether those scores remain profitable trading signals after execution-timing rules, transaction costs, liquidity screens, and capacity limits. It was evaluated on Refinitiv news linked to CRSP stock returns, with a public-data replication arm using GDELT news and Yahoo Finance/Stooq prices.
How profitable is the LLaMA-3 news-trading strategy in this paper?
The paper reports a net long-short Sharpe ratio of 2.85 (95% CI 2.31–3.34) for a LLaMA-3-sentiment portfolio over June 2024–January 2026, after 5-basis-point costs and a 10% volume participation cap, with roughly 180% cumulative return. These figures are far above published news-trading results, and the paper's own public-data replication achieves a much smaller Sharpe of 0.92, so the headline number should be treated with caution until independently verified.
Does this paper handle LLM training-data contamination?
It applies one of the stricter protocols available: the main out-of-sample window starts June 2024, after LLaMA-3's disclosed data cutoff (December 2023) and public release (April 2024); January–May 2024 is used only as a validation buffer; retrieval and browsing are disabled; and near-duplicate articles across train/test boundaries (cosine similarity above 0.80 within 20 trading days) are removed. The authors acknowledge document-level pretraining exposure can never be fully excluded.
Is the code or data for this paper available?
Refinitiv and CRSP are licensed and cannot be redistributed. The paper describes a replication package containing configuration templates, column schemas, public-data scripts (GDELT, Financial PhraseBank, Yahoo/Stooq prices), model checkpoint identifiers, random seeds, portfolio-simulation and statistical-test code, and deterministic output tables; no public repository URL appears in the extracted text.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Calibration plots are referenced but omitted 'for space', and Figure 1 exists only as a text diagram — visual elements could not be assessed.
- The online appendix (transaction-cost grids, factor-loading tables, capacity diagnostics, subperiod panels) is repeatedly referenced but is not part of the extracted text.
- The metadata abstract is truncated mid-sentence; the embedded full text is complete through the references, so all cited numbers come from the full-text tables.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.