Making Alternative Data Work: Context-Augmented LLMs for Financial Forecasting
Jihoon Kwon, Lawrence Liu, Daekyung Park, Sumin Kim, Haverty Jack, Hoyoung Lee, Katherine Bjorkman, Josh McKenney, et al.
cs.AI
Quick verdict
Proposes a two-agent LLM framework integrating alternative data for revenue forecasts; gains look real but rest on small samples and licensed data.
Summary
The paper asks whether large language models can integrate commercial alternative data (card spending, web traffic, foot traffic, and Kalshi prediction-market KPIs) into firm-level revenue forecasting without task-specific training. The authors propose a two-agent framework: a screening agent labels each firm–channel pair INCLUDED/EXCLUDED based on whether the channel plausibly captures the firm's main revenue driver, and a prediction agent (GPT-5.5, medium reasoning effort) forecasts revenue from financial history (H), the alternative-data series (X), and prior earnings-call transcripts (Z), with search tools for background. On 304 test firm-quarters reported December 2025–June 2026 (after the model's knowledge cutoff, mitigating contamination), full H+X+Z context yields the lowest FVU in three of four channels — e.g., 0.021 for web traffic versus 0.122 with history alone — though for KPI markets H+X alone (0.070) slightly beats the full set (0.087). The method outperforms OLS, GBT, and historical-average baselines (Table 3), beats analyst consensus on a majority of surprise events (81% win rate for web traffic; 53–64% elsewhere), and GLM-5.2 replicates the ordering (Appendix A.1). Main limitations: very small per-channel samples (22–175), no significance tests on the headline comparisons, exclusion of failed model outputs from runs, absence of KPI-markets rows in the baseline table, and fully proprietary data restricting reproduction.
Review complete
Assessment at a glance
6
risk flags identified
● high 1
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Adding alternative data in-context alongside financial history and earnings-call transcripts improves LLM revenue forecasting, yielding the lowest FVU in three of four commercial channels (Table 2).
- The two-agent screen-then-predict LLM framework outperforms historical-average, OLS, and GBT baselines trained on the same signals (Table 3).
- The framework's forecasts beat analyst consensus on a majority of revenue-surprise events in every channel, with win rates of 53–81% and lower RMSE throughout (Figure 4).
- LLM screening increases firm-specific channel–revenue correlation from r=0.238 to r=0.281, robust to firm resampling and surrogate placement (Table 4).
- Tool access to firm/channel background materially improves prediction, e.g., web-traffic FVU from 0.079 to 0.021 (Table 5).
Strengths
Contamination-aware evaluation design
All 304 test firm-quarters were reported December 2025–June 2026, strictly after the GPT-5.5 knowledge cutoff, with in-context examples from Feb 2024–Nov 2025 (§5.1); the GLM-5.2 rerun shifts the boundary to that model's Feb 16, 2026 cutoff and shrinks samples accordingly (175→131, 32→27, 75→58, A.1).
Dual-model replication of the core result
The ordering — H+X+Z lowest FVU in three of four channels — holds for both closed GPT-5.5 (Table 2) and open-weight GLM-5.2 (Table 6: card 0.201, web 0.043, foot 0.403), suggesting the effect is not idiosyncratic to one frontier model.
Full ablation suite
Practical benchmark with honest framing in the body
Prompts fully disclosed
Weaknesses
Per-channel samples too small for the precision implied
Incomplete baseline coverage despite universal caption claim
Abstract overclaims uniformity
Screened-subset comparison against a full-coverage benchmark
Exclusion of failed outputs and fully licensed data
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper asks whether a general-purpose LLM can integrate commercial alternative data into firm-level revenue forecasting purely through in-context learning, without task-specific training. Section 3 formalizes the problem: given baseline financial history H (always present), an optional tabular alternative-data source X and textual source Z (earnings-call transcripts), the model predicts revenue targets — year-over-year growth and revenue surprise versus analyst consensus, measured at both an early (~1 week after quarter end) and latest (pre-announcement) snapshot. The claimed contributions are: (1) the idea of in-context learning with alternative data; (2) a two-agent framework in which a screening agent labels firm–channel pairs INCLUDED/EXCLUDED and a prediction agent (GPT-5.5, medium reasoning) forecasts revenue with search-tool access; and (3) evidence across four commercial channels — Carbon Arc card spending, web traffic, and foot traffic, plus Kalshi KPI markets — that alt data improve forecasts, beat feature-based baselines, and beat analyst consensus on a majority of surprise events. A GLM-5.2 replication appears in Appendix A.1.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 6 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What does this paper claim LLMs can do with alternative data?
It claims that GPT-5.5 can forecast a company's quarterly revenue more accurately by reading commercial alternative data — card spending, web traffic, foot traffic, and Kalshi prediction-market KPIs — directly in its prompt, together with financial history and prior earnings-call transcripts. No model is trained; a screening step first keeps only firms a given data channel plausibly covers.
Does the LLM actually beat financial analysts?
On the paper's own test of 304 firm-quarters reported between December 2025 and June 2026, its revenue forecasts were closer to the realized surprise than analyst consensus in 53–81% of events depending on the data channel (81% for web traffic), with lower RMSE in every channel. Caveats: the comparison runs only on the firm subset the LLM's own screening kept, and no statistical significance tests are reported.
Can I reproduce the results with public data?
Not fully. The prompts are published in Appendix A.2 and the paper says code is available on GitHub, but every evaluation input is licensed: Carbon Arc (card spending, web traffic, foot traffic), Kalshi (KPI markets), and FactSet (revenues and analyst consensus). Third parties cannot re-derive the reported tables without these subscriptions.
Which language models were used, and do results depend on one model?
Headline results use GPT-5.5 with medium reasoning effort. Appendix A.1 repeats the experiment with the open-weight GLM-5.2 on a reduced post-cutoff sample and finds the same ordering — combining all sources still works best — suggesting the effect is not specific to one frontier model.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figure 4's axis labels were extracted in interleaved order; the channel-to-number mapping (win rates 54/81/59/56% vs early consensus and 53/78/64/54% vs latest, RMSE pairs per channel) was reconstructed from label sequence and cross-checked against the narrative text, but individual assignments could be swapped between adjacent channels.
- No GitHub URL appears in the extracted text; the statement 'code used in the experiments is available in the Github repository' (§5.1) could not be verified.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.