Target alignment, dilution and forecast selection when cross-sectional forecasts share a common target
econ.EM · cs.LG · q-fin.ST · stat.ME
Quick verdict
Paper shows standardized common-target forecasts split into alignment plus orthogonal parts; careful geometry, but empirics: selection removes dilution yet nothing beats no-info.
Summary
The paper develops an exact geometric framework for standardized cross-sectional forecasts that share one realized target: every forecast decomposes into a target-aligned component plus a target-orthogonal one. Consequences: common-target error correlation largely mirrors forecast correlation (a poor diversity measure); equal weighting beats a no-information forecast only when mean alignment is large relative to composite dispersion; and equal-weight admission rewards dilution, which a scale-free split exposes. A cautious three-way admission rule with HAC simultaneous bounds is validated in seven simulation designs and applied to 24 LLM forecasts of 60 US equity rankings and nine mechanical signals on 42 cross-asset funds. Both panels show no detectable alignment, selection removes most dilution loss, yet no combination beats the no-information forecast; placebo and planted-signal experiments calibrate what these nulls mean. Limitations: retrospective LLM forecasts (possible look-ahead), end-of-sample fixed universe (survivorship), and procedures tuned on the evaluation data.
Review complete
Assessment at a glance
5
risk flags identified
● high 1
● medium 3
● low 1
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Every standardized common-target forecast splits exactly into a target-aligned component plus a target-orthogonal component, with C = γγ' + K holding as an identity with no distributional assumptions (Proposition 1).
- With a common standardized target, common-target deviation correlation is largely a translation of forecast correlation and is therefore a poor diversity measure (Proposition 5; empirical R² = 0.9999 across 276 LLM pairs).
- An equal-weight combination beats the no-information forecast if and only if mean alignment exceeds half the composite's squared norm (Corollary 1; g_w > qw/2).
- Equal-weight admission rewards dilution rather than predictive content — its gains come from the scale-mismatch component (Proposition 9; LLM cross-panel value-reversal batch Δ̂ = −0.130 with scale-free part −0.0001).
- In both empirical panels, selection and weighting remove most dilution loss yet no combination beats the no-information forecast, and no detectable alignment exists.
Strengths
Exact, assumption-light core theory
Propositions 1–9 are algebraic identities holding for any sample and any weights (Table 2), with proofs in Appendix A; the numerical notebook verifies them on synthetic and empirical panels (175 of 177 checks pass, Appendix B). The alignment/orthogonal split (C = γγ' + K) cleanly separates it from estimated factor models.
A decision-relevant closed-form null benchmark
Corollary 1's condition g_w > qw/2 gives an exact phase boundary; Figure 1 (Design D) shows all simulated cells falling on the line R(1/N) − 1 = −2m, e.g. alignment 0.03 with zero correlation loses 0.026 at N=12 but gains 0.017 at N=24, matching the (1−ρ̄)/N term.
Self-critical inference architecture
Power-calibrated nulls, not just nulls
Honest scoping
Weaknesses
The empirical payoff is negative by the paper's own estimand
Contamination is close to untestable in the LLM panel
The null outcome is largely predetermined by design
No inference option comes out clean
Data snooping acknowledged, not solved
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper studies forecast combination when many forecasters score the same units per date against one standardized realized target (equity rankings, cross-asset fund rankings). Its foundation is an exact split: each standardized forecast s_i = γ_i·y + u_i, where γ_i = ⟨s_i, y⟩ is *target alignment* (a correlation) and u_i is target-orthogonal, giving C = γγ' + K (Proposition 1; Table 2 labels Propositions 1–9 exact identities needing no distributional assumptions). Three headline claims follow: (i) common-target deviation correlation is largely a translation of forecast correlation and is a poor diversity measure (Proposition 5; empirically deviation correlation sits within 0.005 of the (1+ρ)/2 zero-alignment benchmark in the LLM panel, R² = 0.9999 across 276 pairs, Figure 3); (ii) an equal-weight combination beats the no-information forecast only when mean alignment exceeds half the composite's squared norm (Corollary 1); (iii) equal-weight admission rewards *dilution* — scale-mismatch shrinkage of the composite — rather than genuine improvement, which a scale-free split (Proposition 9) exposes. The author is notably disciplined: Table 2 and Section 11 ('What is not claimed') explicitly disclaim error-rate control along the adaptive path, that LLMs cannot forecast returns, and that admission failure implies absence of value.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is 'target alignment' in this forecast-combination paper?
For forecasts that rank the same units (e.g., stocks) each date against one standardized realized outcome, target alignment is simply the cross-sectional correlation between a forecaster's standardized scores and the standardized realized target. The paper decomposes every standardized forecast into this aligned part plus a part orthogonal to the target, and shows many forecast-combination questions become simple statements about these two pieces — e.g., an equally weighted average beats the zero (no-information) forecast only if average alignment exceeds half the composite's squared norm.
Do the LLM forecasts in the paper beat a do-nothing benchmark or make money?
No. Across 24 language-model forecasters on 60 US stocks (weekly, 2021–2026) and nine mechanical signals on 42 ETFs (monthly, 2005–2026), no selection or weighting scheme beats the 'no-information' zero forecast in out-of-sample relative-score risk, and top-minus-bottom quintile return spreads are statistically insignificant. The paper frames the LLM experiment as a stress test of its framework, not as evidence about real-time LLM forecasting ability.
Why is error correlation a poor measure of forecaster diversity here?
Because all forecasters on a date are scored against the same standardized target, the correlation between two forecasters' errors is largely a fixed translation of their forecast correlation (at zero alignment, ρ_error = (1+ρ_forecast)/2). In the LLM panel the relationship is nearly perfectly linear (R² ≈ 0.9999), so ranking candidates by error correlation mostly duplicates — and distorts — what forecast correlation already says, while alignment is the part that carries incremental information.
Are the paper's results and code reproducible?
The author states a complete replication package — pipeline code, prompts, cached model responses with hashes, seeds, and per-origin logs — will be deposited in a public repository upon acceptance (it is not yet available at review time). Market prices are excluded for licensing reasons but can be rebuilt from the cited public source. The theory is verifiable by independent reimplementation, so reproducibility prospects are good if the package is delivered.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures are available only as extracted axis labels/captions; visual features (e.g., Figure 1's phase diagram, Figure 3 scatter) were judged from reported numbers and captions.
- The extraction preserves all sections and appendices, so truncation is not a concern; some table alignment could admit minor parsing ambiguity.
- No network access was used; model names (e.g., 'gpt-5-nano', 'gemini-3.5-flash-lite') and stated cutoffs are taken from the paper as given.
- The replication package is promised only upon acceptance, so all reproducibility judgments rest on the paper's description.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.