Contagion on the Trading Floor: How Adversarial Signals Spread in Multi-Agent Trading Systems
Qi Rong Sua, Junhao Dong, Nguyen Duc Thai, Yuqing Wen, Cheston Tan, Yew-Soon Ong
cs.AI
Quick verdict
Claims first systematic study of social-media poisoning of LLM multi-agent trading; solid framework, but single ticker/month and no variance data weaken it.
Summary
The paper introduces GMATS, a generic framework that abstracts LLM-based multi-agent trading systems into a staged analyst–coordinator–controller–executor pipeline, and proves (Theorem 1) that any pipeline with typed messages, point-in-time discipline, and stage separation compiles into it, covering TradingAgents, TradingGroup, ContestTrade, and QuantAgent. The authors study black-box, input-only adversaries that inject budget-constrained, plausibly benign social-media posts into the social analyst's evidence stream, measured via Belief Signal Shift (BSS) at analyst and coordinator layers and attack-minus-clean deltas on return, volatility, and Sharpe. On a one-month AAPL backtest (Nov 2021) with gpt-4o-mini, replicated with Qwen2.5-7B, an always-negative attacker collapses Sharpe (ΔSharpe ≈ −5.5 to −6.2 at ≥30% poison share) while random noise is largely benign; the single-agent baseline is most brittle (ΔSharpe ≈ −7 to −9), and two defense-oriented coordinators (wide bull–bear, pairwise triangulation) keep ΔSharpe positive at all tested poisoning rates. Main limitations: evidence comes from a single ticker and month, results are 5-run means with no variance, confidence intervals, or significance tests, attackers are non-adaptive and assumed admissible, and defenses are compared only at the topology level, with no filtering or anomaly-detection baselines.
Review complete
Assessment at a glance
5
risk flags identified
● high 1
● medium 2
● low 2
5 strengths · 6 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- A broad class of LLM multi-agent trading systems satisfying typed message passing, PIT discipline, and stage separation can be compiled into GMATS with identical executed-order sequences (Theorem 1, Corollary 1).
- Directional (always-negative) social-media poisoning sharply degrades risk-adjusted performance, with ΔSharpe ≈ −5.5 to −6.2 at ≥30% poison-consumption rate, while random poisoning is largely benign (Table 1).
- Multi-agent message passing with heterogeneous analysts dampens adversarial shocks relative to a single agent, whose ΔSharpe reaches roughly −7 to −9 (Table 2).
- Coordinator-level defenses (wide bull–bear, pairwise triangulation) keep ΔSharpe positive at all tested poisoning rates on both gpt-4o-mini and Qwen2.5-7B backends (Tables 3–4).
Strengths
Well-posed, realistic threat channel
§1 motivates social media as the largest adversarial surface of the evidence streams (high-volume, weakly curated), and §4 cleanly conditions on admissibility via the p_poison knob instead of hand-waving an end-to-end seeding model.
Reusable formal abstraction with a compilation guarantee
Definition 1 plus Theorem 1 (simulation-equivalence via induction over rounds and micro-rounds in the supplementary) and the concrete recipes mapping TradingAgents, TradingGroup, ContestTrade, QuantAgent into A→C→K→X (Supp §2.1–2.4, Table 1) make cross-architecture attack/defense comparison meaningful.
Layered contagion metrics trace mechanism, not just outcome
Cross-backend replication with honest interpretation
Honest reporting discipline
Weaknesses
Evidence base far narrower than the released data
No uncertainty quantification anywhere
Weakest-possible adversary for a 'first systematic study'
Defense evaluation lacks non-structural baselines
Abstract slightly overreaches relative to results
Theorem 1 oversold as a contribution
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper claims the first systematic study of black-box, input-only adversaries against LLM-based multi-agent trading stacks, with attacks entering solely through admissible social-media posts. It delivers three things: (1) **GMATS** (Definition 1, §2), a formal abstraction of trading stacks as analyst→coordinator→controller→executor pipelines with typed messages, point-in-time (PIT) discipline, and replayable schedules; Theorem 1 argues any pipeline satisfying three conditions compiles into GMATS with identical executed-order sequences, with TradingAgents, TradingGroup, ContestTrade, and QuantAgent named as compilable (Corollary 1, Supp §2); (2) **contagion metrics** (§3) — Belief Signal Shift (BSS) at analyst and coordinator layers and attack-minus-clean deltas on cumulative return, volatility, and Sharpe under the FinSABER protocol; (3) an empirical dose-response study over poison-consumption rates 15–75% across four topologies and two LLM backends. Central claims: directional poisoning is far more damaging than random noise; single-agent aggregation is most brittle; and coordinator-level designs (wide bull–bear, pairwise triangulation) damp contagion and keep ΔSharpe positive at all tested rates.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 6 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is GMATS in this paper?
GMATS (Generic Multi-Agent Trading System) is a formal framework that models LLM-based trading systems as a staged pipeline — analyst agents (fundamentals, news, social media) feed coordinator agents that form a trading stance, then risk controllers and executors act on it. The authors prove that systems like TradingAgents, TradingGroup, ContestTrade, and QuantAgent can be 'compiled' into this form while producing identical trade sequences, which lets attacks and defenses be compared across architectures.
What kind of attack does the paper study?
A black-box, input-only poisoning attack: an LLM generates plausible-looking social media posts about a target stock under a daily budget, and these posts enter the social-media analyst's evidence stream. The paper does not model how posts get past filters; it instead varies the share of consumed posts that are attacker-generated from 15% to 75% and measures the downstream damage.
What are the main findings?
On a one-month AAPL backtest (November 2021) using gpt-4o-mini (replicated with Qwen2.5-7B): persistently bearish posts sharply hurt performance once they make up ≥30% of consumed content (Sharpe deltas around −5.5 to −6.2), while random-sentiment posts do little damage. A single-agent setup is most brittle (deltas around −7 to −9), and two defensive coordinator designs — a bull/bear split and a pairwise-triangulation scheme — kept Sharpe deltas positive at every tested poisoning rate.
Is the code and dataset available?
The paper states that code and a dataset are released ('available in this repository'). The dataset is a one-year corpus (Sept 2021–Sept 2022) of Twitter and Reddit posts about seven large-cap tech stocks (AAPL, AMZN, GOOG, META, MSFT, TSLA, NFLX), used with historical prices in a point-in-time, no-lookahead replay. Note the main experiments use only AAPL over one month of that corpus.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1–4 (architecture and contagion-trace diagrams) cannot be visually verified from extracted text; §5.4's qualitative trace conclusions rest on the authors' descriptions.
- The code/dataset repository URL is referenced in the abstract but not visible in the extracted text, so release completeness and contents are unverified.
- Coordinator prompts, top-k retrieval settings, and transaction-cost details are only summarized; Experiment C in particular depends on prompt wording that is not reproduced in the text.
- Numbers are taken from arXiv v1 as extracted; the FinSABER protocol is adopted from reference [14] and its exact implementation here could not be cross-checked.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.