Unsound Search with Policy and Value Networks in Legends of Code and Magic
cs.AI
Quick verdict
Claims imitation+PIMC search beats LoCM champion ByteRL 51.35% over 10k pre-registered games; credible protocol, thin margin, agent exceeds memory cap.
Summary
The paper argues that theoretically unsound perfect-information Monte Carlo (PIMC) search is practical in the collectible card game Legends of Code and Magic (LoCM), contradicting prior claims that enumeration-based search is unusable at ~2^101 belief states. Measuring Long et al.'s three structural properties (disambiguation, leaf correlation, bias) on 32 decks, the author finds LoCM in the PIMC-favorable region. Using a Rust reimplementation of the rules (roughly 443x faster than gym-locm) and feed-forward policy/value networks trained by behavior cloning on self-play of ByteRL (champion) and NeteaseOPD (runner-up), the agent runs beam search over turns with worlds sampled from an opponent-deck prior. Under the official referee and 200 ms clock, the strictest configuration (runner-up prior mixed with uniform, no ByteRL data) wins 51.35% (95% CI [50.37, 52.33]) over 10,000 pre-registered games versus 26.8% without search; with the opponent decklist known it reaches 60.4% over 500 games. A replicated best-response attack converging to 90.1% against ByteRL only reaches 61.9-69.3% against search agents. Main limitations: the agent exceeds the competition's 256 MB memory cap and could not be an official entry, results cover only the battle phase with identical decks, exploitability was measured once per defense at reduced search budgets, and no code release is stated.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
4 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- LoCM's measured game structure — strong disambiguation, near-0.5 bias, high leaf correlation (Fig. 1b-d) — places it where PIMC's theoretical defects are empirically cheap, justifying unsound determinized search in CCGs.
- An imitation-trained policy/value agent with decision-time search beats champion ByteRL 51.35% (95% CI [50.37, 52.33]) over 10,000 pre-registered certified games, and 60.4% over 500 games with the opponent decklist known.
- Search, not policy quality, drives the result: the identical network without search certifies at 26.8%, so decision-time compute adds +24.6 points.
- Search buys adversarial robustness: a replicated best-response attack converging at 90.1% against ByteRL only reaches 61.9-69.3% against search configurations (single runs, reduced budgets).
Strengths
Unusually rigorous certification protocol
Section IV.C/V.A: pre-registered game count (10,000) and seed blocks chosen before running, disjoint seeds across tests, official referee with 200 ms timer from an isolated process, Wilson 95% CIs throughout — rare discipline in game-AI comparisons.
Clean isolation of search's contribution
The same network playing policy argmax certifies at 26.8% vs 51.35% with search (+24.6 points, Section V.A), so the margin is attributable to decision-time compute rather than imitation quality.
Strictest-configuration headline minimizes leakage
Claims grounded in measured game structure
Weaknesses
Headline 'beats the champion' is under modified competition conditions
Exploitability conclusions rest on single runs at reduced budgets
Deck-knowledge ablation is conceptually murky
Protocol-fairness claims asserted, not demonstrated
No code or data release stated
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
This single-author paper (arXiv:2609.06816, cs.AI) targets the battle phase of Legends of Code and Magic (LoCM) 1.5, where belief states are ~2^101 and the COG 2022 champion ByteRL plays without any search. Four central claims: (1) LoCM's measured game structure — strong disambiguation, near-0.5 bias, high leaf correlation (Fig. 1b-d) — places it where PIMC's known defects (strategy fusion, non-locality) rarely change decisions; (2) an imitation-trained feed-forward policy/value agent with decision-time determinized search beats ByteRL 51.35% (95% CI [50.37, 52.33]) over 10,000 pre-registered games under the official referee and 200 ms clock, rising to 60.4% over 500 games with the decklist known; (3) search, not policy quality, drives the margin (+24.6 points over the same network's 26.8% argmax play); (4) search buys robustness — a replicated best-response attack converging at 90.1% against ByteRL only reaches 61.9-69.3% against search configurations.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
Does the paper's agent actually beat ByteRL?
Per the paper, yes but narrowly: 51.35% (95% CI [50.37, 52.33]) over 10,000 pre-registered games under the official referee and 200 ms clock, without using ByteRL's data or knowing its decklist; 60.4% over 500 games with the decklist known. However, the agent exceeds the competition's 256 MB memory cap (~0.9 GB peak), so it could not have been an official competition entry.
Why does unsound (PIMC) search work in an imperfect-information card game?
PIMC determinizes hidden information and searches each world with perfect-information methods; it is theoretically flawed (strategy fusion, non-locality) but often plays well. The author measures Long et al.'s three structural properties on LoCM and finds strong disambiguation, near-neutral bias, and high leaf correlation — the same regime as bridge and skat, where PIMC's defects rarely change the correct move.
Is the search-based agent harder to exploit than ByteRL?
In the paper's single replicated best-response attack (the Haluška & Schmid protocol), ByteRL converges to a 90.1% attacker win rate, while the search agents hold the attacker to 61.9-69.3%. Caveats: each defense was attacked once, at reduced search budgets (W=16/64), with no variance reported, and the fully certified configuration was not tested.
Is code or data available for this paper?
No. The paper describes a custom LLM-assisted Rust engine, behavior-cloned networks, and self-play corpora, but states no code, model, or data release. The LoCM referee and gym-locm environment are public, so reproduction would require rebuilding the engine and retraining from the published ByteRL/NeteaseOPD policies.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Numerical values inside figure panels (Fig. 1-3 plot data) are available only via captions and axis labels in the extracted text; all cited numbers were taken from body text and Table I, which are complete in the extraction.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.