FACT: A Forensic Agent with Compiled Tool-Use Trajectories for AI-Generated Image Detection
Jiaoyang Chen, Bin Hu, Jingyu Hu, Kun Zhou, Qin Zhang, Zhengzhe Liu
cs.CV · cs.AI
Quick verdict
FACT turns AIGC image detection into learned tool-use (Evolve–Distill–Refine); strong gains on 6 benchmarks, but key sets are self-built and stats thin.
Summary
The paper reframes AI-generated image detection as an image-conditioned tool-use problem and proposes FACT, an agent that decides which forensic tools to call, interprets returned evidence, and stops when sufficient. Its toolbox pairs five frozen model-based experts (SDAIE, Community Forensics, AIDE, SIDA, Veritas) with eight lightweight signal probes (spectral, wavelet, noise, JPEG statistics). Training follows Evolve–Distill–Refine: GPT-5.1 evolves a textual forensic skill through a propose–execute–verify loop with a held-out replay check; the verified skill is compiled into action–observation trajectories and behaviorally cloned into a Qwen3-VL-8B agent; cost-aware GRPO then optimizes accuracy against tool cost. On two author-built benchmarks (STD3K 91.40%; NewGen-900 82.44%, whose nine generator families were held out from all training stages) and four public ones (Chameleon 87.53%, LOKI 88.35% balanced accuracy, AIGCDetectionBenchmark 98.77%, HydraFake 94.44%), the authors report the best results among compared methods, above the strongest constituent expert and majority voting. Ablations attribute gains to combining probes with experts (82.9%/82.5% to 90.7%) and show GRPO trades 0.13 points on STD3K for lower acquisition cost (1.5500 to 1.3835). Limitations include author-defined internal benchmarks, some baselines quoted from other papers' protocols, no significance testing or variance reporting, reliance on proprietary GPT-5.1, and code/data promised only upon publication.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- AI-generated image detection is better solved by an image-conditioned tool-use policy with learned stopping than by any single fixed detector, as motivated by Table 1's detector specialization.
- An execution-verified textual skill evolved by GPT-5.1 can be compiled into action–observation trajectories and distilled into a compact Qwen3-VL-8B agent that surpasses its skill-guided teacher (81.89% vs 75.10% on NewGen-900).
- Cost-aware GRPO preserves accuracy while reducing normalized acquisition cost from 1.5500 to 1.3835 on NewGen-900.
- FACT achieves the best results among compared methods on all six benchmarks, including 82.44% on NewGen-900 whose nine generator families were held out from all training stages.
- Signal probes and model-based experts are complementary evidence sources: 82.90%/82.50% alone versus 90.70% combined on STD3K+NewGen-900 (Table 7b).
Strengths
Well-motivated problem reframing
Table 1 empirically shows each frozen expert specializes in different generators (SDAIE 1.00 on Z-Image Turbo but 0.36 on GPT-Image-1; AIDE 0.98 vs 0.52 on the same pair), directly motivating image-conditioned evidence acquisition over one-step routing (UniShield) or any fixed detector.
Execution-verified skill evolution
Stage I accepts a skill revision only after rerunning the full tool interaction on a separately sampled verification batch (the Sr+1 replay check), a concrete safeguard against the unfalsified self-improvement that plagues reflexion-style agents.
Honest, informative ablations
Distilled student exceeds its teacher
Held-out generator protocol
Weaknesses
Self-built headline benchmarks
No statistical rigor
Heterogeneous baseline provenance
Below-chance baseline anomaly
Proprietary teacher and deferred release
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper claims three things: (1) AI-generated image detection is better framed as image-conditioned evidence acquisition than as a fixed detector; (2) an Evolve–Distill–Refine pipeline converts image-level labels into a compact tool-using agent (GPT-5.1 teacher distilled into Qwen3-VL-8B); (3) the resulting agent beats its five constituent experts, majority voting, and external published baselines on six benchmarks (Tables 2–6), including nine generator families held out from all training stages (NewGen-900, 82.44%). Scope covers full-image generation plus deepfake and manipulation settings (HydraFake); audio and video are out of scope.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is FACT and how does it detect AI-generated images?
FACT is an agent that detects AI-generated images by adaptively calling a toolbox of forensic tools — five frozen detector models (SDAIE, Community Forensics, AIDE, SIDA, Veritas) plus eight lightweight signal probes — deciding per image which evidence to acquire, interpreting each result, and stopping when evidence suffices. It was trained by distilling GPT-5.1-generated tool-use trajectories into a compact Qwen3-VL-8B model, then refined with cost-aware GRPO that balances accuracy against tool cost.
How accurate is FACT?
The paper reports 91.40% on its STD3K benchmark, 82.44% on NewGen-900 (nine generator families unseen in all training stages), 87.53% on Chameleon, 88.35% balanced accuracy on LOKI, 98.77% on AIGCDetectionBenchmark, and 94.44% on HydraFake — stated as the best among the methods compared in each respective table.
Is the code, model, or dataset for FACT available?
Not yet. The paper states that code, the trained model, and benchmark data will be released 'upon publication, subject to source licenses'. At review time nothing was downloadable, and retraining also depends on access to the proprietary GPT-5.1 teacher used for skill evolution and trajectory synthesis.
How does FACT differ from UniShield or other agentic detectors?
UniShield routes each image once to a predicted forgery domain and a single expert detector. FACT performs a multi-step investigation: it can call several tools sequentially, update its judgment after every returned observation, and decide when to stop, with the stopping policy explicitly optimized for accuracy under a tool-cost budget via cost-aware GRPO.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Supplementary material (evolved skill text, dataset composition and sampling rules, training configurations, baseline provenance details) is referenced throughout but absent from the provided text; judgments that would depend on it are correspondingly limited.
- PDF extraction loses table formatting (bold markers, cell shading in Table 1); all numeric values cited in this review were legible in the extraction.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.