Detecting AI Impostors: How Do Middle Schoolers Identify LLM Agents in a Live Collaborative Setting?
Dan Schumacher, Pragathi Durga Rajarajan, Haven Kotara, Roman Rendon, Kosi Atupulazi, Deepti Tagare, Ismaila Temitayo Sanusi, Fred G. Martin, et al.
cs.CL
Quick verdict
Novel live study: teen AI-detection rises 79.3%→86.7% (p=.10) despite sharper mimicry; well-triangulated but marginal stats and confounds limit certainty.
Summary
The paper introduces DoppelBot, a cooperative social deduction game in which three to five middle schoolers chat anonymously alongside an equal number of LLM-powered 'doppelgängers' that impersonate specific students through a three-agent gpt-4o-mini pipeline (decide-to-respond, response generation, style matching). Across a school-event pilot (29 players) and a two-day summer-camp study, the authors triangulate semi-structured interviews (417 coded utterances, κ=.707), pre/post privacy surveys (n=12–13, descriptive only), behavioral coding of 2,182 chat messages, four stylistic alignment metrics (BERT, LIWC, METEOR, ROUGE-L), and a mixed-effects logistic regression over 215 votes. Detection accuracy rose from 79.3% on Day 1 to 86.7% on Day 2 (paired t-test p=.10, marginal), even as student–bot stylistic alignment increased; two expert annotators scored roughly 92% on both days, suggesting the task did not become easier. Behavioral codes shifted toward coordinated voting and 'meta' strategies exploiting AI limitations, such as embodiment probes. Privacy-survey means rose across all three subscales, but without inferential tests. Key limitations: PIW personalization and repeated exposure were manipulated simultaneously, the sample is small and single-site, the Hawthorne effect inflates suspicion, only gpt-4o-mini was tested, and the headline gain is only marginally significant; the dataset is IRB-gated rather than public.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Middle schoolers' AI-detection accuracy improves with repeated exposure, from 79.3% on Day 1 to 86.7% on Day 2 (paired t-test p=.10), even as their bots become more stylistically aligned with them across BERT, LIWC, METEOR, and ROUGE-L.
- The accuracy gain is driven by a strategic shift from linguistic cues toward coordinated voting and 'meta' strategies exploiting AI limitations such as lack of embodiment (crvt +11.8pp, meta +12.4pp, rand −28.8pp).
- DoppelBot elicits reflection on AI impersonation and data privacy, evidenced by 417 coded interview utterances (privacy concerns 7% of utterances) and descriptive rises in all three privacy-concern subscales.
- Flat expert-annotator accuracy (~92% both days) indicates the Day-2 improvement reflects student learning rather than an easier detection task.
- LLM judges (gpt-4o-mini, Llama-3.1-8B, Qwen3.5-27B) peak at F1 .684 and often fail to beat an always-human baseline of .651, trailing the students' 86.7% vote accuracy.
Strengths
Genuinely unstudied population and setting
First live, collaborative AI-impersonation detection study with adolescents (§1, §2): prior human-detection work uses static text snippets with adults; DoppelBot embeds detection in adversarial multi-player chat where students must both unmask bots and defend their own humanity.
Clever expert-annotation difficulty control
Table 3's post-hoc control (two researchers, κ=.86) scores 91.9%/92.3% on Days 1/2 against students' 79.3%/86.7%, arguing the Day-2 gain reflects student learning rather than an easier detection task — a robustness check most user studies omit.
Multi-method triangulation with strong inter-rater reliability
Transparent, reproducible system + unusually honest limitations
Ethical care with minors
Weaknesses
Headline claim rests on marginal significance with no effect size or CI
Day-2 confound: personalization and repetition manipulated simultaneously
Descriptive-only survey evidence at tiny n
Expert control is not difficulty-matched
Single impostor model bounds all strategy conclusions
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Review: Detecting AI Impostors (DoppelBot)
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
How well can middle schoolers detect AI chatbots impersonating them?
In the DoppelBot studies, students' vote accuracy rose from 79.3% on day one to 86.7% on day two (a marginally significant gain, p=.10), while two expert annotators with full transcripts scored about 92% on both days, suggesting the task itself did not get easier.
What strategies did students use to spot the AI impostors?
Early on they relied on linguistic cues such as repetition, overly correct grammar, or unnatural slang; later they shifted to coordinating votes as a group and probing AI limitations — for example asking about the wall's color or how many people were in the chat, which the bot answered wrongly because it lacked embodied awareness.
Is the DoppelBot dataset publicly available?
No. The source code and all agent prompts are on GitHub, but the game transcripts and voting data involve minors and are provided only to researchers who supply a valid IRB approval (or a letter stating IRB review is not required).
Which language model powered the AI impostors in DoppelBot?
gpt-4o-mini, in a three-stage pipeline: a Decide-to-Respond agent chooses whether to speak (temperature 0.5), a Response Generator writes a generic reply (0.5), and a Stylizer rewrites it to match the target student's past messages (0.9).
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- The provided text is a sampled excerpt with three '[...sampling gap...]' regions (parts of §3.1 game phases, Figure 2's numeric values, portions of the references and appendix prose were skipped); all judgments above rest only on visible text.
- Statistics for the paired bootstrap tests behind the Day 1→Day 2 alignment increase are not present in the provided excerpt (likely within a sampling gap), so their adequacy was not evaluated.
- The split of the 215 votes between the AI EXPO pilot and the summer camp is not visible in the excerpt, so pilot-versus-camp contributions to the headline accuracy could not be verified.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.