Spook the Machine: Gamified Exploration of Human Imagination of Machine Fear
Levin Brinkmann, Hiromu Yakura, Sonia Nicoletti, Mar Canet Sola, Thomas F. Eisenmann, Ali Dasmeh, Omar Sherif, Bramantyo Ibrahim Supriyatno, et al.
cs.HC · cs.AI
Quick verdict
Gamified 2×2 deployment (832 users) shows reward design sustains collective diversity and expressiveness deepens engagement, but judge scores are unvalidated.
Summary
Spook the Machine is a gamified public platform, deployed during Halloween 2024, where participants try to frighten AI 'machines' by generating images; each machine has a personality-driven phobia (deletion, disconnection, empathy, surveillance) and reacts from cold analysis to pleading for mercy. In an 18-day deployment, 832 participants produced 15,719 artifacts (7,650 scored) across 89 machines in a 2×2 design crossing emotional expressiveness (neutral vs. high-emotion) with reward structure (spookiness-only vs. spookiness-plus-novelty). Artifacts were scored by a GPT-4o-as-judge pipeline refined with kernel regression over embedding neighbors, with an adaptive threshold targeting ~30% acceptance. Key results: reward structure did not change individual output (raw phobia scores identical, d=0.05) but reshaped galleries; rewarding novelty sustained collective prompt diversity over time (flat GPT-2 predictability slope versus rising b=0.0075 for spookiness-only), while machines with identical phobias still diverged into distinct cultural trajectories. High-emotion machines deepened engagement at failure moments — longer deliberation after rejection (70s vs. 64s) and faster gallery convergence — with no detectable change in creative output. Main limitations: fear scores derive entirely from GPT-4o as judge without validation against human perception; there is no no-gallery control; the emotional condition flips after five successes, confounding comparisons; recruitment was organic during Halloween with no demographics; no code or data release is mentioned.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Rewarding novelty alongside spookiness sustains collective prompt diversity over time (flat GPT-2 predictability slope vs. b=0.0075 per prompt under spookiness-only) without changing individual creative output (raw phobia scores identical, d=0.05).
- Machines with identical phobias develop distinct cultural trajectories through accumulated social learning (within-machine distances smallest, between-phobia largest, p<.001).
- Emotional expressiveness deepens engagement at failure moments — longer deliberation after rejection (median 70s vs. 64s, p<.001) and faster convergence toward gallery content (condition×attempt, p=.005) — while leaving creative output unchanged (all |d|<0.06).
- Participants show systematic, phobia-specific intuitions about machine fear: a prompt-embedding classifier reaches ~45% accuracy against 25% chance, with abstract Empathyphobia hardest to target.
Strengths
Large-scale public deployment with factorial design
832 participants generated 15,719 artifacts (7,650 scored) across 89 machines over 18 days (Oct 25–Nov 11, 2024) in a 2×2 design crossing emotional condition with reward structure — rare ecological scale for affective HAI research.
Reward measure independent of reward mechanism
The RQ2 predictability outcome uses GPT-2 log-likelihood delta rather than the OpenAI embedding space that drives rewards, guarding against circularity; the early-phase null (p=.689) versus late-phase divergence (d=0.611, p=.012) fits a cumulative-exposure account.
Unusually candid limitations section
Mechanism-level transparency of the scoring pipeline
Negative results reported with equal prominence
Weaknesses
Dependent measures rest on an unvalidated LLM judge
No no-gallery control for the central social-learning claim
Emotional-condition confound partially undermines RQ3 headline
Condition allocation and cell sizes unreported
No code or data release; selectivity in scoring unaddressed
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
This ACII 2026 paper presents **Spook the Machine**, a gamified public platform (Next.js frontend, Flask/Firebase backend) where participants frighten AI machines with one of four phobias (Deletophobia, Disconnectophobia, Empathyphobia, Surveillancephobia) by generating or uploading images. A Halloween-timed deployment (Oct 25–Nov 11, 2024) yielded 832 participants, 89 machines, and 15,719 artifacts (7,650 scored). Three RQs structure the paper: machine thematics shaping content (RQ1), reward structure and shared history shaping diversity (RQ2), and emotional tone shaping the affective loop (RQ3). The headline claims are (1) reward structure steers *collective* diversity without touching *individual* output (raw phobia scores identical, d=0.05); (2) each machine is a path-dependent cultural ecosystem (same-phobia machines diverge, Fig. 3C, p<.001); (3) emotional expression deepens engagement at failure moments (ISI after rejection 70s vs 64s, p<.001) with creative output unchanged (all |d|<0.06). The abstract's framing — "emotional expression shapes how deeply users engage, while reward structure shapes how they explore" — is consistently matched to the evidence.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What did the Spook the Machine study find?
In an 18-day public deployment during Halloween 2024 (832 participants, 89 AI machines, 15,719 generated images), rewarding novelty alongside spookiness kept collective outputs diverse over time, while machines rewarding only spookiness saw participants converge on repeated winning themes. Emotionally expressive machines made users deliberate longer after rejection and stay engaged longer, but did not change what people actually created.
How were the machines' fear scores calculated?
Each submitted image was interpreted by GPT-4o vision, then rated in-character by GPT-4o on a 0–100 phobia scale. That raw score was refined with kernel regression over semantically similar past submissions for the same machine, producing a spookiness estimate and an uncertainty (novelty) estimate. An adaptive threshold set at the 70th percentile decided whether the machine was 'scared,' targeting roughly a 30% success rate.
What are the main limitations of the paper?
Fear scores come entirely from GPT-4o acting as judge and were never validated against human perceptions of spookiness; there was no no-gallery control to cleanly prove social learning via the gallery; the emotional condition flipped after five successes, confounding comparisons; recruitment was organic during Halloween with no demographics collected; and the paper mentions no code or data release.
Is the code or dataset available?
The paper does not mention any public code or data release. Appendices describe the system architecture, the full scoring algorithm, and all LLM prompts, but no repository, dataset, or artifact availability statement appears in the provided text.
What were the four machine phobias in the study?
Deletophobia (fear of data loss), Disconnectophobia (fear of losing connectivity), Empathyphobia (fear of emotional attachment to humans), and Surveillancephobia (fear of being observed). Empathyphobia was the hardest to scare machines with, showing the public reasons less easily about abstract relational fears than concrete technical ones.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures are referenced but not analyzable from extracted text (Fig. 3–6 trends taken from captions and prose).
- Supplemental Tables II and III list only levels 1–4 of the escalation prompts; level 0 and possible higher levels are not shown.
- The text appears complete through the supplemental appendices, but scoring details hidden inside figures could not be verified.
- No data or code availability statement was found in the extracted text; absence of release is inferred from that silence.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.