Does AI Assistance Leave a Temporal Fingerprint? Detecting Overreliance in AI-Assisted Writing and Programming
cs.HC · cs.AI
Quick verdict
AI insertions leave temporal fingerprints in writing and code; simulated delegation is near-perfectly detectable, but only against the authors' simulation.
Summary
The paper asks whether AI assistance leaves a detectable temporal signature in process data and whether it generalizes from writing to programming. The authors reanalyze three public corpora — CoAuthor (1,447 keystroke-level co-writing sessions with provenance-tagged AI insertions), RealHumanEval (219 editor-telemetry traces from 243 programmers), and a pre-LLM CS1 keystroke corpus (1,331 traces, 5.1M keystrokes) as a human-only anchor — plus 550 researcher-simulated wholesale-delegation sessions. AI contributions arrive in bursts far above each author's own baseline (paired dz=1.13 writing, 3.54 programming), and only 0.2% of pre-LLM traces contain any insertion above 100 characters. Post-acceptance engagement diverges: 92.8% of AI-inserted prose characters survive to final documents versus 13.7% of accepted code suggestions surviving intact. Random forests on eight (writing) and seven (programming) observable temporal features classify simulated delegation with F1≥0.997 while misclassifying at most 0.5% of real work as delegation; minimal-AI versus collaborative work remains weakly separable (F1 0.59–0.94). Main limitations, largely self-acknowledged: the delegation class is synthetic and defined by concentrated production, so high F1 partly reflects the authors' own generator; no authentic-coursework data; session-level cross-validation risks within-writer leakage; cross-medium comparisons confound medium with population, tool, and logging granularity; analysis code is promised but not yet deposited.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- AI contributions arrive in bursts far outside each author's own production baseline in both writing (paired dz=1.13) and programming (dz=3.54), and insertions above 100 characters are nearly absent (0.2% of traces) from pre-LLM human-only keystroke data.
- Post-acceptance engagement diverges by medium: 92.8% of AI-inserted prose characters survive to final documents, whereas only 13.7% of accepted code suggestions survive intact.
- Random forests using only observable, label-free temporal features separate simulated wholesale delegation from real work nearly perfectly (F1≥0.997) while misclassifying at most 0.5% of real work as delegation.
- Ordinary AI collaboration remains hard to distinguish from unassisted work, so temporal process evidence flags wholesale delegation rather than assistance.
- These signals must function as conversation-openers over shared timelines, never as automated accusation instruments, pending validation in authentic coursework.
Strengths
First cross-domain test of a real gap
Prior keystroke-based AI-detection evidence covered only English essays (Crossley 2024; Deane 2026; Kundu 2024; Roh 2025); this paper adds the first temporal-signature analysis of AI-inserted code from process telemetry (RealHumanEval snapshot diffing) and the first within-author comparison across mediums (dz=1.13 vs 3.54).
Provenance-exact reanalysis of CoAuthor
Replaying sessions through a character-level provenance buffer yields exact per-origin insertion, deletion, and survival accounting, resolving what the authors identify as a gap left open between Yang et al. (2026) and Zeng et al. (2024); this grounds the 92.8% survival figure precisely rather than by estimation.
Pre-LLM human-only baseline anchors the anomaly claim
Unusually careful inferential hygiene
Responsible interpretive boundary
Weaknesses
Headline delegation result is partly circular
Cross-medium claims rest on confounded comparisons
Real-condition separation is weak and structurally fragile
Limited ecological validity and no outcome link
Deployment math left unquantified
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
Can keystroke or editing logs detect when a student pastes AI-generated work?
According to this study, wholesale pasting produces a categorical temporal signature — very large single insertions followed by a long, nearly flat polishing phase — that simple classifiers detect with F1 above 0.997 in simulation, while under 0.5% of real (unassisted or collaboratively assisted) work was ever misflagged. However, the delegation examples were simulated by the researchers rather than observed in real courses, so field accuracy is unproven, and a student who retypes AI text at a human pace would evade timing-based detection entirely.
Do people engage differently with AI writing suggestions versus AI code completions?
In this paper's data, yes: 92.8% of AI-inserted essay text survived unchanged to the final document, while only 13.7% of accepted code suggestions survived intact — 62.2% of accepted code was discarded. The authors suggest code faces executable feedback that forces verification, whereas prose does not, but they caution the comparison confounds medium with task type, population, suggestion length, and logging granularity.
Was this study conducted with real students in real courses?
No. It reanalyzed three public datasets: crowdworker writers collaborating with GPT-3 (CoAuthor), programmers in short lab-style tasks (RealHumanEval), and a pre-LLM 2019 CS1 keystroke corpus used as a human-only baseline. The wholesale-delegation condition was simulated (pasting AI output with light edits), and the authors explicitly call for validation in authentic coursework before any deployment.
Can students evade temporal AI-use detection?
Partially, yes. The authors acknowledge that deliberately retyping AI-generated text at a human-like pace would evade timing-only features entirely, and cite work on timing-forgery attacks. Their chunked-paste evasion variant (splitting the paste into smaller blocks) was still caught 82.5% of the time, but the paper concludes these signals should never serve as automated accusations — only as starting points for instructor–student conversations.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1–4 could not be analyzed from the extracted text; only their captions were available, so visual claims (e.g., timeline shapes in Fig. 4, cluster separation in Fig. 2c/d) are taken on the prose alone.
- Table 1 is labeled an excerpt, so some observable features reported in the text may be missing from the extracted table.
- Analysis code is promised but not yet deposited, so reproducibility was assessed from the described parameters rather than verified artifacts.
- The paper is an arXiv preprint formatted for AHFE 2026 proceedings; the peer-review status of the venue version is unknown.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.