An Agentic Just-in-Time Adaptive Intervention System for Personalized Sleep Support: Proof-of-Concept Study with N of 1 Data
cs.HC · cs.AI
Quick verdict
Agentic JITAI (Home Assistant + LLM agent) ran 8 times over 7 days with one author: feasibility only—no efficacy, no baseline, no artifacts.
Summary
This proof-of-concept paper presents a just-in-time adaptive intervention (JITAI) for sleep in which an LLM-based agent (Claude Code), scheduled daily inside Home Assistant, reviews the previous 30 days of personal behavioral data (sleep, step counts, smartphone and television use, bedtime routines), evaluates existing smartphone reminders, and decides whether to add, modify, keep, or remove each one, logging every decision for human review. Agent behavior is governed by a human-editable Markdown skill file and constrained to at most three interventions per day; reminders are delivered as push notifications via Home Assistant automations. In an evaluation running eight times across seven consecutive days (August 14–20, 2026) on the first author's own data, the system started automatically, completed every run, and made a single change: adding an afternoon walking reminder on August 16 after detecting a repeated association between lower step counts and shorter sleep. All other runs concluded there was insufficient evidence to intervene. The paper is explicit that the study demonstrates technical feasibility only—there is no statistical analysis, no baseline or control, no evidence that reminders improved sleep, and neither the skill file, code, nor decision logs are published, limiting reproducibility. Tests comparing agent-selected with fixed or rule-based reminders, and larger samples, remain necessary.
Review complete
Assessment at a glance
4
risk flags identified
● high 1
● medium 2
● low 1
4 strengths · 4 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- An LLM agent scheduled inside Home Assistant can autonomously execute the full JITAI cycle — review data, evaluate reminders, revise interventions, log decisions — across eight unattended runs.
- The agent directly revised reminder delivery logic in Home Assistant within human-defined instructions and a three-intervention-per-day cap, distinguishing it from threshold-based JITAIs that only apply predefined rules [3].
- Saved decision logs made each run auditable, separating completed no-change runs from failures.
- The study establishes technical feasibility only and does not show that agent-selected reminders improved sleep.
Strengths
Honest scoping of claims
The paper repeatedly states that the study evaluated whether components could work together, 'rather than whether the reminders improved sleep' (§1, §2.7, §3, §4); efficacy is explicitly deferred to future work.
Transparency-by-design architecture
§2.6 logs the data window reviewed, patterns found, decision, and reasoning for every run; §3 reports the logs even distinguish completed no-change runs from failed runs — directly addressing the classic opacity problem of LLM-driven decisions.
Concrete safety constraints
Completes the full JITAI loop, not just analysis
Weaknesses
Evaluation is minimal by design
No baseline or ablation
Nothing is released
No analysis of LLM reliability
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
This is a proof-of-concept systems paper (§1, §2.7) describing a JITAI in which Home Assistant plays three roles — data store (Recorder), automation/execution engine, and notification channel — while an LLM agent (Claude Code), scheduled daily (§2.1), is governed by a human-readable Markdown skill file (§2.2). At each run the agent reviews 30 days of behavioral data (§2.3), evaluates existing reminders against a user-entered sleep goal, and chooses one of four actions (add/change/keep/remove) under a hard cap of three interventions per day (§2.4), writing changes to automations.yaml (§2.5) and saving a decision log (§2.6). The claim set is deliberately narrow: the paper states repeatedly that it evaluated "whether these components could work together automatically, rather than whether the reminders improved sleep" (§1; reiterated §3, §4). This honesty is commendable and keeps the paper internally consistent — but it also defines the ceiling of what the results can support.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 4 weaknesses, analyzed in detail
- 4 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What does this paper actually build?
It builds a sleep support prototype where an AI agent (Claude Code) runs once per day inside Home Assistant, reviews that person's last 30 days of sleep and behavior data, and autonomously creates, changes, or removes smartphone bedtime reminders, saving a log of every decision for human review.
Did the AI agent improve the user's sleep?
No. The authors state explicitly that the study tested whether the system could run automatically, not whether reminders improved sleep; no sleep outcomes are analyzed, and the single change made in the whole week was adding an afternoon walking reminder.
Is the code or the agent's instruction file available?
No. The paper describes a reusable Markdown 'skill file' that governs the agent, but neither that file, the code, the Home Assistant configuration, nor the decision logs are published, so the system cannot currently be reproduced from the paper.
Why use an AI agent instead of simple automation rules?
The authors argue that rule-based JITAIs need manual re-tuning as routines change, whereas an agent could reinterpret fresh data and revise reminders itself. However, the paper includes no comparison against a rule-based system, so this claimed advantage remains untested.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Full text is available and reads complete, but Figures 1 and 2 are diagrams only described via captions and could not be inspected; Table 1 content is fully captured in the text.
- The actual skill file, prompts, and run logs are not in the text, so agent-internal behavior could only be assessed from the authors' descriptions.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.