DocuTeam: Mixed-Initiative Multi-Agent Discussions around Evolving Documents
Heechan Lee, Juhyeon Choi, Tae Soo Kim, Juho Kim, Joseph Seering
cs.HC · cs.AI
Quick verdict
Claims agent-initiated multi-agent discussion improves novelty, relevance, specificity without added workload; credible, but p-values hover near .05.
Summary
DocuTeam is a Chrome extension over Notion that lets users and multiple LLM agents jointly initiate and steer discussions anchored to specific document regions. Agents monitor edits (2-second debounce), score discussion necessity across three task-oriented modes (Idea, Discussion, Evaluation) with a 0.75 trigger threshold, and autonomously open, redirect, and close agent-to-agent threads, asking clarifying questions when needed; users drag agents onto blocks, inject messages, add or remove members, and drag snippets back into the document. A design workshop (N=15) yielded four design implications. In a within-subjects study (N=20) against a user-initiated side-panel baseline on two event-planning tasks, condition-blind evaluators rated DocuTeam plans higher on novelty (3.20 vs 2.68, p=.047), relevance (3.30 vs 2.78, p=.048), and specificity (3.25 vs 2.28, p=.001), with no workability difference. Participants sent 46% fewer steering messages, changed about 39% more draft words, and reported equal NASA-TLX workload; confidence and satisfaction were unchanged. The authors attribute the gains to an iterative document-mediated refinement loop. Limitations: event-planning tasks only, two-hour sessions, no independent evaluation of the dialogue engine, no multiple-comparison correction across roughly fifteen tests with several p-values just below .05, no effect sizes or confidence intervals, and no stated code release.
Review complete
Assessment at a glance
5
risk flags identified
● high 1
● medium 3
● low 1
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- A mixed-initiative multi-agent system in which agents proactively open document-grounded discussions yields plans rated more novel, relevant, and specific than a user-initiated side-panel baseline (N=20, within-subjects).
- These outcome gains occur without any increase in overall cognitive load (NASA-TLX n.s.), because anchoring and summaries replace chat-based orchestration.
- Users shift from chat-centered idea sourcing to document-centered iterative refinement: 46% fewer steering messages, ~39% more draft word changes, and more messages applied to the document.
- Outcome quality gains do not translate into higher self-reported confidence or satisfaction, because counterarguments and broadened issue spaces increase perceived uncertainty.
- The evolving document can serve as a bidirectional coordination medium between users and an agent team, reducing the orchestration burden of human–multi-agent collaboration.
Strengths
Clear gap articulation
Table 1 (§2.2) taxonomizes prior systems along agent-initiation, autonomous discussion development, and grounding in evolving work, cleanly isolating the gap between hub-and-spoke proactive agents and user-orchestrated multi-agent discussion.
Traceable design lineage
The formative workshop (N=15, §3.2) findings — peripheral awareness, multi-level steering, selective adoption — map one-to-one onto DI1–DI4 and then onto concrete features: modes, anchored bubbles with summaries, steering controls, drag-in adoption (§4.1).
Convergent mixed-methods evidence
Honest reporting of null and inconvenient results
Unusually complete engineering disclosure
Weaknesses
No multiple-comparison correction
No effect sizes, CIs, or verifiable evaluation instrument
Feedback-volume confound
Bundled intervention without ablation
Narrow domain and demand-characteristic risk
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
DocuTeam (KAIST/SNU; arXiv 2609.29309) is a Chrome extension over Notion in which multiple LLM agents monitor document edits and autonomously initiate, run, and redirect discussions anchored to document blocks; users launch discussions by dragging agents onto blocks, steer threads directly (messages, add/remove agents) or indirectly, respond to agent clarification prompts, and drag snippets back into the document with automatic style adaptation (§4.1). Four claims: (1) agent-initiated mixed-initiative discussion yields more novel, relevant, specific outcomes than user-initiated side-panel discussion (N=20 within-subjects); (2) without added cognitive load; (3) interaction shifts from chat-centered ideation to document-centered iterative refinement (46% fewer steering messages, ~39% more draft word changes); (4) outcome gains do not raise confidence or satisfaction.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is DocuTeam?
DocuTeam is a research prototype built as a Chrome extension over Notion in which multiple LLM agents monitor your document edits and proactively start, run, and redirect group discussions anchored to specific document regions, while the user can also launch discussions by dragging agents onto blocks, join and steer conversations, and drag agent ideas directly back into the document.
What did the DocuTeam user study find?
In a within-subjects study with 20 participants on two event-planning tasks, blind evaluators rated plans made with DocuTeam as more novel (3.20 vs 2.68), relevant (3.30 vs 2.78), and specific (3.25 vs 2.28) than plans from a user-initiated side-panel baseline; participants sent 46% fewer steering messages, edited drafts about 39% more, and reported equal NASA-TLX workload — though their confidence and satisfaction did not improve.
What LLM models does DocuTeam use?
The prototype uses OpenAI's gpt-5-mini to generate agent utterances and gpt-4o-mini for all other pipeline components, including intent extraction, moderation decisions, memory updates, summaries, and the discussion-trigger scoring that fires when the average mode score exceeds 0.75.
What are the main limitations of the DocuTeam paper?
The study covers only event-planning tasks in two-hour lab sessions, the dialogue engine itself was never independently evaluated for conversation quality, statistical tests lack multiple-comparison correction and effect sizes (several headline p-values sit just below .05), and no code or data release is stated.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- §5.1.2 study-design details (counterbalancing, evaluator recruitment, rating scale, IRR) were truncated in extraction; some statistical flags may be addressed there.
- Figures (UI walkthrough, Fig 4/5/8 survey charts, prompt screenshots) could not be analyzed from text.
- End of §2.3 and references from [100] onward were cut off, so part of the related-work coverage could not be assessed.
- The blind-evaluation instrument (number of evaluators, scale, anchors, reliability) could not be verified from the available text.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.