PeerPen: AI-Assisted Writing for Online Mental Health Peer Support
Jiwon Kim, Sherry Gong, Maya Ajit, Soorya Ram Shimgekar, Yunhao Yuan, Dong Whi Yoo, Eshwar Chandrasekharan, Koustuv Saha
cs.HC · cs.AI · cs.CL · cs.CY · cs.SI
Quick verdict
PeerPen interviews (n=15) show AI-assisted peer support cuts writing effort but threatens authorship and community trust; solid HCI study, self-selected sample.
Summary
PeerPen is an AI writing-assistance tool embedded in a Reddit-like interface for online mental health peer support. It offers a Generation Feature that drafts a full supportive response from a post and an Improvement Feature that decomposes revisions of a user-written draft into selectable replace/insert suggestions, backed by Mistral-7B models tuned in the authors' prior LLUMI work. Fifteen U.S. participants wrote responses to real community posts while thinking aloud, then were interviewed; transcripts were analyzed via reflexive thematic analysis (three coders, 315 open codes, five themes), complemented by post-task quality surveys. Participants rated outputs highly (readability 4.82 and safety 4.82 for generated, 4.80 and 4.74 for revised responses; actionability lowest at 3.82 for generated versus 4.24 for revised) and reported reduced composing burden and greater confidence, especially among previously passive observers. However, perceived ownership depended on whether the human initiated and directed the writing, and participants anticipated authenticity, trust, and community-homogenization concerns, including an 'authorship spillover' whereby widespread availability could cast doubt even on fully human-written replies. Limitations include 15 participants in a single session, self-selection of AI-open participants, U.S. English-only sampling, no unassisted baseline, and no direct evidence from support seekers.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- AI writing assistance embedded in the community interface reduced the perceived burden of composing supportive responses and increased participants' confidence and willingness to respond (§5.2).
- Participants' sense of authorship depended on process — who supplied communicative intent and directed revision — rather than on the proportion of AI-supplied text (§5.3, §6.2).
- Localized, inspectable revision suggestions supported reflection and learning, but visibility alone did not explain why a change is more supportive (§5.4).
- Widespread availability of such assistance would create an 'authorship spillover' effect in which even entirely human-written responses are subject to doubt, eroding community-level trust (§5.6, §6.3).
- Design implications: graduated scaffolding, in-situ revision guidance with explanations, user steering without ceding authorship, granular provenance, and community-level governance of AI assistance (§6.4).
Strengths
Well-motivated system contrast targeting the authorship question
The four design goals (§3.1) are explicitly derived from prior literature, and the Generation vs. Improvement features (§3.2) operationalize the exact contrast — AI-initiated drafting vs. revision of user text — that the RQ about authorship requires; Table 1 maps each feature to a design goal.
Backend models validated independently before the study
§3.2.3 builds on prior work [37]: the Improvement Model was preferred over original responses in 82.0% of comparisons and the Generation Model performed within ~1% of GPT-5-nano on human ratings (empathy 4.41/5, safety 4.49/5), plus pre-study safety evaluation over diverse r/SuicideWatch posts (§4.4) — so the interface study is not confounded by an obviously broken generator.
Transparent, standards-aligned qualitative analysis
Novel community-level concept
Honest limitations section
Weaknesses
Survey evidence is descriptive and unbalanced
Core benefit claims rest on self-report with no baseline
Sampling limits the very community-trust claims the paper advances
The central 'ordering' claim is not causally isolated
Study materials and artifacts not released
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
This HCI systems-plus-interviews paper (arXiv 2609.14886, cs.HC) introduces PeerPen, an AI writing assistant embedded in a Reddit-replica interface for online mental health communities (OMHCs), and reports a 15-participant qualitative study of how peer supporters perceive AI-assisted supportive writing. Its two features — Generation (GF: full-draft generation from a post) and Improvement (IF: revision decomposed into discrete replace/insert suggestions) — are deliberately contrasted to probe authorship. The central claims: (1) assistance lowers composition burden and raises confidence (§5.2); (2) authorship is process-dependent — ownership held when the human supplied communicative intent and directed revision (§5.3, §6.2); (3) localized revision supports reflection but inspectability is not explanation (§5.4); and (4) 'authorship spillover' — community-wide availability makes even human-written responses suspect, eroding trust beyond any individual interaction (§5.6, §6.3). Claim 4 is the paper's most original contribution and its strongest conceptual move.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is PeerPen and what does it do?
PeerPen is an AI writing-assistance tool embedded in a Reddit-style interface for online mental health peer-support communities. It has two features: a Generation Feature that drafts a complete supportive reply from a post, and an Improvement Feature that revises a user-written draft via individual, selectable replace/insert suggestions. It runs on Mistral-7B models fine-tuned on community interactions in the authors' earlier LLUMI work.
What did the PeerPen user study find?
In interviews with 15 U.S. participants who responded to real mental health community posts, PeerPen lowered the effort of writing supportive replies and increased confidence, especially among people who had mostly lurked before. Participants still wanted to keep authorship, and they raised authenticity and trust concerns — including an 'authorship spillover' effect: if such tools became widespread, people might doubt even replies written entirely by humans.
What are the main limitations of this study?
It involved 15 participants in a single one-hour simulated session; participants self-selected and were at least open to AI assistance (skeptics declined); all were U.S. residents with English as a first language; eight of fifteen were primarily observers rather than support providers; there was no unassisted-writing baseline; and support seekers themselves were not studied, so recipient-side effects are anticipated rather than observed.
Is PeerPen's code or data publicly available?
The paper does not mention any public release of the PeerPen system, interview protocol, or study data. The underlying generation and improvement models come from the authors' prior LLUMI paper (arXiv:2605.30273), which should be consulted for model training and evaluation details.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- The extracted text omitted or truncated several sections (part of §4.3.1 coding detail, part of §6.5 ethics discussion, and part of the reference list), so some procedural details were unavailable.
- Figures 1–3 and Table A1 are described only via captions and surrounding text; the actual screenshots and diagrams could not be visually verified.
- The author affiliation block contains a typographical error ('Espoo, Illinois, Finland'), suggesting extraction or typesetting noise around metadata.
- Interview protocol, post-selection criteria, and appendix materials were not present in the extracted text; judgments about study materials rely on the main-text description only.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.