Toward Complete Hospital Discharge Summarization with Abstract Meaning Representation
Paul Landes, Sitara Rao, Aaron Jeremy Chaise, Barbara Di Eugenio
cs.CL
Quick verdict
AMR-based extractive discharge summarization with sentence-level provenance; novel encounter-level scope but weak eval: no baselines, tiny human study.
Summary
The paper presents an extractive pipeline for generating hospital discharge summaries from all clinical notes of an admission, designed so every output sentence carries explicit provenance links to source spans. Notes are segmented by clinical topic (MEDSECID), parsed into AMR graphs with the THYME clinical parser, and aligned to the discharge summary using CALAMR, a bipartite max-flow alignment framework from the authors' prior work; alignment flow then supervises a BiLSTM (with GatorTron embeddings) that assigns each source sentence to a discharge summary section or discards it (no-section). Experiments use MIMIC-III (3,520 of 11,957 admissions aligned) and a private UIC Health dataset of 11,001 admissions. The section model reaches weighted F1 88.72 on MIMIC-III but macro F1 only 20.41, reflecting dominance of the no-section class. Human Likert evaluation of 133 MIMIC-III summaries (two clinical reviewers) reports correctness 5/5 but readability 3.05, completeness 1, preference 1; only five UIC summaries were evaluated. The authors report ROUGE/BLEU between antecedent notes and gold summaries (ROUGE-1 27.98%) to explain why automatic metrics are low, and decline baseline comparisons citing mismatched evaluation setups. Main limitations: no baselines or ablations, alignment-derived weak supervision not manually audited, severe section-label imbalance, hardware-bound scaling, and a very small UIC evaluation.
Review complete
Assessment at a glance
6
risk flags identified
● high 2
● medium 3
● low 1
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- This is the first work to combine episode-level, multi-section clinical encounter discharge summarization with explicit sentence-level provenance evidence links to source spans (Sec. I, II).
- CALAMR network-flow alignments can serve as weak supervision to train a Source Section Model (BiLSTM + GatorTron embeddings) that reaches weighted F1 88.72 on MIMIC-III (Table III).
- The generated summaries are faithful and traceable because they contain only copied source text with CALAMR-derived evidence links (Sec. VI).
- Human evaluation shows perfect correctness (5/5) but low completeness (1/5) and preference (1/5), framed as evidence of a faithfulness-oriented baseline for future abstractive work (Table IV, Sec. V–VI).
Strengths
Provenance as a first-class output
Every summary sentence carries explicit evidence links to source spans via CALAMR alignment flows (Sec. III-C/D), directly targeting the hallucination/traceability gap the authors identify in LLM-based clinical summarization (Sec. I).
Genuinely hard encounter-level scope
The pipeline consumes all notes of an admission — up to 1,233 notes for a single MIMIC-III admission (footnote 2, Sec. VII) — where prior work covers single sections or single note categories (Sec. II), a setting most LLM approaches cannot fit in context.
Unusually transparent negative reporting
Strong artifact release
Dual-corpus validation with a real-world dataset
Weaknesses
No baseline comparison whatsoever
Methodologically weak human evaluation
Entire supervision chain unvalidated
Headline metric dominated by the trivial class
Completeness far from the title's claim
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
This ICHI 2026 paper (arXiv:2609.13581) tackles discharge summarization at the *encounter* level: given every clinical note of a hospital admission (up to 1,233 notes in MIMIC-III, footnote 2), produce a multi-section discharge summary in which every output sentence carries an explicit evidence link to its source span. The authors position this against LLM-based abstractive summarization, whose hallucination risk they argue is disqualifying for clinical documentation (Sec. I). The headline claims are: (a) this is the first system combining episode-level multi-section discharge summarization with sentence-level provenance (Sec. I–II); (b) CALAMR — the authors' prior bipartite max-flow AMR alignment framework [39] — can supply both the weak supervision and the provenance links; (c) the resulting summaries are faithful (copy-only) and traceable, validated by clinician review (Sec. VI).
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 6 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is CALAMR and how is it used in this paper?
CALAMR (Component ALignment for Abstract Meaning Representation) is the authors' prior framework that represents source notes and a target summary as two AMR semantic-graph components joined into one bipartite graph, scoring connections with a max-flow/network-flow computation. In this paper its alignment flows are used twice: as weak supervision labels for training the section-classification model, and as the explicit provenance (evidence) links from each generated summary sentence back to its source spans.
Does this paper use LLMs to write discharge summaries?
No. The approach is deliberately extractive: sentences are selected from the patient's existing clinical notes, organized into discharge-summary sections by a BiLSTM classifier, and linked back to their sources. The authors argue this avoids the hallucination risk of LLM generation and guarantees traceability, and they propose graph-based abstractive generation (via GraphRNN) only as future work.
What data did the authors use, and is it publicly available?
Two corpora: MIMIC-III v1.4 (a publicly available, credentialed-access ICU database; 3,520 of 11,957 admissions were aligned for experiments) and UICSDS, a private IRB-approved dataset from University of Illinois Health with 11,001 admissions and 607,872 notes. The code (github.com/uic-nlp-lab/calsum), trained models, and 133 generated MIMIC-III summaries with evaluations are released; the UIC hospital data cannot be shared.
How good are the generated discharge summaries?
Mixed. In a Likert study by two clinical raters on 133 MIMIC-III outputs, correctness scored a perfect 5/5 — but only over content actually included — while readability averaged 3.05, completeness 1/5 and preference 1/5, meaning the summaries are faithful and traceable but far too sparse to be clinically useful. Section-classification weighted F1 was 88.72, though macro F1 was only 20.41 due to class imbalance. The authors position the system as a provenance-safe baseline for future abstractive work, not a deployable summarizer.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- PDF extraction interleaves figure-embedded text (Figs. 1–6) with body prose, so figure composition (arrows, layout) could not be assessed; body text itself appears complete (title through reference [56], no mid-sentence cut at the end).
- Per-rater Likert score distributions are not recoverable from the extraction — Table IV means only — but this reflects the paper's reporting, not a truncation of the text.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.