NLP researcher at the University of Pennsylvania working on LLM-as-a-judge evaluation and clinical evidence synthesis
2
papers here
2
reviewed by Evolve
0
first-author
0
corresponding
—
works · OpenAlex
—
h-index · OpenAlex
Bibliometric figures come from OpenAlex (2026-10-02) and cover the author's entire public record, not just the papers listed here. Reproducibility signal comes from Evolve's own reviews.
Research profile
Output—
This snapshot contains no publication-volume data, so output — total paper counts by year and venue — cannot be scored. Only the two reviewed 2026 arXiv papers are observable; a complete publication record would be needed to populate this axis.
Impact—
Citation counts, h-index, and downstream influence are not present in the input data, so impact cannot be scored. This axis would measure how often the researcher's work is cited and adopted; it is honestly left unscored rather than estimated.
Focus0/100
The focus score is 0, derived from an even field distribution: 50% natural language processing (LLM evaluation) and 50% clinical NLP / evidence synthesis. Across the two reviewed 2026 papers the work spans a methods-oriented evaluation strand (rubric judging, cost-efficient classifiers) and an applied clinical strand (AI-assisted data extraction for systematic reviews). This is deliberate two-sided positioning rather than diffusion — but on a strict concentration metric the profile reads as maximally spread.
Collaboration39/100
Collaboration scores 39 from six distinct coauthors in the snapshot, a small pool by bibliometric standards. The coauthors split into an NLP/evaluation cluster — Delip Rao, Sai Karthik Kosuri, and Ankita Shashikant Bhosale — and a clinical/evidence-synthesis cluster with Michael Glick and Alonso Carrasco-Labra, matching the 50/50 field split. Callison-Burch sits in the last author position on both reviewed papers, a senior anchoring role over small teams rather than large consortia.
Activity—
Recent-year publication cadence cannot be computed from this snapshot, which contains only two papers, both dated 2026. The activity axis would measure outputs per year and sustained engagement over time; with a single observation year, it is left unscored.
Reproducibility55/100
Reproducibility scores 55, the mean across the two papers with reproducibility assessments (reproPapers = 2), both rated medium readiness. Evolve's verdicts support this middle band: the JEV paper is called rigorous and honest, yet its equivalence claim is mostly unproven; EviStreams' evaluation is honest but limited by calibration gaps and judge circularity. Transparent analysis, with artifacts not yet sufficient for full replication.
Research summary
Chris Callison-Burch is a researcher at the University of Pennsylvania whose reviewed work sits at the junction of LLM-based evaluation and clinical evidence synthesis. The Evolve catalog contains two 2026 arXiv papers, both with Callison-Burch in the last author position — a senior role consistent with supervising the work rather than hands-on first authorship. The first, “JEV vs. LLMs as Rubric Judges,” reports that a small typed classifier (Jev) matches flash LLM rubric judges at 29–325× lower cost, but that the two error in correlated ways, which limits the value of cost-saving cascades; Evolve rates the work rigorous and honest while noting the equivalence claim remains mostly unproven. The second, “EviStreams,” presents a human-in-the-loop system for AI-assisted data extraction in medical systematic reviews and argues that field specifications, rather than model choice, drive extraction quality; Evolve flags calibration and judge-circularity caveats. Bibliometrically the profile is deliberately two-sided: the field distribution splits evenly (50/50) between LLM evaluation and clinical NLP/evidence synthesis, six distinct coauthors appear across the snapshot, and a reproducibility mean of 55 across two papers, both rated medium readiness, reflects transparent-but-not-yet-fully-replicable evaluation. Output, impact, and activity axes lack underlying data in this snapshot and are left unscored.
Collaboration pattern
Six distinct coauthors appear in this snapshot, splitting into an NLP/evaluation cluster (Delip Rao, Sai Karthik Kosuri, Ankita Shashikant Bhosale) and a clinical evidence-synthesis cluster (Michael Glick, Alonso Carrasco-Labra). The pattern is small-team and senior-anchored: Callison-Burch takes the last author position on both reviewed 2026 papers while collaborating across the two subfields, consistent with the 50/50 field split.
Research keywords
Highlights
Cheap rubric judges that fail together
Human-in-the-loop extraction for medical systematic reviews
Senior author across two research strands
Both 2026 catalog papers place Callison-Burch in the last author position, spanning LLM evaluation and clinical evidence synthesis with a six-person coauthor pool — a compact, supervisory collaboration footprint rather than large-consortium work.
Timeline
- 2026“JEV vs. LLMs as Rubric Judges: Cheaper, Faster, and Wrong in the Same Places” appears on arXiv, with Callison-Burch in the last author position; Evolve finds its cost–error analysis rigorous and honest.
- 2026“EviStreams: Human-in-the-Loop AI Data Extraction for Systematic Reviews in Medicine” is released, again last-authored, arguing that field specifications rather than model choice drive extraction quality.
- 2026Both reviewed outputs land in the same year across two subfields — LLM evaluation and clinical evidence synthesis — with a six-person coauthor pool and both papers rated medium reproducibility readiness (mean 55).
Papers in the Evolve catalog
Frequent co-authors
About this profile
This profile is AI-structured by Evolve.Pub from public bibliographic data (OpenAlex) and Evolve's own paper reviews. It is not written or endorsed by the researcher unless the profile is claimed. Radar dimensions are computed from real bibliometrics with fixed formulas — raw figures are shown alongside; missing dimensions mean insufficient data, never zero. Errors? Claim the profile to correct or hide it.
- Bibliometric scores are computed from the platform's bibliometric snapshot; the focus, collaboration, and reproducibility axes rest on only two reviewed papers.
- Output, impact, and activity axes are unscored because citation counts, h-index, and multi-year publication history are absent from the input data.
- No ORCID or country is provided; this profile is name-resolved from public bibliographic data and may require disambiguation.
- The reproducibility mean (55) is based on only two papers, both rated medium readiness — treat it as indicative rather than definitive.