Peiying Zhu — builds claim-safe, reproducible evaluation methodology for LLM judges, agents, and closed-loop AI systems
4
papers here
2
reviewed by Evolve
3
first-author
0
corresponding
—
works · OpenAlex
0
h-index · OpenAlex
Bibliometric figures come from OpenAlex (2026-10-17) and cover the author's entire public record, not just the papers listed here. Reproducibility signal comes from Evolve's own reviews.
Research profile
Output—
Output cannot be scored: h-index and i10-index are both 0 in the current OpenAlex snapshot, so production volume has no stable signal yet. This axis would measure total and recent publication counts once a fuller, better-linked publication record accumulates.
Impact—
Impact is not yet scoreable: with h-index 0 and i10-index 0, citation-based signals are absent from the OpenAlex snapshot. This axis would measure citation counts, citation-weighted influence, and uptake of the 2026 papers once citations accrue.
Focus0/100
OpenAlex topics spread across five clusters — Ethics and Social Impacts of AI (5 works), Reinforcement Learning in Robotics (3), Simulation Techniques (2), Stock Market Forecasting Methods (2), and Software System Performance and Reliability (2) — while the field distribution splits evenly 0.5/0.5 between AI evaluation methodology and NLP/ML evaluation methodology. That even split drives the focus score of 0: the catalog work is thematically tight around evaluation auditing, but the raw record does not yet show a dominant field.
Collaboration22/100
The OpenAlex record lists only 2 distinct coauthors, with Sidi Chang the most frequent, yielding a low collaboration score of 22. Three of the four catalog papers carry first authorship and one last authorship, none flagged corresponding, which suggests a small, tightly held working mode rather than a large consortium. Collaboration metrics may rise as the 2026 arXiv work becomes linked to richer author metadata.
Activity75/100
Activity is strong: the OpenAlex snapshot records 12 works in the last three years, with countsByYear attributing all 12 to 2026, and all four catalog papers are 2026 arXiv preprints. This concentrated burst yields an activity score of 75. The cadence indicates a productive current phase, though the record spans effectively a single year, so longer-run consistency cannot yet be judged from these counts.
Reproducibility78/100
Reproducibility averages 78 across the 2 catalog papers with sufficient data, the strongest signal in this profile. It tracks the paper-level readiness spread: high for the claim-safe protocol, medium for the LLM-judge ledgers and guardrail audits, and low for ClaimReceipt, whose validation is self-referential with no code released. Preregistration and audit receipts recur as methods, but code availability is uneven.
Research summary
Peiying Zhu is an early-career researcher whose Evolve-reviewed work centers on evaluation methodology for modern AI systems. All four catalog papers are 2026 arXiv preprints: a preregistered claim-safe protocol (refuse/decompose/refresh) for closed-loop AI evaluation; 'Four Ledgers, Not One Score', a forensic audit of how LLM-judge calibration is communicated in biomedical ML; ClaimReceipt, a receipt specification for verifying evidence sufficiency and coverage in agent evaluations; and a construct-validity audit showing guardrail welfare gains in LLM-agent commerce simulations collapse from +87.4 to +7.2 once a schema confound is removed. Zhu is first author on three and last author on one, consistent with a driving role. OpenAlex records 12 works in the last three years, all attributed to 2026, with h-index 0 and i10-index 0, so output and impact axes remain unscored. The field distribution splits evenly (0.5/0.5) between AI evaluation methodology and NLP/ML evaluation methodology, and reproducibility averages 78 across the 2 assessed papers. Recurring themes are construct validity, abstention, provenance, and preregistration; the co-author network is thin (2 distinct coauthors, most frequently Sidi Chang). This profile is name-resolved from public bibliographic data.
Collaboration pattern
Co-authorship in the OpenAlex record is sparse: 2 distinct coauthors over the recent window, with Sidi Chang the only recurring name among frequent coauthors. Authorship positions — first on three of four catalog papers, last on one, no corresponding flags — indicate a lead-your-own-work pattern within a very small circle. The low collaboration score of 22 reflects this thin network; it may be an artifact of recent arXiv-only records rather than an absence of collaboration.
Research keywords
Highlights
A preregistered claim-safe protocol for closed-loop AI evaluation
Reframing LLM-judge calibration as four ledgers, not one score
Exposing construct-validity failures in LLM agent commerce evaluation
ClaimReceipt: receipts that make agent-eval claims recomputable
Timeline
- 2026Publishes 'Refuse, Decompose, Refresh', a preregistered claim-safe protocol for closed-loop AI evaluation, as first author on arXiv.
- 2026Publishes 'Four Ledgers, Not One Score' on responsible communication of LLM-judge calibration in biomedical ML, as last author.
- 2026Publishes ClaimReceipt, a receipt specification for verifying evidence sufficiency and coverage in agent evaluations, as first author.
- 2026Publishes the construct-validity audit of LLM agent commerce evaluation, showing guardrail welfare gains shrink from +87.4 to +7.2 after confound removal.
Papers in the Evolve catalog
- first author
Refuse, Decompose, Refresh: A Claim-Safe Protocol for Closed-Loop AI Evaluation
Four Ledgers, Not One Score: Responsible Communication of LLM-Judge Calibration in Biomedical ML
- first author
ClaimReceipt: Verifying Evidence Sufficiency and Coverage in Agent Evaluations
- first author
When Guardrails Look Effective: Construct Validity Failures in LLM Agent Commerce Evaluation
Frequent co-authors
About this profile
This profile is AI-structured by Evolve.Pub from public bibliographic data (OpenAlex) and Evolve's own paper reviews. It is not written or endorsed by the researcher unless the profile is claimed. Radar dimensions are computed from real bibliometrics with fixed formulas — raw figures are shown alongside; missing dimensions mean insufficient data, never zero. Errors? Claim the profile to correct or hide it.
- Bibliometrics are computed from an OpenAlex snapshot only; h-index (0) and i10-index (0) reflect sparse linking of recent arXiv records, and the output and impact axes are left unscored rather than estimated.
- Identity is name-resolved from public bibliographic data: no institution, country, or ORCID is available, and the breadth of OpenAlex topics (e.g., stock market forecasting, robotics) may reflect name collision.
- Qualitative judgments draw only on the four Evolve-reviewed papers, all 2026 arXiv preprints; the reproducibility mean of 78 covers 2 assessed papers.
- Citation-based impact cannot be assessed: countsByYear contains only 2026 (12 works), so no multi-year citation trajectory exists in the data.