Oxford researcher quantifying AI governance transparency and the measurement validity of LLM-based evaluation
4
papers here
1
reviewed by Evolve
4
first-author
4
corresponding
—
works · OpenAlex
0
h-index · OpenAlex
Bibliometric figures come from OpenAlex (2026-10-17) and cover the author's entire public record, not just the papers listed here. Reproducibility signal comes from Evolve's own reviews.
Research profile
Output—
Not yet scored: OpenAlex reports no usable output totals for this author (h-index and i10-index are both 0). This axis would measure total works, venue mix, and per-year production once indexing of the 2026 preprints matures.
Impact—
Not yet scored: citation evidence is insufficient — h-index and i10-index are both 0, with no citation series available. This axis would measure citations, uptake, and scholarly influence once the 2026 preprints are indexed and begin to be cited.
Focus100/100
Scores 100 on focus. The OpenAlex topic profile is dominated by 'Ethics and Social Impacts of AI' (7 works), with smaller clusters in artificial intelligence applications (3), psychometric methodologies and testing (2), digital platforms and economics (2), and reliability and agreement in measurement (2) — all captured under a single field, AI governance / computational social science. Every cataloged preprint asks how AI systems and their evaluations can be measured, an unusually coherent programme.
Collaboration0/100
Scores 0 on collaboration. The record shows 0 distinct co-authors and an empty frequent-co-author list, and all four cataloged 2026 preprints list this researcher as first and corresponding author. Work so far is effectively solo, which concentrates credit but also the coding, statistics, and review burdens that Evolve's verdicts flag as limiting certainty.
Activity70/100
Scores 70 on activity. OpenAlex records 10 works in the recent three-year window, and the countsByYear series places all 10 in 2026 alone — a first-year burst of arXiv preprints rather than a cumulative record. Productivity is high and current, but the absence of any prior-year baseline keeps the score out of the top band.
Reproducibility55/100
Scores 55 on reproducibility, a mean computed from only 1 assessed paper. The Evolve-reviewed preprints diverge: 'Three Ways Classical Test Theory Misleads for LLM Judges' carries high reproducibility readiness, while 'Silent Revision' is rated medium because its hash-pinned corpus and statistics rest on single-coder coding and n=8 framework pairs. One scored paper is thin evidence.
Research summary
Louis Yiven Zhu is an early-career researcher at the University of Oxford whose entire visible record — 10 works, all dated 2026 in OpenAlex's countsByYear — concentrates on three connected questions: how transparently frontier AI developers change their safety commitments, whether classical measurement theory remains valid when the rater is a language model, and how the price and capability structure of AI services should be indexed. Evidence is strongest in two first-authored, corresponding-author arXiv preprints. 'Silent Revision' reports that 67% of frontier AI safety-framework changes go undisclosed, using a hash-pinned corpus and statistical comparison, though Evolve flags single-coder coding and n=8 framework pairs. 'Three Ways Classical Test Theory Misleads for LLM Judges' shows KR-20, the dependability index Φ(λ), and the Livingston–Lewis method lose attribution with LLM judges, and carries high reproducibility readiness. The bibliometric signature is remarkably concentrated: focus scores 100, activity 70, reproducibility 55 (from one assessed paper), and collaboration 0, with no distinct co-authors recorded. Output and impact cannot yet be scored because h-index and i10-index are both 0. This is a young, coherent, solo research programme — quality controls currently outrun its citation record, making this an early-stage profile.
Collaboration pattern
The record shows an emphatically solo pattern: 0 distinct co-authors, an empty frequent-co-author list, and first-and-corresponding authorship on all four cataloged 2026 preprints, giving the collaboration dimension its score of 0. This concentrates design, coding, and analysis in one researcher — Evolve's single-coder caveat on 'Silent Revision' is a direct consequence. An independent second coder or co-author would be the natural next step.
Research keywords
Highlights
Silent Revision finds 67% of frontier safety-framework changes undisclosed
Classical test theory breaks down with LLM judges
A quality-adjusted price index for AI services
2026 arXiv preprint (first and corresponding author) proposing a quality-adjusted price index for AI services; not yet reviewed by Evolve, so no verdict, keywords, or reproducibility rating are available.
One capability or many? Stress-testing frontier AI evaluation
2026 arXiv preprint (first and corresponding author) testing the economic validity of frontier AI evaluation — whether evaluations measure a single capability or many; not yet reviewed by Evolve, so findings remain unverified.
Timeline
- 2026OpenAlex records 10 works in 2026 — the entire visible output lands in a single year, with no prior entries in countsByYear.
- 2026Publishes 'Silent Revision: Measuring Undisclosed Change in the Safety Frameworks of Frontier AI Developers' as first and corresponding author (arXiv).
- 2026Publishes 'Three Ways Classical Test Theory Misleads for LLM Judges' as first and corresponding author (arXiv); Evolve rates its reproducibility readiness high.
- 2026Publishes 'The Price of Intelligence: A Quality-Adjusted Price Index for AI Services' as first and corresponding author (arXiv).
- 2026Publishes 'One Capability or Many? Testing the Economic Validity of Frontier AI Evaluation' as first and corresponding author (arXiv).
Papers in the Evolve catalog
- first authorcorresponding
Silent Revision: Measuring Undisclosed Change in the Safety Frameworks of Frontier AI Developers
- first authorcorresponding
Three Ways Classical Test Theory Misleads for LLM Judges
About this profile
This profile is AI-structured by Evolve.Pub from public bibliographic data (OpenAlex) and Evolve's own paper reviews. It is not written or endorsed by the researcher unless the profile is claimed. Radar dimensions are computed from real bibliometrics with fixed formulas — raw figures are shown alongside; missing dimensions mean insufficient data, never zero. Errors? Claim the profile to correct or hide it.
- All bibliometrics come from OpenAlex only (identity: openalex); citation and impact data appear unindexed (h-index and i10-index both 0) and may not yet reflect real uptake.
- Only 2 of the 4 cataloged papers carry Evolve review verdicts; 'The Price of Intelligence' and 'One Capability or Many?' are title-level entries without keywords, field labels, or reproducibility ratings.
- The reproducibility score (55) is a mean over just 1 assessed paper and should be read as preliminary.
- Identity is name-resolved from public bibliographic data (OpenAlex profile linked to ORCID 0009-0001-5579-0340); biography beyond institution and country is out of data scope.