LLM researcher at the University of Washington: pretraining, knowledge distillation, MoE scaling, and agent evaluation.
4
papers here
3
reviewed by Evolve
0
first-author
0
corresponding
—
works · OpenAlex
—
h-index · OpenAlex
Bibliometric figures come from OpenAlex (2026-10-17) and cover the author's entire public record, not just the papers listed here. Reproducibility signal comes from Evolve's own reviews.
Research profile
Output—
This axis would measure total research output — publication volume, venue tiers, and annual counts — but the underlying bibliometric data is not yet available for this profile, so no score is assigned.
Impact—
Impact would capture citation counts, h-index-style aggregates, and downstream uptake of the reviewed papers; these signals are absent from the current snapshot, so this axis remains unscored.
Focus0/100
The focus score of 0 reflects the raw field distribution: the catalogued papers split evenly, about 0.33 each, across LLM agent evaluation and test-time scaling, machine learning/NLP, and language model pretraining. Within 2026 alone the work moves from agent benchmarking to byte-level tokenization, MoE overfitting, and distillation timing — breadth across the LLM stack rather than concentration in one subfield, though all lines share a concern with how LLMs are trained and evaluated.
Collaboration68/100
Collaboration scores 68, computed from a raw count of 29 distinct coauthors across just four catalogued papers. The frequent-coauthor list pairs University of Washington colleagues — Jacqueline He, Howard Yen, Shuyue Stella Li, Pang Wei Koh — with repeated collaborators such as Margaret Li, Wen-tau Yih, Hanqing Zeng, Yinglong Xia, Benyu Zhang, Zhuokai Zhao, Qiang Zhang, and Kaiyuan Liu. Authorship positions (middle on three papers, last on one, none corresponding) suggest a senior contributor embedded in larger team efforts.
Activity—
Activity would track publication cadence, such as papers per year and gaps between releases. The snapshot provides only paper years — all four from 2026 — without a full publication history, so this axis cannot be scored yet.
Reproducibility55/100
Reproducibility readiness averages 55 (reproMean) over three papers with a computed score. Catalog ratings run from medium to high: the controlled OLMo-2 mid-training distillation study is rated high, while the Elo-per-token agent analysis, byte-model distillation study, and MoE-overfitting study are each rated medium. Evolve flags single-seed grids and conclusions resting on extrapolation as the main reproducibility limits.
Research summary
Luke Zettlemoyer is a machine learning and natural language processing researcher at the University of Washington whose catalogued work concentrates on how large language models are trained, compressed, and evaluated. Four 2026 arXiv papers reviewed by Evolve split evenly across three subfields: LLM agent evaluation and test-time scaling, machine learning/NLP with an LLM-training emphasis, and language model pretraining. On the training side, controlled OLMo-2 experiments show that knowledge distillation applied during mid-training trades factual recall for reasoning, with an entropy-routed Switch Distillation largely correcting the trade-off at 1B scale; a companion byte-level study predicts that distilled byte models can beat token distillation by about 4% asymptotically via exact <eot> logit conversion. In pretraining, a last-authored study argues that Mixtures-of-Experts overfit faster than dense models under data repetition, driven by total parameter count. On evaluation, an Elo-per-token framework quantifies how agents trade tokens for quality when comparing test-time scaling against repeated sampling. The catalog records 29 distinct coauthors and a collaboration score of 68; he is a middle author on three papers and last author on one, with no corresponding-author role captured. Reproducibility readiness averages 55 over three assessed papers, with the mid-training distillation study rated high. Citation-level impact metrics are not yet available in this snapshot.
Collaboration pattern
Across four 2026 papers, Zettlemoyer collaborates with 29 distinct coauthors, yielding a collaboration score of 68. The frequent-coauthor list mixes University of Washington colleagues — Jacqueline He, Howard Yen, Shuyue Stella Li, Pang Wei Koh — with repeat collaborators including Margaret Li, Wen-tau Yih, Hanqing Zeng, Yinglong Xia, Benyu Zhang, Zhuokai Zhao, Qiang Zhang, and Kaiyuan Liu. His positions (middle author three times, last author once, no corresponding role) fit a senior contributor who plugs into larger team efforts spanning agent evaluation, distillation, and pretraining.
Research keywords
Highlights
Elo-per-token: a yardstick for agent test-time strategies
Byte distillation predicted to break the token ceiling
MoEs overfit more to repeated pretraining data
Mid-training distillation trades recall for reasoning
Timeline
- 2026Publishes an Elo-per-token analysis of LLM agents' test-time strategies, framing test-time scaling against repeated sampling.
- 2026Proposes byte-model distillation with exact <eot> logit conversion, predicted to surpass token distillation asymptotically.
- 2026Last-authors a study showing Mixtures-of-Experts overfit more than dense models under repeated pretraining data.
- 2026Shows via controlled OLMo-2 experiments that mid-training distillation favors reasoning over factual recall, with Switch Distillation as a fix.
Papers in the Evolve catalog
When Agents Slow Down: Understanding LLM Agents' Test-Time Strategies via Elo-per-token Analysis
Breaking the Token Ceiling: Distilling Smaller, Stronger Byte Models
Data Scarcity and Model Sparsity: Mixtures-of-Experts Overfit More to Repeated Data
Knowledge Distillation During Mid-Training Favors Reasoning over Factual Recall
Frequent co-authors
About this profile
This profile is AI-structured by Evolve.Pub from public bibliographic data (OpenAlex) and Evolve's own paper reviews. It is not written or endorsed by the researcher unless the profile is claimed. Radar dimensions are computed from real bibliometrics with fixed formulas — raw figures are shown alongside; missing dimensions mean insufficient data, never zero. Errors? Claim the profile to correct or hide it.
- Bibliometric scores (collaboration, focus, reproducibility) are computed from OpenAlex-derived data; citation counts, h-index, and venue-tier metrics are absent from this snapshot.
- The profile covers only four Evolve-reviewed arXiv papers, all from 2026; earlier and non-arXiv work is outside this profile's scope.
- Output, impact, and activity dimensions lack underlying data and are deliberately left unscored.
- Identity is name-resolved from public bibliographic data; the listed ORCID is the value provided in the source snapshot and was not independently verified.