ETH Zurich researcher in machine translation evaluation — adversarial stress tests, document-level metrics, benchmarks.
3
papers here
3
reviewed by Evolve
1
first-author
0
corresponding
—
works · OpenAlex
9
h-index · OpenAlex
Bibliometric figures come from OpenAlex (2026-10-16) and cover the author's entire public record, not just the papers listed here. Reproducibility signal comes from Evolve's own reviews.
Research profile
Output—
No output score is computed yet — the indexed publication data available is insufficient for this composite. The axis would measure publication volume and trajectory against field norms; visible OpenAlex yearly counts rise from 1 (2019) to 62 (2025) but remain unscored.
Impact—
No impact score is available: citation-side data is insufficient for scoring beyond the raw h-index of 9 and i10-index of 9. This axis would measure citation uptake, influence on evaluation practice, and downstream reuse; it should not be inferred from the current data.
Focus100/100
Focus scores 100/100 on an extremely narrow field assignment: the OpenAlex record places this researcher in a single field, natural language processing (machine translation evaluation), and all three Evolve-reviewed papers sit in that same field. Topic counts reinforce the concentration around NLP techniques (155), topic modeling (142), and translation studies and practices (82); the cataloged 2026 work is uniformly evaluation methodology.
Collaboration84/100
With 67 distinct co-authors across 141 works in the last three years, the record shows a strongly collaborative mode, scoring 84/100. Twelve frequent co-authors recur — including Jannis Vamvas, Sara Papi, Pinzhen Chen, Chanjun Park, Niyati Bafna, and Ona de Gibert — and the three catalog papers place the researcher in first, middle, and last author positions, a pattern of rotating roles inside large international machine translation teams.
Activity100/100
Activity scores 100/100 on a steep publication ramp: 1 work in 2019, 4 in 2020, 13 in 2021, 11 in 2022, then acceleration to 22 in 2023, 39 in 2024, and 62 in 2025, with 40 already recorded for 2026. The 141 works in the trailing three-year window confirm a sustained, top-of-axis research pace rather than a single-year spike.
Reproducibility85/100
The three Evolve-reviewed papers carry a mean reproducibility score of 85. Two — the adversarial GRPO generation paper and the document-level evaluation study — are rated high readiness, backed by explicit metrics and blinded counterfactual designs; the first-authored Last Translation Benchmark is rated medium because it relies on unquantized LLM verifiers. Methods and protocols across the set are described in replication-usable detail.
Research summary
Vilém Zouhar is a machine translation evaluation researcher based at ETH Zurich, whose OpenAlex-linked record shows a steeply accelerating output curve — from 1 work in 2019 to 13 in 2021, 39 in 2024, and 62 in 2025, with 141 works over the most recent three-year window and 67 distinct co-authors. Bibliometric topics are dominated by natural language processing techniques (155) and topic modeling (142), with translation studies and practices (82) and speech-related clusters (32 and 11) as secondary strands. The Evolve-reviewed catalog concentrates on machine translation evaluation. As first author, Zouhar led the Last Translation Benchmark (2026), a community-built benchmark of 3,456 difficult exceptions with rule-based verifiers that separates frontier models from human rankings, though it relies on unquantized LLM verifiers. In a middle position, the researcher contributed to a blinded counterfactual study showing that document-level evaluation protocols fail to register coherence. In the last position, the researcher co-authored GRPO-based adversarial text generation that cuts MT quality (average COMET 0.63 to 0.48 on WMT25 En→It) while preserving grammar. Reproducibility is a strength: the three reviewed papers average 85, with high readiness on two and medium on the first-authored benchmark.
Collaboration pattern
The record is highly collaborative: 67 distinct co-authors over the last three years and a recurring circle of twelve frequent collaborators, including Jannis Vamvas, Maike Züfle, Sara Papi, Pinzhen Chen, Chanjun Park, Niyati Bafna, Ona de Gibert, and Bhavitvya Malik. Across the three 2026 catalog papers the authorship position rotates between first, middle, and last — a pattern of contributing across multiple international machine translation evaluation teams rather than one fixed group.
Research keywords
Highlights
First-authored community benchmark separates frontier MT from human rankings
Blinded counterfactual study finds document-level MT evaluation blind to coherence
GRPO-generated adversarial texts cut MT quality from COMET 0.63 to 0.48
Timeline
- 2026First-authored the Last Translation Benchmark, a community-built MT benchmark of 3,456 difficult exceptions with rule-based verifiers that separates frontier models from human rankings.
- 2026Co-authored The Blindness of Document-Level Translation Evaluation, a large blinded counterfactual study showing document-level MT evaluation protocols fail to register coherence.
- 2026Co-authored Generating Adversarial Texts for Machine Translation via GRPO, degrading average COMET from 0.63 to 0.48 on WMT25 En→It while keeping grammar.
Papers in the Evolve catalog
Frequent co-authors
About this profile
This profile is AI-structured by Evolve.Pub from public bibliographic data (OpenAlex) and Evolve's own paper reviews. It is not written or endorsed by the researcher unless the profile is claimed. Radar dimensions are computed from real bibliometrics with fixed formulas — raw figures are shown alongside; missing dimensions mean insufficient data, never zero. Errors? Claim the profile to correct or hide it.
- Bibliometrics come from the OpenAlex record only; citation and venue coverage may lag arXiv-first work.
- Output and impact dimension scores are not computed; the only citation-side figures provided are h-index 9 and i10-index 9.
- All three reviewed papers are 2026 arXiv preprints; peer-review venue status is not captured in the data.
- Affiliation metadata (ETH Zurich; country listed as GB) comes from the OpenAlex identity record and shows an inconsistency worth verifying.