IBM researcher in NLP: LLM evaluation, machine translation benchmarks, and multilingual pretraining
3
papers here
3
reviewed by Evolve
0
first-author
0
corresponding
—
works · OpenAlex
17
h-index · OpenAlex
Bibliometric figures come from OpenAlex (2026-10-16) and cover the author's entire public record, not just the papers listed here. Reproducibility signal comes from Evolve's own reviews.
Research profile
Output—
This axis measures overall publication volume and authorship roles (first, middle, last, corresponding), but the underlying scoring data is not yet available for this profile, so no score is shown. Volume should not be inferred from the three reviewed catalog papers alone.
Impact—
This axis would measure citation-based influence, including total and field-normalized citation impact, but the underlying data is not yet available for this profile, so it is left unscored rather than estimated from the available bibliometric snapshot.
Focus0/100
The focus score is 0 because output is spread evenly rather than concentrated: field assignment splits exactly into thirds — NLP for LLM evaluation, for machine translation evaluation, and for LLM pretraining and multilinguality (0.33 each). Topic assignments reinforce the breadth, spanning topic modeling (148 records), general NLP techniques (121), domain adaptation and few-shot learning (40), multimodal applications (21), and speech and dialogue systems (20).
Collaboration84/100
Scoring 84, this axis reflects a highly collaborative profile: 66 distinct coauthors across 116 works in the most recent three years. The recurring-partner list spans at least twelve names, including Shachar Don-Yehiya, Omri Abend, Vilém Zouhar, Niyati Bafna, Mukund Choudhary, Maike Züfle, Sara Rajaee, Pinzhen Chen, Jannis Vamvas, Sara Papi, Ona de Gibert, and Bhavitvya Malik — many mid-sized teams rather than one fixed duo.
Activity100/100
With a score of 100, activity is the strongest axis: the per-year record climbs from 9 works in 2018 to 11 in 2021, 25 in 2022, and 43 in 2024, with 28 in 2025 and 45 in 2026. The recent three-year window alone holds 116 works, indicating a steep, sustained acceleration in publication rate rather than a single spike.
Reproducibility55/100
The reproducibility score is 55, computed from 3 Evolve-reviewed papers (reproMean 55), and all three catalog papers carry a medium reproducibility rating. Verdicts flag concrete limits: each cross-lingual pretraining result is a single run, the user-feedback study rests on one judge family with κ = 0.24 ground truth, and the translation benchmark relies on unquantified LLM verifiers — tractable, but requiring care.
Research summary
Leshem Choshen is a researcher at IBM (United States) whose OpenAlex-indexed work sits in natural language processing, split evenly across three assigned fields: LLM evaluation, machine translation evaluation, and LLM pretraining with multilinguality (about 33% each). The snapshot records an h-index of 17, an i10-index of 36, 116 works over the most recent three years, and 66 distinct coauthors. Annual output rises from 9 works in 2018 to 43 in 2024, 28 in 2025, and 45 in 2026. Topic assignments are led by topic modeling (148) and natural language processing techniques (121), followed by domain adaptation and few-shot learning (40), multimodal applications (21), and speech and dialogue systems (20). The three Evolve-reviewed papers, all 2026 arXiv preprints, cover why pretraining fails to share cross-lingual knowledge (last author), a community-built Last Translation Benchmark with 3,456 hard exceptions and rule-based verifiers, and user feedback as an improvement signal that LLM judges cannot detect. All three carry medium reproducibility readiness, with verdicts citing single-run evidence, κ = 0.24 ground truth, and unquantified LLM verifiers. The output and impact radar axes are unscored pending data.
Collaboration pattern
OpenAlex records 66 distinct coauthors across 116 works in the last three years, pointing to many mid-sized, project-specific teams rather than a single fixed partnership. Recurring names include Shachar Don-Yehiya, Omri Abend, Vilém Zouhar, Niyati Bafna, Mukund Choudhary, Maike Züfle, Sara Rajaee, Pinzhen Chen, Jannis Vamvas, Sara Papi, Ona de Gibert, and Bhavitvya Malik, consistent with the community benchmark and evaluation efforts in the catalog.
Research keywords
Highlights
Disjoint token spaces drive cross-lingual knowledge compartmentalization
A community-built benchmark for machine translation's hardest cases
User feedback as an improvement signal LLM judges cannot detect
Timeline
- 2026Last author on 'Why Pretraining Fails to Share Cross-Lingual Knowledge' (arXiv), showing that disjoint token spaces alone cause cross-lingual knowledge compartmentalization in pretraining.
- 2026Middle author on 'Last Translation Benchmark' (arXiv), a community-built machine translation benchmark with 3,456 hard exceptions and rule-based verifiers.
- 2026Middle author on 'User Feedback Provides a Unique Signal that LLMs Can not Detect' (arXiv), arguing user feedback is a potent improvement signal masked from LLM judges.
Papers in the Evolve catalog
Frequent co-authors
About this profile
This profile is AI-structured by Evolve.Pub from public bibliographic data (OpenAlex) and Evolve's own paper reviews. It is not written or endorsed by the researcher unless the profile is claimed. Radar dimensions are computed from real bibliometrics with fixed formulas — raw figures are shown alongside; missing dimensions mean insufficient data, never zero. Errors? Claim the profile to correct or hide it.
- Bibliometric figures are computed from OpenAlex records only (identity: openalex); citations indexed in other sources are not reflected.
- The output and impact radar axes lack sufficient underlying data and are shown as unscored rather than estimated.
- Paper-level analysis rests on 3 Evolve-reviewed 2026 arXiv preprints; this reviewed set is not a complete publication list.
- The profile is name-resolved from public bibliographic data (ORCID 0000-0002-0085-6496); name collisions with other researchers are possible.