Microsoft Research Montréal researcher building RL-trained coding agents and verifiable SWE task benchmarks
2
papers here
2
reviewed by Evolve
0
first-author
0
corresponding
—
works · OpenAlex
2
h-index · OpenAlex
Bibliometric figures come from OpenAlex (2026-10-16) and cover the author's entire public record, not just the papers listed here. Reproducibility signal comes from Evolve's own reviews.
Research profile
Output—
Output scoring is not yet available in the current snapshot: a complete corpus of works is missing. This axis would measure cumulative publication volume and venue distribution over time; only 8 recent three-year works are visible so far.
Impact—
Impact scoring is unavailable: citation data is too sparse for a reliable estimate. This axis would measure citation counts and uptake by follow-up work; the visible h-index of 2 and i10-index of 0 mark an early-stage record, not a score.
Focus0/100
Topical concentration is low (focus score 0/100). The record spreads across five OpenAlex topics — Topic Modeling (6 works), Natural Language Processing Techniques (4), Domain Adaptation and Few-Shot Learning (3), and two software-engineering clusters (2 each) — and recent output splits evenly (0.5/0.5) between AI-for-software-engineering and LLM/RL agent work. This suggests breadth across NLP and coding-agent research rather than a single narrow thread.
Collaboration60/100
Collaboration scores 60/100, based on 19 distinct coauthors in the record. Work is team-based within Microsoft Research Montréal: 12 recurring collaborators include Marc-Alexandre Côté, Alessandro Sordoni, Xingdi Yuan, and Young Jin Kim. Both reviewed 2026 papers are multi-authored group efforts, with Shi in a middle author position on FrogNano and last author on ProgramDistill, suggesting growing seniority within the team.
Activity64/100
Activity scores 64/100. The counts-by-year record shows 2 works in 2024, 4 in 2025, and 2 in 2026 — 8 works across the most recent three-year window, with 2025 the most productive visible year. Both 2026 catalog entries are arXiv preprints, indicating the current rhythm is centered on rapid preprint release of coding-agent research.
Reproducibility38/100
Reproducibility scores 38/100, a mean across 2 reviewed papers. ProgramDistill is rated low: the pipeline is strongly engineered but offers thin statistics and no public release yet. FrogNano is rated medium: the training recipe is credible, but validation-set reuse, base-score discrepancies, and missing significance testing temper its headline 61.5% SWE-bench Verified result.
Research summary
Zhengyan Shi is a researcher at Microsoft Research Montréal whose recent work sits at the intersection of large language model agents and software engineering. The OpenAlex-resolved record shows 8 works over the most recent three years — 2 in 2024, 4 in 2025, and 2 in 2026 — with an h-index of 2 and an i10-index of 0, a footprint consistent with an early-stage or recently refocused research line rather than an established citation record. The 19 distinct coauthors, 12 of them recurring collaborators, point to steady team-based research inside a large industrial lab. Topical keywords span Topic Modeling (6 works), Natural Language Processing Techniques (4), Domain Adaptation and Few-Shot Learning (3), and software engineering clusters (4 combined), while recent output splits evenly (0.5/0.5) between AI for software engineering (coding agents) and machine-learning approaches (LLM agents, reinforcement learning for software engineering). Two 2026 arXiv preprints reviewed by Evolve illustrate the direction: FrogNano trains a 4B coding agent via online task synthesis, reporting 61.5% on SWE-bench Verified, and ProgramDistill converts 26 interactive web apps into 4,063 replay-verified agent tasks. Both carry methodological caveats flagged in the reviews, with reproducibility readiness rated medium and low respectively, yielding a reproducibility mean of 38.
Collaboration pattern
The record lists 19 distinct coauthors with a stable core of 12 recurring collaborators at Microsoft Research Montréal, including Marc-Alexandre Côté, Alessandro Sordoni, Xingdi Yuan, Young Jin Kim, and Emiliano Penaloza. Both reviewed 2026 papers are group productions from this circle, with Shi positioned middle on FrogNano and last on ProgramDistill, consistent with team-based experimentation inside a large industrial lab rather than solo work.
Research keywords
Highlights
FrogNano: a 4B coding agent trained via online task synthesis
ProgramDistill: 26 web apps turned into 4,063 verified SWE tasks
Timeline
- 2024Earliest works in the visible record: 2 publications recorded in 2024.
- 2025Most productive visible year: 4 works recorded in 2025.
- 20262 works recorded, including both Evolve-reviewed arXiv preprints (FrogNano and ProgramDistill) on coding agents and benchmark construction.
Papers in the Evolve catalog
Frequent co-authors
About this profile
This profile is AI-structured by Evolve.Pub from public bibliographic data (OpenAlex) and Evolve's own paper reviews. It is not written or endorsed by the researcher unless the profile is claimed. Radar dimensions are computed from real bibliometrics with fixed formulas — raw figures are shown alongside; missing dimensions mean insufficient data, never zero. Errors? Claim the profile to correct or hide it.
- Bibliometric figures come from an OpenAlex-resolved profile; 'Zhengyan Shi' is not a rare name, and identity is resolved from public bibliographic data only.
- The citation footprint (h-index 2, i10-index 0) is very low, consistent with early-stage output or incomplete author disambiguation in OpenAlex.
- Evolve's qualitative verdicts cover only the 2 reviewed 2026 arXiv preprints, not the researcher's full body of work.
- No ORCID or country-level metadata was provided; the institutional affiliation is as stated in the input data.