Nanjing University researcher building benchmarks and training paradigms for LLM coding agents and agent self-evolution
5
papers here
2
reviewed by Evolve
0
first-author
0
corresponding
—
works · OpenAlex
0
h-index · OpenAlex
Bibliometric figures come from OpenAlex (2026-10-17) and cover the author's entire public record, not just the papers listed here. Reproducibility signal comes from Evolve's own reviews.
Research profile
Output—
Output is unscored: the OpenAlex record carries no reliable publication-volume indicators beyond yearly counts, with h-index and i10-index both registering zero for this very recent profile, so total productivity cannot yet be measured.
Impact—
Impact is unscored: citation counts, h-index and i10-index are zero or unavailable in the OpenAlex snapshot — expected for 2026 arXiv preprints. This axis would measure citations, adoption and scholarly influence once citations accrue.
Focus0/100
The focus score is 0/100, the low end of the radar. OpenAlex assigns this record an even 50/50 field split between machine learning for LLM agents and software-engineering evaluation of coding agents, and topics spread across five clusters — Topic Modeling and Software Engineering Research (7 each), AI-based Problem Solving and Planning (5), Reinforcement Learning in Robotics and Artificial Intelligence in Games (3 each) — a profile that reads as deliberately broad rather than narrowly specialized.
Collaboration86/100
Collaboration scores 86/100. The OpenAlex record lists 73 distinct co-authors across 15 works in the last three years, with 12 recurring collaborators — including Zexuan Wang, Jinkai Liu, Wenhao Huang, Ge Zhang and Jiaheng Liu — indicating stable team-based work. All five Evolve-cataloged 2026 preprints carry Lei in a middle author position, consistent with contributing to larger benchmark-building teams rather than solo or corresponding-author leadership.
Activity81/100
Activity scores 81/100, driven by a sharp recent ramp: yearly counts rise from 1 work in 2022 to 4 in 2025 and 11 in 2026, with 15 works in the last three years. Five of the 2026 outputs are arXiv preprints already reviewed in the Evolve catalog, indicating an intensive, benchmark-heavy publishing phase currently underway rather than a long cumulative record.
Reproducibility78/100
Reproducibility scores 78/100, computed by Evolve across 2 reviewed papers with a mean of 78. GameLogicBench rates high — a rigorously validated, deterministic tick-level benchmark — while FLARE, HarnessDev and S3Gym rate medium and Aspire low, with Evolve noting single-run or small-sample evaluation protocols in several of these studies.
Research summary
Xinping Lei is a researcher at Nanjing University whose work centers on evaluating and training large language model agents for software engineering. OpenAlex bibliographic data, from which this profile is name-resolved, records 16 works with a steep recent ramp — 1 in 2022, 4 in 2025, and 11 in 2026 — including 15 works in the last three years and 73 distinct co-authors, a strongly collaborative mode. Topic assignments split between Topic Modeling and Software Engineering Research (7 works each), with AI-based Problem Solving and Planning (5), Reinforcement Learning in Robotics (3), and Artificial Intelligence in Games (3); the field distribution divides evenly between machine learning for LLM agents and software-engineering evaluation of coding agents. Five 2026 arXiv preprints appear in the Evolve catalog, all with Lei in a middle author position: FLARE (dense generative-reward-model supervision for coding agents), GameLogicBench (tick-level game-logic benchmark, high reproducibility readiness), HarnessDev, Aspire, and S3Gym. Citation indicators (h-index, i10-index) currently register as zero in OpenAlex, so output and impact are left unscored rather than estimated, and Evolve's reproducibility reviews of two papers average 78/100.
Collaboration pattern
This is a strongly collaborative profile: 73 distinct co-authors across 15 works in the last three years, with a stable core of 12 frequent collaborators including Zexuan Wang, Jinkai Liu, Wenhao Huang, Yuhao Wu, Ge Zhang, Jiaheng Liu and Junting Zhou. On all five Evolve-cataloged 2026 preprints, Lei appears in a middle author position and not as corresponding author, consistent with team-based benchmark and paradigm-building work.
Research keywords
Highlights
FLARE: dense generative-reward-model supervision for long-horizon coding agents
GameLogicBench: tick-level game-logic evaluation for coding agents
HarnessDev: can LLMs build and evolve their own agent harness?
Aspire: honest negative results on vague-goal self-evolution
S3Gym: self-testing and self-judging only sometimes yield self-improvement
Timeline
- 2022First recorded work appears in the OpenAlex record, with 1 publication in the year.
- 2025Publication activity resumes after a gap, with 4 works recorded in the year.
- 2026Peak activity: 11 works recorded, bringing the last-three-year total to 15.
- 2026Evaluation benchmarks appear: GameLogicBench (tick-level game-logic testing of coding agents) and HarnessDev (LLM-created and evolved agent harnesses).
- 2026Self-evolution and training line lands: Aspire (vague-goal self-evolution), S3Gym (self-testing and self-judging), and FLARE (full-lifecycle dense supervision via generative reward model).
Papers in the Evolve catalog
FLARE: A Full-Lifecycle Dense Supervision Paradigm for Long-Horizon Coding Agents via Generative Reward Model
GameLogicBench: Evaluating Coding Agents on Runtime Game Logic with Tick-Level State Assertions
HarnessDev: Can LLMs Create and Evolve Their Own Agent Harness?
Aspire: Can Models Self-Evolve from Vague Goals?
S3Gym: Can LLMs Turn Self-Testing and Self-Judging into Self-Improvement?
Frequent co-authors
About this profile
This profile is AI-structured by Evolve.Pub from public bibliographic data (OpenAlex) and Evolve's own paper reviews. It is not written or endorsed by the researcher unless the profile is claimed. Radar dimensions are computed from real bibliometrics with fixed formulas — raw figures are shown alongside; missing dimensions mean insufficient data, never zero. Errors? Claim the profile to correct or hide it.
- Bibliometric indicators come from OpenAlex only (identity: openalex); the profile is name-resolved from public bibliographic data, so same-name researchers may be conflated.
- No ORCID was provided, so author disambiguation relies solely on the OpenAlex record.
- h-index and i10-index register as zero and the output and impact axes are unscored; nearly all cataloged works are 2026 arXiv preprints for which citations have not yet accrued.
- Evolve assessments cover only the five reviewed 2026 preprints; other works in the OpenAlex record were not evaluated.