Mingwei Liu

Sun Yat-sen University · China

Unclaimed · AI-structured profileORCID OpenAlex

Sun Yat-sen University researcher building benchmarks and efficient evaluation for LLM coding agents

papers here

reviewed by Evolve

first-author

corresponding

—

works · OpenAlex

h-index · OpenAlex

Bibliometric figures come from OpenAlex (2026-10-17) and cover the author's entire public record, not just the papers listed here. Reproducibility signal comes from Evolve's own reviews.

Research profile

Output—

No publication-output metrics were available for this axis at snapshot time, so it is unscored. It would normally measure total works, venue quality, and output consistency across the career rather than recent-year counts alone.

Impact—

Citation-impact data was insufficient for this axis at snapshot time, so it is unscored. It would measure citation counts, h-index trajectory, and uptake of the benchmark and evaluation artifacts in follow-up work.

Focus100/100

This is the strongest axis in the radar (score 100). The field distribution collapses to a single field — software engineering (AI for code) — and the topic profile reinforces it: Software Engineering Research (21), Software Testing and Debugging Techniques (11), Software Engineering Techniques and Practices (6), with Topic Modeling (10) and NLP Techniques (4) as supporting methods. All three Evolve-reviewed papers sit on the LLM-for-SE line, showing an unusually tight research identity.

Collaboration55/100

The collaboration score is 55. The recent three-year record lists 15 distinct coauthors, and the frequent-coauthor list is anchored by a stable Sun Yat-sen cluster — Yanlin Wang, Kefeng Duan, Dewu Zheng, Jiachi Chen, Ensheng Shi, Zibin Zheng — plus external collaborators such as Terry Yue Zhuo. All three Evolve-reviewed papers place this researcher in a middle author position without corresponding status, consistent with contributing specialist work inside a larger team rather than leading it.

Activity91/100

Activity scores 91, the second-highest axis. The counts-by-year series runs 1 (2024), 1 (2025), 20 (2026), and recent3yWorks totals 22 — meaning essentially the entire visible record was produced in a single year. Twenty works in 2026, including three arXiv preprints reviewed by Evolve, indicates an intensive current sprint, though the steepness of the ramp means longevity is not yet demonstrated.

Reproducibility55/100

Reproducibility scores 55 (reproMean), but coverage is thin: the score is computed from a single evaluated paper (reproPapers: 1). Evolve rates all three catalog entries medium for reproducibility, flagging single-run statistics in SWE-Gate, small agent pools in the IRT-based evaluation, and missing significance tests and selective-RAG baselines in the retrieval work. The axis should be read as a provisional signal rather than a settled track record.

Research summary

Mingwei Liu is a researcher at Sun Yat-sen University (China) working at the intersection of software engineering and large language models. The OpenAlex-derived record shows a concentrated, fast-moving profile: 22 works in the last three years, with per-year counts of 1 (2024), 1 (2025), and 20 (2026), and topic mass led by Software Engineering Research (21), Software Testing and Debugging Techniques (11), and Topic Modeling (10). An h-index of 2 and i10-index of 0 reflect a record that has just accelerated rather than an established citation history. Three 2026 arXiv papers reviewed by Evolve outline the agenda: SWE-Gate argues that passing functional tests is not enough for software engineering agents and introduces a 303-instance benchmark for review-constraint compliance; a trajectory-aware evaluation study uses trajectory summaries as privileged information within item response theory to make SWE-agent benchmarking cheaper; and a critical token-aware retrieval method reports 8.4–15.4% relative Pass@5 gains for repository-level code generation. All three entries are middle-author preprints rated medium for reproducibility, with Evolve flagging single-run statistics, small agent pools, and missing significance tests. Collaboration is anchored in a stable Sun Yat-sen-centered group among 15 distinct coauthors.

Collaboration pattern

The record lists 15 distinct coauthors over the recent three-year window. The frequent-coauthor list is dominated by a stable Sun Yat-sen cluster — Yanlin Wang, Kefeng Duan, Dewu Zheng, Jiachi Chen, Ensheng Shi, Xilin Liu, Yuchi Ma, Zibin Zheng — with external collaborators such as Terry Yue Zhuo. All three Evolve-reviewed papers are middle-author, non-corresponding works, suggesting this researcher currently contributes specialist expertise within a larger team rather than leading projects.

Research keywords

LLM agentsSWE-BenchRepository-level code generationRepository-level repairCode reviewBenchmark constructionConstraint complianceItem response theoryEfficient benchmarkingRetrieval-augmented generationDense retrievalSoftware testing

Highlights

SWE-Gate: functional tests alone miss review constraints for coding agents
The paper argues that passing functional tests is insufficient for software engineering agents, and introduces the SWE-Gate benchmark with 303 instances that capture review constraints. Evolve finds the benchmark supports the claim, but notes single-run statistics and synthetic-task realism need scrutiny.
PTA-IRT: cheaper SWE-agent evaluation via trajectory-aware IRT
This work feeds trajectory summaries into item response theory as privileged information, yielding consistent wins across 4 benchmarks while lowering the cost of SWE-agent benchmarking. Evolve cautions that the small agent pools limit certainty in the reported gains.
Critical token-aware retrieval lifts repository-level code generation
The method triggers retrieval only at critical tokens during repository-level code generation, reporting 8.4–15.4% relative Pass@5 improvements. Evolve deems the gains plausible but notes the absence of significance tests and selective-RAG baselines.

Timeline

2024First year in the OpenAlex counts-by-year series, with 1 recorded work.
20251 recorded work, in the year before the sharp output ramp.
202620 recorded works, including three arXiv preprints reviewed by Evolve: SWE-Gate, trajectory-aware agent evaluation, and critical token-aware retrieval for repository-level code generation.

Papers in the Evolve catalog

Frequent co-authors

Yanlin Wang×3 Kefeng Duan×2 Dewu Zheng×2 Jiachi Chen×2 Ensheng Shi×2 Xilin Liu×2 Yuchi Ma×2 Zibin Zheng×2 Terry Yue Zhuo×1 Jianxing Yu×1 Xin He×1 Jiachi Chen×1

About this profile

This profile is AI-structured by Evolve.Pub from public bibliographic data (OpenAlex) and Evolve's own paper reviews. It is not written or endorsed by the researcher unless the profile is claimed. Radar dimensions are computed from real bibliometrics with fixed formulas — raw figures are shown alongside; missing dimensions mean insufficient data, never zero. Errors? Claim the profile to correct or hide it.

Bibliometric indicators (h-index, i10-index, counts by year, topics, coauthors) come from an OpenAlex-derived snapshot; identity is resolved from public bibliographic data for a common-style name.
The output and impact radar axes are unscored because the underlying metrics were insufficient at snapshot time; h-index (2) and i10-index (0) are quoted verbatim but not turned into impact claims.
All three reviewed papers are 2026 arXiv preprints with middle, non-corresponding authorship; no peer-reviewed venue record appears in the input data.
The reproducibility score (55) rests on a single evaluated paper (reproPapers: 1), so coverage is too thin for a track-record claim.