LLM agent self-evolution benchmark builder; three 2026 arXiv benchmarks with honest, mostly negative findings
3
papers here
0
reviewed by Evolve
0
first-author
0
corresponding
—
works · OpenAlex
0
h-index · OpenAlex
Bibliometric figures come from OpenAlex (2026-10-17) and cover the author's entire public record, not just the papers listed here. Reproducibility signal comes from Evolve's own reviews.
Research profile
Output—
Not yet scored: author-level output data is insufficient in the current snapshot. This axis would measure total publication volume, venue distribution, and first/corresponding-author share beyond the three reviewed 2026 arXiv preprints.
Impact—
Not yet scored: citation and influence data is unavailable in this snapshot. This axis would measure citation counts, h-index trajectory, and uptake of the three benchmark papers by follow-up self-evolution and agent-harness research.
Focus—
Not yet scored: the focus dimension lacks sufficient structured data here. It would measure thematic concentration — whether the clustering around LLM self-evolution benchmarks persists or broadens toward other recorded OpenAlex topics such as topic modeling and robotics.
Collaboration76/100
Scored 76/100. The OpenAlex record lists 45 distinct coauthors, and Evolve's catalog shows a recurring core — Xinping Lei, Zexuan Wang, Wenhao Huang, Yuhao Wu, Jiajun Shi, Yuxuan Zhang, Jiaheng Liu, Shen Yan, Ge Zhang, Jingyuan Zhang and Junting Zhou — with Jingyuan Zhang appearing repeatedly. All three 2026 papers carry middle authorship without corresponding authorship, consistent with a contributor role inside larger team-based benchmark-building efforts.
Activity40/100
Scored 40/100. Recorded output is concentrated in 2026: counts-by-year shows 3 works in 2026 and the three-year window also totals 3, all arXiv preprints, with no earlier indexed activity in the snapshot. h-index and i10-index are both 0. The shape suggests a recent, burst-like start rather than sustained multi-year productivity — consistent with an early-career profile whose activity score reflects limited indexed history.
Reproducibility—
Not yet scored: no aggregate author-level reproducibility metric exists in this snapshot. Per-paper, Evolve rates the reviewed benchmarks medium, low, and medium for reproducibility readiness; the axis would measure code, data, and evaluation-protocol release across the full body of work.
Research summary
Jinkai Liu (name-resolved from OpenAlex public bibliographic data; no institution, country, or ORCID available) is an early-stage researcher focused on benchmarking LLM agent self-evolution. Evolve's catalog holds three 2026 arXiv papers, each with the author in a middle, non-corresponding position: HarnessDev asks whether LLMs can create and evolve their own agent harnesses; Aspire tests whether models can self-evolve from vague goals; S3Gym probes whether self-testing and self-judging become self-improvement. Evolve's reviews consistently credit rigorous, transparent benchmark protocols and honest reporting of largely negative results: evolution gains are inconsistent, self-improvement is task-dependent, and model-built harnesses still trail human-engineered systems on coding and search, including SWE-bench Pro. The OpenAlex topic profile leads with AI-based Problem Solving and Planning (3 works), then Topic Modeling (2), with single works in Reinforcement Learning in Robotics, Artificial Intelligence in Games, and Evolutionary Algorithms. Measured collaboration is broad — 45 distinct coauthors, a 76/100 collaboration score, and recurring partners such as Xinping Lei, Zexuan Wang, Wenhao Huang, and Ge Zhang. Citation indicators remain empty (h-index 0, i10-index 0) with only 3 works in the past three years, so this profile should be read as early-stage, scoped to indexed bibliographic data plus Evolve's three reviewed preprints.
Collaboration pattern
Collaboration is the strongest measured dimension (76/100): the OpenAlex record lists 45 distinct coauthors, and the reviewed papers share a recurring core including Xinping Lei, Zexuan Wang, Wenhao Huang, Yuhao Wu, Ge Zhang, and Jingyuan Zhang, who appears twice in the frequent-coauthor list, suggesting multiple joint works. All three 2026 papers place the author in a middle position without corresponding authorship, indicating work within larger team-based benchmark projects rather than solo-led studies.
Research keywords
Highlights
HarnessDev: testing whether LLMs can build their own agent harnesses
Aspire: a benchmark for self-evolution from vague goals
S3Gym: self-testing and self-judging do not guarantee self-improvement
Timeline
- 2026Publishes Aspire on arXiv — a benchmark testing whether LLMs can self-evolve from vague goals, judged against a hidden expert evaluation set.
- 2026Publishes S3Gym on arXiv — a benchmark finding that self-testing and self-judging translate into self-improvement only in a task-dependent way.
- 2026Publishes HarnessDev on arXiv — showing LLMs can create runnable agent harnesses but lag human systems on coding and search, with unstable evolution gains.
Papers in the Evolve catalog
Frequent co-authors
About this profile
This profile is AI-structured by Evolve.Pub from public bibliographic data (OpenAlex) and Evolve's own paper reviews. It is not written or endorsed by the researcher unless the profile is claimed. Radar dimensions are computed from real bibliometrics with fixed formulas — raw figures are shown alongside; missing dimensions mean insufficient data, never zero. Errors? Claim the profile to correct or hide it.
- Bibliometric indicators (h-index, i10-index, topics, coauthor counts) come from OpenAlex only and may be incomplete.
- Profile is name-resolved from public bibliographic data; Jinkai Liu is a common name, so the OpenAlex record may not fully match this researcher's works.
- No institutional affiliation, country, or ORCID was available to confirm identity.
- Evolve's qualitative assessments cover only three 2026 arXiv preprints; peer-reviewed versions and other works are not captured.