LLM evaluation and agentic-AI researcher behind a payment-domain benchmark and a large code-derived skill bank
2
papers here
2
reviewed by Evolve
0
first-author
0
corresponding
—
works · OpenAlex
0
h-index · OpenAlex
Bibliometric figures come from OpenAlex (2026-10-16) and cover the author's entire public record, not just the papers listed here. Reproducibility signal comes from Evolve's own reviews.
Research profile
Output—
Not yet scored: publication-volume metrics (total works, yearly distribution, author position) are incomplete for this OpenAlex record, so overall output cannot be sized. This axis would measure the number and cadence of papers, including preprints and venue types.
Impact—
Not yet scored: citation-based measures (h-index, i10-index, citation counts) are missing or unresolved in the underlying record, so influence cannot be assessed. This axis would measure citations, uptake, and downstream use of this researcher's benchmarks and methods.
Focus0/100
The focus score is 0, reflecting an even 50/50 split between natural language processing / LLM evaluation and artificial intelligence (LLM-based agents / software engineering). Topic counts confirm the breadth: NLP techniques (4), topic modeling (3), reinforcement learning in robotics (2), AI problem solving and planning (2), and software engineering research (1). Work spans evaluation, agents, and robotics rather than a single niche.
Collaboration58/100
Collaboration scores 58, based on 17 distinct coauthors in the record. Twelve recur frequently, including Sijie Dong, Wei Ren, Xuanwei Hu, Jiawei Luo, Zifan Wang, Xiaoyun Feng, Hui Cai, Lyuxin Xue, Peng Lu, Xin Zhang, Wei Wu, and Yongqi Tong. Both cataloged 2026 papers list this researcher in a middle author position, consistent with team-based rather than solo work.
Activity61/100
Activity scores 61. The record shows 7 works in 2026 and 7 works over the last three years, so all indexed output falls within a single recent year — a concentrated recent burst rather than a long publishing history. Both reviewed papers are 2026 arXiv preprints.
Reproducibility78/100
Reproducibility scores 78, a mean over 2 papers with reviewed readiness. The payment-domain benchmark BENCHCOMPASS rates high, with a 306-item design separating knowledge, context use, and attack robustness; the skill-synthesis paper rates medium, with +11.7% relative gains across 8 benchmarks and 9 model settings but missing contamination and significance checks.
Research summary
Jianshe Li is an early-career researcher working at the intersection of LLM evaluation and LLM-based agents, with an OpenAlex record that splits evenly (50/50) between natural language processing / LLM evaluation and artificial intelligence (LLM-based agents / software engineering). The profile is name-resolved from public bibliographic data. Two 2026 arXiv papers anchor the record. BENCHCOMPASS is a 306-item payment-domain benchmark that separates domain knowledge, context use, and task-input attack robustness, and converts benchmark scores into signals for training and harness decisions; Evolve rates its reproducibility readiness high while noting that small subsets and judge-family overlap temper the claims. The second paper mines code at scale to build a 1M-skill bank for agentic intelligence, reporting +11.7% relative improvement across 8 benchmarks and 9 model settings; its reproducibility is rated medium, with contamination and significance checks still missing. Activity metrics show a concentrated recent burst: 7 works in 2026, all within the last three years, and 17 distinct coauthors of whom 12 recur frequently. Topic data adds breadth beyond the catalog papers — topic modeling (3), reinforcement learning in robotics (2), and AI problem solving and planning (2). Citation-based measures are not yet available, so output and impact dimensions remain unscored.
Collaboration pattern
Jianshe Li works in teams: the OpenAlex record lists 17 distinct coauthors, 12 of whom recur, including Sijie Dong, Wei Ren, Xuanwei Hu, Jiawei Luo, Zifan Wang, Xiaoyun Feng, Hui Cai, Lyuxin Xue, Peng Lu, Xin Zhang, Wei Wu, and Yongqi Tong. Both 2026 catalog papers place this researcher in a middle author position, suggesting contribution within larger groups rather than first- or corresponding-author leadership. The collaboration dimension scores 58.
Research keywords
Highlights
BENCHCOMPASS: a payment-domain benchmark that turns scores into signals
One-million-skill bank mined from code for agentic intelligence
Evaluation-first agenda with strong reproducibility signals
Across the two reviewed papers, the reproducibility mean is 78 over 2 papers: one high-readiness benchmark and one medium-readiness method paper, whose gains are reported across 8 benchmarks and 9 model settings, with evaluation gaps flagged rather than hidden.
Timeline
- 2026BENCHCOMPASS, a 306-item payment-domain LLM benchmark, appears on arXiv with Jianshe Li in a middle author position.
- 2026Grounded Skill Synthesis from Code at Scale appears on arXiv, reporting a 1M-skill bank and +11.7% relative gains across 8 benchmarks and 9 model settings.
- 2026OpenAlex indexes 7 works for this year — all of the output recorded in the last three years — alongside 17 distinct coauthors.
Papers in the Evolve catalog
Frequent co-authors
About this profile
This profile is AI-structured by Evolve.Pub from public bibliographic data (OpenAlex) and Evolve's own paper reviews. It is not written or endorsed by the researcher unless the profile is claimed. Radar dimensions are computed from real bibliometrics with fixed formulas — raw figures are shown alongside; missing dimensions mean insufficient data, never zero. Errors? Claim the profile to correct or hide it.
- Bibliometric scores and identity resolution come from OpenAlex only; no institutional or ORCID verification is in scope.
- h-index and i10-index display as 0, which likely reflects citation-coverage gaps in the source rather than measured zero impact; the impact dimension is therefore left unscored.
- Both cataloged papers are 2026 arXiv preprints with this researcher in a middle, non-corresponding position; no peer-reviewed venue data is in scope.
- Jianshe Li carries common-name risk: this profile is name-resolved from public bibliographic data without author-disambiguation confirmation.