Shen Yan

ByteDance (China) · China

Unclaimed · AI-structured profileOpenAlex

ByteDance researcher on LLM self-evolution: agent self-improvement benchmarks and reasoning-guided data augmentation

papers here

reviewed by Evolve

first-author

corresponding

—

works · OpenAlex

h-index · OpenAlex

Bibliometric figures come from OpenAlex (2026-10-17) and cover the author's entire public record, not just the papers listed here. Reproducibility signal comes from Evolve's own reviews.

Research profile

Output—

Output metrics are not yet available: publication counts beyond the recent snapshot, venue profile, and total works are insufficient in the current OpenAlex record, so this axis cannot be scored. It would measure cumulative research productivity.

Impact—

Impact cannot be assessed yet: the record shows h-index 0 and i10-index 0 with no citation history, which reflects the recency of this output rather than measured influence. This axis would track citations, citation velocity, and uptake of the benchmarks.

Focus—

Focus concentration cannot be scored: topic labels exist (Topic Modeling, NLP Techniques, Problem Solving and Planning), but the sample is too small to quantify a stable research focus. This axis would measure how concentrated output is across topics and venues.

Collaboration72/100

Collaboration scores 72 on the radar, backed by 36 distinct coauthors in the record and a recurring group of 12 frequent collaborators, including Jiajun Shi, Jiaheng Liu, Ge Zhang, Wenhao Huang, Yuhao Wu, and Renming Pang. All three cataloged papers are team-written arXiv efforts with Yan in a middle author position, suggesting a contributor role inside larger ByteDance-affiliated LLM research groups rather than solo or corresponding-author leadership.

Activity52/100

Activity scores 52, reflecting a narrow but concentrated recent record: OpenAlex lists 5 works in the last three years, all dated 2026 in this snapshot, and all three Evolve-cataloged papers appeared on arXiv in 2026. There is no earlier publication year in the record, so cadence over a longer horizon cannot be judged and the score stays moderate.

Reproducibility—

Reproducibility cannot be scored from bibliometric data alone. Evolve's manual reviews offer hints — S3Gym is rated medium readiness and Aspire and REER-PT low — but no standardized reproducibility score exists yet. This axis would measure code, data, and evaluation-harness release quality.

Research summary

Shen Yan is a researcher at ByteDance (China) whose reviewed work centers on how large language models can improve themselves — and on measuring honestly whether they actually do. Three 2026 arXiv papers in the Evolve catalog define this profile: Aspire asks whether models can self-evolve from vague goals, using a benchmark with a hidden expert evaluation set; S3Gym tests whether self-testing and self-judging translate into self-improvement for LLM agents; and REER-PT reverse-engineers reasoning traces to guide perplexity-optimized pre-training data augmentation. Evolve's reviews consistently note honest, sometimes negative findings — self-improvement gains appear rare and task-dependent — alongside a shared methodological weakness: small evaluation sets and single-run results without significance testing. OpenAlex records show 5 works in the last three years, all dated 2026 in this snapshot, with topic mass in Topic Modeling and Natural Language Processing Techniques (3 each), AI-based Problem Solving and Planning and Multimodal Machine Learning Applications (2 each), plus Software Engineering Research. Citation indicators are still at zero (h-index 0, i10-index 0), consistent with a very recent output record rather than an established citation footprint. With 36 distinct coauthors recorded, collaboration is the strongest radar axis (72), while activity scores 52. All three cataloged papers list Yan in a middle author position, none as corresponding author.

Collaboration pattern

Shen Yan works in sizeable, team-based settings: the record lists 36 distinct coauthors and a recurring core of 12 frequent collaborators, including Jiajun Shi, Jiaheng Liu, Ge Zhang, Wenhao Huang, Yuhao Wu, Yuxuan Zhang, and Renming Pang. All three cataloged 2026 arXiv papers place Yan in a middle author position with no corresponding-author role, consistent with contributing specialist work — benchmarks, evaluation, data augmentation — within ByteDance-affiliated LLM research groups.

Research keywords

LLM self-evolutionLLM agentsBenchmark designVague goalsSelf-testing and self-judgingSelf-improvementAutonomous post-trainingPre-training data augmentationPerplexity optimizationReasoning annotationsSynthetic dataIn-context learning

Highlights

Aspire: a benchmark for self-evolution from vague goals
Introduces a benchmark with a hidden expert evaluation set for testing whether LLMs can autonomously post-train toward vague goals; Evolve's review finds it honest but statistically thin — self-evolution gains are rare and the harness scores below the reference.
S3Gym: can self-testing and self-judging become self-improvement?
A benchmark probing whether LLM agents can convert self-testing and self-judging into real gains; the protocol is sound and findings honest — improvement is task-dependent — but tiny 3-episode evaluations lack statistical rigor. Reproducibility readiness is rated medium.
REER-PT: perplexity-guided pre-training data augmentation via reverse-engineered reasoning
Reverse-engineers chain-of-thought style reasoning annotations to augment pre-training data under a perplexity objective; perplexity drops and 680M-model gains look strong, but results come from single runs without significance tests, and reproducibility readiness is low.

Timeline

2026Aspire: Can Models Self-Evolve from Vague Goals? released on arXiv — a benchmark with a hidden expert evaluation set for vague-goal LLM self-evolution.
2026S3Gym released on arXiv, testing whether LLM agents can turn self-testing and self-judging into self-improvement.
2026REER-PT released on arXiv, applying reverse-engineered reasoning annotations to perplexity-guided pre-training data augmentation.
2026OpenAlex snapshot records 5 works in the last three years, all dated 2026, with 36 distinct coauthors.

Papers in the Evolve catalog

Frequent co-authors

Jiajun Shi×3 Jiaheng Liu×3 Ge Zhang×3 Wenhao Huang×3 Yuhao Wu×2 Yuxuan Zhang×2 Xinping Lei×2 Zexuan Wang×2 Jinkai Liu×2 Haoran Que×1 Ting Huang×1 Renming Pang×1

About this profile

This profile is AI-structured by Evolve.Pub from public bibliographic data (OpenAlex) and Evolve's own paper reviews. It is not written or endorsed by the researcher unless the profile is claimed. Radar dimensions are computed from real bibliometrics with fixed formulas — raw figures are shown alongside; missing dimensions mean insufficient data, never zero. Errors? Claim the profile to correct or hide it.

Bibliometric indicators (h-index, i10-index, topics, coauthor counts) come from an OpenAlex snapshot only; citation data may lag, and this common-style name may be mis-resolved.
Qualitative assessment draws only on the three Evolve-reviewed arXiv papers (all 2026); other works by this researcher are not reviewed here.
All cataloged papers list a middle, non-corresponding author position, so individual contribution cannot be isolated from team output.
Identity is resolved from public bibliographic data (OpenAlex) without ORCID; "Shen Yan" is a common name and records may include homonyms.