Early-career LLM agent evaluation researcher: controllable user simulation and synthetic decision labs
2
papers here
2
reviewed by Evolve
0
first-author
0
corresponding
—
works · OpenAlex
0
h-index · OpenAlex
Bibliometric figures come from OpenAlex (2026-10-16) and cover the author's entire public record, not just the papers listed here. Reproducibility signal comes from Evolve's own reviews.
Research profile
Output—
Output measures publication volume — total works, yearly output, and venue footprint. This axis is not yet scored: the OpenAlex record behind this name is too sparse (h-index and i10 index both reported as 0) to support a reliable estimate.
Impact—
Impact measures citation uptake — citations, h-index trajectory, and influence on follow-up work. No score is available: citation coverage for this OpenAlex record is too thin (both reported indices are 0), so any impact estimate would be guesswork.
Focus0/100
The focus score is 0, and the raw data explains why: field weight is split evenly at 0.5/0.5 between LLM agent evaluation and agent-based evaluation methodology, while five topics top out at two works each (topic modeling and speech/dialogue systems at 2; NLP techniques, usability, and software testing at 1). The portfolio is coherent — both 2026 preprints target agent evaluation — but the record is too young to concentrate.
Collaboration46/100
Collaboration scores 46. The recent window shows 9 distinct coauthors, and recurring names include Rahul Khedar, Eshita, Sneha Teja Sree Reddy Thondapu, Arup Kumar Das, Jitesh Chandra Mishra, Arun Menon, Avinash Karn, and Mouli V. Both reviewed 2026 preprints carry middle authorship without a corresponding-author role, consistent with a contributing team member rather than a solo lead.
Activity47/100
Activity scores 47. The record shows 4 works in the last three years, and the counts-by-year data concentrates them in 2026 (4 works) alongside a single earlier work recorded in 2015 — a long quiet gap followed by a burst. Both catalog papers are 2026 arXiv preprints, so current output is real but recent and thin.
Reproducibility55/100
Reproducibility scores 55, the mean over the 2 papers Evolve has assessed, both rated medium readiness. Review notes flag concrete gaps: the persona-vector paper lacks a flat-prompt baseline, significance tests, and any check beyond an LLM judge, while Augur rests on 50 real cases — a deliberate but small-n design.
Research summary
Mayank Malhotra is an early-stage researcher working at the intersection of large language model (LLM) agent evaluation and computational social simulation; the profile is resolved from public OpenAlex bibliographic data. Evolve has reviewed two of his 2026 arXiv preprints, both with middle authorship. A Three-Tier Persona Vector for Controllable User Simulation in Agentic Evaluation proposes a 23-dimension persona specification backed by production-scale evidence from 64,698 conversations, though the review flags no flat-prompt baseline, no significance tests, and LLM-judge-only evaluation. Augur: A Synthetic Decision Lab for Rehearsing Reactions to Product and Policy Changes builds a synthetic environment for rehearsing product and policy changes; across 50 real cases it argues that frontier-versus-open gaps are mostly prompt artifacts rather than capability differences — judged credible and honest, but small-n. His record shows 4 works in the last three years, 9 distinct coauthors, and topics spanning topic modeling, speech and dialogue systems, NLP techniques, usability, and software testing, with field weight split evenly (0.5/0.5) between LLM agent evaluation and agent-based evaluation methodology. Radar scores put collaboration at 46, activity at 47, and reproducibility at 55 (mean over 2 papers, both medium readiness); output and impact remain unscored because citation coverage is too sparse (h-index and i10 index both 0).
Collaboration pattern
The record lists 9 distinct coauthors in the recent window, with recurring collaborators including Rahul Khedar, Eshita, Sneha Teja Sree Reddy Thondapu, Arup Kumar Das, Jitesh Chandra Mishra, Arun Menon, Avinash Karn, and Mouli V. Both reviewed 2026 preprints place him in a middle authorship position without corresponding role, suggesting he works as a contributing member of a small team rather than a solo or supervising lead; the collaboration radar score is 46.
Research keywords
Highlights
Three-tier persona vector brings controllable user simulation to production scale
Augur decision lab argues frontier-vs-open gaps are mostly prompt artifacts
Timeline
- 2015Earliest work in the OpenAlex counts-by-year record: 1 publication.
- 2026Co-authored (middle position) the Augur synthetic decision lab preprint on arXiv for rehearsing reactions to product and policy changes.
- 2026Co-authored (middle position) the three-tier persona vector paper for controllable user simulation, with evidence from 64,698 conversations.
- 2026Most active year on record: 4 works in the counts-by-year data, with 9 distinct coauthors across the recent window.
Papers in the Evolve catalog
Frequent co-authors
About this profile
This profile is AI-structured by Evolve.Pub from public bibliographic data (OpenAlex) and Evolve's own paper reviews. It is not written or endorsed by the researcher unless the profile is claimed. Radar dimensions are computed from real bibliometrics with fixed formulas — raw figures are shown alongside; missing dimensions mean insufficient data, never zero. Errors? Claim the profile to correct or hide it.
- Bibliometric scores come from OpenAlex records only; this profile is name-resolved from public bibliographic data, and Mayank Malhotra is a relatively common name, so identity collisions are possible.
- Output and impact dimensions are unscored because citation coverage is too sparse (h-index and i10 index both reported as 0).
- Only two 2026 arXiv preprints (both middle authorship) have been reviewed by Evolve; no peer-reviewed venue, affiliation, or ORCID information is available.
- Radar scores reflect a very young record and should be re-read as the publication history accumulates.