LLM-based retrieval evaluation researcher at JPMorgan Chase auditing coverage and cost in production RAG systems
2
papers here
2
reviewed by Evolve
1
first-author
0
corresponding
—
works · OpenAlex
—
h-index · OpenAlex
Bibliometric figures come from OpenAlex (2026-10-16) and cover the author's entire public record, not just the papers listed here. Reproducibility signal comes from Evolve's own reviews.
Research profile
Output—
No publication-count data is available in this snapshot, so the output axis is unscored. Once present, it would measure the volume and venue spread of this researcher's publications over time, beyond the two arXiv preprints reviewed here.
Impact—
Citation counts and influence metrics are not available in this snapshot, so the impact axis is unscored. It would measure how often this researcher's evaluation methods are cited, adopted, or built upon by other RAG evaluation work.
Focus0/100
The field distribution splits this researcher evenly between 'natural language processing / information retrieval' (0.5) and 'information retrieval (LLM-based evaluation for RAG)' (0.5), and the computed focus score is 0 — the radar's way of flagging that recorded output spans two labeled areas rather than one. In practice the two are adjacent: both reviewed papers sit inside LLM-based retrieval evaluation for RAG, so the low score reflects taxonomy split more than genuine topic scattering.
Collaboration32/100
The collaboration dimension scores 32 out of 100, computed from a small raw base of 4 distinct coauthors in the snapshot. Recurring partners are Hanoz Bhathena, Saket Sharma, and Max Nelson, all within an industrial setting at JPMorgan Chase. The pattern suggests a compact, stable internal team rather than broad external networking: authorship alternates between first-author lead on Re:CAP and middle authorship on the pooled-evaluation paper.
Activity—
Recent output cadence cannot be assessed from this snapshot, so the activity axis is unscored. It would measure publication frequency and recency across successive years; both reviewed papers date to 2026, which alone cannot establish a trend.
Reproducibility55/100
Reproducibility scores 55, the mean readiness rating across the 2 papers Evolve has scored (reproPapers = 2), with both rated 'medium'. Both works are arXiv preprints proposing LLM-based evaluation pipelines, where reproduction hinges on released prompts, judge configurations, and pooled judgment data; the verdicts flag concrete limits — a TREC-COVID recall regression in Re:CAP and sparse gold qrels in the pooled-selection study — consistent with the mid-range score.
Research summary
Aviral Joshi is a researcher at JPMorgan Chase & Co (United States) working at the intersection of natural language processing and information retrieval, focused on LLM-based evaluation for retrieval-augmented generation (RAG). Evolve's catalog contains two 2026 arXiv preprints. The first, Re:CAP — Auditing Retrieval Coverage in Production RAG Pipelines, is first-authored; it claims that reference-free LLM probing recovers relevant material that flat RAG retrieval misses, and Evolve's verdict calls it unusually rigorous and honest while noting that TREC-COVID recall regresses. The second, Incremental Pooled LLM Evaluation for Cost-Effective Retrieval Model Selection, is a middle-authorship paper claiming that pooled LLM judging with incremental pool growth delivers stable retrieval model selection at 4.9× lower cost; validation is described as solid, though sparse gold qrels limit certainty. The radar's present dimensions rest on a small base: 4 distinct coauthors yield a collaboration score of 32, an even 0.5/0.5 field split across two adjacent IR areas corresponds to a focus score of 0, and a reproducibility mean of 55 across 2 scored papers reflects medium readiness for both. The output, impact, and activity axes lack underlying bibliometric data and are left unscored. Taken together, this is an early-stage, industrially grounded profile of a researcher building evaluation and auditing infrastructure for production RAG systems.
Collaboration pattern
Collaboration is compact and internal: the snapshot records 4 distinct coauthors, with Hanoz Bhathena, Saket Sharma, and Max Nelson recurring (Max Nelson appears twice in the source list). All activity sits inside JPMorgan Chase, with no external or academic-affiliation partners visible. Authorship roles vary — first author on Re:CAP, middle author on the pooled-evaluation paper — suggesting a small team that shares lead duties rather than a wide co-authorship network.
Research keywords
Highlights
Re:CAP: auditing retrieval coverage in production RAG pipelines
4.9× cheaper retrieval model selection via pooled LLM judging
A candor thread across both papers: reporting the method's own failure modes
Timeline
- 2026First-authored Re:CAP appears on arXiv, proposing reference-free LLM probing to audit retrieval coverage in production RAG pipelines.
- 2026Co-authors the incremental pooled LLM evaluation study, claiming stable retrieval model selection at 4.9× lower cost.
- 2026Both recorded outputs (both 2026, both arXiv) fall in LLM-based retrieval evaluation for RAG, under JPMorgan Chase affiliation.
Papers in the Evolve catalog
Frequent co-authors
About this profile
This profile is AI-structured by Evolve.Pub from public bibliographic data (OpenAlex) and Evolve's own paper reviews. It is not written or endorsed by the researcher unless the profile is claimed. Radar dimensions are computed from real bibliometrics with fixed formulas — raw figures are shown alongside; missing dimensions mean insufficient data, never zero. Errors? Claim the profile to correct or hide it.
- Bibliometric scores are derived from an OpenAlex-based snapshot only; output, impact, and activity metrics are absent, so those radar axes are unscored rather than estimated.
- No ORCID is linked; the profile is name-resolved from public bibliographic data, so identity certainty is limited.
- The evidence base is 2 arXiv preprints (both 2026) reviewed by Evolve; no peer-reviewed venue record is in scope.
- Radar dimensions rest on very small samples (2 scored papers, 4 distinct coauthors, and a frequentCoauthors list containing a duplicated entry), so scores are statistically fragile.