JPMorgan Chase NLP researcher advancing LLM-as-judge retrieval evaluation and coverage auditing for production RAG.
2
papers here
2
reviewed by Evolve
0
first-author
0
corresponding
—
works · OpenAlex
—
h-index · OpenAlex
Bibliometric figures come from OpenAlex (2026-10-16) and cover the author's entire public record, not just the papers listed here. Reproducibility signal comes from Evolve's own reviews.
Research profile
Output—
Output data is insufficient: total publication counts, venues, and yearly volume are not available, so this axis — which would measure overall scholarly production volume across indexed venues — is left unscored.
Impact—
Impact indicators such as citation counts, h-index, and downstream adoption are absent from the available data; this axis would measure how widely the two 2026 RAG-evaluation papers are cited and reused, and remains unscored.
Focus0/100
Focus scores 0 on the 0-100 radar, reflecting a field distribution split exactly evenly — 0.5 natural language processing / information retrieval and 0.5 information retrieval (LLM-based evaluation for RAG). Both recorded 2026 papers target the same underlying problem, though: LLM-based evaluation of RAG retrieval. The zero therefore reads as a two-bucket split in raw field labels rather than genuinely scattered work; the two papers are methodologically contiguous.
Collaboration32/100
Collaboration scores 32, with 4 distinct coauthors recorded across the two indexed papers. Frequent collaborators are Aviral Joshi, Hanoz Bhathena, and Max Nelson, and on both 2026 papers Sharma sits in the last author position without corresponding-author status. The small, stable co-author pool suggests a compact industry team producing RAG-evaluation work together rather than a wide external network; the low score is consistent with that narrow reach.
Activity—
Activity data is insufficient: per-year publication cadence and recent-output trends cannot be computed from the snapshot. This axis would measure publishing rhythm over time — including whether 2026 marks a burst or a continuation — and is left unscored.
Reproducibility55/100
Reproducibility is the strongest scored axis at 55 (reproMean across 2 papers, both rated medium reproducibility readiness). Evolve's reviews support this middling reading: Re:CAP is called unusually rigorous and honest yet reports a TREC-COVID recall regression, while the pooled-evaluation paper is described as solidly validated but limited by sparse gold qrels. Both artifacts appear reusable in principle, with caveats on each side.
Research summary
Saket Sharma is an applied natural language processing researcher at JPMorgan Chase & Co (United States) whose visible research centers on evaluation methodology for retrieval-augmented generation (RAG) systems. The Evolve catalog lists two 2026 arXiv papers, on both of which Sharma appears as last author without corresponding-author status. The first, Re:CAP, audits retrieval coverage in production RAG pipelines using reference-free LLM probing; Evolve's review calls it unusually rigorous and honest while noting a recall regression on TREC-COVID. The second proposes incremental pooled LLM evaluation for cost-effective retrieval model selection, claiming stable selection at 4.9x lower cost; the review judges the validation solid but flags sparse gold qrels as limiting certainty. Bibliometrically the profile is narrow: 4 distinct coauthors (frequent collaborators include Aviral Joshi, Hanoz Bhathena, and Max Nelson), a collaboration score of 32, and a field distribution split evenly between natural language processing / information retrieval and LLM-based evaluation for RAG. Reproducibility is the strongest scored axis at 55, computed across the 2 papers, each rated medium reproducibility readiness. Output, impact, and activity axes lack sufficient data and remain unscored, so this should be read as an early-stage, industry-anchored profile.
Collaboration pattern
The recorded collaboration footprint is compact: 4 distinct coauthors across the two indexed 2026 papers, giving a collaboration score of 32. Frequent collaborators are Aviral Joshi, Hanoz Bhathena, and Max Nelson. On both papers Sharma appears as last author — a placement conventionally associated with senior or supervisory roles — but is not corresponding author. The pattern points to a small, stable team at JPMorgan Chase iterating on RAG evaluation together, with little external network visible in the data.
Research keywords
Highlights
Auditing retrieval coverage without gold references
Stable retrieval model selection at 4.9x lower cost
A coherent LLM-as-judge evaluation agenda
Both cataloged 2026 papers apply LLM-as-judge methodology to RAG retrieval evaluation — one auditing coverage in production pipelines, the other cutting selection cost — with Sharma in the last-author position on each.
Timeline
- 2026Published Re:CAP on arXiv, auditing retrieval coverage in production RAG pipelines, with Sharma as last author.
- 2026Published incremental pooled LLM evaluation for cost-effective retrieval model selection on arXiv, again as last author.
- 2026Both papers entered the Evolve catalog rated medium reproducibility readiness, yielding a reproducibility mean of 55.
Papers in the Evolve catalog
Frequent co-authors
About this profile
This profile is AI-structured by Evolve.Pub from public bibliographic data (OpenAlex) and Evolve's own paper reviews. It is not written or endorsed by the researcher unless the profile is claimed. Radar dimensions are computed from real bibliometrics with fixed formulas — raw figures are shown alongside; missing dimensions mean insufficient data, never zero. Errors? Claim the profile to correct or hide it.
- Dimension scores come from a narrow public bibliometric snapshot; output, impact, and activity axes lack sufficient data and are intentionally unscored.
- Coverage is limited to the two 2026 arXiv papers reviewed by Evolve; any other publications by this researcher are not reflected.
- Identity is name-resolved from public bibliographic data via ORCID 0000-0002-2541-0079; 'Saket Sharma' is a relatively common name.
- Both papers are arXiv preprints rated medium reproducibility readiness; the reproducibility score (55) rests on only two reviewed papers.