Urja Pawar

Brooklyn Navy Yard · United States

Unclaimed · AI-structured profileOpenAlex

Early-career LLM evaluation researcher: self-report faithfulness, hallucination detection, behavioral explainability

papers here

reviewed by Evolve

first-author

corresponding

—

works · OpenAlex

h-index · OpenAlex

Bibliometric figures come from OpenAlex (2026-10-16) and cover the author's entire public record, not just the papers listed here. Reproducibility signal comes from Evolve's own reviews.

Research profile

Output—

No output score is available yet. This axis would measure total publication volume, venue tiers, and the balance between peer-reviewed papers and preprints. OpenAlex records 7 recent works, but the underlying data is insufficient to compute the dimension.

Impact—

No impact score is available. This axis would measure citations, h-index, and downstream uptake of the work. The snapshot records h-index 0 and i10-index 0 with no citation accumulation yet, so impact cannot be scored honestly at this stage.

Focus0/100

The focus score is 0, the lowest on this profile, and the raw data explains why: OpenAlex's field distribution splits evenly at one third each across LLM uncertainty quantification, LLM evaluation and AI safety, and explainable AI / LLM evaluation, while output spreads across five topic clusters. This axis measures topic concentration, not quality; the portfolio reads as a deliberate multi-angle attack on one evaluation theme rather than aimless drift.

Collaboration48/100

Collaboration scores 48 on the 0-100 scale, with 10 distinct coauthors recorded in the recent window. Frequent collaborators include Rajitha Ramanayake, Owen O'Neill, Nabeel Kemal, Houssem Chatbri, and Phil Blandfort. Across the three reviewed papers Pawar is first author twice and last author once, indicating a core, team-embedded contributor rather than an isolated author; the mid-range score reflects a focused, recurring group rather than a broad network.

Activity61/100

Activity scores 61. OpenAlex counts 7 works in the recent three-year window — 1 in 2025 rising to 6 in 2026 — a steep single-year acceleration. All three Evolve-reviewed papers are 2026 arXiv preprints, confirming output concentrated in the most recent year. The pace is typical of an early career: modest absolute volume, but the 2026 cluster signals a research program that is intensifying quickly.

Reproducibility70/100

Reproducibility is the strongest scored axis at 70, computed as a mean over 3 reviewed papers. Evolve rates the pre-registered, 12-model self-report study 'high' for reproducibility readiness, and the hallucination-detection and explanation-faithfulness papers 'medium' — the latter limited by synthetic data, three-trial scores, and no data release. Consistent methodological disclosure across all three papers underpins this above-average score.

Research summary

Urja Pawar is an early-stage researcher in large language model evaluation, based at the Brooklyn Navy Yard in the United States. OpenAlex records seven works in the recent three-year window — one in 2025 and six in 2026 — while citation indicators (h-index 0, i10-index 0) have not yet accumulated, consistent with a career at its start. Evolve has reviewed three 2026 arXiv preprints, all with leading authorship roles: first author on 'From Tokens to Semantics', which fuses semantic entropy with token log-probabilities for black-box hallucination detection; first author on 'Necessary or Sufficient?', which tests whether the top-3 features LLMs cite match measured necessity and sufficiency of their decisions; and last author on 'Strangers to Themselves', a pre-registered 9-evaluation, 12-model study arguing that model self-reports are generic AI theory plus favorable bias rather than self-knowledge. Topic clusters span ethics and social impacts of AI (5 works), topic modeling (3), adversarial robustness (3), explainable AI (3), and psychology of moral and emotional judgment (2). Field distribution splits evenly across three areas, and the profile is completed by 10 distinct coauthors and a reproducibility mean of 70 across three reviewed papers.

Collaboration pattern

OpenAlex records 10 distinct coauthors in the recent window, and the frequent-collaborator list is stable: Rajitha Ramanayake, Owen O'Neill, Nabeel Kemal, Houssem Chatbri, Christopher Martin, Vadim Pertsovskiy, Abhishek Mandal, Phil Blandfort, Ashwin Kandath, and Guillaume Bourgeon. Pawar is first author on two reviewed papers and last author on one, suggesting a core contributor within a persistent team rather than ad-hoc partnerships. The mid-range collaboration score of 48 reflects a focused, recurring group.

Research keywords

LLM evaluationhallucination detectionsemantic entropyuncertainty quantificationLLM self-explanationsexplanation faithfulnessbehavioral evaluationLLM self-knowledgeexplainable AI (XAI)counterfactual interventionscalibrationAI safety

Highlights

Pre-registered study argues LLM self-reports are generic, not self-knowledge
As last author, Pawar contributed to a 12-model, 9-evaluation pre-registered study concluding that what language models say about themselves reflects generic AI theory plus favorable bias rather than genuine introspective access. Evolve notes the evidence largely supports the claim, and reproducibility readiness is rated high.
Behavioral test of whether LLM explanations are necessary or sufficient
First-authored work checks whether the top-3 features LLMs cite as reasons actually match the measured necessity and sufficiency of their decisions via counterfactual interventions. Evolve calls the design sound but notes synthetic data, three-trial scores, and no data release.
Fusing token and semantic signals for black-box hallucination detection
First-authored paper proposes TopK, Gated, and Stacked fusion of semantic entropy with token log-probabilities to detect hallucinations without access to model internals. Evolve finds it rigorous, but with modest gains and a small LLM-judged dataset.

Timeline

2025OpenAlex records the first work in the snapshot: a single 2025 publication marks the start of the recorded record.
2026Recorded output reaches 6 works in 2026, a sixfold rise over the previous year per OpenAlex countsByYear.
2026First-authored preprint 'From Tokens to Semantics' appears: fusing semantic entropy with token log-probabilities for black-box hallucination detection.
2026First-authored preprint 'Necessary or Sufficient?' appears: behavioral evidence testing of LLM explanation faithfulness.
2026Last-authored preprint 'Strangers to Themselves' appears: a pre-registered study arguing model self-reports are generic rather than self-knowledge.

Papers in the Evolve catalog

2026-09-09
Strangers to Themselves: What Language Models Say About Themselves Is Generic
2026-09-04first author
Necessary or Sufficient? Evaluating LLM Explanations With Behavioural Evidence
2026-09-02first author
From Tokens to Semantics: Leveraging Complementary Signals for Hallucination Detection in Black-Box LLMs

Frequent co-authors

Rajitha Ramanayake×2 Owen O'Neill×2 Nabeel Kemal×2 Houssem Chatbri×2 Christopher Martin×2 Vadim Pertsovskiy×2 Abhishek Mandal×1 Phil Blandfort×1 Ashwin Kandath×1 Guillaume Bourgeon×1

About this profile

This profile is AI-structured by Evolve.Pub from public bibliographic data (OpenAlex) and Evolve's own paper reviews. It is not written or endorsed by the researcher unless the profile is claimed. Radar dimensions are computed from real bibliometrics with fixed formulas — raw figures are shown alongside; missing dimensions mean insufficient data, never zero. Errors? Claim the profile to correct or hide it.

Bibliometric indicators are computed from OpenAlex only; no ORCID is linked, so identity is name-resolved from public bibliographic data.
h-index and i10-index are recorded as 0 and no citation totals are available; the output and impact dimensions cannot be scored.
All three reviewed papers are 2026 arXiv preprints; peer-review status and venue acceptance fall outside this snapshot.
The focus score of 0 reflects an evenly split three-way field distribution (one third each); it measures topic concentration, not research quality.