Melanie Mitchell

Santa Fe Institute · United States

Unclaimed · AI-structured profileORCID OpenAlex

ARC-focused AI researcher (Santa Fe Institute) probing LLM abstraction, rule induction, and benchmark validity

papers here

reviewed by Evolve

first-author

corresponding

—

works · OpenAlex

h-index · OpenAlex

Bibliometric figures come from OpenAlex (2026-10-15) and cover the author's entire public record, not just the papers listed here. Reproducibility signal comes from Evolve's own reviews.

Research profile

Output—

The output axis is unscored: the snapshot provides no career-wide publication counts or per-year productivity series beyond a single 2026 count. It would measure cumulative publication volume, output trajectory, and authorship positions across the researcher's career.

Impact—

Impact is unscored: citation-based influence cannot be computed reliably from this snapshot, whose only related signals are an h-index of 2 and an i10-index of 0. This axis would measure citations, field-normalized influence, and uptake of the benchmark work.

Focus0/100

Focus scores 0 on the radar, the lowest of the available axes. The raw field distribution is nonetheless narrow: reviewed work splits evenly (0.5/0.5) between LLM evaluation and benchmarking, and machine learning for AI reasoning benchmarks and interpretability. Topic clusters add spread — Psychiatry/Mental Health/Neuroscience leads with 5, followed by AI-based Problem Solving and Planning (4), AI Applications (3), Topic Modeling (2), and NLP Techniques (2) — and the clinical cluster does not appear in the reviewed papers.

Collaboration39/100

The collaboration axis scores 39, based on 6 distinct co-authors in the snapshot. Both reviewed papers list the researcher as last author, though neither as corresponding author. The frequent co-author list centres on Adrien Deliège, Claas Beger, Marc Van Droogenbroeck, and Ryan Yi; the appearance of the researcher's own name in that list signals name-disambiguation noise. The small, recurring pool suggests a compact, repeat-partner working style rather than a broad collaboration network.

Activity67/100

Activity scores 67. The counts-by-year series contains exactly one entry — 9 works in 2026 — and the recent-3-year total is also 9, meaning all recorded output falls inside the current window. Both reviewed papers appeared on arXiv in 2026, consistent with a burst of recent, benchmark-oriented production; with no earlier years recorded, this reflects a short observable window rather than a long-term productivity trend.

Reproducibility55/100

Reproducibility scores 55, a mean computed over 2 papers with reproducibility data. Both reviewed papers carry a medium repro-readiness rating. Evolve's verdicts temper confidence: PotARCin is called rigorous but with acknowledged confounds, and VARC's controlled-dataset evidence is strong while its single-run results lack variance estimates. This places the work at mid-range reproducibility — careful benchmark design, but with reporting gaps to address for independent replication.

Research summary

This is a name-resolved profile, built from OpenAlex bibliographic data, for a researcher listed at the Santa Fe Institute in the US (ORCID 0000-0001-8881-3505). Recorded output is recent and compact: the snapshot counts 9 works in 2026 — the only year with recorded counts — and a recent-3-year total of 9, with an h-index of 2 and an i10-index of 0. Both Evolve-reviewed papers are 2026 arXiv preprints on which the researcher is the last author, and both sit at the intersection of abstract reasoning and LLM evaluation. PotARCin extends the Abstraction and Reasoning Corpus (ARC) into a five-dimension skill-acquisition benchmark, exposing 25-52 percentage-point drops and self-contradiction; VARC contributes Embed-TTT, rule-aligned test-time task embeddings for implicit rule induction. The field distribution splits evenly (0.5/0.5) between LLM evaluation/benchmarking and machine learning for AI reasoning benchmarks/interpretability. Radar scores place activity at 67, reproducibility at 55, collaboration at 39, and focus at 0, while output and impact lack sufficient data to score. Because the name is widely shared and even appears in its own co-author list, identity confidence is limited.

Collaboration pattern

The snapshot records 6 distinct co-authors, and both reviewed papers are last-authored without corresponding-author status. The recurring pool — Adrien Deliège, Claas Beger, Marc Van Droogenbroeck, and Ryan Yi — suggests a small set of repeat partners across the 2026 benchmark projects. Notably, the researcher's own name also appears in the frequent-coauthor list, a signature of name-disambiguation noise, so collaboration patterns should be read as provisional pending stronger identity resolution.

Research keywords

abstract reasoningAbstraction and Reasoning Corpus (ARC)LLM evaluationbenchmark designPotemkin understandingprogram synthesistest-time trainingtask embeddingsrule inductionVision ARC (VARC)interpretability

Highlights

PotARCin extends ARC to five rule-use dimensions, exposing 25-52pp capability drops
This 2026 arXiv benchmark study evaluates skill acquisition in abstract reasoning across five dimensions of rule use, reporting performance drops of 25-52 percentage points alongside self-contradiction — a Potemkin-understanding pattern. Evolve calls it a rigorous benchmark study with acknowledged confounds; the researcher is last author.
Embed-TTT yields rule-aligned test-time task embeddings in ARC-like tasks
In the Vision ARC (VARC) setting, this 2026 arXiv paper induces rules implicitly through test-time task embeddings that align with underlying rules. Evolve rates the controlled-dataset evidence strong, while cautioning that single-run results lack variance estimates.
Self-contradiction as a probe of Potemkin understanding
Beyond aggregate accuracy drops, the PotARCin evaluation documents self-contradiction in model behavior, echoing the Potemkin-understanding concern that benchmark success can mask missing competence in abstract reasoning. This makes the benchmark a diagnostic tool rather than a plain leaderboard.

Timeline

2026PotARCin appears on arXiv: a five-dimension evaluation of skill acquisition in abstract reasoning tasks, last-authored.
2026The VARC paper appears on arXiv, introducing Embed-TTT rule-aligned test-time task embeddings for ARC-like tasks, last-authored.
2026OpenAlex snapshot records 9 works for this year — the only year with recorded counts, equal to the recent-3-year total.

Papers in the Evolve catalog

Frequent co-authors

Adrien Deliège×1 Claas Beger×1 Marc Van Droogenbroeck×1 Melanie Mitchell×1 Claas Beger×1 Ryan Yi×1

About this profile

This profile is AI-structured by Evolve.Pub from public bibliographic data (OpenAlex) and Evolve's own paper reviews. It is not written or endorsed by the researcher unless the profile is claimed. Radar dimensions are computed from real bibliometrics with fixed formulas — raw figures are shown alongside; missing dimensions mean insufficient data, never zero. Errors? Claim the profile to correct or hide it.

All bibliometrics come from an OpenAlex name-resolved snapshot (identity: openalex); citation counts and the h-index may be incomplete or mixed across same-name records.
“Melanie Mitchell” is a widely shared name, and the researcher's own name appears among the frequent co-authors — identity confidence is limited, and this profile should not be assumed to describe any specific well-known individual.
Recorded output exists only for 2026 (9 works); no earlier career history is available, and the output and impact radar axes lack sufficient data to score.
The reproducibility score averages only 2 papers, and topic clusters (e.g., Psychiatry/Mental Health/Neuroscience, 5 works) partly reflect disambiguation noise rather than the reviewed AI work.