Rachel: A general-purpose language model directs and revises retrosynthetic routes
Qisheng Li, Shunchao Jiang, Chen Qi, Xin Su, Da Han, Guangyong Chen
physics.chem-ph · cs.AI
Quick verdict
Claims GPT-5.5 strictly closes 111/120 retrosynthesis routes in its stateful Rachel env; ablations support LLM agency, but route quality rests on LLM judges.
Summary
Rachel is a stateful software environment in which a general-purpose LLM — GPT-5.5 at high reasoning effort — autonomously directs multistep retrosynthetic planning: it proposes transformations, accepts or rejects them, designates terminal precursors, and revises a persistent route plan, while RDKit-based checks validate proposed chemistry without imposing any search policy or stopping rule. Strict closure required a complete route plus an independent post-hoc audit matching every terminal precursor to PubChem CID and vendor records. GPT-5.5 within Rachel strictly closed 111 of 120 PaRoutes120 targets and 24 of 25 RF25 difficult targets (24 of 25 RF25 source studies postdate the model's knowledge cutoff), versus 95 for a direct-prompt LLM baseline, and above AOT* (95) and SyntheLite (83). Replacing LLM decisions with fixed policies collapsed closure to 6–15 of 120, and four planning-support ablations on RF25 reduced it to 2–18 of 25. On the shared common58 subset, dual forward models (ReactionT5v2 and RXNGraphormer) supported 57.4% of Rachel's steps at Both@10 versus 71.7% for reference routes, and both method-blinded LLM evaluators gave Rachel the highest mean overall route score. Main limitations: no human expert chemical assessment (LLM judges only), forward-model support falls to 21.8% on RF25 routes, single runs per target without variance estimates, and closure measures source availability rather than synthetic feasibility.
Review complete
Assessment at a glance
6
risk flags identified
● high 0
● medium 4
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Within the stateful Rachel environment, GPT-5.5 alone constructs routes achieving strict closure for 111/120 PaRoutes120 targets and 24/25 RF25 difficult targets, without reference routes or route-level solutions.
- Replacing LLM route decisions with fixed policies (closure 6–15/120) or restricting planning support (2–18/25 on RF25) sharply reduces closure, supporting a causal role for LLM direction in complete-route construction.
- On the shared common58 subset, Rachel's recorded chemistry receives dual forward-model support exceeding direct LLM and AOT*, and the highest mean overall route score from both method-blinded LLM evaluators.
- Recorded trajectories show genuine strategy revision: 35/120 PaRoutes and 23/25 RF25 routes contain at least one recorded plan revision, and revised plans are registered for subsequently committed steps in nearly all revised routes.
Strengths
Rigorous, game-resistant closure endpoint
Strict closure requires post-freeze, independent audit of every terminal precursor against PubChem CIDs and vendor records (Methods, S3); audit information was never fed back into planning, and all 120 targets remain in each denominator including failed runs — much harder to game than self-reported success.
Causal ablation design cleanly isolates LLM decision-making
Three mechanical policies reuse Rachel's identical execution and validation (22,577/32,295 candidates passed; 2,894/3,016 decision opportunities committed, Fig. 5a) yet closure falls from 111/120 to 6–15/120; four RF25 component ablations drop 24/25 to 2–18/25 (Fig. 5c), separating scaffolding from LLM direction.
Knowledge-cutoff-aware difficult cohort
Unusual transparency and auditability
Honest negative diagnostics
Weaknesses
No human expert chemical assessment
Majority of steps lack independent computational support
Single-run design, one planner backbone
Survivorship-conditioned quality comparisons
Endpoint semantics: procurement evidence, not feasibility
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
Rachel is a stateful planning environment (public code, commit 57c1716) in which a general-purpose LLM — GPT-5.5 at High reasoning effort — performs de novo retrosynthesis with no system-imposed search policy or stopping rule. The model proposes transformations (predefined bond/FGI operations or free-form 'custom' chemistry), accepts or rejects candidates, designates terminals, and maintains or revises a persistent route plan; RDKit-based checks (atom balance, template execution, topology, functional-group compatibility) gate commitment. The central claims: (1) strict closure — a complete route plus a post-freeze independent audit resolving every terminal to a PubChem CID and vendor record — for 111/120 PaRoutes120 targets and 24/25 RF25 difficult targets; (2) this outcome is attributable to LLM direction, since fixed-policy replacements close only 6–15/120 and four planning-support ablations on RF25 fall to 2–18/25; (3) recorded trajectories demonstrate strategy revision, with 35/120 and 23/25 routes revised and revised plans registered for nearly all retained steps in revised routes (Fig. 4c: 34/35 and 23/23).
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 6 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What does 'strict closure' mean in the Rachel retrosynthesis paper?
A route counts as strictly closed only if construction is complete and every terminal precursor is independently matched after planning to a PubChem compound identifier (CID) plus vendor-listing evidence. It is a computational source-resolution endpoint — evidence that each building block exists in catalogs — not proof that the route works in a laboratory.
Did the authors perform any real chemical experiments?
No. The study is entirely computational: routes are planned by GPT-5.5 inside the Rachel software environment, checked with RDKit-based structural rules, and audited against PubChem and vendor databases. No laboratory synthesis was performed, and the paper explicitly states that closure does not establish synthetic feasibility.
How is Rachel different from AOT* or SyntheLite?
Those systems couple LLM proposals to explicit search machinery — AND-OR tree search (AOT*) or LLM blueprints guiding template-based search (SyntheLite) — with search policies and stopping rules built in. Rachel instead lets the LLM itself decide which transformations to accept, when to stop, and how to revise strategy, while the environment only executes, validates and records the chemistry.
Which language model powers Rachel, and is the code available?
All headline results use GPT-5.5 with reasoning effort set to High. The Rachel harness itself is model-agnostic code (state tracking, RDKit checks, audits) released on GitHub at github.com/ChazenLi/Rachel under CC BY-NC 4.0, with a Route Atlas and Source Data for all 145 evaluated targets.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures are available only as captions in the extracted text; distributions, score magnitudes and per-target panels could not be visually inspected.
- The abstract is truncated mid-sentence in the extraction; its final sentence was reconstructed from the identical framing in the Introduction and Conclusion.
- Supplementary Data (Route Atlas HTML, Source Data ZIP) are referenced but not inspectable here; several comparator runtime settings are only 'indexed' in accompanying files.
- PaRoutes reference-route comparator numbers (71.7% Both@10) rely on the text's summary of Fig. 3c; underlying per-step data could not be re-verified.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.