Agentic programs: an emerging form of scientific software in computational materials science
Yunsung Lim, Haekwan Jeon, Jaesun Kim, Jisu Kim, Seungwu Han
cond-mat.mtrl-sci · cs.AI
Quick verdict
Position paper defining 'agentic programs'; DeMARS case study is compelling but 100-CIF validation is qualitative, with no error rates or baselines reported.
Summary
This position paper argues that LLM-based agent harnesses enable a new class of scientific software, 'agentic programs,' defined by four characteristics: combining deterministic algorithms with bounded LLM judgment, bounding responsibility so task-specific verification gates become possible, maturing the system by codifying episodic failures into persistent artifacts, and achieving complete delegation in production. The concept is illustrated with DeMARS, a program that converts experimentally measured disordered crystal structures (CIFs with partial occupancies) into minimal atomistic models — relevant because roughly half of ICSD entries are disordered. DeMARS's architecture routes only underdetermined decisions (enumeration specification, energy-distribution acceptance, mechanism classification) to an LLM analyst operating between deterministic interfaces, while gates for charge neutrality, occupancy fidelity, and connectivity check consequences, and an adversarial reviewer agent inspects finished records. After maturation on ~800 disordered CIFs, DeMARS processed 100 randomly selected held-out CIFs end-to-end without human intervention; the authors report all certified models were chemically reasonable upon manual inspection and declines were verifiable. A concrete maturation episode (Sr2LiMoO5.5, ICSD 166728) shows the reviewer catching unsampled anti-site disorder. Main limitations: validation is qualitative with no error rates, certified/declined counts, or baseline comparisons; code is not yet released; and LLM nondeterminism and hallucination risk are acknowledged but unresolved.
Review complete
Assessment at a glance
5
risk flags identified
● high 1
● medium 2
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- LLM-based agent harnesses enable a new class of scientific software, 'agentic programs,' defined by bounded responsibility, hybrid algorithm/LLM judgment, task-specific verification, episodic maturation, and complete delegation in production.
- DeMARS demonstrates this paradigm for constructing atomistic models from disordered CIFs by restricting LLM judgment to underdetermined decision points checked by deterministic gates and an adversarial reviewer.
- After maturation on ~800 disordered CIFs, DeMARS processed 100 randomly selected held-out CIFs end-to-end without human intervention, with certified models judged chemically reasonable and declines verifiable from their own records.
- Computational materials science is particularly amenable to agentic programs because inputs and deliverables are well defined and the underlying physical/chemical knowledge is relatively stable.
Strengths
Crisp conceptual framework with explicit differentiation
The four characteristics (algorithm+judgment, bounded responsibility, episodic maturation, complete delegation) are cleanly separated from interactive assistants and human-supervised agentic workflows; the paper candidly concedes the individual elements are not new, locating novelty in their composition for end-to-end delegation of a bounded task.
Principled architecture: LLM between deterministic interfaces
In DeMARS (Fig. 3), the analyst LLM neither generates the quantitative facts it judges on nor certifies its own output — code supplies evidence (①) and independently checks consequences (⑥), and energy above hull is deliberately an interpretive metric, not a pass/fail gate, avoiding penalizing valid metastable disorder models.
Concrete, verifiable maturation episode
Honest treatment of failure and limitations
Practical problem with careful test design
Weaknesses
Headline validation is entirely qualitative
No baseline or ablation isolating LLM judgment
Code unavailable and model stack unspecified
Single case study supporting a general paradigm
Unquantified coverage envelope
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
This is a **position paper with a case study**, not an empirical benchmark. The authors (Seoul National University / KIAS) propose 'agentic programs' as an emerging class of scientific software in computational materials science, defined by four characteristics: (1) deterministic algorithms combined with *bounded* LLM judgment confined to underdetermined decision points; (2) bounded responsibility enabling task-specific verification gates; (3) 'episodic maturation' — root-causing failures into versioned artifacts (skills, doctrines, gates, deterministic code); (4) complete delegation in production, including the right to *decline* to certify. The concept is instantiated in DeMARS, which converts experimentally measured disordered crystal structures (CIFs with partial occupancies; ~half of ICSD entries per ref [31]) into minimal atomistic models. The central empirical claim: after maturation on ~800 disordered CIFs, DeMARS processed 100 randomly selected held-out CIFs end-to-end without human intervention, with certified models found 'chemically reasonable' upon manual inspection and declines 'verifiable from the record's own evidence.'
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is an 'agentic program'?
A term proposed in this paper for scientific software that runs an entire bounded scientific task without routine human help: deterministic algorithms handle rule-based steps, while a bounded LLM agent makes judgment calls at decision points that cannot be fully specified in advance. Its judgments are checked by task-specific verification gates, and recurring failures are codified into permanent improvements (rules, gates, code) so the program matures over time.
What does DeMARS do?
DeMARS (De-averaged Minimal Atomistic Representation System) converts experimentally measured disordered crystal structures — crystallographic information files (CIFs) with partially occupied sites — into minimal atomistic models suitable for simulation. About half of the entries in the ICSD crystal structure database are reported as disordered, so automating this step addresses a real bottleneck in computational materials science.
How well does DeMARS perform?
The paper reports a qualitative test: after maturation on roughly 800 disordered CIFs, DeMARS processed 100 randomly selected held-out CIFs end-to-end with no human intervention. The authors' manual inspection found all certified models chemically reasonable and the reasons for declining cases verifiable. No numeric error rates, certified/declined counts, or baseline comparisons are provided.
Is the DeMARS code available?
Not at the time of this preprint. The paper states that the source code, agent definitions, and versioned skills will be released publicly upon publication of the peer-reviewed version of the work.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1-3 appear in the extraction only as fragmented text plus captions; panel-level quantitative content is not recoverable (no results table exists in the body text, so nothing numeric was lost).
- The extracted text reads complete from abstract through references, acknowledgements, and author contributions; omissions noted in this review are the paper's, not truncation artifacts.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.