ChipMEM: Verification-Grounded Memory for EDA Agents
Abdulrahman AlRabah, Joshua Mabry, Dilek Hakkani-Tür, Abdussalam Alawini, Hamid Shojaei, Kartik Hegde, Sandesh Adhikary
cs.LG · cs.CL
Quick verdict
Claims verification-gated memory improves EDA-agent transfer; honest reporting, but small-n, best-of-selection and proprietary agent weaken the evidence.
Summary
ChipMEM is a memory layer for LLM-based EDA agents that converts tool-verified experience into two reusable stores. Procedural memory distills a trajectory into a skill only after the deterministic domain harness (synthesis, simulation, or formal equivalence) issues a verified PASS; statistical memory maintains hierarchical Beta estimates over tool-call retries and recovery strategies, advising the agent when retries are unlikely to succeed. An agent-agnostic adapter applies the same interface to RTL optimization (proprietary Cadence ChipStack 2.0 agent with GPT-5.5) and testbench generation, with OpenTitan runs using Qwen3.8-27B. Under matched settings, RTLRewriter-Bench short-suite equivalence passes rise from 33/49 to 37/49 (39/54 vs 35/54 including five long cases) and mean area gain from 5.66% to 8.69%; RTL-OPT mean area improvement rises from 6.18% to 8.79% under best-valid-per-design selection across three result sets. With a frozen read-only library, held-out CVDP transfer improves from 18/20 to 20/20. OpenTitan full-gate passes rise from 1/10 to 4/10 with 11-15% fewer LLM and tool calls; the five-design ablation shows combined memory (5/5 passes) beats either component alone (4/5 each). Main limitations: no significance tests for outcome comparisons, n=5 ablation, single-evaluation held-out runs, best-of-three selection, marginal custom-suite gains (0.14% to 0.31% area), a proprietary core agent, and one observed harmful transfer (+283.3% area).
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Skills distilled by an LLM only after a deterministic EDA harness (synthesis, simulation, formal equivalence) issues a verified PASS form transferable procedural memory, unlike self-assessed skill libraries.
- ChipMEM raises equivalence-passing outputs on the RTLRewriter-Bench short suite from 33/49 to 37/49 and mean area gain from 5.66% to 8.69% under matched model and tool settings.
- A frozen, read-only procedural library transfers to unseen CVDP testbench tasks, improving acceptance from 18/20 to 20/20 in a single evaluation.
- Bayesian statistical memory cuts OpenTitan interaction cost (LLM calls -11.2%, tool calls -15.0%, tool time -12.0%) while full-gate passes rise from 1/10 to 4/10.
- A common agent adapter lets one memory interface serve both RTL-optimization and testbench-generation agents without modifying either agent or its acceptance gate.
Strengths
Verification-gated learning rule
Skills are stored only after the deterministic domain harness (synthesis, simulation, formal equivalence) returns a verified PASS (Algorithm 1, lines 14-19), directly addressing the failure mode of skill libraries that learn from LLM self-judgment; PPA acceptance further requires 'successful synthesis, functional equivalence, and a positive audited improvement'.
Clean transfer protocol
The frozen, read-only library evaluated once on 20 held-out CVDP tasks (Table 2) cleanly separates transfer from task-specific revision — the exact gap the introduction identifies in prior EDA memory work that 'evaluate on the tasks that produced the experience'.
Unusually honest negative reporting
Fully specified Bayesian component
Component ablation plus breadth
Weaknesses
Underpowered outcome claims
Unmatched best-of selection on RTL-OPT
Marginal gains where the flow is realistic
Proprietary core and legacy-rule contamination
Harmful transfer is real but unquantified
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is ChipMEM and what problem does it solve?
ChipMEM gives LLM agents that write and optimize hardware (RTL) a memory of verified lessons. A skill file is stored only after EDA tools — synthesis, simulation, or formal equivalence checking — confirm a passing result, never on the model's own judgment. A Bayesian side-table tracks which tool retries and recovery strategies actually worked, so later sessions start with informed priors instead of repeating failed calls.
How much does ChipMEM improve results?
Under matched settings it raises RTLRewriter-Bench short-suite equivalence passes from 33/49 to 37/49 with mean area gain 5.66% to 8.69%; RTL-OPT mean area improvement goes from 6.18% to 8.79%; frozen-memory transfer on held-out CVDP testbench tasks goes from 18/20 to 20/20; and OpenTitan full-PPA gate passes go from 1/10 to 4/10 while using 11-15% fewer LLM and tool calls. Gains on the authors' custom 20-design suite are much smaller (0.14% to 0.31% area).
Is the code or data available?
The paper links a GitHub repository (github.com/aalrabah/ChipMEM) and all benchmarks are public (RTL-OPT, RTLRewriter-Bench, CVDP, OpenTitan), but the core agent is the proprietary Cadence ChipStack 2.0 and experiments use GPT-5.5, so fully independent replication requires commercial tools and paid model access.
Does the memory ever hurt performance?
Yes. The held-out PPA study reports one design whose mapped area regressed by 283.3% after retrieved skills, and per-design tables show several designs where ChipMEM loses attempts or strict passes versus the memory-off baseline (e.g., 11 arbiter and 02 fifo sync in the custom suite). The paper acknowledges skills from dissimilar designs can be counterproductive but does not quantify an overall harm rate.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 3-6 are images in the extraction; per-design directions and plotted distributions could not be verified against the text.
- Abstract numbers (39/54 vs 35/54) combine the 49-design short suite (Table 3b: 37/49 vs 33/49) with the 5-design long suite (2/5 both) — scopes differ across sections.
- The core EDA agent (Cadence ChipStack 2.0) is proprietary, so baseline behavior cannot be independently audited from the paper.
- PDF table extraction may be lossy; numbers were cross-checked only where duplicated across abstract, body, and appendix.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.