ZAPS: Zero-Cost Active Proxy Search for Neural Architecture Search
Hassan Touayouch, Rabie Najem, Mohammed Benjelloun
cs.LG
Quick verdict
ZAPS fuses selected zero-cost proxies with topology in active-learning NAS; strong tabular gains over BANANAS/TPE are credible but partly oracle-calibrated.
Summary
The paper introduces ZAPS, a four-stage NAS pipeline that fuses zero-cost proxy signals with architectural topology inside an active-learning loop. PROXYFIT greedily selects six non-redundant proxies (Spearman relevance ordering, τ=0.85 redundancy threshold); a hybrid K-means seeds half the initialization budget in proxy-favored regions and half across the space; proxies are re-selected each iteration by a bootstrapped vote; and an XGBoost ensemble over normalized proxy ranks plus one-hot cell encodings drives UCB acquisition. On NAS-Bench-201 with B=200 evaluations, ZAPS recovers 52.3% of the true top-100 on CIFAR-10 and 65.8% on CIFAR-100 (200 seeds), ahead of Random Search, Local Search, REA, BANANAS and TPE, with lower seed variance; the CIFAR-100 margin narrows to +0.6 pp by B=500, while on NAS-Bench-101 it widens to +26 pp. Ablations attribute most of the gain to proxy ranks (removing them costs ~21 pp) versus 4.5–7.1 pp for topology and 15.6–22.6 pp for structured initialization. The authors disclose that PROXYFIT uses whole-space ground-truth correlations and show transferred subsets perform comparably. Limitations: self-implemented baselines, tabular benchmarks only, NB-101 restricted to 9,781 annotated architectures, and no regret separation from BANANAS on CIFAR-10.
Review complete
Assessment at a glance
4
risk flags identified
● high 0
● medium 2
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Fusing a selected, non-redundant zero-cost proxy subset with one-hot topology encodings in a single XGBoost surrogate, inside a UCB active-learning loop, beats both proxy-only aggregation and encoding-only surrogates under tight evaluation budgets.
- At B=200 on NAS-Bench-201, ZAPS recovers 52.3% (CIFAR-10) and 65.8% (CIFAR-100) of the true top-100 over 200 seeds, ahead of Random Search, Local Search, REA, BANANAS and TPE with lower run-to-run variance.
- Proxy ranks are the dominant signal (removing them costs 21.2–22.4 pp vs 4.5–7.1 pp for topology, Table 7), and structured initialization is the dominant pipeline component (15.6–22.6 pp, Table 6).
- The PROXYFIT-selected subset transfers across datasets and search spaces with ≤4.2 pp difference, so whole-space ground-truth calibration does not drive the results.
Strengths
Statistical reporting above the field's norm
200 seeds per cell, one-sided Mann–Whitney U with Cliff's δ effect sizes (Section 4.4); the paper reports where significance and practical relevance diverge (CIFAR-100 B=300 vs TPE: p<0.01 but δ=0.14 negligible) and where tests fail entirely (regret vs BANANAS, p=0.25).
Transparent handling of oracle leakage
Section 3.1 explicitly discloses PROXYFIT consumes whole-space ground-truth correlations and quantifies the concern via cross-dataset/cross-space transfer (Section 4.8, Table 9: native vs transferred subsets differ by ≤4.2 pp in P@100).
Genuinely informative ablations
Useful negative result on proxy aggregation
High reproducibility readiness
Weaknesses
All baselines are self-implementations
Headline configuration uses oracle calibration
Tabular-only evaluation on a biased NB-101 subset
No variance or significance reporting on NAS-Bench-101
Hyperparameter sensitivity unexamined
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
ZAPS (arXiv 2609.14184, cs.LG) is a method paper on evaluation-efficient Neural Architecture Search. Its central claim: combining a *selected*, non-redundant subset of zero-cost proxies with one-hot topological encodings in a single surrogate, driven by UCB active learning, recovers more of the true top-100 architectures under tight budgets than either proxy-only aggregation or encoding-only surrogates (BANANAS, NASBOWL). The primary criterion is P@100 (Eq. 1) — the fraction of the benchmark's true top-100 that a method actually evaluated within budget B — with relative regret as a secondary metric. Experiments cover NAS-Bench-201 (15,625 architectures; CIFAR-10/CIFAR-100, ImageNet16-120 for the transfer study) and NAS-Bench-101 (9,781 proxy-annotated of 423,624 architectures), 200 seeds per configuration.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 4 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is ZAPS in neural architecture search?
ZAPS (Zero-cost Active Proxy Search) is a NAS method that finds top networks using a small budget of full trainings (e.g. 200 evaluations). It combines cheap zero-cost proxy scores — computed at initialization without training — with the architecture's structural encoding inside an XGBoost surrogate model, and actively chooses which architectures to train next via an Upper Confidence Bound rule.
How much better is ZAPS than BANANAS or TPE?
On NAS-Bench-201 with a budget of 200 trained architectures, ZAPS recovers 52.3% of the true top-100 architectures on CIFAR-10 (BANANAS: 34.7%, TPE: 35.4%) and 65.8% on CIFAR-100 (BANANAS: 48.0%, TPE: 50.5%), averaged over 200 random seeds, with lower run-to-run variance. The advantage shrinks as the budget grows on CIFAR-100 but widens on NAS-Bench-101.
Does PROXYFIT need access to ground-truth accuracies?
Yes — as defined, PROXYFIT computes proxy–accuracy correlations over the entire benchmark table, which the authors disclose explicitly. They show that a proxy subset calibrated on a different dataset performs nearly identically (within ~4–5 pp), arguing the calibration is a reusable one-off rather than label leakage driving the results.
Was AI used to write this paper?
Yes, and the paper contains an unusually detailed AI use statement: an LLM (Claude) audited the baseline code and found bugs, proposed the fusion ablation experiment, drafted substantial text portions and figure scripts. The authors state they wrote the core implementation, reviewed all corrections, and re-derived every numerical claim from result files with a verification script.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures are not analyzable from the extracted text (only axis labels and legends survive extraction); judgments rest on the in-text tables.
- The abstract is truncated mid-sentence in the provided feed ('The advant…'), though the full body through the references is present.
- Code is promised for release and 'available from the authors on request' but was not inspectable at review time; reproducibility rating assumes the artifacts materialize.
- The AI-use statement implies result provenance depends heavily on the authors' verification scripts, which cannot be audited from the text alone.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.