CoRA-NAS: Coarse Ranking and Anchor-Residual Refinement for Neural Architecture Search
Yifan Yang, Zhaoyan Wang, Zheng Gao, Xiaoyu Li, Jiaojiao Jiang
cs.LG · cs.CV
Quick verdict
Label-free two-stage NAS ranking claims cross-space robustness on four vision benchmarks; evidence largely supports it, though size-space ties best proxies.
Summary
The paper proposes CoRA-NAS, a two-stage neural architecture search ranking framework. Stage 1 (CoRA-Rank) percentile-ranks five off-the-shelf zero-cost proxies covering capacity (#Params, Synflow) and structure-at-initialization (AZ-NAS expressivity/trainability views, NASWOT/jacov), combines them in an equal-weight log-rank consensus, and applies a label-free gate that drops proxies anti-correlated with the leave-one-out consensus. Stage 2 (CoRA-Refine) trains a stratified ~6.4% anchor set for a short epoch prefix (~1% of full-training cost), extrapolates early validation curves (LLW+pow3 blend), and propagates the residual to the static prior with an ExtraTrees regressor. On NAS-Bench-201, NAS-Bench-101, TransNAS-Bench-101 and NATS-SSS, mean Spearman correlations are 0.946, 0.715, 0.786 and 0.894; the worst-space 0.715 is the highest floor among compared methods, including target-aware LIBRA-NAS. Selection on NB201/CIFAR-100 reaches 73.32% versus the 73.37% ground-truth best. On the size space the method beats #Params decisively but only ties Synflow/MeCo. A real DARTS-space search reaches 2.81% CIFAR-10 error, beating AZ-NAS and random yet trailing gradient methods. Limitations are candidly scoped: no-weak-regime holds only for the four vision spaces (NAS-Bench-NLP fails; ViT-Bench-101 trails specialist L-SWAG), and no code release is stated.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
4 strengths · 4 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- A single-configuration, label-free two-stage ranker achieves the highest worst-space Spearman (0.715) across four vision NAS benchmarks, with every compared baseline collapsing on at least one space.
- Stage 2's ≈1% anchor-curve residual recovers the size space where the static prior loses to #Params (NATS-SSS 0.747→0.894), decisively beating #Params but only tying Synflow/MeCo.
- A clean equal-weight prior yields a learnable residual, making it a better refinement basis than the stronger but opaque AZ-NAS score (final 0.951 vs 0.929; CV-R² 0.43 vs 0.37, Table 4).
- On the real DARTS space, the method's selected cell reaches 2.81±0.05% CIFAR-10 test error trained from scratch, beating AZ-NAS and random on a shared pool (5 seeds, paired p<0.015) while trailing gradient/one-shot references (2.50-2.76%).
Strengths
Unified evaluation protocol with audited reproduction caveats
Section 4 re-runs ~19 baselines on one L40S, cross-checks nine GPU-dependent proxies on an RTX-4090 (|ΔSPR|≤0.009 over 27 cells), and Table 13 itemizes every >0.02 discrepancy from published values with a root cause (e.g., T-CET reports KT not SPR; AZ-NAS ImageNet16-120 not reproducible from released code).
Unusually candid boundary mapping
Section 4.3/5 report NAS-Bench-NLP as a failure case (prior 0.27, encoding→performance diagnostic 0.25, Refine within run-noise) and ViT-Bench-101 trailing the specialist L-SWAG (mean 0.365 vs 0.395, worst target 0.002), explicitly scoping 'no weak regime' to four vision spaces.
Deep ablation of every load-bearing component
Statistical care on the closest comparisons
Weaknesses
Size-space contribution is a tie, not a win
Headline SPR partially evaluates on training data
Selection story depends on a protocol patch
Mixed-provenance cross-space table
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
CoRA-NAS attacks the NAS performance-estimation bottleneck with a coarse-to-fine design: a zero-cost static prior plus a ~1%-budget learning-curve correction. Its four headline claims are (1) the highest worst-space Spearman across four vision benchmarks from one configuration (0.715 on NB101, vs floors of 0.608 for AZ-NAS and 0.667 for target-aware LIBRA-NAS); (2) label-free operation — no fully trained accuracy labels fit the ranker, only early validation curves; (3) the residual stage recovers the size space where the prior loses to #Params (0.747→0.894); (4) a clean prior keeps the residual learnable (Table 4). The paper is unusually explicit about scope: 'no weak regime' is claimed only over NB201, NB101, TransNAS-Bench-101, and NATS-SSS, with NAS-Bench-NLP reported as an honest failure (prior 0.27, encoding→performance 0.25, Refine within noise) and ViT-Bench-101 trailing the label-using specialist L-SWAG (0.365 vs 0.395 mean; worst target 0.002).
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 4 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is CoRA-NAS and how does it work?
CoRA-NAS is a two-stage method for ranking neural architectures cheaply. Stage 1 scores every architecture with five existing zero-cost proxies (covering model capacity and structure-at-initialization), removes any proxy that anti-correlates with the others on the current search space, and merges the rest via an equal-weight rank consensus. Stage 2 then trains about 6.4% of architectures (stratified across the ranking) for a few epochs — roughly 1% of the cost of training the whole space — extrapolates their early validation curves, and uses a tree ensemble to propagate the correction to all architectures. It never uses fully trained accuracy labels to fit its ranker.
How accurate is CoRA-NAS compared to full training and other NAS methods?
On NAS-Bench-201, NAS-Bench-101, TransNAS-Bench-101 and NATS-SSS its mean Spearman correlations with true accuracy are 0.946, 0.715, 0.786 and 0.894; its worst space (0.715) is the highest floor among all compared methods. On NB201/CIFAR-100 its selected architecture reaches 73.32% accuracy versus a 73.37% ground-truth best. However, on the pure size space it only ties the strongest capacity proxies (Synflow/MeCo) and on the real DARTS space it beats AZ-NAS and random search (2.81% vs 3.11%/3.09% error) but not gradient-based methods (2.50-2.76%).
Does CoRA-NAS work outside CNN image-classification benchmarks?
Not fully. The paper's 'no weak regime' claim is explicitly scoped to four vision benchmarks. On the recurrent NAS-Bench-NLP space both the proxies and the architecture encoding are weak and refinement does not help (rankings around 0.3). On the vision-transformer ViT-Bench-101, its zero-cost stage trails the transformer specialist L-SWAG by about 0.03 mean Spearman, with a worst target of 0.002. The authors present both as honest boundary cases.
Is CoRA-NAS code available?
The paper does not state a code or data release. The method is fully specified (algorithms, thresholds, feature lists), and all benchmarks used are public, so reimplementation is feasible, but anchor training and proxy computation require GPU resources comparable to the authors' single L40S setup.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1-3 are present only as extracted captions; plotted point-level data (e.g., the exact cost-quality frontier in Fig. 1) cannot be verified from the text, though all headline numbers cited here also appear in tables.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.