Benchmarking Active Spot Selection for Cost-Efficient Spatial Transcriptomics
Zheyu Zhu, Junchao Zhu, Fengbei Liu, Tianyuan Yao, Gelei Xu, John Cannon, Haichun Yang, Yuankai Huo, et al.
cs.CV · cs.LG · q-bio.OT
Quick verdict
Finds four active-learning spot strategies trail Random at 5-10% ST budgets with small mixed gains later; honest null, single-seed design limits certainty.
Summary
This paper presents a retrospective, pool-based benchmark asking whether active learning can reduce the number of measured spots in spatial transcriptomics without sacrificing prediction quality. Using two fully profiled public cohorts from HEST-1k — HER2-positive breast cancer (8 patients, 36 slides) and cutaneous squamous cell carcinoma (4 patients, 12 slides) — the authors mask expression vectors and simulate six-round selection at budgets of 5%, 10%, 30%, and 50% of the fold-wide training pool. Four standard strategies (MC-dropout, temporal output discrepancy, CoreSet, and a TypiClust-inspired variant) are compared against uniform Random sampling under a shared ST-Net-style DenseNet-121 predictor, patient-level cross-validation, and matched selection schedules, yielding 160 configurations. Evaluation covers per-gene within-slide Pearson correlation, expression-cluster agreement (ARI), and Moran's I fidelity. Pooled differences from Random are negative at 5% and 10% on both cohorts (HER2: -0.0176 and -0.0117) and small positive at 30-50%; on HER2, CoreSet and MC-dropout show lower PCC but higher ARI at small budgets, a reversal that does not reproduce on cSCC. The honest null conclusion is weakened by single-seed runs, non-converged small-budget training, unnormalized targets, and a single backbone and platform; no code release is stated.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
4 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Under matched spot budgets, a shared predictor, and identical selection schedules, the four evaluated active-selection strategies do not consistently outperform Random sampling at 5-10% of the fold-wide training pool on two ST cohorts.
- Mean PCC advantages at 30-50% budgets are small and not robust: pooled +0.0056/+0.0057 on HER2 versus Random, with cSCC results varying by strategy and budget.
- Pointwise accuracy (PCC) and spatial/cluster measures (ARI, Moran's I fidelity) can rank strategies oppositely at small budgets on HER2, so single-metric evaluation of spot selection is misleading.
- Future region-selective ST studies should report acquisition budget, training horizon, and evaluation measure, and validate computational selection in a physically realizable assay.
Strengths
Honest, precisely-scoped negative result
The abstract and Section 6 repeatedly hedge conclusions to 'the reported fixed training horizons', state the full-label reference is 'not schedule-matched', and quantify pooled gains as only +0.0056/+0.0057 at 30%/50% on HER2 — no overclaiming of the crossover as a general budget threshold.
Carefully controlled matched design
Within each budget all strategies share the quota (q = floor(betaN/R)), acquisition epochs (0, 5, 8, 11, 14, 17), one ST-Net-style DenseNet-121 predictor, and the training schedule; Table 2 even documents optimizer steps per budget (715 at 5% vs 3,156 at 50%). Patient-level 4-fold CV prevents slide leakage.
Three-axis evaluation surfaces a real finding
Unusually complete reproducibility documentation
Weaknesses
Single seed with effect sizes dwarfed by fold variance
Confounded optimization exposure at small budgets
Unnormalized targets confound both scoring and evaluation
Two of four strategies are non-canonical
Editorial hygiene and no code statement
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
This conference-format paper (arXiv:2609.27208) asks a narrow, well-posed question: under matched spot budgets, do standard active-learning (AL) acquisition strategies select more useful training spots than uniform Random sampling for spatial transcriptomics (ST)? The authors run a retrospective, pool-based simulation on two fully profiled HEST-1k cohorts — HER2-positive breast cancer (8 patients, 36 slides, 13,620 spots) and cutaneous squamous cell carcinoma (4 patients, 12 slides, 8,671 spots) — masking expression vectors and selecting spots over six rounds at 5%, 10%, 30%, and 50% of the fold-wide training pool. Four strategies (MC-dropout, TOD, CoreSet, TypiClust-inspired) plus Random yield 160 configurations under patient-level 4-fold CV. The central claim is negative: active selection does not consistently beat Random at small budgets (pooled PCC deltas of -0.0176/-0.0117 at 5%/10% on HER2), and gains at 30-50% are small and metric-dependent. The paper is commendably explicit that conclusions hold only 'under the reported fixed training horizons'.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
Does active learning beat random sampling for spatial transcriptomics spot selection?
In this retrospective benchmark of four standard strategies (MC-dropout, TOD, CoreSet, TypiClust-inspired) on two public HEST-1k cohorts, no at small budgets: pooled per-gene PCC differences versus Random were negative at 5% and 10% (HER2: -0.0176 and -0.0117), and only marginally positive at 30-50%, with rankings depending on the evaluation metric.
What data does the paper use?
Two public legacy Spatial Transcriptomics cohorts from HEST-1k: HER2-positive breast cancer (8 patients, 36 slides, 13,620 spots) and cutaneous squamous cell carcinoma (4 patients, 12 slides, 8,671 spots), evaluated with patient-level 4-fold cross-validation and the first 300 alphabetically ordered genes.
Which active learning methods were compared and how?
MC-dropout uncertainty, temporal output discrepancy (TOD), CoreSet greedy k-center diversity, and a TypiClust-inspired clustering strategy were simulated over six selection rounds at 5-50% budgets, all sharing one ST-Net-style DenseNet-121 morphology-to-expression predictor, and compared against uniform Random sampling (160 configurations total).
Is code available for this benchmark?
The paper does not state a code release. However, it documents the seed (42), software versions, augmentation policy, and per-budget training schedules in enough detail that the benchmark can be reimplemented from the public HEST-1k data at single-GPU scale.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures could not be inspected from the text; the Figure 1 'editorial annotation' observation is inferred from embedded caption/annotation text and should be verified against the PDF.
- The abstract in the source text is truncated mid-sentence; conclusions were cross-checked against the body and tables instead.
- Conference-format paper (4-page style): methods are compressed, and some implementation details (e.g., HEST-1k preprocessing choices) are inherited from the dataset rather than specified.
- The 'TypiClust-inspired' strategy is a deliberate deviation from published TypiClust, so judgments about that method's canonical behavior do not apply.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.