TopU-LBVS: A Realistic Multi Target Benchmark for Ligand Based Virtual Screening
Surbhi Kumar, Yuhe Zhou, Varun Shiralkar, Niu Huang, Baris Coskunuzer
cs.LG · cs.AI
Quick verdict
A well-built 93-target hard-negative LBVS benchmark; its ~4x random-vs-hard decoy gap is real but partly by design, and label noise remains.
Summary
This paper introduces TopU-LBVS, a ligand-based virtual screening benchmark of 93 protein targets across seven protein classes, built from curated ChEMBL 35 data with property-matched, structurally similar hard decoys at a fixed 1:40 active-to-decoy ratio. It defines three fixed protocols: full (training on historical ChEMBL* SAR, testing on hard libraries), low (low-data learning within the hard distribution), and mini (seven targets with a paired random-decoy control that changes only test decoys). Across ten baselines spanning fingerprints, GNNs, hybrids, and pretrained models, mean EF@1% on the mini control falls from 21.2 with random decoys to 5.5 with hard decoys — a nearly fourfold drop seen in 67 of 70 paired comparisons — and in the low-data protocol all baselines remain near random enrichment (best: MolFormer EF@10% 1.12). Fingerprint baselines lead the full protocol but collapse in the low-data regime, and fingerprint augmentation reverses from helpful to harmful. Main limitations: the hardness gap is partly built into the decoy-construction objective, the pChEMBL > 4.9 activity threshold admits label noise, small libraries make EF@1% noisy, and statistical testing is limited to three-seed means without significance tests.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 2
● low 3
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Testing on hard, property-matched decoys instead of random ChEMBL* decoys cuts mean EF@1% from 21.2 to 5.5 (nearly 4x) across ten models on seven mini targets, with random decoys winning in 67 of 70 paired comparisons (Sec. 6.3, Table 5).
- Learned molecular models (GIN/GAT/GPS, their fingerprint hybrids, D-MPNN, MolFormer) do not reliably outperform Morgan-RF and Tanimoto-NN fingerprint baselines under hard-negative screening (Sec. 6.1, 6.4).
- In the low-data hard-negative regime (TopU-LBVS-low), all ten baselines perform at or near random enrichment (best overall EF@10% 1.12 for MolFormer; Tier 1 overall 0.80), i.e., the regime remains largely unsolved (Sec. 6.2, Table 4).
- TopU-LBVS is the first LBVS resource to jointly provide a large multi-class target panel, explicit hard negatives, screening-level imbalance, and fixed supervised protocols with released splits (Sec. 2, Table 1).
Strengths
Clean paired-control experimental design
TopU-LBVS-mini changes only the test decoys while holding training data, validation, test actives, and the 1:40 ratio fixed; the resulting mean EF@1% drop from 21.2 to 5.5 (67/70 pairs, Table 5) is an unusually direct quantification of random-decoy inflation.
Scale and protocol rigor vs prior benchmarks
93 targets across 7 classes with fixed splits (seed 2026), released evaluation code, and three defined protocols; Table 1 shows DUD-E, DEKOIS 2.0, MUV, and LIT-PCBA lack learning protocols and fixed splits, while FS-Mol lacks hard negatives and is roughly balanced.
Substantial curation and audit work
Complete per-target reporting
Honest scoping
Weaknesses
Circularity between decoy construction and headline finding
Permissive, bias-prone activity labels
Small-test-set variance undermines per-target claims
Thin modern-baseline coverage and no inferential statistics
Editorial inconsistencies
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
TopU-LBVS is a benchmark paper for ligand-based virtual screening (LBVS). From ChEMBL 35 the authors curate ChEMBL*, a cleaned bioactivity set covering 93 protein targets in 7 classes (cytochrome P450s, GPCRs, ion channels, kinases, nuclear receptors, proteases, other enzymes; Sec. 3.1, App. A.5). Each target gets a TopU library: curated actives plus hard decoys at a fixed 1:40 ratio (410-14,432 compounds; median 53 actives, minimum 10). Three protocols are fixed: **full** (train on ChEMBL* SAR at 1:10, test on TopU), **low** (6:2:2 or 7:1:2 scaffold splits within TopU, tiered at 50 actives), and **mini** (7 targets with a paired random-decoy control that changes only test decoys; Sec. 4, Table 2). Central claims: (1) random-decoy evaluation inflates LBVS performance nearly 4x — mean EF@1% 21.2 vs 5.5 across ten models and seven mini targets, with random decoys better in 67/70 pairs (Sec. 6.3, Table 5); (2) learned models do not consistently beat fingerprint baselines under hard negatives; (3) the low-data hard-negative regime remains near random enrichment for all ten baselines.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is TopU-LBVS?
A public benchmark for ligand-based virtual screening with 93 protein targets from ChEMBL 35. Each target has a hard screening library of known actives plus property-matched, structurally similar inactive decoys at a 1:40 ratio, together with fixed train/validation/test splits, evaluation code, and results for ten reference models.
Why do AI molecular models score well on older benchmarks but poorly on TopU-LBVS?
Older benchmarks use random or easily separated decoy compounds, so models can rank highly by exploiting simple physicochemical differences rather than recognizing true activity. On TopU's hard decoys, the paper reports mean EF@1% falls from 21.2 to 5.5 across ten models on seven mini targets, indicating much of the earlier performance was an artifact of easy negatives.
Is the TopU-LBVS code and data available?
Yes. The paper states that data, fixed splits, TopU construction scripts, evaluation code, and implementations of all ten baselines are released via a GitHub repository (topu-benchmark/topu-lbvs) and a HuggingFace dataset, with a fixed random seed (2026) for reproducibility. Availability was not independently verified in this review.
What does hard negative mean in this benchmark?
An inactive (decoy) compound chosen to closely match known actives in physicochemical properties and substructure fingerprints, so activity cannot be guessed from simple molecular descriptors. Decoy sets are optimized to minimize the statistical distinguishability between actives and decoys, then audited so easily separable libraries are rejected.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Middle sections were partially truncated in extraction: parts of Sec. 5 baseline descriptions, the Sec. 6.1 aggregate results table (Table 3), and portions of Appendices B/C/H are missing; the review rests on the remaining narrative and per-target tables.
- Figure 1 (pipeline diagram) is not analyzable from the extracted text.
- GitHub and HuggingFace release links are quoted as stated in the paper but could not be verified offline.
- Some appendix result tables (e.g., Tables 27-31) are cut off mid-table in the extraction.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.