MolSC: Leveraging Substituent Contributions to Enhance Fine-grained Molecular Understanding in LLMs
Hyuntae Park, Sooyeon Kim, Jiwon Park, SangKeun Lee
cs.LG · cs.AI
Quick verdict
MolSC (181K substituent contributions) sharply improves LLM substituent-effect prediction, though downstream gains partly stem from its dual-view training.
Summary
The paper introduces MolSC, a training dataset of 181,098 substituent contributions derived from ChEMBL: for each molecule-scaffold pair obtained via Hussain-Rea fragmentation, it records signed property changes across structural alerts (PAINS, Glaxo, BMS, Dundee, MLSMR), same-assay target bioactivity (eight activity types on the pchembl scale), and five physicochemical descriptors. After filtering over-represented scaffolds and trivial substituents (~78% removed), the data is verbalized into forward (attachment) and backward (detachment) multi-turn instruction conversations with a dual-view SMILES+SELFIES representation, and used to LoRA fine-tune Llama-3.2-1B/3B. A held-out benchmark, MolSC-Bench (1,541 examples, disjoint at scaffold/substituent/molecule levels), shows existing molecular LLMs near chance (direction accuracy 0.146-0.454) and GPT-5.2/Gemini-3-Flash at 0.706/0.722, while the trained 3B model reaches 0.923 with propMAE 0.307 on profiling. Downstream, the 3B model attains 0.948 exact match on forward reaction prediction, and the 1B model beats GPT-4o and o3-mini on FGBench. However, ablations show QM9 gains stem mainly from dual-view rather than substituent supervision, and the raw backbone already scores 0.778 on forward reaction prediction, complicating claims of surpassing larger baselines. MAE aggregation across heterogeneous property scales is unexplained, and measured-bioactivity noise is acknowledged. Code and data are released.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Existing molecular LLMs and proprietary frontier models (GPT-5.2 dirAcc 0.706, Gemini-3-Flash 0.722) fail to reliably predict substituent-induced property changes on MolSC-Bench Task 2.
- Training on MolSC (181,098 substituent contributions) lifts direction accuracy to 0.923 (3B) and generalizes to scaffolds, substituents, and molecules unseen in training.
- MolSC training transfers to downstream tasks using only 1D representations, with the 3B model reaching 0.948 exact match on forward reaction prediction and the 1B model beating GPT-4o/o3-mini on FGBench.
- Substituent effects are scaffold-context-dependent, and MolSC-trained models capture direction flips that GPT-5.2 misses (conflict-pair accuracy 0.582 vs 0.145, Table 8).
Strengths
Genuinely novel supervision signal
Explicit signed substituent-level contributions (§3.1) from measured ChEMBL data across three axes fill a real gap: prior molecular datasets pair molecules with whole-molecule labels only. Same-assay pairing for bioactivity deltas (citing Kalliokoski et al. 2013) and dual forward/backward conversations with 5 paraphrasings per turn show construction care.
Unusually thorough leakage control
MolSC-Bench is disjoint from training at scaffold, substituent, and molecule levels (§3.2, Table 1: 1,541 examples), and downstream evaluation molecules are excluded from training (footnote 3, §4.1), making the 0.923 vs 0.706 dirAcc gap (Table 2) a fair generalization test.
Honest attribution where statistics undercut the story
Context-dependence analysis is the most convincing evidence
1D-only representations competitive with graph-based models
Weaknesses
Pooled MAE across heterogeneous scales is uninterpretable
Backbone-strength confound in downstream claims
In-distribution format advantage on the headline benchmark
Thin statistical reporting
Bioactivity axis weaknesses
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
MolSC (arXiv 2609.23073) targets fine-grained structure-property reasoning in molecular LLMs. The authors claim three contributions: (1) a training dataset of 181,098 substituent contributions extracted from ChEMBL, where a contribution is the signed property difference between a molecule and its Hussain-Rea scaffold across structural alerts (PAINS/Glaxo/BMS/Dundee/MLSMR), same-assay target bioactivity (8 activity types on the pchembl scale), and five physicochemical descriptors; (2) MolSC-Bench, 1,541 held-out examples disjoint at scaffold, substituent, and molecule levels; (3) evidence that existing molecular LLMs and GPT-5.2/Gemini-3-Flash fail at substituent-effect prediction (dirAcc 0.146–0.722, Table 2) while MolSC training reaches 0.923 (3B) and transfers downstream. The core claims are mostly supported, with two attribution caveats developed below.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is the MolSC dataset?
MolSC is a training dataset of 181,098 substituent contributions extracted from ChEMBL. Each molecule is split into a scaffold and a substituent, and the contribution is the signed change in structural alerts, target bioactivity, and physicochemical descriptors caused by attaching that substituent. It is used to instruction-tune molecular LLMs (Llama-3.2-1B/3B) to predict how local structural edits change molecular properties.
How much better is the MolSC-trained model than GPT-5.2?
On MolSC-Bench Task 2 (predicting signed property changes upon substituent attachment), the 3B model reaches 0.923 direction accuracy versus 0.706 for GPT-5.2 and 0.722 for Gemini-3-Flash. However, on absolute property profiling (Task 1), GPT-5.2 remains better on the bioactivity error metric (0.657 vs 0.731 MAE).
Is the code and data for MolSC available?
The paper states that data and code are available at github.com/Park-ing-lot/MolSC, and the underlying source database (ChEMBL) is public, so the dataset pipeline can in principle be re-derived with RDKit-based fragmentation.
Does MolSC really beat larger molecular LLMs like 7B graph-based models?
Partly. The 3B model reports 0.948 exact match on forward reaction prediction, above 7B-scale graph-augmented baselines. But the paper's own appendix shows the unmodified 1B backbone already scores 0.778 there, and QM9 property-regression gains come mostly from the dual-view SMILES+SELFIES training trick rather than substituent supervision — so the dataset-specific downstream gain is smaller than the headline suggests.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1–3 are only partially interpretable from extracted text (visual layout, blue/red direction markings in Table 7 inferred from caption).
- Some appendix tables (9–24) are dense and formatting may have been flattened during extraction; individual cell values were cross-checked against prose where possible.
- Proprietary model versions (GPT-5.2, Gemini-3-Flash) are evaluated via API at an unspecified date; exact API prompts beyond Figures 5–6 are not shown.
- The review cannot verify the contents of the released GitHub repository.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.