PU classification under Non-SCAR: clustering-assisted logistic model with oversampling enhancement
Konrad Furmańczyk, Kacper Paczutkowski
stat.ML · cs.AI · cs.LG
Quick verdict
Claims SMOTE-enhanced cluster-Lasso PU methods win under SCAR violations; plausible, but tested on one synthetic SAR mechanism with no significance tests.
Summary
This paper addresses positive–unlabeled (PU) classification when the SCAR assumption is violated, i.e., the SAR setting where labeling probability depends on features. The authors extend their previously proposed cluster-based 'pecking' cleaning procedure — mixing a fraction of labeled positives into the unlabeled pool, deriving cleaning labels by 2-means clustering, and fitting logistic regression averaged over 5 repetitions — with strict and non-strict Lasso variants, adding SMOTE oversampling before the clustering step to counter class imbalance. Evaluation covers 13 UCI datasets and one synthetic dataset, with label frequencies c = 0.3/0.5/0.8 and a feature-dependent surrogate-labeling mechanism built from the two highest-variance features; baselines are the naive approach, Spy-EM, and LassoJoint. Reported results: SMOTE improved AUC in 69% of experiments for the cluster method and about 56% for the Lasso-cluster variants, versus 16% for naive; at c = 0.8, AUC gains exceed 10% and relative accuracy gains reach about 61%. LassoJoint is largely insensitive to SMOTE. Cluster-Lasso methods are competitive under non-SCAR, while LassoJoint remains strongest under SCAR. Main limitations: a single constructed SAR mechanism, no significance tests or confidence intervals, accuracy-heavy reporting on imbalanced data, and no SAR-specific baselines such as SAR-EM or double logistic regression despite citing them.
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 3
● low 2
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Integrating SMOTE before cluster-based cleaning improves PU classification when SCAR is violated: AUC improved in 69% of experiments for the clust method (Table 1) and about 56% for the strict/non-strict Lasso-cluster variants.
- At label frequency c = 0.8, SMOTE yields AUC gains above 10% and relative accuracy gains up to 60.8% for the cluster-Lasso methods (Table 2).
- LassoJoint is largely insensitive to SMOTE (improved AUC in only 35.4% of experiments) and remains near-optimal under SCAR but is not consistently superior under non-SCAR.
- Under non-SCAR, cluster-Lasso methods outperform naive and Spy-EM especially in Accuracy and F1 (e.g., +30.5% relative ACC over naive at c = 0.3, Table 3), while naive stays competitive in AUC as c grows.
- The 2-means-based cleaning first step extends beyond single-sample PU to case-control PU learning (stated as prospective applicability, not tested).
Strengths
Systematic SMOTE ablation at scale
Table 1 quantifies improved/worsened/equal counts over roughly 3,660 experiments per method (SMOTE improved AUC in 69% (2,526) of clust experiments vs 16.4% (600) for naive), across both SCAR and non-SCAR schemes and c ∈ {0.3, 0.5, 0.8}.
Controlled label-frequency design
Section 3.3 varies c = 0.3/0.5/0.8 and the conclusions consistently interpret the transition toward full supervision (naive gaining AUC as c grows, LassoJoint near-optimal under SCAR).
Strong reproducibility package
Negative results reported plainly
Computationally cheap pipeline
Weaknesses
Single hand-constructed SAR mechanism
No uncertainty quantification
Accuracy-dominant, threshold-sensitive reporting
Missing SAR-specific baselines
Editorial defects in results reporting
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper (arXiv:2609.14675, stat.ML) studies positive–unlabeled (PU) classification when the SCAR assumption P(S=1|Y=1,X)=c fails — the SAR regime where labeling propensity e(x) depends on features. Stated contributions (Sections 1–2): (i) integrate SMOTE oversampling into the authors' previously proposed cluster-based 'pecking' cleaning procedure (their ref. [10]) with strict and non-strict Lasso variants, and (ii) assess the robustness of LassoJoint [7,23] under SCAR violations. Central claims: SMOTE materially helps clustering-based PU methods under non-SCAR; cluster-Lasso hybrids beat naive and Spy-EM there; LassoJoint stays stable but does not benefit from SMOTE.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is PU learning and the SCAR assumption?
Positive–unlabeled (PU) learning is classification when only some positive examples are labeled (S = 1) and all remaining samples are unlabeled (S = 0), a mix of positives and negatives. SCAR ('selected completely at random') assumes positives become labeled with a constant probability c independent of their features. When the labeling probability depends on features, the setting is called SAR (selected at random) and is substantially harder; this paper targets that harder setting.
Does SMOTE help in positive-unlabeled learning according to this paper?
For clustering-based cleaning methods, yes: SMOTE improved AUC in 69% of the paper's experiments for the cluster method and about 56% for its strict/non-strict Lasso variants, versus only 16% for naive logistic regression on surrogate labels. The joint LassoJoint method was largely unaffected. Gains were largest at label frequency c = 0.8: AUC improvements above 10% and relative accuracy improvements up to roughly 61%.
Is the code and data for this paper available?
Yes. The authors state that all R code is included in Supplement 1 and hosted on GitHub (github.com/kapacc/mdai26). Experiments use 13 public UCI Machine Learning Repository datasets plus one synthetic dataset, with standard R libraries including glmnet (Lasso) and the smotefamily SMOTE implementation, run on the ICM UW Topola server.
What does 'non-SCAR' mean in this paper?
It means the probability that a positive instance receives a label depends on its features — the SAR (selected at random) setting, violating SCAR's constant labeling rate. The authors simulate this by constructing surrogate labels from the two highest-variance features using cumulative sums of the true label, with a linear approximation tuned to hit a target labeling frequency c.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figure 1 (AUC, SMOTE scheme, non-SCAR) is unrecoverable from the extracted text, so per-dataset method rankings rest on the prose rather than the figure.
- The supplementary material (absolute per-dataset tables, q-sensitivity in Sections 1.1/2.1, and the non_scar_labelling code) was not part of the extraction, so several robustness claims could not be verified.
- In-text table cross-references appear off by one, so attributing each number to its table required verification against the printed captions; the mapping used here reflects the captions.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.