When Features Become Instances: Inverted Contrastive Learning for Unsupervised Feature Selection
Utsab Ghosh, Roshni Chakraborty
cs.AI · cs.CL
Quick verdict
Claims best clustering accuracy on 10/12 datasets via feature-wise contrastive norm ranking; plausible but gains small, no significance tests, sloppily edited.
Summary
The paper reformulates unsupervised feature selection (UFS) as a contrastive representation-learning problem over features rather than samples. ICLFS transposes the data matrix so each feature is an instance described by its sample-profile vector, builds four masked positive views and one sample-shuffled negative view, and trains a single-head self-attention encoder with a residual MLP projector under an InfoNCE objective plus an inter-feature decorrelation regularizer. Features are ranked by the L2 norm of their unnormalized projector embeddings — motivated by evidence that InfoNCE training inflates norms of "prototypical" instances — and the ranking is refined post hoc by Laplacian-Gated Ranking Correction, which swaps out top-ranked features with poor Laplacian scores. On 12 scikit-feature benchmarks, evaluated by k-means clustering accuracy (20 runs, six cardinalities), ICLFS is best on 10 datasets, tied-second on RELATHE, and loses on ALLAML to LS-CAE (66.60% vs 71.11%); largest gains on PROSTATE (83.97% vs 60.78%) and NCI9 (52.33% vs 41.08%). Main limitations: differences of ~0.4–0.5% on several datasets are within one standard deviation and receive no significance testing; results rest on a single training run per configuration; hyperparameters (two sample-aware regimes, lambda_decorr, LGRC's alpha and q) were tuned on the same benchmarks; DUFS and SSFS are discussed but not compared; no runtime analysis despite O(d^2) attention; and the text contains leftover editing notes.
Review complete
Assessment at a glance
6
risk flags identified
● high 2
● medium 3
● low 1
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Reformulating UFS as feature-wise contrastive learning — features as instances, masked positives, a same-feature shuffled negative — yields feature rankings more effective than neighborhood-, cluster- and reconstruction-based criteria.
- ICLFS achieves the best k-means clustering accuracy on 10 of 12 scikit-feature benchmarks against LS, MCFS, NDFS, SPEC, CAE and LS-CAE, tying second on RELATHE and losing only on ALLAML.
- Projector-space embedding norm is a more reliable saliency signal than encoder norm or score-level fusions (Table 6), leveraging InfoNCE's documented norm-growth behavior.
- All pipeline components contribute: decorrelation, attention and structured multi-view masking account for most gains, while LGRC adds a small last-mile refinement (Table 5).
Strengths
Clean and novel problem reformulation
Transposing X so each feature's sample-profile becomes a contrastive instance (Sec. 3.3), with masked positives plus a same-feature shuffled negative (Sec. 3.4), is a well-motivated inversion of standard sample-wise contrastive learning; the shuffled negative is a sharper control than an unrelated-feature negative.
Principled saliency signal with supporting ablation
Ranking by projector-norm is grounded in cited evidence on InfoNCE norm growth (Refs. 14–18), and Table 6 shows z-norm beats h-norm and two fusion rules on 4/5 datasets (e.g., WARPPIE10P 44.14 vs 43.36/41.38), not just an asserted design choice.
Honest, informative ablation ladder
Transparent negative results
Reproducibility assets
Weaknesses
No statistical support for the headline claim
Training variance unquantified
Hyperparameters tuned on the evaluation benchmark
Missing closely related baselines
Editorial defects and no cost analysis
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
This is a method paper proposing ICLFS, an unsupervised feature selection (UFS) framework that treats features — not samples — as contrastive learning instances. The authors claim three things: (1) a novel reformulation, transposing X ∈ R^{n×d} to R^{d×n} and learning consistency over structured masked views (Secs. 3.3–3.4); (2) projector-space embedding norm as the saliency signal, motivated by published evidence that InfoNCE training inflates the norms of 'prototypical' instances (Refs. 14–18, Secs. 2.3, 3.6.1); and (3) empirical superiority — best k-means clustering accuracy on 10 of 12 scikit-feature benchmarks against LS, MCFS, NDFS, SPEC, CAE and LS-CAE (Table 4). Claims 1–2 are clearly argued; claim 3 is where the evidence is statistically thinnest.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 6 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is different about ICLFS compared to standard contrastive learning?
Standard contrastive learning treats samples as instances. ICLFS transposes the data matrix so each feature becomes an instance described by its values across all samples; positive views are masked versions of that feature profile, the negative view is the same profile with sample order shuffled, and features are ranked by the norm of their learned projector embeddings instead of by reconstruction or graph-based scores.
How well does ICLFS perform?
On 12 standard UFS benchmarks it reports the best k-means clustering accuracy on 10 datasets, ties for second on RELATHE, and trails LS-CAE on ALLAML (66.60% vs 71.11%). Gains are largest on PROSTATE (83.97% vs CAE's 60.78%) and NCI9 (52.33% vs LS's 41.08%); several other wins are smaller than one percentage point and untested for significance.
Is code and data available for ICLFS?
The authors state the implementation is public at github.com/neilghos/ICLFS, and all 12 datasets come from the public scikit-feature repository; the full hyperparameter specification (architectures, learning rates, masking ratios, LGRC settings) is listed in Section 4.3 of the paper.
What is Laplacian-Gated Ranking Correction (LGRC)?
A post-processing step applied after contrastive training: it takes the top alpha·k features from the norm-based ranking, removes any candidate whose Laplacian score (a locality-preservation measure on the original data) falls above the 75th-percentile threshold, refills from the remaining ranked list, then returns the top-k features by original norm order. The ablation shows it contributes only 0.34–1.14% accuracy.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figure-internal labels of Figs. 1–3 are extraction-garbled (diagram text dumped as blocks); all substantive content was reconstructed from captions and body text. The provided text ends cleanly at reference [24] with no truncation.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.