Coarse-Graining Hidden Representations: Unsupervised Neuron Selection via Mapping Entropy
Margherita Mele, Andrea Castagna, Roberto Menichetti, Raffaello Potestio, Alessandro Ingrosso
cs.LG · cond-mat.dis-nn · cond-mat.stat-mech
Quick verdict
Paper applies mapping-entropy minimization to pick neuron subsets; results are coherent on small synthetic tasks but scope is narrow and baselines thin.
Summary
The paper recasts neuron selection as coarse-graining of a trained network's hidden layer. Hidden activations are binarized by sign, and candidate subsets of ncg neurons are scored by the mapping entropy (ME), a KL divergence quantifying the loss of distinguishability when the full configuration ensemble is replaced by a reduced one; ME-minimizing subsets are found by exhaustive search or simulated annealing (EXCOGITO), and Wang-Landau sampling maps the landscape. In teacher-student soft committee machines (M=2,K=10; M=5,K=25), at exact replica matching (η=0) the ME optimum is the minimal balanced subset (ncg=M), and the minimal balanced size grows with the mismatch η and with the trained network's residual mismatch (η_eff≈0.04). In a nonlinear Gaussian-process classifier whose 30 hidden neurons split into localized and oscillatory classes, ME selects coherent single-class subsets whose preferred class flips from localized to oscillatory during training, confirmed by a density-of-states analysis. On pruning for NLGP and translation-augmented MNIST (1 vs 7, K=30), ME-selected subsets outperform random subsets of equal size, most clearly at strong compression (ncg≈8-12 of 30), with only the output bias re-optimized. Limitations: very small single-hidden-layer networks, no comparison with supervised pruning criteria, no error bars or statistical tests, and no ablation of the binarization choice.
Review complete
Assessment at a glance
4
risk flags identified
● high 0
● medium 4
● low 0
4 strengths · 0 weaknesses · 5 author questions
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Minimizing the mapping entropy over binarized hidden configurations yields a fully unsupervised neuron-selection criterion that depends only on activation statistics.
- In teacher-student settings, ME minimization recovers the minimal teacher-consistent representation at exact replication, and the minimal balanced subset size directly tracks the residual mismatch of the hidden layer.
- In the NLGP task, ME selects coherent mappings drawn from a single functional class (localized or oscillatory), and the preferred class shifts from localized to oscillatory over training.
- ME-selected subnetworks outperform equal-size random subsets after pruning on NLGP and translation-augmented MNIST, with the advantage clearest under strong compression.
Strengths
Controlled ground-truth validation
TS construction with tunable replica mismatch η (Eq. 7) gives a falsifiable target: at η=0 the ME optimum is balanced at ncg=M (FIG. 1e,f), and trained networks with η_eff≈0.04 (FIG. 1d) land on the same in-silico branches — a clean validation loop most unsupervised-selection papers lack.
Landscape-level evidence, not just point estimates
The Wang-Landau 1/t density of states at fixed ncg=15 (FIG. 2f) shows a bimodal landscape at epoch 300 and basin migration to purely oscillatory mappings by epoch 1000, with exact combinatorial normalization — ruling out a sampling artifact of the annealing.
Quantified mechanism for the NLGP class competition
Calibrated claims and fair evaluation protocol
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper (Mele, Castagna, Menichetti, Potestio, Ingrosso; arXiv:2609.05126) imports the mapping-entropy optimization workflow (MEOW) from biomolecular coarse-graining into neural-network analysis. Hidden activations of trained one-hidden-layer networks are binarized by sign (Eq. 1); a neuron subset of size ncg defines a decimation mapping scored by the mapping entropy Smap (Eq. 2), a KL divergence between the empirical microscopic distribution and the uniform back-mapped reconstruction. Central claims: (i) ME minimization is a fully unsupervised neuron-selection criterion; (ii) in teacher-student (TS) settings it recovers the minimal teacher-consistent subset at exact replication and tracks residual mismatch thereafter; (iii) in the nonlinear Gaussian-process (NLGP) task it selects coherent single-functional-class subsets whose preferred class shifts during training; (iv) ME-selected subnetworks beat equal-size random subsets under pruning, most clearly at strong compression. Claims (i)-(iii) are well supported in scope; (iv) is supported only against a random baseline.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 0 weaknesses, analyzed in detail
- 4 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is mapping entropy for neuron selection?
It is the KL divergence between the full distribution of binarized hidden-activation patterns of a trained network and the distribution reconstructed after keeping only a subset of neurons. A lower value means the kept subset preserves more of the original configurations' distinguishability, so minimizing it over subsets of a fixed size selects the most informative neurons — using only activation statistics, no labels.
Does this neuron-selection method need labels or gradients?
No. The method (called MEOW) scores candidate neuron subsets solely from the statistics of hidden-layer activations, making it fully unsupervised. Labels are used only afterwards, to evaluate the pruned network's accuracy against random-subset baselines.
How well does mapping-entropy pruning perform?
On 30-neuron networks for a synthetic Gaussian-process classification task and for MNIST digits 1 vs 7 with random translations, ME-selected subsets beat random subsets of the same size. The advantage is clearest under strong compression (keeping about 8-12 of 30 neurons) and shrinks as more neurons are kept; after pruning only the output bias is re-optimized.
Is code available for this paper?
The text contains no code or data availability statement. It relies on EXCOGITO, a previously published coarse-graining toolbox by some of the same authors, for the mapping-entropy optimization and Wang-Landau sampling.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figure images (curves, heatmaps) are not part of the extraction — only captions and in-text numbers were available, so all quantitative judgments rely on values stated in the text (e.g., Smap 0.26 vs 0.31 at ncg=15).
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.