Entropy-Punctured Bloom Filters for Memory-Efficient Machine Learning
John Cartmell, Mihaela Cardei, Ionut Cardei
cs.LG · cs.AI
Quick verdict
Entropy-punctured Bloom filters for compact regression encodings; plausible, leakage-controlled results, but weak statistical testing and no code release.
Summary
This paper extends Bloom Filter (BF) feature encoding from classification to regression and adds a compression step: discretized features are encoded into fixed-length bit vectors, then bit positions whose marginal entropy falls below a threshold are removed, exploiting the fact that near-constant bits carry little variability. On eight public datasets spanning tabular, time-series, and text regression, ridge regression, XGBoost, and an MLP are compared against raw features, PCA, and random projection under leakage-free, storage-matched protocols, with a 'predictive efficiency' metric (R² per encoded byte) proposed as a summary. Punctured BFs beat the size-matched PCA/RP fidelity on four of eight datasets and raw features on only one (Yelp, 0.675 vs 0.669), trailing raw features by up to 0.115 (Airline); because encodings are far smaller, they rank first on predictive efficiency on seven of eight datasets (random projection wins on Bike). On Parkinson's, τ=0.15 puncturing halves storage (64 to about 32 bytes) at R²=0.8830±0.0083. Main limitations: most results lack variance estimates or significance tests; the PE metric is nonstandard and gated by a 90%-of-best fidelity rule; the feature-hashing baseline covers only two datasets with fixed dimensions; the parameter-selection heuristic is underspecified; and no code is released.
Review complete
Assessment at a glance
4
risk flags identified
● high 0
● medium 2
● low 2
4 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Bloom Filter encodings of quantized features support regression fidelity competitive with PCA and random projection at substantially smaller per-sample storage (Tables III/IV).
- Removing bit positions with marginal entropy below threshold τ halves the Parkinson's encoding (64→~32 bytes) with negligible fidelity loss (R²=0.8830±0.0083, Section V-E).
- Punctured BFs achieve the highest predictive efficiency (R² per byte) on seven of eight datasets under matched storage budgets (Table IV).
- Low-entropy bit removal is far less damaging than high-entropy or random removal, locating predictive signal in variable bit positions (Figure 6).
- At matched storage, PBF retains much higher fidelity than signed feature hashing on California (0.7622 vs 0.0844) and Parkinson's (0.8830 vs 0.3654) (Table VI).
Strengths
Disciplined leakage-free protocol
Sections III-B/III-C and IV-B state that entropy statistics, puncturing masks, BF parameter selection, and PCA/RP fits are computed on training partitions only; Bike uses a chronological 50/50 split and Airline forward splits; fixed seeds are reported for hashing, shuffling, and model init.
Unusually honest metric self-critique
Section IV-I explicitly calls PE (R²/bytes) a descriptive ratio that 'can favor extremely small representations,' gates PBF configs at ≥90% of best dataset R², and the paper repeatedly warns (Sections III-B, V-A, VI) that marginal entropy is label-independent, not a relevance measure.
Informative structural ablations
Broad evaluation surface
Weaknesses
Near-total absence of uncertainty quantification
Headline rests on a metric that structurally favors PBF
Feature-hashing baseline narrow and possibly under-tuned
Parameter selection not reproducible
Single-dataset structural sweeps
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper (arXiv 2609.14187, cs.LG) extends the authors' prior Bloom-filter (BF) feature-encoding work for classification to regression and adds a structural compression step. The pipeline: numeric, categorical, and text features are discretized or tokenized into 'feature=value' symbols, encoded into an m-bit BF with k hash functions, and bit positions whose empirical marginal entropy Hj falls below a threshold τ are removed ('punctured'). Three contributions are claimed: (i) the entropy-punctured BF (PBF) method, (ii) a storage-normalized 'predictive efficiency' metric PE = R²/B (R² per encoded byte), and (iii) a controlled structural analysis covering bit-removal strategies, permutation, OR-folding, and hash families. Evaluation spans eight public datasets (Table I) with ridge regression, XGBoost, and an MLP, against raw features, PCA, and random projection (RP) at approximately matched byte budgets, plus a focused feature-hashing (FH) control on two datasets.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 4 strengths & 5 weaknesses, analyzed in detail
- 4 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is an entropy-punctured Bloom filter?
It is a compressed feature encoding for machine learning: data attributes are first hashed into a fixed-length Bloom filter bit vector, then bit positions that are almost always 0 or 1 across the training data (low empirical entropy) are deleted. This shrinks the encoding — for example from 64 to about 32 bytes per sample on one benchmark — while largely preserving the accuracy of downstream regression models.
Does it beat PCA or random projection?
Not on raw accuracy: raw features achieve the best R² on six of eight benchmark datasets, and the punctured Bloom filter beats the size-matched PCA/random-projection baseline on four of eight. Its advantage is accuracy per stored byte: because the encoding is much smaller, it ranks first on that efficiency measure on seven of eight datasets.
Is code or data available for this paper?
The paper does not mention a public code release. All eight datasets are public (UCI, Kaggle, Yelp), per-dataset filter sizes and entropy thresholds are listed in a table, and fixed random seeds are reported, but the heuristic used to choose those parameters is only described qualitatively.
What is 'predictive efficiency' in this paper?
The paper's summary metric: the R² of a regression model divided by the encoded representation size in bytes per sample. It is a descriptive, storage-normalized measure rather than an information-theoretic quantity, and the authors interpret it jointly with R² because tiny encodings can otherwise inflate the ratio.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1-7 are not analyzable from the extracted text; the parameter-sweep, occupancy, and removal-strategy findings rest on figure content not visible here.
- Table II extraction is slightly garbled (m/Bytes header alignment); values were interpreted as encoded sizes in bytes.
- The abstract is truncated mid-sentence ('yielding improved...') in the source text; conclusions were cross-checked against the body.
- No appendix or supplementary detail on the parameter-selection heuristic is visible in the provided text.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.