Shapley Value Estimation for Multi-Site Data with Blockwise-Missing Features
Siqi Li, Wangxuan Fan, Yiming Li, Doudou Zhou, Molei Liu
stat.ML · cs.LG
Quick verdict
FUSHAP: imputation-free Shapley estimation under multi-site blockwise missingness via influence-function control variates; strong results, informal theory.
Summary
This paper tackles feature-level Shapley value (SV) estimation when evaluation data are split across sites that observe different feature blocks. The authors first show that impute-then-estimate pipelines inject coalition-dependent bias, then propose FUSHAP: starting from an importance-weighted WLS (KernelSHAP-style) estimator computed on a small labeled-complete site, they derive its exact influence function and use regressions of that influence function on partially observed (feature-block, outcome) pairs at auxiliary sites as control variates, reducing variance without imputation. A studentized permutation test screens out distributionally misaligned sites, and closed-form calibration weights size each site's contribution. In Gaussian simulations (p=10, n=300), FUSHAP attains 3.1-8.5x lower MSE than the single-site estimator and roughly 2-3x lower than mean/MICE imputation; screening detects a misaligned site with 82% power at delta=0.25 and 100% at delta>=0.5 while keeping aligned-site false rejections near nominal (4.3%). On Beijing air quality and NACC Alzheimer's data with artificially imposed complementary missingness, MSE improves roughly 2.6-7.4x over single-site, and imputation can underperform the single-site baseline. Limitations: no formal guarantees for the augmented estimator (density-ratio condition assumed, permutation calibration empirical), some experiments violate the alignment assumption, real-data missingness is synthetic, and scale is modest (p<=10).
Review complete
Assessment at a glance
5
risk flags identified
● high 0
● medium 4
● low 1
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Imputing blockwise-missing features before Shapley estimation introduces systematic, coalition-dependent bias that persists across standard imputers and can raise MSE up to 3.1x above imputation-free alternatives (Section 1, Table 1, Model III: 0.062 vs 0.020).
- The constrained WLS (KernelSHAP-style) Shapley estimator admits an exact influence function Sigma^-1[g + lambda*1], enabling per-observation linear decomposition of its estimation error (Section 3.2, Appendix B).
- Control-variate augmentation from partially observed auxiliary sites reduces MSE 3.1-8.5x versus the single-site estimator in simulations (Table 1) and roughly 2.6-7.4x on real data (Tables 3-4), without imputation-induced bias.
- A studentized permutation screening test detects misaligned sites with 82% power at Delta=0.25 and 100% at Delta>=0.5, while aligned-site false rejections average 4.3% at the nominal 5% level (Section 4.2.3, Table 5).
- In NACC data, standard mean imputation can increase MSE above the single-site baseline for 3 of 4 models, up to 3.0x for the MLP (Table 4: 21.53 vs 7.07).
Strengths
Novel, well-posed problem with careful related-work delineation
Section 1.1 cleanly separates feature-level SV under blockwise missingness from site-level data valuation (Wang et al. 2020; Zheng et al. 2023) and from scalar/moment-based data-fusion targets (Xue and Qu 2021; Jin and Rothenhäusler 2023), showing the 2^p coalition structure is genuinely new.
Exact influence-function theory for the preliminary estimator
Appendix B exploits exact linearity of the WLS solution in (b,c) (Eq. 30) to derive the IF psi = Sigma^-1[g + lambda*1] (Eqs. 14-15) with a per-observation Lagrange term enforcing the efficiency constraint; Remark 2 extends to covariate shift via density-ratio weighting.
Complete, principled estimation machinery
Comprehensive and transparent evaluation design
Honesty about regime of validity
Weaknesses
No formal guarantees for the core augmented estimator
Real-data missingness is synthetic
Internal inconsistency in the baseline description
No uncertainty quantification on headline metrics
Modest scale and narrow imputation baselines
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper studies feature-level Shapley value (SV) estimation when the labeled observations evaluating the coalitional game are fragmented across sites with blockwise missingness — each auxiliary site observes the outcome plus a different subset of features — alongside a large unlabeled complete pool (UC) defining the target population (Section 2.1). Four claims: (1) impute-then-estimate injects systematic, coalition-dependent bias (Section 1, Table 1); (2) the constrained-WLS (KernelSHAP/SAGE) estimator admits an exact influence function enabling imputation-free variance reduction via control variates from partially observed sites — FUSHAP (Section 3, Appendix B); (3) a studentized permutation screen removes misaligned sites (Section 3.4); (4) closed-form calibration weights optimally combine sites (Section 3.5, Appendix C). Empirically: 3.1-8.5x lower MSE than the single-site estimator and roughly 2-3x lower than mean/MICE imputation in simulation (Table 1), and about 2.6-7.4x on two real datasets (Tables 3-4).
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 5 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is FUSHAP and what problem does it solve?
FUSHAP is a method for estimating feature-level Shapley values when the data used to evaluate the attribution game come from multiple sites that each record only a subset of features (blockwise missingness). It starts from a Shapley estimate on a small fully observed labeled site, then borrows strength from larger auxiliary sites that observe the outcome plus partial features, using them as control variates — without ever imputing the missing features.
Why is imputation considered harmful for Shapley values?
Shapley attributions depend on the joint covariance structure of the features. Imputation replaces the true distribution with a surrogate one, so every coalition-level value function is evaluated under the wrong distribution. The paper's simulations (Table 1) show impute-then-estimate pipelines carry large squared bias (e.g., 0.111 vs FUSHAP's 0.003 in the linear model), and in NACC data imputation was worse than ignoring the auxiliary sites for 3 of 4 models.
Is the FUSHAP code and data available?
Yes. Code is available at github.com/siqili0325/FUSHAP. The Beijing multi-site air quality dataset is public via the UCI Machine Learning Repository (DOI 10.24432/C5RK5G); the NACC Alzheimer's data are available only upon request through naccdata.org.
When does FUSHAP help the most, and when can it fail?
It helps most when the complete labeled site is small (e.g., n around 300), auxiliary sites are large, the influence function is well predicted from the observed feature blocks (e.g., polynomial-type models), and auxiliary sites are conditionally aligned with the target population. It can fail when the complete site is very small (n=100 caused control-variate overfitting in the paper's Figure 1c) or when a misaligned site is included — which the permutation screening step is designed to catch.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- Figures 1-2 were extracted as text only; curve shapes are inferred from quoted numbers, not from the plots.
- Table 2 (computational cost) is truncated at p=16 in the provided text; the p-scaling claim beyond p=12 cannot be verified.
- The abstract in the metadata is cut mid-sentence; the full embedded text was used for the assessment.
- Reference list formatting is partially garbled by extraction; citation years of a few cited preprints (marked 2026) were taken as printed.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.