Importance Weighting for Unlabeled-unlabeled Learning under Distribution Shift

Summary

The paper addresses distribution shift adaptation for unlabeled-unlabeled (UU) learning, where a binary classifier is learned from two unlabeled sets with different class-priors, and the training and test distributions may differ arbitrarily. The authors extend importance weighting to this setting by rewriting both the test risk and the squared-error objective for relative density-ratio estimation as UU risks computable from unlabeled data (Sections 4.2-4.3), assuming the class-priors are known. A weighted combination of the test-distribution UU risk and the importance-weighted training UU risk (Eq. 21) is optimized by alternating importance-weight estimation and classifier learning with a shared feature extractor (Algorithm 1). Because UU learning subsumes PN, PU, noisy-label, pairwise-comparison, and similarity learning, the method covers these problems under shift within one framework without shift-type assumptions. Experiments on MNIST, FMNIST and CIFAR10 with constructed support and input-output shifts, plus DIABETES and FOODSTAMP with real shifts, show best-or-comparable accuracy in 6 of 7 configurations against five baselines (Table 1), including settings where supervision differs between training and test distributions (Table 2). Limitations: no formal consistency or error-bound analysis; baselines are largely author-constructed variants; all class-priors must be known, with accuracy dropping by up to about 10 points under prior noise (Table 4); and many improvements are within statistical noise of the strongest baselines.