Conformalized Quantile Regression and Minimax Limits of Fixed-Score Calibration under Known Covariate Shift
Rustam Isaev, Anton Conrad, Denis Belomestny, Eric Moulines, Sergey Samsonov
math.ST · cs.LG · stat.ML
Quick verdict
Derives finite-sample Lp bounds for CQR length/coverage under known covariate shift, with matching minimax fixed-score benchmarks; rigorous, honest, assumption-bounded theory.
Summary
This theory paper studies split conformalized quantile regression (CQR) with finite-sample, high-probability Lp bounds (p ∈ [1,∞]) on interval-length deviation from the equal-tailed oracle and on the realized conditional-coverage profile, jointly over training and calibration folds (Proposition 1, Theorem 2; localization-free L1 case in Proposition 3). The only learner-specific input is an Lp error bound for the two fitted quantile endpoints, instantiated for sparse-ReLU pinball ERM at rate n^{-β/(2β+d)}(log n)^{3/2} (Theorem 10). Under known, bounded covariate shift, weighted CQR inherits Lp(QX) length and coverage guarantees with calibration radius ~sqrt((1+χ²)/m) (Theorem 5). Two constructed fixed-score calibration benchmarks yield matching expected minimax bounds: Le Cam two-point lower bound c_α·sqrt((1+κ)/m) with the weighted rule achieving (3/2)·sqrt((1+κ)/m), and a K-atomic Varshamov–Gilbert/χ²-Fano lower bound sqrt((1+κ)α(1−α)K/m) holding with probability ≥ 1−2e^{−K/32} for K ≥ 23, matched for finite p. Numerics are exact Binomial–Beta evaluations plus a small learned-CQR simulation. Main limitations: known bounded likelihood ratio, oracle-fixed score in the benchmarks, and no p=∞ upper bound for the K-threshold problem.
Review complete
Assessment at a glance
3
risk flags identified
● high 0
● medium 0
● low 3
5 strengths · 5 weaknesses · 5 author questions
AI text signal = Binoculars zero-shot detector score over 20 sampled passages (Fast-DetectGPT co-reported) — absence does not prove human authorship, and a flag does not imply misconduct.
Levels reflect verification complexity — not a quality score. Evolve never rates your paper.
Key claims under review
- Split CQR satisfies finite-sample high-probability Lp(PX) bounds on oracle-length deviation and realized conditional-coverage profile, for every p ∈ [1,∞], under local mass conditions (A1) plus any learner meeting an Lp endpoint error rate (A2(p)).
- Under known bounded covariate shift, weighted split CQR controls the realized Lp(QX) coverage profile and length with calibration radius of order sqrt((1+χ²(QX‖PX))/m).
- On the single-atom fixed-score benchmark, the weighted conformal rule is minimax-rate optimal in expectation for every p ∈ [1,∞], matching the Le Cam lower bound up to α-dependent constants.
- On the K-atomic benchmark, every calibration rule suffers expected loss ≥ (1/64)√((1+κ)α(1−α)K/m) with probability ≥ 1−2e^{−K/32} (K ≥ 23), and the atomwise split conformal rule matches this rate for every finite p.
- Sparse-ReLU pinball ERM verifies the endpoint condition A2(2) at rate n^{-β/(2β+d)}(log n)^{3/2}, yielding end-to-end Corollaries 11–12 for CQR with and without shift.
Strengths
Matching minimax results with fully worked constants
Theorem 6 gives lower bound c_α·sqrt((1+κ)/m) with explicit c_α = (1/8)√(log 2 · α(1−α)); Proposition 7 achieves (3/2)·sqrt((1+κ)/m) via a Binomial–Beta identity, and Appendix E reports the exact Gaussian-limit constant √(2α(1−α)/π) ≈ 0.2394 at α = 0.1.
Clean modularity of assumptions
A2(p) is the only learner-specific condition: any endpoint learner with a high-probability Lp(PX) error bound plugs into Proposition 1/Theorem 2. Theorem 10 instantiates it for sparse-ReLU pinball ERM via Schmidt-Hieber approximation plus an ε-net Bernstein inequality (Appendix F).
High-probability Fano lower bound with exponential confidence
Complete, verifiable proofs including constant verification
Honest, precise scoping of claims
Weaknesses
Known bounded likelihood ratio is a strong assumption
Fixed-score benchmarks fix the score at the oracle
No p = ∞ upper bound for the K-threshold problem
Localization conditions required for the p > 1 CQR bounds
Numerical study is small-scale and partly illustrative
Full Review Report
$1.99 · one-timeUnlock the complete preliminary analysis Evolve produced for this paper.
Report preview
Scope & Claims
The paper targets a precise gap: split CQR (Romano et al., NeurIPS 2019) guarantees only marginal coverage, and conditional validity is impossible distribution-free (Vovk 2012; Bian–Barber). The authors instead ask a *post-hoc profiling* question: after both folds are realized, how far are the interval length and the full conditional-coverage function from the equal-tailed oracle, in Lp(PX) for every p ∈ [1,∞]? The answer (Proposition 1, Theorem 2, eqs. 10–11) decomposes into an endpoint term εp(n, δ/4), a DKW calibration term ηm(δ) = √(log(4/δ)/2m), and the explicit rank slack sm ∈ [(1−α)/m, (2−α)/m] (eq. 8). Proposition 3 removes localization for the L1 coverage bound. Under covariate shift with known bounded w (L1), Theorem 5 transfers these controls to Lp(QX) with radius ηw_m(δ) (eq. 21) scaling as √((1+χ²)/m). The minimax section then asks whether the √((1+χ²)/m) dependence is an artifact of the weighted rule; the two fixed-score benchmarks prove it is not. Claims are carefully scoped — all minimax statements are tied to the constructed benchmarks, and the K ≥ 23 condition and m > m⋆(K) sample-size requirement (eq. 31) are stated in the theorem, not buried in proofs.
The remaining sections cover Methodology Assessment, Statistical Evidence, Results & Reproducibility, Limitations & Risks, and Verdict & Recommendations.
- All 5 strengths & 5 weaknesses, analyzed in detail
- 3 risk flags with severity, section & reviewer questions
- Methods & statistical evidence assessment
- 5 questions a peer reviewer would ask
- Full six-dimension evaluation report & recommendations
One-time purchase · this paper only
One-time purchase, this paper only. AI-generated preliminary analysis — not peer review.
Frequently asked questions
What is conformalized quantile regression (CQR) and what does this paper add?
CQR fits the two conditional quantile endpoints on a training set and then expands or contracts both endpoints by one common scalar calibrated on a held-out calibration set, giving marginal coverage 1−α. This paper adds finite-sample, high-probability Lp bounds on how far the realized interval length is from the ideal (equal-tailed oracle) length and on how uneven the conditional coverage is across covariates, including under covariate shift where the test inputs follow a different distribution.
What does covariate shift mean here and why does the χ² divergence appear in the rates?
Covariate shift means training/calibration inputs follow distribution PX while test inputs follow QX, with the response mechanism Y|X unchanged. The rates depend on 1+χ²(QX‖PX), which equals the expected squared likelihood ratio and quantifies how 'far' the target distribution is from the source: the effective calibration size becomes m/(1+χ²), so stronger shift costs proportionally more calibration data. This is proved assuming the likelihood ratio w = dQX/dPX is known and bounded.
Are the minimax lower bounds proved for CQR in general?
No. They are proved for two specifically constructed 'fixed-score' benchmarks: a single-target-atom problem (one scalar threshold, Le Cam two-point method) and a K-target-atom problem (K thresholds, Fano method). In both, the conformal score is held fixed at its oracle value and only the response kernel varies. The results show calibration alone already incurs the √((1+χ²)/m) and √((1+χ²)K/m) costs, not that every CQR problem has these limits.
Is code available for the numerical experiments?
The paper does not state any code or data release. However, experiments E1 and E2 are exact evaluations of Binomial–Beta order-statistic identities (no simulation needed beyond Monte Carlo cross-checks), and E3 is a synthetic-data simulation with fully specified parameters (α = 0.1, R = 200 replications, seed 20260826), so the figures are in principle reproducible from the text.
About this review
This is an AI-generated preliminary review by Evolve.Pub, produced from the openly available text of the paper. It is not peer review and does not represent the authors' views. The canonical record of the paper is at the publisher link above.
- The extracted text has gaps: the statements of Proposition 4, Theorem 5, and Corollary 12 (shifted main theorems) are partially elided, though their full proofs in Appendices C and F.8 are present and were used to reconstruct the claims.
- Figures (1–3) are available only as caption text; plot quality and visual agreement between curves could not be assessed.
- Section 7 and Appendix F rely on cited external results (Schmidt-Hieber 2020; Madrid Padilla et al. 2022); those papers' exact statements were not verified.
- Reference [18] and [30] carry 2026 dates and could not be cross-checked; the related-work positioning rests on the authors' descriptions.
Go deeper on this paper
Wrote a paper? Stress-test it before reviewers do.
Upload your manuscript for the same structured scrutiny — reasoning, statistics, evidence, figures, and reproducibility.