Empirical bayes oracle uncertainty quantification for regression

EDUARD BELITSER, SUBHASHIS GHOSAL

Research output: Contribution to JournalArticleAcademicpeer-review

86 Downloads (Pure)

Abstract

We propose an empirical Bayes method for high-dimensional linear regression models. Following an oracle approach that quantifies the error locally for each possible value of the parameter, we show that an empirical Bayes posterior contracts at the optimal rate at all parameters and leads to uniform size-optimal credible balls with guaranteed coverage under an "excessive bias restriction"condition. This condition gives rise to a new slicing of the entire space that is suitable for ensuring uniformity in uncertainty quantification. The obtained results immediately lead to optimal contraction and coverage properties for many conceivable classes simultaneously. The results are also extended to high-dimensional additive nonparametric regression models.

Original languageEnglish
Pages (from-to)3113-3137
Number of pages25
JournalAnnals of Statistics
Volume48
Issue number6
Early online date11 Dec 2020
DOIs
Publication statusPublished - Dec 2020

Funding

PROOF OF COROLLARY 4. Consider the case pî = pˇi, the other case is similar. Note that θˆ has support Î. By part (ii) of Corollary 1, Î has cardinality |Î| ≤ τ0s(θ0) ≤ τ0s for some constantτ0>0withprobabilityatleast1−exp{−c′′s(θ0)log(ep/s(θ0))},wherec′′>0isa constant. Since (θˆ − θ0) is supported within an index set of cardinality at most (τ0 + 1)s(θ0), by the definition of the compatibility coefficient φ2, it follows that ‖X(θˆ − θ0)‖ ≥ φ2((τ0 + 1)s(θ0))‖X‖max‖θˆ − θ0‖. This leads to the first claim of the corollary. The proof for the ℓ1-case is similar. The last relation is inherited from Theorem 2. □ Acknowledgments. Research of the second author is partially supported by National Science Foundation (NSF) grant number DMS-1510238. A visit to VU Amsterdam was supported by visitor STAR grant from the Netherlands Organisation for Scientific Research (NWO).

FundersFunder number
National Science FoundationDMS-1510238
National Science Foundation
Nederlandse Organisatie voor Wetenschappelijk Onderzoek

    Keywords

    • Coverage
    • Credible ball
    • Empirical Bayes
    • Excessive bias restriction
    • Oracle rate

    Fingerprint

    Dive into the research topics of 'Empirical bayes oracle uncertainty quantification for regression'. Together they form a unique fingerprint.

    Cite this