Sticky PDMP samplers for sparse and local inference problems

Joris Bierkens, Sebastiano Grazzi*, Frank van der Meulen, Moritz Schauer

*Corresponding author for this work

Research output: Contribution to JournalArticleAcademicpeer-review

Abstract

We construct a new class of efficient Monte Carlo methods based on continuous-time piecewise deterministic Markov processes (PDMPs) suitable for inference in high dimensional sparse models, i.e. models for which there is prior knowledge that many coordinates are likely to be exactly 0. This is achieved with the fairly simple idea of endowing existing PDMP samplers with “sticky” coordinate axes, coordinate planes etc. Upon hitting those subspaces, an event is triggered during which the process sticks to the subspace, this way spending some time in a sub-model. This results in non-reversible jumps between different (sub-)models. While we show that PDMP samplers in general can be made sticky, we mainly focus on the Zig-Zag sampler. Compared to the Gibbs sampler for variable selection, we heuristically derive favourable dependence of the Sticky Zig-Zag sampler on dimension and data size. The computational efficiency of the Sticky Zig-Zag sampler is further established through numerical experiments where both the sample size and the dimension of the parameter space are large.

Original languageEnglish
Article number8
Pages (from-to)1-31
Number of pages31
JournalStatistics and Computing
Volume33
Issue number1
Early online date28 Nov 2022
DOIs
Publication statusPublished - Feb 2023

Bibliographical note

Funding Information:
this work is part of the research programme Bayesian inference for high dimensional processes with project number 613.009.034c, which is (partly) financed by the Dutch Research Council (NWO) under the Stochastics—Theoretical and Applied Research (STAR) grant. J. Bierkens acknowledges support by the NWO for the research project Zig-zagging through computational barriers with project number 016.Vidi.189.043.

Publisher Copyright:
© 2022, The Author(s).

Funding

this work is part of the research programme Bayesian inference for high dimensional processes with project number 613.009.034c, which is (partly) financed by the Dutch Research Council (NWO) under the Stochastics—Theoretical and Applied Research (STAR) grant. J. Bierkens acknowledges support by the NWO for the research project Zig-zagging through computational barriers with project number 016.Vidi.189.043.

Keywords

  • Bayesian variable selection
  • Big-data
  • High-dimensional problems
  • Monte Carlo
  • Non-reversible jump
  • Piecewise deterministic Markov process
  • Spike-and-slab

Fingerprint

Dive into the research topics of 'Sticky PDMP samplers for sparse and local inference problems'. Together they form a unique fingerprint.

Cite this