Giacomo Zanella

Scalable computation for Bayesian hierarchical models

Abstract: we study MCMC algorithms for Bayesian hierarchical models, the computational complexity of which scales linearly with the number of observations and of parameters in the model. We focus on crossed random effects and nested multilevel models, which are used ubiquitously in applied statistics, and consider methodologies built around Gibbs sampling, sparse linear algebra and belief propagation. For certain combinations of algorithm and model we establish theoretical guarantees for scalability and for others the lack thereof, leveraging connections to random graphs theory and statistical asymptotics. We illustrate the computational methodology on real data analyses on predicting electoral results and real estate prices, comparing with off-the-shelf variational approximations and Hamiltonian Monte Carlo. Our theoretical results, although partial, are useful in suggesting interesting methodologies and lead to conclusions that our numerics suggest to hold well beyond the scope of the underlying assumptions.