Accelerating MCMC for Computationally Intensive Models by Natesh Pillai

Program

Advances in Applied Probability II (ONLINE)

ORGANIZERS: Vivek S Borkar (IIT Bombay, India), Sandeep Juneja (TIFR Mumbai, India), Kavita Ramanan (Brown University, Rhode Island), Devavrat Shah (MIT, US) and Piyush Srivastava (TIFR Mumbai, India)

DATE: 04 January 2021 to 08 January 2021

VENUE: Online

Applied probability has seen a revolutionary growth in research activity, driven by the information age and exploding technological frontiers. Applications include the internet and the world wide web, social networks, integrated supply chains in manufacturing networks, the highly intertwined international economies, and so on. The common thread running through these is that they are large interconnected systems that are emergent with very little top down design to optimize them. Probabilistic methods with limit theorems as their mainstay are best suited to find structure and regularity to help model, analyze and optimize such systems. Interface of probability with learning theory, optimization and control, and statistics has been the driving force behind the emerging paradigms, techniques and mathematics to address the huge scale of problems seen in such technological and commercial applications, not to mention several in biological or physical systems. In this six day program on advances in applied probability, we will have some of the leading researchers from the field conduct short courses in emerging areas, including:

Statistical learning theory
High dimensional computation
Monte Carlo methods
Discrete probability
Percolation
Empirical methods in probability

The program will include sixteen research talks mostly between 5:30 pm to 8:30 pm IST, spread over five days.

0:00:11 Accelerating MCMC for computationally intensive models
0:01:45 Focus of the Talk
0:03:27 Outline
0:03:43 Motivating Problem: Intractable Likelihoods from Computer Experiments
0:03:52 A terrestrial example: ice sheet dynamics in western Antarctica
0:04:36 How readily is heat absorbed by the ice? How much mixing occurs near the ice-ocean interface? Can we infer these parameters to calibrate our models?
0:05:00 Solves Boussinesq, hydrostatic so-called primitive equations of ocean dynamics, simplified from Navier-Stokes: DVn
0:05:57 Other Applications
0:08:04 Markov chain Monte Carlo (MCMC)
0:08:37 Posterior contours
0:09:48 Given Xo O, simulate chain {Xt}ten according to transition kernel: MH Kernel Koo(x, .)
0:11:33 Applications to large data sets
0:11:46 Given Xo O, simulate chain {X+}ten according to transition kernel: MH Kernel Koo(x, .)
0:12:06 Surrogates: A popular idea
0:15:37 Asymptotically exact MCMC via local approximations
0:16:28 Sampling from the exact posterior: We take a different approach:
0:18:15 Given Xo, initialize a sample set So, then simulate chain {Xt} with kernel:
0:19:52 Our algorithm
0:20:53 Idea I: Local approximations
0:21:45 Advantages: Approximation converges locally under loose conditions [Cleveland] For example, quadratic approximations over BR
0:23:13 Early times
0:23:48 Random refinement Bt With probability Bt, such that Et Bt = co,
0:27:40 Locally space filling refinement To space fill near St = Xt or Et = qt,
0:28:48 Our algorithm entails 3 natural statistical ideas! Local Approximations Refinement: True model evaluations
0:29:50 Theoretical Guarantees
0:29:59 Theorem (Conrad, Marzouk, P., Smith 2018) Let the log-posterior be approximated with local quadratic models;
0:31:29 Sketch of exactness proof
0:32:55 In some set O' C O, an infinite number of model evaluations happen
0:34:27 Proof slightly different when is compact 2 Need uniform control of approximations in the tails.
0:35:48 Then, if the underlying Markov chain Ktrue is ergodic for 7,
0:35:58 Theoretical Insights
0:37:03 Connections to Adaptive MCMC II
0:37:18 Convergence of Adaptive MCMC involves checking two conditions (Roberts and Rosenthal, 2007) 2 Containment Condition:
0:37:55 Many algorithmic variations: Target of approximation Forward model: f(0)
0:38:25 Example: elliptic PDE inverse problem
0:38:34 Karhunen-Loeve expansion: log k(x) = _=1 0; vidi(x). Standard Gaussian priors on 0;
0:41:00 Example: genetic toggle switch
0:41:06 Genetic toggle switch posterior
0:41:19 Genetic toggle switch: accuracy versus cost
0:41:49 Coupled system of nonlinear PDEs relates ice stream velocities to basal friction, ice thickness,
0:42:00 Showing 6 of 12 parameters:
0:42:10 MacAyeal ice stream model: computational performance
0:42:52 Possible Pitfalls
0:43:46 MCMC is generally a serial algorithm and parallelization remains a challenge Fortunately,
0:43:52 Conclusions
0:46:03 Does approximation affect mixing? Does approximation affect tail behaviour and recurrence/transience? Is the approximation so non-uniform that the implied constants in our