Bayesian Computation and Inference with Misspecified Models

Satellite Workshop BayesComp 2025

Date: 16-17 June 2025.

Location: National University of Singapore.

This BayesComp 2025 Satellite Workshop is about methods, theory and applications for Bayesian inference with misspecified models.

Outline

A common justification for the use of Bayesian inference is that Bayes’ theorem is the optimal way to update beliefs based on new observations, and that representing beliefs through a posterior distribution is desirable for uncertainty quantification. However, standard posterior distributions are only meaningful when the model or likelihood is well-specified, which is not the case in the presence of outliers, adversarial contaminations, or faulty measurement instruments. This realisation has led to an increased focus on generalisations of Bayesian inference which aim to produce ‘generalised posterior distributions’ that provide some representation of uncertainty but also overcome some of the lack of robustness of standard posteriors. The aim of this workshop will be to give a broad overview of this topic, touching on both foundational questions and algorithmic advances, and inviting the BayesComp community to take a more active role in solving some of the remaining open challenges in this area.

Call for Poster Sessions Abstracts

We would like to invite the submisison of abstracts for the poster session on the second afternoon. Abstracts can be submitted here and should be limited to 200 words. The deadline for abstract submission is 1st May 2025. A selection of accepted posters will be given the oppourtunity to present their work in the spotlight session on Monday afternoon.

Programme

The tentative programme for this workshop is below. This programme is subject to change.

Monday 16 June

• 09.00-10.30 Tutorial: Jeremias Knoblauch - post-Bayesian Inference
• 10.30-11.00 Coffee
• 11.00-11.45 Invited Talk: Antonietta Mira
• 11.45-12.30 Invited Talk: Jeremie Houssineau - Robust Bayesian inference with possibility theory
• 12.30-14.00 Lunch
• 14.00-15.30 Poster Spotlight Talks
• 15.30-16.00 Coffee
• 16.00-17.00 Invited Talk: Sonia Petrone
• 17.00-18.00 Invited Talk: David Frazier

Tuesday 17 June

• 09.00-09.45 Invited Talk: Eli Weinstein
• 09.45-10.30 Invited Talk: Aretha Teckentrup
• 10.30-11.00 Coffee
• 11.00-11.45 Invited Talk: Edwin Fong - Model misspecification in martingale posteriors
• 11.45-12.30 Invited Talk: Harita Dellaporta - Model-based Distributionally Robust Optimisation: Bayesian Ambiguity Sets and Model Misspecification
• 12.30-14.00 Lunch
• 14.00-15.30 Poster Session
• 15.30-16.00 Coffee
• 16.00-17.00 Invited Talk: Pierre Alquier - A new mutual information bound for satistical inference
• 17.00-18.00 Invited Talk: TBC

Speakers

• Jeremias Knoblauch (University College London, UK).
  
  Title: Post-Bayesian Inference.
  Abstract: In this talk, I provide my perspective on our community's efforts to develop inference procedures with Bayesian characteristics that go beyond Bayes' Rule as an epistemological principle. I will explain why these efforts are needed, as well as the forms which they take. Focusing on some of my own contributions to the field, I will trace out some of the community's most important milestones, as well as the challenges that lie ahead. Throughout, I will provide success stories of the field, and emphasise the new opportunities that open themselves up to us once we dare to go beyond orthodox Bayesian procedures.
  
• Antonietta Mira (Università della Svizzera italiana, Switzerland).
  
  
• Jeremie Houssineau (Nanyang Technological University, Singapore).
  
  Title: Robust Bayesian inference with possibility theory
  Abstract: In Bayesian inference, the marginal likelihood is one of the main quantities for model selection based on its ability to capture the fitness of the model. Yet, when reformulating Bayesian inference in the context of possibility theory, the marginal likelihood no longer favours simpler models and cannot be used directly for model selection. What it can do however, is capture the coherence between the model and the observation(s). This possibilistic marginal likelihood therefore provides a simple solution to make the underlying inference more robust, leveraging useful properties that are specific to possibility theory. Illustrations will be given for multiple problems of varying difficulty, focusing on robustness to outliers.
  
• Sonia Petrone (Bocconi University, Italy).
  
  
• David Frazier (Monash University, Australia).
  
  
• Eli Weinstein (Columbia University).
  
  
• Aretha Teckentrup (University of Edinburgh, UK).
  
  
• Edwin Fong (University of Hong Kong).
  
  Title: Model misspecification in martingale posteriors
  Abstract: The martingale posterior is a framework where posterior uncertainty is directly generated through predictive imputation. The Bayesian model can then be directly specified using a sequence of predictive distributions, bypassing the need for explicit likelihood and prior specifications. Given the absence of a likelihood however, it is not immediately clear what the definition or impact of misspecification is for martingale posteriors. This talk will investigate predictive notions of misspecification, the consequences they have on parameter inference, and potential remedies.
  
• Harita Dellaporta (University College London, UK).
  
  Title: Model-based Distributionally Robust Optimisation: Bayesian Ambiguity Sets and Model Misspecification.
  Abstract: Decision making under uncertainty is challenging as the data-generating process (DGP) is often unknown. Bayesian inference proceeds by estimating the DGP through posterior beliefs about the model's parameters. However, minimising the expected risk under these posterior beliefs can lead to sub-optimal decisions due to model uncertainty or limited, noisy observations. This talk will address this problem by introducing Distributionally Robust Optimisation with Bayesian Ambiguity Sets (DRO-BAS) which hedges against model uncertainty by optimising the worst-case risk over a posterior-informed ambiguity set. Simulations show DRO-BAS Pareto dominates existing Bayesian DRO formulations when evaluating the out-of-sample mean-variance trade-off, while achieving faster solve times. However, when the model is misspecified, this can lead to over-conservative decisions as the DGP might not be contained anymore in the ambiguity set. I will briefly discuss how Bayesian Ambiguity Sets can be easily adjusted to address this challenge by introducing DRO with Robust, to model misspecification, Bayesian Ambiguity Sets. These are expected Maximum Mean Discrepancy ambiguity sets under a robust posterior that incorporates beliefs about the DGP. The resulting optimisation problem obtains a dual formulation in the Reproducing Kernel Hilbert Space for any choice of model family.
  
• Pierre Alquier (ESSEC Business School, Singapore).
  
  Title: A new mutual information bound for satistical inference.
  Abstract: Recent advances in statistical learning theory have revealed profound connections between mutual information (MI) bounds, PAC-Bayesian theory, and Bayesian nonparametrics. This work introduces a novel mutual information bound for statistical inference. The derived bound has wide-ranging applications. It yields improved contraction rates for fractional posteriors in Bayesian nonparametrics. It can also be used to study variational inference or Maximum Likelihood Estimation (MLE). By bridging these diverse areas, this work advances our understanding of the fundamental limits of statist ical inference and the role of information in learning from data. We hope that these results will not only clarify connections between statistical inference and information theory but also help to develop a new toolbox to study a wide range of estimators.
  

Organisation

• François-Xavier Briol, UCL (f.briol@ucl.ac.uk)
• Jack Jewson, Monash University (jack.jewson@monash.edu)
• Jeremias Knoblauch, UCL (j.knoblauch@ucl.ac.uk)