Gaussian processes and related topics

7 juil. 2025, 14:00 → 9 juil. 2025, 18:00 Europe/Paris

Amphithéâtre Schwartz (Institut de mathématiques de Toulouse)

Scientific / organization commitee 

• François Bachoc (Chair)
• Agnès Lagnoux
• Andrés López-Lopera
• Pierre Neuvial
• Olivier Roustant

 

Mini-courses:

• Gaussian processes in nonparametric statistics, by Ismaël Castillo and Elie Odin
• Constrained Gaussian processes, by Andrés López-Lopera and Mathis Deronzier

 

Confirmed speakers:

• Denis Allard
• François Bachoc 
• Nathalie Bartoli
• Christophette Blanchet-Scalliet
• Iain Henderson
• Agnès Lagnoux
• Conor Moriarty Osborne
• Henry Moss
• Thibault Randrianarisoa
• Paul Rosa
• Botond Szabo

lun. 7 juillet

14:00 → 14:10 Introduction

Orateur: François Bachoc (Institut de Mathématiques de Toulouse)

14:10 → 15:10 Mini-course 1: lecture “Introduction to Gaussian processes” (Chair: Pierre Neuvial)

Orateur: François Bachoc (Institut de Mathématiques de Toulouse)

Bachoc_GAP.pdf

15:10 → 15:55 Long talk: Modeling and simulating spatio-temporal, multivariate and nonstationary Gaussian Random Fields: a Gaussian mixtures perspective (Chair: Pierre Neuvial))

Gaussian Random Fields (GRFs) play a critical role in modeling and simulating environmental and climate-driven processes. The simulation of GRFs enables the representation of the variability of the process under study through the generation of multiple equally plausible realizations. Gaussian vectors corresponding to a sample of moderate size of a GRF can easily be simulated using the Cholesky decomposition of the associated covariance matrix, but this approach is limited to vectors of moderate size. To overcome this limitation, an interesting alternative is to rely on spectral methods that are based on the decomposition of the target GRF into spectral waves. This approach has been recently extended in various directions in order to make it more versatile, including in spatial, multivariate and spatio-temporal settings. To further increase the versatility of spectral simulation methods, we propose to revisit them adopting a Gaussian mixture perspective. This work leverages the Gaussian mixture perspective to propose extensions covering new classes of covariance functions for nonstationary (univariate or multivariate) spatio-temporal GRFs, as well as simulation algorithms for those that are currently missing in the framework of spectral simulation. All simulation methods are translated into pseudo-code algorithms, and an illustration is provided for a bivariate nonstationary spatio-temporal example

Orateur: Denis Allard (INRAE, Avignon)

ALLARD_GaussMixtures.pdf

15:55 → 16:25 Coffee break

16:25 → 16:45 Short talk: On the equivalence and orthogonality of zero-mean Gaussian measures (Chair: Pierre Neuvial)

We discuss some key considerations that are helpful in identifying orthogonal measures, including both established and novel approaches.

Orateur: Michael Hediger (University of Zurich)

16:45 → 18:15 Poster session

mar. 8 juillet

09:00 → 10:15 Mini-course 1: lecture “Constrained Gaussian processes” (Chair: Agnès Lagnoux)

Orateurs: M. Andrés LOPEZ-LOPERA (UPHF), Mathis Deronzier (IMT)

High_dimensional_additive_Gaussian_processes_under_monotonicity_constraints.pdf

slidesConstrGPs_GAP_2025.pdf

10:15 → 10:35 Coffee break

10:35 → 12:00 Mini-course 1: lab session “Gaussian processes and constrained Gaussian processes”

Orateurs: M. Andrés LOPEZ-LOPERA (UPHF), Mathis Deronzier (IMT)

12:00 → 13:15 Lunch break

13:15 → 13:45 Poster session

13:45 → 15:15 Mini-course 2: lecture “Gaussian processes in nonparametric statistics” (Chair: Agnès Lagnoux)

Orateurs: Elie Odin (Institut de Mathématiques de Toulouse), Ismaël Castillo (Sorbonne Université)

Mini_cours_GPs_BNP_final.pdf

15:15 → 15:35 Short talk: On L^2 posterior contraction rates in Bayesian nonparametric regression models (Chair Olivier Roustant))

The nonparametric regression model with normal errors has been extensively studied, both from the frequentist and Bayesian viewpoint. A central result in Bayesian nonparametrics is that under assumptions on the prior, the data-generating distribution (assuming a true frequentist model) and a semi-metric d(.,.) on the space of regression functions that satisfy the so called testing condition, the posterior contracts around the true distribution with respect to d(.,.), and the rate of contraction can be estimated. In the regression setting, the semi-metric d(.,.) is often taken to be the Hellinger distance or the empirical L^2 norm (i.e., the L^2 norm with respect to the empirical distribution of the design) in the present regression context. Typical examples of priors include Gaussian processes for which the theory can be elegantly simplified. However, extending contraction rates to the "integrated" L^2 norm usually requires more work, and has previously been done for instance under sufficient smoothness or boundedness assumptions, which may not necessarily hold. In this work we show that, for priors based on truncated random basis expansions and in the random design setting, a high probability two sided inequality between the empirical L^2 norm and the integrated L^2 norm holds in appropriate spaces of functions of low frequencies, under mild assumptions on the underlying basis (which can be for instance a Fourier, wavelet or Laplace eigenfunction basis), allowing us to directly deduce an L^2 contraction rate from an empirical L^2 one without further assumption on the true regression function. We also discuss extensions to semi supervised learning on graphs, where the basis is estimated from the data itself.

Orateur: Paul Rosa (University of Cambridge)

slides.pdf

15:35 → 16:05 Coffee break

16:05 → 16:50 Long talk: Bayesian optimization applied to constrained black box problems for aeronautical applications (Chair: Olivier Roustant)

Orateur: Nathalie Bartoli (ONERA Toulouse)

ANR_GAP_NBartoli_split.pdf

16:50 → 17:10 Short talk: Convergence rates of deep Gaussian processes (Chair: Olivier Roustant)

Gaussian processes have proven to be powerful and flexible tools for various statistical inference and machine learning tasks. However, they can be limited when the underlying datasets exhibit non-stationary or anisotropic properties. Deep Gaussian processes extend the capabilities of standard Gaussian processes by introducing a hierarchical structure, where the outputs of one Gaussian process serve as inputs to another. This hierarchical approach enables deep Gaussian processes to model complex, non-stationary behaviours that standard Gaussian processes may struggle to capture. In this talk, we introduce deep Gaussian processes and explore their use as priors in interpolation and regression tasks. We present results on the convergence rates of deep Gaussian processes in terms of the number of known data points.

Orateur: Conor Osborne (University of Edinburgh)

GPs_and_related_topics_Toulouse.pdf

17:10 → 18:30 Poster session

19:30 → 22:30 Gala dinner

Les caves de la Maréchale

mer. 9 juillet

08:30 → 10:00 Mini-course 2: lecture “Gaussian processes in nonparametric statistics” (Chair François Bachoc)

Orateurs: Elie Odin (Institut de Mathématiques de Toulouse), Ismaël Castillo (Sorbonne Université)

10:00 → 10:30 Coffee break

10:30 → 11:15 Long talk: Vecchia gaussian processes: Probabilistic properties, minimax rates and methodological developments (Chair François Bachoc)

Gaussian Processes (GPs) are widely used to model dependency in spatial statistics and machine learning, yet the exact computation suffers an intractable time complexity of O(n^3). Vecchia approximation allows scalable Bayesian inference of GPs in O(n) time by introducing sparsity in the spatial dependency structure that is characterized by a directed acyclic graph (DAG). Despite the popularity in practice, it is still unclear how to choose the DAG structure and there are still no theoretical guarantees in nonparametric settings. In this paper, we systematically study the Vecchia GPs as standalone stochastic processes and uncover important probabilistic properties and statistical results in methodology and theory. For probabilistic properties, we prove that the conditional distributions of the Mat\'{e}rn GPs, as well as the Vecchia approximations of the Mat\'{e}rn GPs, can be characterized by polynomials. This allows us to prove a series of results regarding the small ball probabilities and RKHSs of Vecchia GPs. For statistical methodology, we provide a principled guideline to choose parent sets as norming sets with fixed cardinality and provide detailed algorithms following such guidelines. For statistical theory, we prove posterior contraction rates for applying Vecchia GPs to regression problems, where minimax optimality is achieved by optimally tuned GPs via either oracle rescaling or hierarchical Bayesian methods. Our theory and methodology are demonstrated with numerical studies. Based on a joint work with Yichen Zhu

Orateur: Botond Szabo (Bocconi University)

GP_workshop_Toulouse_2025.pdf

11:15 → 12:00 Long talk: To BayesOpt and Beyond (Remote speaker) (Chair François Bachoc)

Bayesian optimisation (BO) pairs Gaussian-process surrogates with exploration-aware acquisition rules to locate the optimum of costly, black-box functions in just a handful of trials. In this introductory talk we unpack how GPs supply calibrated uncertainty that powers the explore-exploit trade-off, walk through the classical BO loop and its staple acquisition functions, and outline practical considerations for noisy, constrained, and moderately high-dimensional settings. We then cast an eye to the GenAI era, sketching how BO’s core ideas adapt to this new landscape.

Orateur: Henry Moss (Lancaster University)

4Francois.pdf

12:00 → 13:30 Lunch break

13:30 → 14:15 Long talk: Kernel based sensitivity analysis for set-valued models (Chair Andrés Felipe Lopez Lopera))

The goal of this work is to use goal-directed sensitivity analysis in order to
reduce the cost of solving a robust optimization problem. Specifically, we focus on
quantifying the impact of uncertain inputs on feasible sets, which are subsets of the
design domain. While most sensitivity analysis methods deal with scalar outputs,
we introduce a novel approach for performing sensitivity analysis with setvalued
outputs. We propose a kernel designed for set-valued outputs and use the Hilbert-Schmidt Independence Criterion (HSIC) . The proposed methodology is implemented to carry out an uncertainty analysis for time-averaged concentration maps of pollutants using a Gaussian Processs Regression as an emulator.

Orateur: Christophette Blanchet-Scalliet (École Centrale de Lyon)

ANR_Toulouse_Blanchet_v2.pdf

14:15 → 15:00 Long talk: Physics-informed Gaussian process priors (Chair Andrés Felipe Lopez Lopera)

Orateur: Iain Henderson (ISAE-SUPAERO)

presentation_anr_gap-1.pdf

15:00 → 15:25 Coffee break

15:25 → 16:10 Long talk: Asymptotics for constrained Gaussian processes (Chair Andrés Felipe Lopez Lopera)

Orateur: Agnès LAGNOUX

lagnoux_GAP_07_25_GB.pdf

16:10 → 16:30 Short talk: Variational Gaussian processes for linear inverse problems (Chair Andrés Felipe Lopez Lopera)

By now Bayesian methods are routinely used in practice for solving inverse problems. In inverse problems the parameter or signal of interest is observed only indirectly, as an image of a given map, and the observations are typically further corrupted with noise. Bayes offers a natural way to regularize these problems via the prior distribution and provides a probabilistic solution, quantifying the remaining uncertainty in the problem. However, the computational costs of standard, sampling based Bayesian approaches can be overly large in such complex models. Therefore, in practice variational Bayes is becoming increasingly popular. Nevertheless, the theoretical understanding of these methods is still relatively limited, especially in context of inverse problems.

In our analysis we investigate variational Bayesian methods for Gaussian process priors to solve linear inverse problems. We consider both mildly and severely ill-posed inverse problems and work with the popular inducing variable variational Bayes approach. We derive posterior contraction rates for the variational posterior in general settings and show that the minimax estimation rate can be attained by correctly tuned procedures. As specific examples we consider a collection of inverse problems (heat equation, Volterra operator and Radon transform) and inducing variable methods based on population and empirical spectral features.

Orateur: Thibault Randrianarisoa (University of Toronto)

RandrianarisoaGAPWorkshop2025.pdf