Solving image inverse problem with deep generative models

par Jean Prost (IRIT)

mardi 30 juin 2026, 11:15 → 12:15 Europe/Paris

Salle K. Johnson (1R3, 1er étage)

Image inverse problems involve recovering a clean image from a degraded observation. Such problems arise in numerous applications, including photography, medical imaging, and astrophysics. These problems are typically ill-posed because part of the image information is lost or corrupted during the acquisition process.

Under a Bayesian framework, the posterior distribution of the solution can be modeled by combining information from the degraded observation with prior knowledge about the solution. Deep generative models provide powerful priors, but working with the resulting posterior distribution remains challenging due to the complexity of these models. 

I will present an optimization algorithm for computing an (augmented) maximum a posteriori (MAP) estimator when the prior is induced by a hierarchical variational autoencoder (HVAE). The proposed method leverages the HVAE encoder to avoid backpropagation through the generative model, thereby reducing the computational cost of inference.

I will then introduce a posterior sampling algorithm based on a text-to-image latent consistency model (LCM). By exploiting the fast generation capabilities of LCMs, the method produces high-quality posterior samples in only a few iterations (approximately eight). The text prompt can either be specified in advance to guide the reconstruction or automatically recovered through a prompt-optimization procedure that seeks the most likely textual description of a degraded observation. The proposed approach outperforms competing methods based on other text-to-image generative models while being significantly faster and more memory-efficient.