Séminaire de Probabilités commun ICJ/UMPA

Architectural bias in a transport-based generative model : an asymptotic perspective

par Hugo Cui

Europe/Paris
435 (ENS de Lyon)

435

ENS de Lyon

Description

We consider the problem of learning a generative model parametrized by a two-layer auto-encoder, and trained with online stochastic gradient descent, to sample from a high-dimensional data distribution with an underlying low-dimensional structure. We provide a tight asymptotic characterization of low-dimensional projections of the resulting generated density, and evidence how mode(l) collapse can arise.  On the other hand, we discuss how in a case where the architectural bias is suited to the target density, these simple models can efficiently learn to sample from a binary Gaussian mixture target distribution. Based on joint works with Yue M Lu, Cengiz Pehlevan, Lenka Zdeborová, Florent Krzakala and Eric Vanden-Eijnden.