BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//CERN//INDICO//EN
BEGIN:VEVENT
SUMMARY:High-dimensional Limit Theorems for Stochastic Gradient Descent: E
 ffective Dynamics and Critical Scaling
DTSTART:20230414T120000Z
DTEND:20230414T140000Z
DTSTAMP:20260614T002400Z
UID:indico-event-9653@indico.math.cnrs.fr
CONTACT:cecile@ihes.fr
DESCRIPTION:Speakers: Gérard Ben Arous (NYU & IHES)\n\nProbability and an
 alysis informal seminar This is a joint work with Reza Gheissari (Northwe
 stern) and Aukosh Jagannath (Waterloo)\, Outstanding paper award at NeurIP
 S 2022. We study the scaling limits of stochastic gradient descent (SGD) 
 with constant step-size in the high-dimensional regime. We prove limit the
 orems for the trajectories of summary statistics (i.e.\, finite-dimensiona
 l functions) of SGD as the dimension goes to infinity. Our approach allows
  one to choose the summary statistics that are tracked\, the initializatio
 n\, and the step-size. It yields both ballistic (ODE) and diffusive (SDE) 
 limits\, with the limit depending dramatically on the former choices. Inte
 restingly\, we find a critical scaling regime for the step-size below whic
 h the effective ballistic dynamics matches gradient flow for the populatio
 n loss\, but at which\, a new correction term appears which changes the ph
 ase diagram. About the fixed points of this effective dynamics\, the corre
 sponding diffusive limits can be quite complex and even degenerate. We dem
 onstrate our approach on popular examples including estimation for spiked 
 matrix and tensor models and classification via two-layer networks for bin
 ary and XOR-type Gaussian mixture models. These examples exhibit surprisin
 g phenomena including multimodal timescales to convergence as well as conv
 ergence to sub-optimal solutions with probability bounded away from zero f
 rom random (e.g.\, Gaussian) initializations. ========Pour être informé
  des prochains séminaires vous pouvez vous abonner à la liste de diffusi
 on en écrivant un mail à sympa@listes.math.cnrs.fr avec comme sujet: "su
 bscribe seminaire_mathematique PRENOM NOM"(indiquez vos propres prénom et
  nom) et laissez le corps du message vide.\n\nhttps://indico.math.cnrs.fr/
 event/9653/
LOCATION:Amphithéâtre Léon Motchane (IHES)
URL:https://indico.math.cnrs.fr/event/9653/
END:VEVENT
END:VCALENDAR