BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//CERN//INDICO//EN
BEGIN:VEVENT
SUMMARY:Rodrigo Maulen (Laboratoire de Probabilités Statistique & Modéli
 sation) "Attention-based clustering"
DTSTART:20251119T093000Z
DTEND:20251119T103000Z
DTSTAMP:20260510T014600Z
UID:indico-event-15163@indico.math.cnrs.fr
DESCRIPTION:Transformers have emerged as a powerful neural network archite
 cture capable of tackling a wide range of learning tasks. In this work\, w
 e provide a theoretical analysis of their ability to automatically extract
  structure from data in an unsupervised setting. In particular\, we demons
 trate their suitability for clustering when the input data is generated fr
 om a Gaussian mixture model. To this end\, we study a simplified two-head 
 attention layer and define a population risk whose minimization with unlab
 eled data drives the head parameters to align with the true mixture centro
 ids. This phenomenon highlights the ability of attention-based layers to c
 apture underlying distributional structure..\n\nhttps://indico.math.cnrs.f
 r/event/15163/
URL:https://indico.math.cnrs.fr/event/15163/
END:VEVENT
END:VCALENDAR