Oct 17 – 20, 2023
LAAS
Europe/Paris timezone

Clustering in large language models: an interacting particle systems perspective

Oct 20, 2023, 9:00 AM
1h
Salle de Conférence (LAAS)

Salle de Conférence

LAAS

Speaker

Borjan Geshkovski

Description

With remarkable empirical success, Transformers enable large language models to compute succinct representations of data using the self-attention mechanism. We model these architectures as interacting particle systems in the spirit of models in collective behaviour and opinion dynamics, allowing us to show the appearance of various clustering/coagulation phenomena. Associated control problems will also be discussed. Based on joint work with Cyril Letrouit, Yury Polyanskiy, and Philippe Rigollet.

Presentation materials