14h15-15h15 Gabriel Peyré (ENS)
The Expressive Power of Large Language Models
Abstract: Large language models process vast sequences of input tokens by alternating between classical multi-layer perceptron layers and self-attention mechanisms. While the approximation capabilities of perceptrons are relatively well understood, those of attention mechanisms remain less explored. In this talk, I will compare the proof techniques and approximation results associated with these two types of layers, emphasizing key open questions that connect large language models with approximation theory in infinite-dimensional spaces representing input token distributions.
15h30-16h30 Vianney Perchet (Criteo et ENSAE)
AI-powered AI. Robustness, Consistency and Smoothness of Algorithms.
Abstract: AI systems no longer operate as isolated algorithms, machines, or agents; instead, they
interact with each other. An algorithm designed for a specific task might use the output or
prediction of another as its input. This has serious consequences: blindly relying on another
algorithm can be extremely helpful if it is accurate, but devastating if it is not, as a single error -
even the tiniest one - could break down the whole system. New algorithms will need to incorporate
these predictions to improve their consistency, while remaining robust against errors. We will use
simple toy examples to show how the generic framework of learning-augmented algorithms (LAA) is
perfectly suited for this challenge, highlighting the complexity of designing practical and potentially
vastly more efficient algorithms.