TBA

• Quentin Berthet (Google DeepMind)

Room: Centre de conférences Marilyn et James Simons TBA

The Expressive Power of Large Language Models

• Gabriel Peyré (CNRS, DMA, École Normale Supérieure)

Room: Centre de conférences Marilyn et James Simons Large language models process vast sequences of input tokens by alternating between classical multi-layer perceptron layers and self-attention mechanisms. While the approximation capabilities of perceptrons are relatively well understood, those of attention mechanisms remain less explored. In this talk, I will compare the proof techniques and approximation results associated with these two types of layers, emphasizing key open questions that connect large language models with approximation theory in infinite-dimensional spaces representing input token distributions.

ReLU and Softplus Neural Nets as Zero-Sum, Turn-Based, Stopping Games

• Yiannis Vlassopoulos (Athena Research Center & IHES)

Room: Centre de conférences Marilyn et James Simons Neural networks are for the most part treated as black boxes. In an effort to understand the mathematical structure that underlies them we will explain how ReLU neural nets can be interpreted as zero-sum, turn-based, stopping games. The game runs in the opposite direction to the net. The input to the net is the terminal reward of the game, the output of every neuron turns out to be equal to the value of the game at a corresponding state. The weights are used to define state-transition probabilities and the biases to define rewards. Running the ReLU net becomes the same as running the Shapley-Bellman backwards recursion (which in this case is minimax dynamic programming) for the value of the game. As an application, we obtain bounds for the output of every neuron of the net, given bounds for the input to the net. Moreover, the game interpretation links the ReLU net with statistical mechanics, interpreting the output of every neuron as a discrete path integral. We will also explain consequences of the game point of view, to interpretability of the net considered as a classifier. Adding an entropic regularization to the ReLU net game, allows us to interpret Softplus neural nets as games in an analogous fashion. This is joint work with Stéphane Gaubert.

Why AI Needs Formal Mathematics

• Edward Lockhart (Google DeepMind)

Room: Centre de conférences Marilyn et James Simons Current reinforcement learning methods train Large Language Models to generate outputs that satisfy an automated judge. While this drives impressive feats of reasoning, it inadvertently incentivises the superficial appearance of correctness. Models may learn to "reward hack" by glossing over logical flaws or confidently making false claims. In this talk, I will explore how some AI researchers are turning to formal verification to solve this illusion of competence. By pairing LLMs with proof assistants, we can shift AI training from adversarial reward-maximisation to a cooperative process where reward hacking becomes impossible. I will also examine the broader implications of this emerging capability, discussing how "formalisation on-demand" can serve as a substitute for human social credibility and lay the groundwork for fully autonomous AI mathematical research.