3rd Edition of Mathematics for and by Large Language Models

jeudi 28 mai 2026, 09:00 → 15:00 Europe/Paris

Centre de conférences Marilyn et James Simons (Le Bois-Marie)

The goal of this conference is to advance the dialogue and interactions between the LLM community and the larger world of mathematics in order to further the mathematical understanding of LLMs and contribute to solving some of the outstanding problems in the new field of LLMs.

 

In particular we intend to investigate mathematical structures that can be used to understand LLMs in terms of what they implicitly learn and how. 

 

At the same time, in the opposite direction the use of LLMs in order to do mathematics will be investigated.

 

Registration is free and open until May 20, 2026.

Invited speakers:
Quentin Berthet (Google DeepMind)
Edward Lockhart (Google DeepMind)
Gabriel Peyré (CNRS, DMA, École Normale Supérieure)
Yiannis Vlassopoulos (Athena Research Center & IHES)

Organizers: 
Michael Douglas (Harvard University & IHES), Amaury Hayat (CERMICS), Julio Parra-Martinez (IHES) and Yiannis Vlassopoulos (Athena Research Center & IHES)

The math and LLM day and the workshop on AI for the study of Amplitudes, are supported by the Google DeepMind AI for Math Initiative, and the IHES thanks Google DeepMind for their support.

 

Math-LLM-Conf-2026.pdf

09:00 → 09:30 Welcome coffee

09:30 → 10:30 TBA

TBA

Orateur: Quentin Berthet (Google DeepMind)

10:30 → 11:30 The Expressive Power of Large Language Models

Large language models process vast sequences of input tokens by alternating between classical multi-layer perceptron layers and self-attention mechanisms. While the approximation capabilities of perceptrons are relatively well understood, those of attention mechanisms remain less explored. In this talk, I will compare the proof techniques and approximation results associated with these two types of layers, emphasizing key open questions that connect large language models with approximation theory in infinite-dimensional spaces representing input token distributions.

Orateur: Gabriel Peyré (CNRS, DMA, École Normale Supérieure)

11:30 → 12:00 Coffee break

12:00 → 13:00 ReLU and Softplus Neural Nets as Zero-Sum, Turn-Based, Stopping Games

Neural networks are for the most part treated as black boxes.
In an effort to understand the mathematical structure that underlies them we will explain how ReLU neural nets can be interpreted as zero-sum, turn-based, stopping games.

The game runs in the opposite direction to the net. The input to the net is the terminal reward of the game, the output of every neuron turns out to be equal to the value of the game at a corresponding state. The weights are used to define state-transition probabilities and the biases to define rewards.
Running the ReLU net becomes the same as running the Shapley-Bellman backwards recursion (which in this case is minimax dynamic programming) for the value of the game.

As an application, we obtain bounds for the output of every neuron of the net, given bounds for the input to the net.

Moreover, the game interpretation links the ReLU net with statistical mechanics, interpreting the output of every neuron as a discrete path integral.
We will also explain consequences of the game point of view, to interpretability of the net considered as a classifier.

Adding an entropic regularization to the ReLU net game, allows us to interpret Softplus neural nets as games in an analogous fashion.

This is joint work with Stéphane Gaubert.

Orateur: Yiannis Vlassopoulos (Athena Research Center & IHES)

13:00 → 14:00 Lunch - Buffet

14:00 → 15:00 TBA

TBA

Orateur: Edward Lockhart (Google DeepMind)