May 9 – 13, 2022
Le Bois-Marie
Europe/Paris timezone

Remote talk - BI for AI: Energy Conserving Dynamics for Optimization and Sampling (T: 50mn + Q: 10mn)

May 10, 2022, 4:30 PM
1h
Centre de conférences Marilyn et James Simons (Le Bois-Marie)

Centre de conférences Marilyn et James Simons

Le Bois-Marie

Speaker

Prof. Eva SILVERSTEIN (Stanford University)

Description

We introduce a novel framework for optimization based on energy-conserving Hamiltonian dynamics in a strongly mixing (chaotic) regime and establish some of its key properties analytically and numerically. The prototype is a discretization of Born-Infeld dynamics, with a squared relativistic speed limit depending on the objective function. This class of frictionless, energy-conserving optimizers proceeds unobstructed until slowing naturally near vanishing loss (up to a self-tunable hyper-parameter shift), which dominates the phase space volume of the system. Building from studies of chaotic systems such as dynamical billiards, we formulate a specific algorithm with good performance on machine learning and PDE-solving tasks, including generalization (so far studied at small scale). In progress are experiments on applications to computational chemistry, sampling, and larger-scale ML, along with further theoretical study of its impact on representation/feature learning. An application of this ML-inspired method to numerical PDE solving for string compactifications blends three of Mike’s many far-reaching insights.

Presentation materials