Orateur
Description
We introduce a novel framework for optimization based on energy-conserving Hamiltonian dynamics in a strongly mixing (chaotic) regime and establish some of its key properties analytically and numerically. The prototype is a discretization of Born-Infeld dynamics, with a squared relativistic speed limit depending on the objective function. This class of frictionless, energy-conserving optimizers proceeds unobstructed until slowing naturally near vanishing loss (up to a self-tunable hyper-parameter shift), which dominates the phase space volume of the system. Building from studies of chaotic systems such as dynamical billiards, we formulate a specific algorithm with good performance on machine learning and PDE-solving tasks, including generalization (so far studied at small scale). In progress are experiments on applications to computational chemistry, sampling, and larger-scale ML, along with further theoretical study of its impact on representation/feature learning. An application of this ML-inspired method to numerical PDE solving for string compactifications blends three of Mike’s many far-reaching insights.
