Description
Title: Towards Frugal Deep Learning: Alternative to backprop, Data Sketching, and Causal Attention
Abstract: This talk presents recent research conducted by the MILES team, within the framework of the PEPR SHARP project, toward the development of frugal deep learning. As contemporary AI systems continue to grow in scale, computational cost, and energy consumption, there is a need for learning paradigms that are more efficient and sustainable. Our work explores this challenge from three complementary perspectives: (i) learning algorithms that can approximate gradient-based optimization while maintaining strong learning capabilities; (ii) on the way to data efficiency, we explore dynamical sketching to summarize and organize data during the learning process; (iii) trading complexity for expressivity in the design of attention mecanism for causal decoding.
Bio: Since November 2019, Alexandre Allauzen has been Professor at
ESPCI Paris (École Supérieure de Physique et de Chimie Industrielles de la Ville de Paris). He is affiliated with LAMSADE (Laboratoire d’Analyse et de Modélisation de Systèmes pour l’Aide à la Décision) at Université Paris-Dauphine and leads the MILES (Machine Intelligence and Learning Systems) research team. His work focuses on frugal deep learning and its applications to speech processing and natural language processing. More recently, his research has expanded to machine learning for the sciences,
with a particular emphasis on physics, in collaboration with the Institut Langevin (Smart-Waves program). Since September 2025, he has also served as Vice-President for Academic Affairs and Training at Université Paris Sciences et Lettres (PSL), where he contributes to the institution’s academic strategy and educational development.