Florent Krzakala - TBA
Marc Lelarge - TBA
Alexander S. Wein - Mini-Course on Random Tensor Models
Abstract: A tensor is a multi-dimensional array of numbers, for instance, a "matrix" of size n-by-n-by-n. These objects naturally describe multi-dimensional datasets that occur in various machine learning application. Tensors also arise when applying the method of moments, a cornerstone of statistical inference. I will give a pedagogical overview of some of the basic models and methods that allow us to understand how to best extract useful information from large tensors. A tentative list of topics includes: tensor network notation, the spiked tensor model (tensor PCA), spectral methods based on tensor contractions and flattenings, Kikuchi matrices, tensor decomposition, barriers to efficient computation in tensor problems.
Prerequisites: The course will be self-contained and assume only basic knowledge of linear algebra and probability. Some familiarity with random matrix theory and concentration inequalities may help, but is not required.
Zhenyu Liao - Topics in Random Matrix Theory and Its Applications in Modern Machine Learning
Abstract: The first part of the course will focus on the theory of large-dimensional random matrices. Through an exploration of the sample covariance matrix model, we will elucidate the connections and distinctions between large-dimensional analysis within the double-asymptotic regime using random matrix theory (RMT) and traditional asymptotic statistics. Emphasizing a unified theoretical framework, we will introduce corresponding proof techniques to apply RMT effectively.
The second part of the course will revolve around the applications of RMT in machine learning (ML), particularly in (deep) neural networks. We will delve into recent research advancements in high-dimensional statistics and RMT as well as their intersection with both the theory and practice of deep neural networks. Topics will include, "nonlinear neural network performance analysis", "double descent", and "neural tangent kernel", among others. Additionally, we will outline potential opportunities and challenges associated with leveraging RMT to address fundamental issues in ML.
Prerequisites: Basic knowledge of probability, linear algebra, and familiarity with machine learning concepts.
