In recent years, machine learning has motivated the study of "nonlinear random matrices”, that is, random matrices that involve the entry-wise application of some deterministic nonlinear functions. A prominent example is the matrix YY* with Y = f(WX), where W and X are random rectangular matrices with i.i.d. centered entries representing weights and data in neural networks, and f is a nonlinear activation function. This matrix, often referred to as “random feature matrix” or “conjugate kernel” in the literature, has been extensively studied in the case where W and X have light-tailed distributions. In such cases, the limiting spectral distribution of YY* corresponds to the asymptotic empirical spectral distribution of an information-plus-noise type matrix (Pennington-Worah 2017, Benigni-Péché 2021). In this talk, we focus on the case where W has heavy-tailed variables while X remains light-tailed. This setting is motivated by empirical studies on well-trained neural networks, which reveal the emergence of strong correlations in weights, and heavy-tailed distributions offer a realistic framework for modeling these correlations. The resulting entries of Y present significantly stronger correlations than in the light-tailed case, making the spectral analysis completely new. This talk is based on ongoing work with Alice Guionnet.
