Learning low-dimensional functions with neural networks: random biases and hidden structure

par Elisabetta Cornacchia (Inria Ens Paris)

jeudi 11 déc. 2025, 14:00 → 15:00 Europe/Paris

The success of deep learning in high-dimensional settings is often attributed to low-dimensional structure in real-world data, which reduces both the statistical and computational complexity of learning. Standard theoretical models typically assume that this structure lies in the target function, for example in single- or multi-index models that depend on a low-dimensional projection of otherwise unstructured inputs. In the first part of the talk, I will show that, under mild conditions, additive noise in the data, in the form of random biases, can make certain low-dimensional targets tractable, even when they are challenging to learn in the noiseless setting. In the second part, I will introduce a model that places structure directly in the input space, as observed in natural data, rather than only in the target. Specifically, I will consider targets that depend on a small number of latent Boolean variables, with input features grouped into clusters and correlated with these latent variables. Under an identifiability assumption, I will show that the sample complexity scales with the number of latent variables and—when the signal-to-noise ratio is sufficiently high—is essentially independent of the ambient input dimension.