Choose timezone
Your profile timezone:
Description
We discuss the training dynamics of deep linear networks from a geometric and dynamical-systems perspective. Gradient descent induces a Riemannian gradient flow on the end-to-end network map, revealing a slow-fast structure in the evolution of its singular values. This provides a dynamical explanation for the implicit bias toward low-rank solutions and connects learning dynamics with invariant-manifold and model-reduction ideas. We show how this structure can inform initialization strategies for accelerating training.
The seminar will exceptionnally take place on Zoom.
Room: https://univ-cotedazur.zoom.us/j/81351738512?pwd=CbI2kc9Q4sGbYbxlhKk3uyQghJVnBJ.1
Password: 776759