Séminaire de Statistique et Optimisation

Straight-Through meets Sparse Recovery: the Support Exploration Algorithm

par Mimoun Mohamed (I2M AMU)

Europe/Paris
Salle K. Johnson, 1er étage (1R3)

Salle K. Johnson, 1er étage

1R3

Description

The straight-through estimator (STE) is commonly used to optimize quantized neural networks, yet its contexts of effective performance are still unclear despite empirical successes.
To make a step forward in this comprehension, we apply STE to a well-understood problem: sparse support recovery. 
We introduce the Support Exploration Algorithm (SEA), a novel algorithm promoting sparsity, and we analyze its performance in support recovery (a.k.a. model selection) problems. 
SEA explores more supports than the state-of-the-art, leading to superior performance in experiments, especially when the columns of A are strongly coherent.
The theoretical analysis considers recovery guarantees when the linear measurements matrix A satisfies the Restricted Isometry Property (RIP).
The sufficient conditions of recovery are comparable but more stringent than those of the state-of-the-art in sparse support recovery. Their significance lies mainly in their applicability to an instance of the STE.