Incremental Optimization in Machine Learning

par Prof. Nicolas Couellan (Université Paul Sabatier)

vendredi 6 déc. 2013, 10:30 → 11:30 Europe/Paris

X203 (XLIM-Université de Limoges)

After a brief review of support vector machines (SVM) classification formulations and the main available optimization methods to perform training, we will present new first order constrained approaches. The methods exploit the structure of the SVM training problem and combine ideas of incremental gradient technique, gradient acceleration and successive simple calculations of Lagrange multipliers. Both primal and dual formulations will be presented and compared numerically. We will also discuss comparisons with an unconstrained large scale learning algorithm based on stochastic sub-gradient to emphasize that the proposed methods can remain competitive for large scale learning due to the very special structure of the training problem.