Orateur
Description
Distributionaly robust optimization with Wassersein-distance uncertainty sets proves to be an outstanding tool to handle data heterogeneity and distribution shifts; see (Kuhn et al.)[2].
Recently, (Azizian et al.)[1] studied regularizations of WDRO problems. From a risk minimization problem $\min_{\theta\in\Theta}\mathbb{E}_\xi[\ell_\theta(\xi)]$ (ERM), it provides a dual formula (WDRO) for a robust estimator:
$$
\begin{align*}
\min_{\theta\in\Theta;\lambda \geq 0} & \lambda \rho + \varepsilon ~ \frac1{N}\sum_{i=1}^N \left[ \log\left(\frac1{M}\sum_{j = 1}^M
\left[ \exp{\left(\frac{\ell_\theta(\zeta_{ij}) - \lambda \|\xi_i - \zeta_{ij}\|^2}{\varepsilon }\right)}\right]\right) \right]\\
& \xi_i ~ \text{(N samples from the dataset)}\\
& \zeta_{ij} \sim \mathcal{N}(\xi_i, \sigma^2 I) ~ \text{(M "robustness" test points per sample).}
\end{align*}
$$
In this talk, we introduce $\texttt{SkWDRO}$ (Vincent et al.)[3], a python library that solves this problem efficiently, with multiple benefits:
- it provides fine treatment of numerical aspects and choices of hyper-parameters
- it offers a scikit-learn interface to solve some popular convex problems
- it revolves around a PyTorch interface to reformulate a given $\ell_\theta$ loss module into its robust counterpart in only a few lines of code.
[1]: Azizian Waïss, Iutzeler Franck and Malick, Jérôme: **Regularization for Wasserstein distributionally robust optimization**. *ESAIM COCV, 2023*
[2]: Wiesemann Wolfram, Kuhn Daniel and Sim Melvyn: **Distributionally robust convex optimization**. *Operations research, INFORMS, 2014*
[3]: Florian Vincent and Waïss Azizian and Franck Iutzeler and Jérôme Malick: **$\texttt{skwdro}$: a library for Wasserstein distributionally robust machine learning**. ArXiV, 2024
