Orateur
Description
This talk introduces the foundations of fairness in artificial
intelligence (AI), from its legal definitions and practical
operationalizations to its applications and limitations. The notion of
fairness naturally raises the question of auditing: since fairness
objectives may conflict with predictive performance or other business
incentives, AI providers cannot necessarily be expected to prioritize
ethical considerations voluntarily. Regulatory frameworks therefore
increasingly establish requirements for AI systems, alongside potential
sanctions for non-compliance. However, ensuring that these requirements
are effectively implemented requires independent and reliable auditing
mechanisms.
We will explore what constitutes a fairness audit, how effective
existing auditing approaches can be, which methodologies are best suited
to different contexts, and to what extent an auditee with incentives to
conceal discriminatory practices might manipulate or circumvent the
auditing process. In particular, we will discuss vulnerabilities arising
from the manipulation of evaluation datasets and the challenges they
pose to reliable fairness assessments [1, 2].
Finally, we will present an example of an evaluation protocol for
assessing the alignment of large language models (LLMs). In this
experiment, we ask different LLMs to infer the sociodemographic
characteristics (e.g., gender and ethnicity) of singers based solely on
their song lyrics. This evaluation provides a way to investigate the
cultural assumptions and biases embedded in model outputs. Using this
experiment as a case study, we will highlight evidence of prevalent
cultural alignment across models and discuss what it reveals about the
relationship between AI alignment, fairness, and the reproduction of
societal biases [3].
[1] Lafargue, V., Laurindo Monteiro, A., Claeys, E., Risser, L., &
Loubes, J.-M. (2026). Exposing the Illusion of Fairness: Auditing
Vulnerabilities to Distributional Manipulation Attacks. ECML PKDD 2026,
Applied Data Science Track.
[2] Lafargue, V., Claeys, E., & Loubes, J.-M. (2025). Fairness Is in the
Details: Face Dataset Auditing. ECML PKDD 2025, Applied Data Science
Track.
[3] Lafargue, V., Guerra-Adames, A., Claeys, E., Vuichard, E., & Loubes,
J.-M. (2026). Probing Cultural Signals in Large Language Models through
Author Profiling. Proceedings of EMNLP 2026.