Orateur
Description
Thanks to modern data infrastructures, large amounts of images describing vegetation are available at multiple scales: at the individual scale (e.g., Pl@ntNet images), at the landscape scale (e.g., LUCAS images), and at the ecosystem scale (e.g., Sentinel 2 data). Usually, these different types of visual content are processed separately. Images of individual plants are typically used to train species identification models. Images of communities are rather used for ecotope characterization. Remote sensing images are used for land monitoring, or more recently for species distribution modeling. The combination of these three scales, however, has not yet been leveraged, or only in very specific scenarios. We propose to combine those multi-scale vegetation images to derive biodiversity-relevant features using deep contrastive learning models, and incorporate these multi-scale multi-modal features in species distribution models to more accurately address species mapping at EU scale and derive biodiversity indicators at local scales for better land management. We first train our model on a pair-matching pretext task to align GPS coordinates with images in a common representation space; before evaluating it on a downstream species prediction task. We train and evaluate our method over the CBN-Med region of France, by using Sentinel-2A satellite imagery, landscape images form the EU LUCAS dataset, Pl@ntNet images and citizen science plant observations from the GeoLifeCLEF2024 dataset. The code, model and parts of the dataset are publicly available on GitHub through the Malpolon framework. We show that enriching our classification model with a contrastive pre-training task gives similar or better performances on a species prediction task compared to training a BCE loss from scratch. This opens more possibilities to training species prediction models in areas with few or no observations (few-shot learning).
