DINOv2 SALAD: Optimal Transport Aggregation for Visual Place Recognition

Sergio Izquierdo, Javier Civera

University of Zaragoza, I3A, Spain

2024 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

...

TLDR : We propose a novel pipeline for Visual Place Recognition. We leverage DINOv2 features as backbone and propose SALAD (which creates an assignment of features to clusters based on optimal transport) for aggregation, obtaining state-of-the-art results on several benchmarks.

Abstract

The task of Visual Place Recognition (VPR) aims to match a query image against references from an extensive database of images from different places, relying solely on visual cues. State-of-the-art pipelines focus on the aggregation of features extracted from a deep backbone, in order to form a global descriptor for each image. In this context, we introduce SALAD (Sinkhorn Algorithm for Locally Aggregated Descriptors), which reformulates NetVLAD's soft-assignment of local features to clusters as an optimal transport problem. In SALAD, we consider both feature-to-cluster and cluster-to-feature relations and we also introduce a 'dustbin' cluster, designed to selectively discard features deemed non-informative, enhancing the overall descriptor quality. Additionally, we leverage and fine-tune DINOv2 as a backbone, which provides enhanced description power for the local features, and dramatically reduces the required training time. As a result, our single-stage method not only surpasses single-stage baselines in public VPR datasets, but also surpasses two-stage methods that add a re-ranking with significantly higher cost.

Results

...
  • DINOv2 SALAD outperforms relevant baselines on challenging and varied benchmarks
  • It allows for different descriptor sizes, all of them with competitive performance
  • Single-stage pipeline (no re-ranking)
  • Fast to train (30 mins)
...

BibTex

@inproceedings{izquierdo2023sfmttr,
  title={Optimal Transport Aggregation for Visual Place Recognition},
  author={Izquierdo, Sergio and Civera, Javier},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  year={2024}
}