COMPUTER VISION

Self-supervised Correspondence Estimation via Multiview Registration

January 03, 2023

Abstract

Video provides us with the spatio-temporal consistency needed for visual learning. Recent approaches have utilized this signal to learn correspondence estimation from close-by frame pairs. However, by only relying on close-by frame pairs, those approaches miss out on the richer long-range consistency between distant overlapping frames. To address this, we propose a self-supervised approach for correspondence estimation that learns from multiview consistency in short RGB-D video sequences. Our approach combines pairwise correspondence estimation and registration with a novel SE(3) transformation synchronization algorithm. Our key insight is that self-supervised multiview registration allows us to obtain correspondences over longer time frames; increasing both the diversity and difficulty of sampled pairs. We evaluate our approach on indoor scenes for correspondence estimation and RGB-D pointcloud registration and find that we perform on-par with supervised approaches.

Download the Paper

AUTHORS

Written by

Benjamin Graham

Andrea Vedaldi

David Novotny

Ignacio Rocco

Justin Johnson

Natalia Neverova

Mohamed El Banani

Publisher

WACV

Research Topics

Computer Vision

Related Publications

June 20, 2024

COMPUTER VISION

ICON: Incremental CONfidence for Joint Pose and Radiance Field Optimization

Weiyao Wang, Pierre Gleize, Hao Tang, Xingyu Chen, Kevin Liang, Matt Feiszli

June 20, 2024

June 17, 2024

COMPUTER VISION

Move Anything with Layered Scene Diffusion

Jiawei Ren, Frost Xu, Jerry Wu, Ziwei Liu, Tao Xiang, Antoine Toisoul

June 17, 2024

June 14, 2024

COMPUTER VISION

Decomposed evaluations of geographic disparities in text-to-image models

Abhishek Sureddy, Dishant Padalia, Nandhinee Periyakaruppa, Oindrila Saha, Adina Williams, Adriana Romero Soriano, Megan Richards, Polina Kirichenko, Melissa Hall

June 14, 2024

June 05, 2024

COMPUTER VISION

Cache Me if You Can: Accelerating Diffusion Models through Block Caching

Felix Wimbauer, Bichen Wu, Edgar Schoenfeld, Ji Hou, Zijian He, Artsiom Sanakoyeu, Peizhao Zhang, Sam Tsai, Jonas Kohler, Christian Rupprecht, Daniel Cramers, Peter Vajda, Jialiang Wang

June 05, 2024

Help Us Pioneer The Future of AI

We share our open source frameworks, tools, libraries, and models for everything from research exploration to large-scale production deployment.