COMPUTER VISION

Identifying and Disentangling Spurious Features in Pretrained Image Representations

October 12, 2023

Abstract

Neural networks employ spurious correlations in their predictions, resulting in decreased performance when these correlations do not hold. Recent works suggest fixing pretrained representations and training a classification head that does not use spurious features. We investigate how spurious features are represented in pretrained representations and explore strategies for removing information about spurious features. Considering the Waterbirds dataset and a few pretrained representations, we find that even with full knowledge of spurious features, their removal is not straightforward due to entangled representation. To address this, we propose a linear autoencoder training method to separate the representation into core, spurious, and other features. We propose two effective spurious feature removal approaches that are applied to the encoding and significantly improve classification performance measured by worst-group accuracy.

Download the Paper

AUTHORS

Written by

Aram H. Markosyan

Hrant Khachatrian

Hrayr Harutyunyan

Rafayel Darbinyan

Publisher

ICML

Research Topics

Computer Vision

Related Publications

March 20, 2024

COMPUTER VISION

SceneScript: Reconstructing Scenes With An Autoregressive Structured Language Model

Armen Avetisyan, Chris Xie, Henry Howard-Jenkins, Tsun-Yi Yang, Samir Aroudj, Suvam Patra, Fuyang Zhang, Duncan Frost, Luke Holland, Campbell Orme, Jakob Julian Engel, Edward Miller, Richard Newcombe, Vasileios Balntas

March 20, 2024

February 13, 2024

GRAPHICS

COMPUTER VISION

IM-3D: Iterative Multiview Diffusion and Reconstruction for High-Quality 3D Generation

Luke Melas-Kyriazi, Iro Laina, Christian Rupprecht, Natalia Neverova, Andrea Vedaldi, Oran Gafni, Filippos Kokkinos

February 13, 2024

January 25, 2024

COMPUTER VISION

LRR: Language-Driven Resamplable Continuous Representation against Adversarial Tracking Attacks

Felix Xu, Di Lin, Jianjun Zhao, Jianlang Chen, Lei Ma, Qing Guo, Wei Feng, Xuhong Ren

January 25, 2024

December 08, 2023

COMPUTER VISION

Learning Fine-grained View-Invariant Representations from Unpaired Ego-Exo Videos via Temporal Alignment

Sherry Xue, Kristen Grauman

December 08, 2023

Help Us Pioneer The Future of AI

We share our open source frameworks, tools, libraries, and models for everything from research exploration to large-scale production deployment.