Aaron Defazio

RESEARCH SCIENTIST | NEW YORK CITY, UNITED STATES

Aaron's research focuses on improving the practice of machine learning through the development of more reliable and theoretically sound methods such as performance optimization, initialization, and normalization. He also drives current research frontiers in applied areas and is currently involved in MRI imaging reconstruction and automated theorem proving.

Aaron's Publications

November 06, 2024

THEORY

CORE MACHINE LEARNING

The Road Less Scheduled

Aaron Defazio, Alice Yang, Harsh Mehta, Konstantin Mishchenko, Ahmed Khaled, Ashok Cutkosky

November 06, 2024

November 13, 2023

CORE MACHINE LEARNING

Mechanic: A Learning Rate Tuner

Aaron Defazio, Ashok Cutkosky, Harsh Mehta

November 13, 2023

June 13, 2023

CORE MACHINE LEARNING

Learning-Rate-Free Learning by D-Adaptation

Aaron Defazio, Konstantin Mishchenko

June 13, 2023

May 13, 2022

A Scaling Calculus for the Design and Initialization of ReLU Networks

Aaron Defazio, Leon Bottou

May 13, 2022

October 14, 2021

The Power of Factorial Powers: New Parameter settings for (Stochastic) Optimization

Aaron Defazio, Robert Gower

October 14, 2021

November 18, 2019

RESEARCH

SPEECH & AUDIO

On the Ineffectiveness of Variance Reduced Optimization for Deep Learning

Aaron Defazio, Leon Bottou

November 18, 2019

November 18, 2019

RESEARCH

SPEECH & AUDIO

On the Curved Geometry of Accelerated Optimization

Aaron Defazio

November 18, 2019