RESEARCH
SPEECH & AUDIO
On the Ineffectiveness of Variance Reduced Optimization for Deep Learning
November 18, 2019
Abstract
The application of stochastic variance reduction to optimization has shown remarkable recent theoretical and practical success. The applicability of these techniques to the hard non-convex optimization problems encountered during training of modern deep neural networks is an open problem. We show that naive application of the SVRG technique and related approaches fail, and explore why.
AUTHORS
Publisher
NeurIPS
Related Publications
September 24, 2026
REINFORCEMENT LEARNING
RESEARCH
MaD-RL: Matching Distributions for Calibrating LLMs with Reinforcement Learning
Sourabh Kulkarni, Ksheeraj Sai Vepuri, Basar Demir, Jason Bohrer, Emily Shen, Jianfa Chen, Nan Jiang, Ankit Jain, Harihar Subramanyam, Mannat Singh, Chirag Nagpal
September 24, 2026
September 06, 2026
SPEECH & AUDIO
Alignment-Free Text-Audiobox for Voice Dubbing and Full-Duplex Dialogue Synthesis
Min-Jae Hwang, Sho Inoue, Bokai Yu, David Kant, Dongmin Hyun, Dorian Desblancs, Gregory Antonovsky, Oleg Repin, Zehai Tu, Juan Pino, Wei-Ning Hsu
September 06, 2026
July 17, 2026
CONVERSATIONAL AI
REINFORCEMENT LEARNING
Learning to Reason by Analogy via Retrieval-Augmented Reinforcement Fine-Tuning
Zilin Xiao, Qi Ma, Jason Chen, Xintao Chen, Avinash Atreya, Hanjie Chen, Vicente Ordonez
July 17, 2026
July 13, 2026
AR/VR
RESEARCH
S-EMBER: A Large-Scale Benchmark for Streaming Egocentric Memory Retrieval
Xiaodong Wang, Xuanyi Zhao, Pedro Rodriguez, Devendra Singh Sachan, Barlas Oguz, Seungwhan Moon, Shang-Wen Li, Gargi Ghosh, Xin Dong, Wen-Tau Yih
July 13, 2026
Help Us Pioneer The Future of AI
We share our open source frameworks, tools, libraries, and models for everything from research exploration to large-scale production deployment.
Meta © 2026
