Research
Improving Treatment Effect Estimators Through Experiment Splitting
February 21, 2019
Abstract
We present a method for implementing shrinkage of treatment effect estimators, and hence improving their precision, via experiment splitting. Experiment splitting reduces shrinkage to a standard prediction problem. The method makes minimal distributional assumptions, and allows for the degree of shrinkage in one metric to depend on other metrics. Using a dataset of 226 Facebook News Feed A/B tests, we show that a lasso estimator based on repeated experiment splitting has a 44% lower mean squared predictive error than the conventional, unshrunk treatment effect estimator, a 18% lower mean squared predictive error than the James-Stein shrinkage estimator, and would lead to substantially improved launch decisions over both.
Related Publications
May 14, 2025
Core Machine Learning
UMA: A Family of Universal Models for Atoms
Brandon M. Wood, Misko Dzamba, Xiang Fu, Meng Gao, Muhammed Shuaibi, Luis Barroso-Luque, Kareem Abdelmaqsoud, Vahe Gharakhanyan, John R. Kitchin, Daniel S. Levine, Kyle Michel, Anuroop Sriram, Taco Cohen, Abhishek Das, Ammar Rizvi, Sushree Jagriti Sahoo, Zachary W. Ulissi, C. Lawrence Zitnick
May 14, 2025
May 13, 2025
Human & Machine Intelligence
Dynadiff: Single-stage Decoding of Images from Continuously Evolving fMRI
Marlène Careil, Yohann Benchetrit, Jean-Rémi King
May 13, 2025
April 25, 2025
NLP
ReasonIR: Training Retrievers for Reasoning Tasks
Rulin Shao, Qiao Rui, Varsha Kishore, Niklas Muennighoff, Victoria Lin, Daniela Rus, Bryan Kian Hsiang Low, Sewon Min, Scott Yih, Pang Wei Koh, Luke Zettlemoyer
April 25, 2025
April 17, 2025
Human & Machine Intelligence
Conversational AI
Collaborative Reasoner: Self-improving Social Agents with Synthetic Conversations
Ansong Ni, Ruta Desai, Yang Li, Xinjie Lei, Dong Wang, Ramya Raghavendra, Gargi Ghosh, Daniel Li (FAIR), Asli Celikyilmaz
April 17, 2025
April 08, 2021
Responsible AI
Integrity
Towards measuring fairness in AI: the Casual Conversations dataset
Caner Hazirbas, Joanna Bitton, Brian Dolhansky, Jacqueline Pan, Albert Gordo, Cristian Canton Ferrer
April 08, 2021
April 30, 2018
The Role of Minimal Complexity Functions in Unsupervised Learning of Semantic Mappings | Facebook AI Research
Tomer Galanti, Lior Wolf, Sagie Benaim
April 30, 2018
April 30, 2018
Computer Vision
NAM – Unsupervised Cross-Domain Image Mapping without Cycles or GANs | Facebook AI Research
Yedid Hoshen, Lior Wolf
April 30, 2018
December 11, 2019
Speech & Audio
Computer Vision
Hyper-Graph-Network Decoders for Block Codes | Facebook AI Research
Eliya Nachmani, Lior Wolf
December 11, 2019
Help Us Pioneer The Future of AI
We share our open source frameworks, tools, libraries, and models for everything from research exploration to large-scale production deployment.
Our approach
Latest news
Foundational models
Meta © 2025