Learning Compiler Pass Orders using Coreset and Normalized Value Prediction

July 26, 2023


Finding the optimal pass sequence of compilation can lead to a significant reduction in program size. Prior works on compilation pass ordering have two major drawbacks. They either require an excessive budget (in terms of the number of compilation passes) at compile time or fail to generalize to unseen programs. In this work, instead of predicting passes sequentially, we directly learn a policy on the pass sequence space, which outperforms the default -Oz flag by an average of 4.5% over a large collection (4683) of unseen code repositories from diverse domains across 14 datasets. To achieve this, we first identify a small set (termed coreset) of pass sequences that generally optimize the size of most programs. Then, a policy is learned to pick the optimal sequences by predicting the normalized values of the pass sequences in the coreset. Our results demonstrate that existing human-designed compiler passes can be improved with a simple yet effective technique that leverages pass sequence space which contains dense rewards, while approaches operating on the individual pass space may suffer from issues of sparse reward, and do not generalize well to held-out programs from different domains. Website:

Download the Paper


Written by

Youwei Liang

Kevin Stone

Chris Cummins

Mostafa Elhoushi

Jiadong Guo

Pengtao Xie

Hugh Leather

Yuandong Tian



Research Topics

Systems Research

Related Publications

November 07, 2023



The Framework Tax: Disparities Between Inference Efficiency in NLP Research and Deployment

Jared Fernandez, Jacob Kahn, Clara Na, Yonatan Bisk, Emma Strubell

November 07, 2023

August 21, 2023


GraphAGILE: An FPGA-Based Overlay Accelerator for Low-Latency GNN Inference

Bingyi Zhang, Hanqing Zeng, Viktor Prasanna

August 21, 2023

June 19, 2023


MODeL: Memory Optimizations for Deep Learning

Benoit Steiner, Mostafa Elhoushi, Jacob Kahn, James Hegarty

June 19, 2023

April 26, 2023



Green Federated Learning

Ashkan Yousefpour, Shen Guo, Ashish Shenoy, Sayan Ghosh, Pierre Stock, Kiwan Maeng, Schalk Krüger, Mike Rabbat, Carole-Jean Wu, Ilya Mironov

April 26, 2023

Help Us Pioneer The Future of AI

We share our open source frameworks, tools, libraries, and models for everything from research exploration to large-scale production deployment.