EgoMimic: Georgia Tech PhD student uses Project Aria Research Glasses to help train humanoid robots

Imagine having help completing everyday tasks in your home such as doing the laundry, washing dishes, and making repairs. We already use tools to help with these tasks, like washing machines, dishwashers, and electric drills. But what if you could have an even more powerful and flexible tool in the form of a humanoid robot that could learn from you and accelerate any number of physical projects on your to-do list?

Even if you had the available hardware system, teaching a robot to do everyday tasks can only be achieved through a slow and clunky data collection method called robot teleoperation. Until now. By using the Project Aria Research Kit, Professor Danfei Xu and the Robotic Learning and Reasoning Lab at Georgia Tech use the egocentric sensors on Aria glasses to create what they call “human data” for tasks that they want a humanoid robot to replicate. They use human data to dramatically reduce the amount of robot teleoperation data needed to train a robot’s policy—a breakthrough that could some day make humanoid robots capable of learning any number of tasks a human could demonstrate.

“Traditionally, collecting data for robotics means creating demonstration data,” says Simar Kareer, a PhD student in Georgia Tech’s School of Interactive Computing. “You operate the robot’s joints with a controller to move it and achieve the task you want, and you do this hundreds of times while recording sensor data, then train your models. This is slow and difficult. The only way to break that cycle is to detach the data collection from the robot itself.”

Today, robot policy models are trained with large amounts of targeted demonstration data specific to each narrow task at a high cost. Kareer hypothesizes that passively collected data from many researchers, like the data captured by Aria glasses, could instead be used to enable data creation for a much broader set of tasks to create more generally useful robots in the future.

Inspired by Project Aria and Ego-Exo4D which includes a massive egocentric dataset of over 3K hours of video recordings of daily-life activities, Kareer developed EgoMimic, a new algorithmic framework that utilizes human data and robot data for humanoid robot development.

In the future, humanoid robots could be trained at scale using egocentric data in order to perform a variety of tasks in the same way humans do.

“We look at Aria as an investment in the research community,” says James Fort, a Reality Labs Research Product Manager at Meta. “The more that the egocentric research community standardizes, the more researchers will be able to collaborate. It’s really through scaling with the community like this that we can start to solve bigger problems around how things are going to work in the future.”

Kareer will present his paper on EgoMimic at the 2025 IEEE Engineers’ International Conference on Robotics and Automation (ICRA) in Atlanta.