ROBOTICS

REINFORCEMENT LEARNING

Galactic: Scaling End-to-End Reinforcement Learning for Rearrangement at 100k Steps-Per-Second

June 18, 2023

Abstract

We present Galactic, a large-scale simulation and reinforcement-learning (RL) framework for robotic mobile manipulation in indoor environments. Specifically, a Fetch robot (equipped with a mobile base, 7DoF arm, RGBD camera, egomotion, and onboard sensing) is spawned in a home environment and asked to rearrange objects – by navigating to an object, picking it up, navigating to a target location, and then placing the object at the target location. Galactic is fast. In terms of simulation speed (rendering + physics), Galactic achieves over 421,000 steps-per-second (SPS) on an 8-GPU node, which is 54x faster than Habitat 2.0 [55] (7699 SPS). More importantly, Galactic was designed to optimize the entire rendering+physics+RL interplay since any bottleneck in the interplay slows down training. In terms of simulation+RL speed (rendering + physics + inference + learning), Galactic achieves over 108,000 SPS, which 88x faster than Habitat 2.0 (1243 SPS). These massive speed-ups not only drastically cut the wall-clock training time of existing experiments, but also unlock an unprecedented scale of new experiments. First, Galactic can train a mobile pick skill to > 80% accuracy in under 16 minutes, a 100x speedup compared to the over 24 hours it takes to train the same skill in Habitat 2.0. Second, we use Galactic to perform the largest-scale experiment to date for rearrangement using 5B steps of experience in 46 hours, which is equivalent to 20 years of robot experience. This scaling results in a single neural network composed of task-agnostic components achieving 85% success in GeometricGoal rearrangement, compared to 0% success reported in Habitat 2.0 for the same approach. The code is available at github.com/facebookresearch/galactic .

Download the Paper

AUTHORS

Written by

Vincent-Pierre Berges

Andrew Szot

Devendra Singh Chaplot

Aaron Gokaslan

Dhruv Batra

Eric Undersander

Publisher

CVPR

Related Publications

May 04, 2023

ROBOTICS

REINFORCEMENT LEARNING

MoDem: Accelerating Visual Model-Based Reinforcement Learning with Demonstrations

Nicklas Hansen, Yixin Lin, Hao Su, Xiaolong Wang, Vikash Kumar, Aravind Rajeswaran

May 04, 2023

March 31, 2023

ROBOTICS

REINFORCEMENT LEARNING

PIRLNav: Pretraining with Imitation and RL Finetuning for ObjectNav

Ram Ramrakhya, Dhruv Batra, Erik Wijmans, Abhishek Das

March 31, 2023

March 29, 2023

ROBOTICS

Where are we in the search for an Artificial Visual Cortex for Embodied Intelligence?

Franziska Meier, Aravind Rajeswaran, Dhruv Batra, Jitendra Malik, Karmesh Yadav, Oleksandr Maksymets, Sergio Arnaud, Sneha Silwal, Vincent-Pierre Berges, Aryan Jain, Claire Chen, Jason Ma, Yixin Lin

March 29, 2023

March 29, 2023

ROBOTICS

Adaptive Skill Coordination for Robotic Mobile Manipulation

Akshara Rai, Alexander William Clegg, Dhruv Batra, Eric Undersander, Naoki Yokoyama, Sehoon Ha

March 29, 2023

Help Us Pioneer The Future of AI

We share our open source frameworks, tools, libraries, and models for everything from research exploration to large-scale production deployment.