Open-sourcing PyRobot to accelerate AI robotics research

Frameworks such as PyTorch have dramatically accelerated the progress of research in AI subfields such as computer vision and natural language processing. These tools provide a level of abstraction that makes it easier to build off of state-of-the-art systems, use shared libraries and tools, and optimize performance. To provide similar utility in AI robotics research, we’ve collaborated with researchers at Carnegie Mellon University to build PyRobot.

PyRobot is a framework and ecosystem that enables AI researchers and students to get up and running with a robot in just a few hours, without specialized knowledge of the hardware or of details such as device drivers, control, and planning. PyRobot will help Facebook AI advance our long-term robotics research, which aims to develop embodied AI systems that can learn efficiently by interacting with the physical world. We are now open-sourcing PyRobot to help others in the AI and robotics community as well.

PyRobot is a lightweight, high-level interface on top of the robot operating system (ROS). It provides a consistent set of hardware-independent midlevel APIs to control different robots. PyRobot abstracts away details about low-level controllers and interprocess communication, so machine learning (ML) experts and others can simply focus on building high-level AI robotics applications.

PyRobot will make it easier for the research community to use robotics datasets, algorithm implementations, and models, and also help them set benchmarks, compare their work, and build on each others’ results. When paired with low-cost robot platforms such as LoCoBot, PyRobot will reduce the entry barrier into robotics and help democratize robotics research. In addition to open-sourcing PyRobot, we are also announcing a request for proposals focused on using PyRobot with LoCoBot, with winners receiving a LoCoBot for use in their research.

A simple, shared interface for ML and robotics research

Just setting up a robot and making it wave its arm can often take several days or even a week of tinkering with native robotics software. A common joke in the robotics community says you add one year for every robot you touch in your PhD thesis. PyRobot addresses this challenge by providing a common API across multiple robots and simulators. Using a robot is as simple as writing a “from pyrobot import Robot; bot = Robot(‘locobot’)” command. Moreover, the same high-level code works on any robot supported by the PyRobot framework. (To start, this includes LoCoBot and Sawyer, and support for additional robots will be added.)

PyRobot has common utility functions for all robots, such as joint position control, joint velocity control, joint torque control, Cartesian path planning, forward kinematics and inverse kinematics (based on the robot URDF file), path planning, and visual SLAM, among other features. Though it abstracts away the complexity of the underlying software stack, users still have the flexibility to use components at varying levels of the hierarchy, such as bypassing a planner in order to command low-level velocities and torques. PyRobot can be used to implement a variety of high-level AI applications. For example, we have experimented with point-goal navigation, manipulation tasks like pushing and grasping, and teleoperation for data collection using robots.

Something Went Wrong

We're having trouble playing this video.

Learn more

Something Went Wrong

We're having trouble playing this video.

Learn more

Something Went Wrong

We're having trouble playing this video.

Learn more

PyRobot provides an easy way to use AI and ML algorithms with robotics. For example, one can use the interface with a physical robot to collect data for a task such as grasping, train a deep learning model using PyTorch, and then execute the algorithm on the robot using PyRobot. In some ongoing projects, PyRobot has simplified testing of sim2real transfer of policies for end-effector control and point-goal navigation.

A scalable tool for research and education

The cost of hardware and the complexity of specialized software has limited the scale of robotics research. The high-level API of PyRobot, when used along with relatively inexpensive robots like LoCoBot, will help address this, however. With lower barriers to entry, researchers can deploy several robots that collect data and learn in parallel, for example. By providing a common framework across different hardware, PyRobot will lead to development of benchmarks in robotics (similar to other areas in AI) and quantify the pace of progress. We plan to work with others in the robotics research community to move toward building these benchmarks.

PyRobot can also help scale AI education initiatives, making it easier to provide practical robot experience to students with diverse backgrounds. PyRobot will bring in people with fresh perspectives and ideas, encouraging collaboration and enabling researchers to iterate faster on robotics applications. Its simple API will speed the process of open-sourcing and sharing code for research projects that utilize PyRobot, and foster an ecosystem in which different researchers can easily build on top of one another’s work and compare their results. We look forward to having multiple datasets, pretrained models, and algorithmic implementations hosted on PyRobot.

As a start, we are providing implementations of pretrained models of several existing navigation, grasping, and pushing algorithms. As with our work creating other open source AI research tools — including PyTorch, AI Habitat, TorchVision, and ELF — we believe open collaboration is an important part of advancing the field and delivering the benefits of AI broadly. We will look for opportunities to share robotics research code and datasets via PyRobot framework.

Expanding the PyRobot platform and advancing AI research

In the immediate future, we will continue to add functionality to PyRobot by interfacing with simulators (like AI Habitat, Gibson, and MuJoCo). We will also work with the research community to provide support for more hardware, such as Universal Robots. Apart from open-sourcing our own research, Facebook AI also plans to share open source implementations of other state-of-the-art robotics algorithms and release robotics datasets.

Robotics is important for advancing AI overall, because it enables researchers to explore challenges like teaching machines to learn predictive models and develop “commonsense” knowledge in the real world. Solving these problems will lead to more capable AI that could one day be useful for applications like intelligent assistants. With its flexible, open design, PyRobot will help Facebook AI researchers and others in the field accelerate progress toward these goals.