ExecuTorch Adoption in Reality Labs: Powering On-Device AI Across Meta Devices

Reality Labs’ goal is to build the next computing platform. To that end, the teams across Reality Labs develop next-generation hardware, including Ray-Ban Meta and Oakley Meta glasses, Meta Ray-Ban Display glasses and the Meta Neural Band, and Meta Quest headsets. These products require AI models to run efficiently on a wide range of hardware — from high-performance systems-on-a-chip (SoCs) to ultra-low-power microcontrollers. The modular design of ExecuTorch along with its portability, small footprint, and hardware abstraction make it an ideal fit for the diverse product ecosystem at Reality Labs.

On-device AI deployment presents a fundamental challenge that requires us to balance supporting researchers and engineers with quick experimentation with optimization flows across diverse hardware targets, while not sacrificing performance and productivity. Traditional approaches require converting PyTorch models to other formats, which introduces numerical mismatches and costly debug cycles that break the tight iteration loop researchers need. This challenge becomes particularly acute where there's a diverse set of product lines and AI use cases.

ExecuTorch addresses this challenge by eliminating the conversion step and providing an entirely PyTorch-native flow. It uses PyTorch’s export capability to create a portable representation of the model that can execute both on-device and in PyTorch for pre-deployment validation. Developers can then further compile the graph for specific hardware — for example, quantizing the model to optimize for memory constraints or applying compiler passes to optimize for latency on target SoCs. One of the important benefits of ExecuTorch is that most optimizations can still be validated within PyTorch before deployment. This is made possible by tight integration with hardware partners who contribute by adhering to ExecuTorch’s API and design principles.

With this design and its hardware abstraction, ExecuTorch enables Reality Labs to deploy models consistently across different products and chipsets with minimal modifications. Here’s a look at how this works in our Reality Labs hardware.

ExecuTorch enables Meta Quest 3 and 3S to run advanced AI workloads — such as depth estimation and scene understanding — directly on the device. This local processing ensures fast, reliable performance, allowing features like Passthrough to seamlessly blend the physical world with virtual content. The result is a natural, realistic interaction with your environment, where the boundaries between virtual and physical spaces fade away. With high-performance AI features that are both responsive and convenient, these headsets deliver a next-level experience.

Real-time AI models enabled by ExecuTorch also drive important features like hand tracking and controller tracking. These capabilities form the foundation for accurate UI controls, gesture recognition, and virtual keyboards, making interactions intuitive and precise.

Another standout feature enabled by ExecuTorch is persistent room memory. Thanks to efficient on-device inference, Meta Quest 3 and 3S can remember up to 15 different rooms, each with its own unique layout and boundaries. This flexibility saves time and ensures a consistent, personalized experience wherever you use your headset.

Because the glasses see what you see, they can instantly identify someone’s region of interest from the scene and recognize relevant text. This enables translation, read-outs, and contextual actions from documents, menus, and street signs, providing immediate translation, dictation, and contextual actions right on the display. Someone wearing the glasses can walk into a restaurant, glance at a menu in another language, and get the translation in seconds, or point at specific items on the menu and ask for more information without ever reaching for their phone.

Meta Ray-Ban Display’s reading assistant feature takes this even further: By simply pointing to physical text, a person can trigger instant translations, definitions, or dictation, all thanks to real-time hand tracking and advanced OCR models running directly on the device. These capabilities make Meta Ray-Ban Display glasses a helpful companion for navigating the world and accessing information in the moment.

ExecuTorch has rapidly grown beyond Meta into a vibrant community of developers, researchers, and hardware partners who share a vision for open, efficient, and portable on-device AI. From chip vendors and device makers to app developers and AI researchers, contributors across the ecosystem are helping expand ExecuTorch’s reach.

The project’s open source development ensures transparency and collaboration, with contributions from companies like Apple, Arm, Cadence, Intel, MediaTek, NXP Semiconductors, Qualcomm Technologies Inc., and Samsung. ExecuTorch backends for these platforms are publicly available. In addition to hardware partners, ExecuTorch is now an important part of the AI ecosystem. For example, in the recent General Availability release, ExecuTorch demonstrated model export coverage with Hugging Face transformers and Ultralytics libraries, interoperability with Unsloth AI and torchao frameworks, and integration into SDKs such as Liquid AI, NimbleEdge, and React-Native-Executorch. See the Success Stories page for more up-to-date information.

Openness is important because on-device AI spans multiple platforms, chips, and use cases. By working in the open, ExecuTorch lets the entire ecosystem innovate together, ensuring PyTorch models can run efficiently anywhere — from phones and AI apps to future AR glasses and beyond.