Using AI to create real-time depth maps for occlusions in AR

In order to make augmented reality (AR) more realistic, we need to be able to apply AR effects both in front and behind objects in a scene. Most AR systems, however, are based on SLAM, which only tracks a few sparse point features. This allows for overlays in front of objects, but not behind them. We developed a method that increases the density of the depth from the sparse point track locations to all other pixels, creating dense depth maps for every frame. This method helps AR effects fully interact with a scene geometry, such as enabling occlusions by real objects in the scene.

Watch the video:

Something Went Wrong

We're having trouble playing this video.

Learn more

Read the full paper:

Fast Depth Densification for Occlusion-aware Augmented Reality