July 02, 2024
We present Meta 3D AssetGen (AssetGen), a significant advancement in text-to-3D generation which produces faithful, high-quality meshes with texture and material control. Compared to works that bake shading in the 3D object's appearance, AssetGen outputs physically-based rendering (PBR) materials, supporting realistic relighting. AssetGen generates first several views of the object with factored shaded and albedo appearance channels, and then reconstructs colours, metalness and roughness in 3D, using a deferred shading loss for efficient supervision. It also uses a sign-distance function to represent 3D shape more reliably and introduces a corresponding loss for direct shape supervision. This is implemented using fused kernels for high memory efficiency. After mesh extraction, a texture refinement transformer operating in UV space significantly improves sharpness and details. AssetGen achieves 17% improvement in Chamfer Distance and 40% in LPIPS over the best concurrent work for few-view reconstruction, and a human preference of 72% over the best industry competitors of comparable speed, including those that support PBR. Project page with generated assets: https://assetgen.github.io.
Written by
Yawar Siddiqui
Tom Monnier
Mahendra Kariya
Yanir Kleiman
Emilien Garreau
Andrea Vedaldi
Roman Shapovalov
David Novotny
Publisher
arXiv
July 02, 2024
Raphael Bensadoun, Tom Monnier, Yanir Kleiman, Filippos Kokkinos, Yawar Siddiqui, Mahendra Kariya, Omri Harosh, Roman Shapovalov, Emilien Garreau, Animesh Karnewar, Ang Cao, Idan Azuri, Iurii Makarov, Eric-Tuan Le, Antoine Toisoul, David Novotny, Oran Gafni, Natalia Neverova, Andrea Vedaldi
July 02, 2024
July 02, 2024
Raphael Bensadoun, Yanir Kleiman, Idan Azuri, Omri Harosh, Andrea Vedaldi, Natalia Neverova, Oran Gafni
July 02, 2024
June 20, 2024
Weiyao Wang, Pierre Gleize, Hao Tang, Xingyu Chen, Kevin Liang, Matt Feiszli
June 20, 2024
June 17, 2024
Jiawei Ren, Frost Xu, Jerry Wu, Ziwei Liu, Tao Xiang, Antoine Toisoul
June 17, 2024
Product experiences
Foundational models
Product experiences
Latest news
Foundational models