Text-To-4D Dynamic Scene Generation
source link: https://make-a-video3d.github.io/
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
Text-To-4D Dynamic Scene Generation
Text-To-4D Dynamic Scene Generation
Uriel Singer*
Shelly Sheynin*
Adam Polyak*
Oron Ashual
Iurii Makarov
Filippos Kokkinos
Naman Goyal
Andrea Vedaldi
Devi Parikh
Justin Johnson
Yaniv Taigman
*Equal Contribution
Meta AI
Abstract
We present MAV3D (Make-A-Video3D), a method for generating three-dimensional dynamic scenes from text descriptions. Our approach uses a 4D dynamic Neural Radiance Field (NeRF), which is optimized for scene appearance, density, and motion consistency by querying a Text-to-Video (T2V) diffusion-based model. The dynamic video output generated from the provided text can be viewed from any camera location and angle, and can be composited into any 3D environment. MAV3D does not require any 3D or 4D data and the T2V model is trained only on Text-Image pairs and unlabeled videos. We demonstrate the effectiveness of our approach using comprehensive quantitative and qualitative experiments and show an improvement over previously established internal baselines. To the best of our knowledge, our method is the first to generate 3D dynamic scenes given a text description.
For optimal viewing of dynamic 3D videos, please access our website from a desktop using Chrome.
Recommend
About Joyk
Aggregate valuable and interesting links.
Joyk means Joy of geeK