Vector Institute and Apple Researchers Develop GSN for Reference-Free 3D Environment Generation
Researchers from the Vector Institute and Apple have developed a new machine learning model called Generative Scene Networks (GSN), capable of creating detailed 3D environments without requiring reference images. Built upon Neural Radiance Fields (NeRFs), GSNs overcome previous limitations by hallucinating new details and modeling entire scenes, such as transitioning from a house to a garage, even when unconstrained. The technology allows users to provide partial scenes for the model to complete, offering a grounded representation of reality. Led by Vector Faculty Member Graham Taylor and his student Terrance DeVries, the team highlights potential applications in the construction industry, video game development, real estate, and design. Taylor notes that while current LiDAR technology is expensive for small construction firms, GSNs could democratize 3D modeling. Although startup Origami XR currently uses NeRFs, Taylor believes GSNs offer significant future potential for modeling entire buildings. The research exemplifies the growing Canadian AI ecosystem, linking academic research with industrial deployment and economic opportunity, similar to the impact of StyleGAN2 on 2D image editing tools.
Wire timeline
Vector Institute and Apple Researchers Develop GSN for Reference-Free 3D Environment Generation
Researchers from the Vector Institute and Apple have developed a new machine learning model called Generative Scene Networks (GSN), capable of creating detailed 3D environments without requiring reference images. Built upon Neural Radiance Fields (NeRFs), GSNs overcome previous limitations by hallucinating new details and modeling entire scenes, such as transitioning from a house to a garage, even when unconstrained. The technology allows users to provide partial scenes for the model to complete, offering a grounded representation of reality. Led by Vector Faculty Member Graham Taylor and his student Terrance DeVries, the team highlights potential applications in the construction industry, video game development, real estate, and design. Taylor notes that while current LiDAR technology is expensive for small construction firms, GSNs could democratize 3D modeling. Although startup Origami XR currently uses NeRFs, Taylor believes GSNs offer significant future potential for modeling entire buildings. The research exemplifies the growing Canadian AI ecosystem, linking academic research with industrial deployment and economic opportunity, similar to the impact of StyleGAN2 on 2D image editing tools.
Vector Institute for Artificial Intelligence