Apple & UCSB Scientists: Pioneering Deep Neural Networks to Usher in New Era of 3D Graphics

Apple & UCSB Scientists: Pioneering Deep Neural Networks to Usher in New Era of 3D Graphics

Apple & UCSB Scientists: Pioneering Deep Neural Networks to Usher in New Era of 3D Graphics

As Seen On

As we delve into the realm of 3D graphics and its myriad applications, it’s difficult not to appreciate the vast strides in technology that have brought us here. From mesmerizing animated films to lifelike video game designs, 3D modeling plays an essential part in our technologically driven world. However, as groundbreaking as these applications may already seem, tech giant Apple in conjunction with researchers from the University of California, Santa Barbara (UCSB), are plowing ahead to push the boundaries of such graphical interpretations even further.

Traditionally, creating 3D models has been a time-consuming process that required the heavy lifting of designing point clouds and polygonal meshes. The myriad of complexities present in an organic environment made the task even more daunting, and often, the resulting 3D models fell short of perfectly capturing the essence of the original subjects.

This is where deep neural networks come in, signifying a revolutionary leap forward in 3D modeling technology. Pioneering scientists at Apple and UCSB have proposed a method to directly infer scene-level 3D geometry with the help of such networks. Their approach lies in the prediction of the truncated signed distance function (TSDF) associated with a scene by using a 3D convolution neural network.

TSDF is a volumetric scene representation that naturally incorporates occlusion reasoning. The neural network deployed to predict this, known as a Convolutional Neural Network (CNN), is capable of creating smooth, consistent surfaces, which take 3D models several notches closer to their real-world twins.

The research teams relied on a strategy called tri-linear interpolation in their voxel grid application. This, however, was not void of complications. Initially, it caused unintentional blurring and random noise in the 3D surfaces. The researchers pulled themselves back up by introducing supervised predictions, leading to striking improvements in 3D model quality.

Voxels and their shortcomings are well-known to 3D professionals and enthusiasts alike. As a 3D equivalent of pixels, voxels do an amazing job of rendering 3D spaces but often fall short in capturing the fine details of a scene. The researchers’ revolutionary method addresses this issue by applying a novel dense back-projection strategy.

Nevertheless, one substantial hurdle remained – blurring in the back-projection volume. The solution they found was using an initial multi-view stereo depth estimation to keep things in focus, thus perfecting the depth perception.

One of the most exciting features of this method is its ability to learn fine details and enable free selection of output resolution. It simply means an unprecedented level of accuracy, featuring down to the most minute details of a scene in 3D models. This paves the way for intricate, true-to-life 3D graphics unlike any we have seen before.

The impact of this pioneering research is bound to be monumental in the 3D graphics industry, revolutionizing everything from video games to virtual reality and beyond. Indeed, the creation of remarkably detailed and accurate 3D models will open a world of possibilities in creating even more immersive virtual experiences that could reshape how we interact with technology.

We invite you to dive deeper into this groundbreaking research by following the GitHub link, where you can find the teams’ paper and code. As we continue to navigate the thrilling world of 3D graphics and deep learning, we hope you’ll join us in our online communities to share ideas and developments in this rapidly evolving field. Enjoy the journey, and stay tuned for what’s to come!

Casey Jones Avatar
Casey Jones
8 months ago

Why Us?

  • Award-Winning Results

  • Team of 11+ Experts

  • 10,000+ Page #1 Rankings on Google

  • Dedicated to SMBs

  • $175,000,000 in Reported Client

Contact Us

Up until working with Casey, we had only had poor to mediocre experiences outsourcing work to agencies. Casey & the team at CJ&CO are the exception to the rule.

Communication was beyond great, his understanding of our vision was phenomenal, and instead of needing babysitting like the other agencies we worked with, he was not only completely dependable but also gave us sound suggestions on how to get better results, at the risk of us not needing him for the initial job we requested (absolute gem).

This has truly been the first time we worked with someone outside of our business that quickly grasped our vision, and that I could completely forget about and would still deliver above expectations.

I honestly can't wait to work in many more projects together!

Contact Us


*The information this blog provides is for general informational purposes only and is not intended as financial or professional advice. The information may not reflect current developments and may be changed or updated without notice. Any opinions expressed on this blog are the author’s own and do not necessarily reflect the views of the author’s employer or any other organization. You should not act or rely on any information contained in this blog without first seeking the advice of a professional. No representation or warranty, express or implied, is made as to the accuracy or completeness of the information contained in this blog. The author and affiliated parties assume no liability for any errors or omissions.