Have you heard about volumetric capture yet? It's used to create "holograms" and other three-dimensional models that can be used for lifelike augmented and virtual reality applications as well as desktop applications. While some of these sound pretty crazy, even most mobile phones can run AR experiences using holograms, making motion capture increasingly popular in entertainment and advertising.
Here, we'll talk about what volumetric capture is, how it differs from other kinds of image production, and how it is used.
What Is Volumetric Capture?
"Volumetric capture" (also known as "volumetric video") refers to recording a physical place, object, person, or even event in a way that makes it appear to take up three-dimensional space. In the case of a recording of an object or person, this allows a viewer to rotate or move around the end experience. If you've read about other spatial recording and display technologies, volumetric capture may not sound that special.
For example, light field photography can capture a scene in a way that appears to have depth, but a user can't navigate through the scene in a convincing way. LiDAR and ToF (Time of Flight) sensors on devices like mobile phones can both create spatial models from objects and scenes that can be navigated in three dimensions. However, unlike those systems, volumetric capture can create spatial models of moving objects and people in real-time.
But how?
How Does Volumetric Capture Work?
With LiDAR and ToF, the recorder can create a volumetric capture of a stationary scene or object by moving around that scene or object and essentially capturing lots of two-dimensional data from a lot of different angles. Think of it as like taking one photograph of each side of a cube and then gluing the photos together into a cube shape. You now have a spatial model created from two-dimensional representations of the real thing.
Volumetric capture works by using a whole bunch of cameras from a whole bunch of different angles that are all filming at the same time. Then, computer algorithms stitch the views from these angles together to create volumetric images. Microsoft's Mixed Reality Capture Studio, one of the biggest players in volumetric capture, calls these "holograms." But, who would need a hologram, and what for?
What Is Volumetric Capture Used For?
Volumetric capture is currently primarily used in advertising and entertainment. In advertising, holograms give consumers access to information, products, and spokespeople in immersive brand engagement opportunities that they can experience from anywhere.
Similarly, volumetric capture is increasingly being used in entertainment for things like remote concerts. Performers can do a live performance just like they normally would and have their holograms broadcast to fans around the world in an experience that is more immersive than watching or listening to a concert live on a platform like YouTube.
Some use cases utilize a whole studio, like those located around the world through Microsoft's initiative. However, they aren't the only name in the game. TetaVi deploys portable volumetric capture rigs that can be deployed "on location."
The Future of Volumetric Capture
Like most mixed reality applications and technologies, volumetric capture has a lot of potential that hasn't yet been realized. However, as the technology becomes smaller, easier to use, and more affordable, there are a lot of uses that could materialize in the near future.
For example, TetaVi's volumetric capture system consists of four depth cameras and eight RGB cameras. Imagine if it were as easy and inexpensive to install these in a space as it is to install microphones. Concerts, plays, meetings, and social events could all be volumetrically captured and broadcast to larger audiences or preserved for future historians.
Furthermore, there are already some use cases that have huge potential to grow, like gaming and social media applications. In the case of gaming, volumetric capture is largely used to create content for VR experiences that allow the user to walk around a volumetric capture of a physical person, object, or location in a virtually created or recreated environment.
Have You Experienced Volumetric Capture?
Right now, the chances that you have experienced or even made a volumetric capture are pretty slim. However, it’s highly likely that you will in the future. Volumetric capture isn't a part of everyone's life yet, but there's nothing wrong with understanding something early on.