Read part one of this blog series to learn the basics of volumetric capture!
As a process, volumetric capture (VC) is already establishing a foothold in entertainment, with application and adoption rising steeply with each passing year. VC’s market share is increasing as more productions across the industry begin adapting it to their needs. By 2026, the VC market is estimated to grow from $1.5 billion (circa 2021) to $4.9 billion USD in 2026 (at a CAGR of 26.9%). Along with the use of virtual production stages and new work practices, this technology’s nature is leading to the adoption of new workflows for the entertainment industry. In many ways, it is a full-scale reimagining of how a production team plans, stages, shoots, and edits their project. With volumetric capture, a director can map out their film in the prep stage and an editor can decide on framing and camera movement during post—long after the film was shot. The popular show Ted Lasso recently used similar technology to fill stadium sequences that were shot during the pandemic, capturing background performers from multiple angles and plugging them into the crowd as necessary. The production was even able to compose advanced “previz” maps of each episode, which helped with planning and prepared all departments in advance of the cameras rolling. Intel, the technology giant, had a 10,000-square-foot geodesic dome—the world’s largest immersive media hub—in Los Angeles, where each of their 96 high-resolution 5K cameras can capture action in two dimensions before specially designed algorithms convert trillions (with a “T”) of pixels into a three-dimensional 360º virtual environment. (Fun fact: Intel’s studio was the largest volumetric capture facility not owned by a classic Hollywood studio.) And they have also developed tech called “True View,” which effectively captures actors, players, and objects in volumetric 3-D—a true game changer for productions. Neill Blomkamp’s 2021 pandemic-era film, Demonic, used the technology for scenes where its protagonist enters the subconscious mind of a comatose woman—a feat accomplished by using 250 4K cameras to shoot about 15 terabytes of footage each day. This process allowed Blomkamp the freedom to place his actor into any environment.
As it becomes more common, volumetric capture is finding its place in other arenas as well. Its potential in the gaming industry is obvious enough, paring with technologies like motion capture and 3-D animation to help performers lend their own likeness to a project in a more immersive way. Currently, there’s a popular trend that has seen A-list actors like Keanu Reeves and Rami Malek lend their likeness to major titles. Critics have noted that the inadequacies in the motion capture technology used to insert these well-known actors into games has hindered their performances at times, leading to the unwanted Uncanny Valley effect (a term used to describe the phenomenon when a humanoid figure elicits a sense of unease due to its nearly lifelike nature, positing that we only accept realistic artificial life up to a point before the noted differences make us uneasy). Volumetric capture can bolster game creators’ efforts, lending more realism to their work and helping stars deliver performances worthy of their talents. On top of that, the technology can help developers fully realize the rich worlds they create much more efficiently than with traditional computer graphics.
Of course, sports are an obvious opportunity for VC. Imagine watching the Super Bowl from any perspective you’d like. Currently, the NFL uses static cameras placed in fixed positions to capture play action, and that footage is then stitched together to create unique perspectives for replays. Using volumetric capture as a complementary (albeit entirely different) process, we would theoretically be able to join the huddle, analyze facial expressions, or trek through the stadium crowd at will. This all serves to bring fans closer to the action in a more visceral way. To record just one minute of an NFL game, Intel’s aforementioned “True View” technology records 3 terabytes of data (meaning that in just 15 minutes, the tech captures the data equivalent of the entire Library of Congress). Is the technology perfect? No, not yet. It can be patchy and full of problems, but the industry is pushing the limits further and further with every new project. Instead of a hundred cameras, imagine the depth of a thousand. Maybe even more.
I'm sure we all remember how mind-blowing it was to see Tupac Shakur appear onstage at Coachella 15 years after he passed via hologram. Volumetric capture continues to find a foothold in the music business with things like Madonna’s live performance at the 2019 Billboard Music Awards (where the singer danced alongside four volumetrically captured holograms of herself at varying points in her career). And the music business is colliding with the gaming world, where motion capture is already being used for in-platform concerts (in Fortnight and on Snapchat). VC will make these digital environments more realistic and seamless. And with 10.7 million players attending a February 2019 Marshmello concert and 12.3 million more participating in an April 2020 Travis Scott gig, this technology will give throngs of excited fans more value as they enter the fourth dimension.
As with any innovative technology, time will tell how it is adopted, enhanced, and utilized. Still, the goal remains constant: to provide a more immersive experience and engage audiences. It is difficult to assert with total confidence that this technology will rewrite the way films and television are made or how concerts are thrown, but it certainly opens a door to possibilities that were barely imaginable a few decades ago. There was a time when the idea of a completely computer-generated character leading a major studio tentpole was unfathomable. Today, you are hard-pressed to escape the glut. One can easily imagine the ways that volumetric capture can be integrated into these established creative processes to add depth and reality to an endeavor that has (since its inception) tricked the eye and defied the mind. Creativity is magic, and entertainment magicians are now getting access to one of the greatest tricks they’ve ever know.