On the Horizon: Virtual Production and Entertainment Part Three

Virtual production is revolutionizing the way we make films and television! 

Yeah. Sure. Of course. 

But that’s not to say that VP is without its limitations. As of now, a full stage cannot be rendered in high resolution in real time. This means that only the camera’s specific field of view can see a photo-real image at any given time. The rest of the stage sits in a low-resolution stasis while it waits to be useful. There are also issues with latency, as computers are constantly tasked with relaying an astounding amount of camera information. This often leads to a 10—15-frame delay, limiting the speed of fast camera movements and often making it so they can’t be done on LED walls. As with any new technology, filmmakers must often balance ease and convenience with their truest artistic intent. 

More and more, you’re seeing productions hit the market that have made extensive use of these techniques and technologies. Shows like Game of Thrones and Westworld are bringing virtual production into the spotlight, with the former notably adopting the technology for virtual location scouting and previs planning and the latter recreating costly European locations for more affordable in-studio pickup shots. More often, the programs we watch are demonstrating the potential of LED walls and their ability to create (or recreate) nearly flawless worlds. LED and projection technology was also used in major motion pictures like Oblivion, Rogue One, and First Man. And in addition to these films, the Covid-19 pandemic changed Hollywood’s mindset; suddenly, locations that would have had to be practical or green screened were suddenly available without the need for travel, boarding, and regulation. The limits of the pandemic forced studios and networks to reconfigure many of their productions (such as Star Trek: Discover, Thor: Love and Thunder, and Bullet Train) to accommodate new safety and travel regulations. Beyond the practical, VP is finding a foothold in the world of animation as well, with the recent The Lion King and The Jungle Book remakes crafting their worlds almost entirely in the virtual space. In fact, Forbes recently suggested that Pixar’s Toy Story 4 might have cut its giant budget by 50% (saving around $100 million) had it utilized virtual production practices. 

In television, the tangible effect of the technology can be felt with shows like FX’s Snowfall, a series set in the 1980s that saves nearly $50,000 per episode by using virtual technology to simulate sets that would have been painstakingly recreated just ten years ago. Similarly, Disney+’s upcoming Percy Jackson series uses a Canadian VP stage to tell a fantastical story steeped in Greek Mythology. And at Germany’s Studio Babelsberg, Netflix just wrapped a series called 1899 that built a turntable (which allowed crew to move rapidly through complex coverage and turnarounds) in the volume. VP is also finding a home in the world of sports, as decisionmakers believe it could improve live game coverage by using camera technology to display a full range of stats as dynamic 3D objects during the broadcast. And in music, we’re already seeing VP in promotional videos and performances. In 2020, Katy Perry stole the show with a live American Idol performance that used XR to create a 3D virtual environment consisting of a house, chair, and flowers, with which she interacted in real time. Even metal icons Metallica have gotten in on the act, using enormous LED walls for 40th anniversary gigs. Different venues, different circumstances, but all the same technology. 

Back in the gaming space, home of the inimitable Unreal Engine, the systems are primarily used for building virtual environments for entertainment and exploration. Again, this is the same set of processes as virtual production in the real world. There is an unavoidable connection between the two applications; the change of medium simply shifts the focus and tools. With industry stalwart Epic Games, Unreal marks a “Virtual Production Week” when the two entertainment industries come together for a conversation about evolving virtual production tools. And during a recent Game Awards ceremony, production company The Mill used motion capture and Unreal Engine to bring characters from the popular Apex Legends game to life. The process used an actor in a mocap suit who interacted with the show’s host in real time. This performer was then seamlessly replaced using virtual production technologies. 

It’s hard to know exactly where virtual production technology will lead the creative space over the next decade, but it’s not just entertainment seeing potential in the process. Advertisers, buyers, and media professionals across all industries are seeing the potential in this new format. Imagine fashion shows, car launches, art spaces, exhibitions, concerts, news broadcasts, and architectural tours taking full advantage of the experience VP provides. When Hurricane Florence became a threat in 2018, the Weather Channel used virtual production processes to demonstrate the storm’s potential danger to affected audiences. And while the technology used to bring live broadcasts to life (two perpendicular backgrounds or a single curved screen rather than a 360º stage), the weighty effect is the same. 

Of course, every hot new industry technology poses practical and philosophical questions. For all its convenience, there is an element of alchemy productions might lose by relying too heavily on these digital sets. While it may seem indistinguishable now, a growing chorus of cinephiles yearns for the days of location shooting and constructed sets. In the same way early CGI characters wowed us in the late 90s only to look silly and outdated today, might these false environments lack a sense of scope in twenty years? After all, the best character CGI of the 90s doesn’t hold a candle to some of the most mediocre puppetry of the late 80s, so will that be the case with the images committed to camera in virtual productions? Why do Star Wars fans prefer Jabba the Hutt’s appearance in Return of the Jedi to his infamous deleted scene from A New Hope? Most cineastes still prefer the charm and on-set presence of a well-made puppet to a perfectly rendered digital creature that we all know came from the electric heart of a computer processor. But more and more, the line is being blurred. It’s hard to deny the upside of allowing tentpole filmmakers complete control over the things they’re committing to camera, but there is certainly an argument to be made that relying too heavily on these virtual worlds could have long-term negative effect on the scope of the whole filmic enterprise. As with any revolutionary tool or process, virtual production’s inevitable form will be shaped by the artists and creators that push the technology to its limits.