On the Horizon: Extended Reality and Entertainment

Technological evolution. It’s an immutable law of human existence. And with innovation comes artistry. Filmmakers, musicians, and live performers have always looked to modern technology to enhance the experiences they deliver, helping them more effectively transport us to new worlds. Entertainment is an escape—a way to enter a new reality, even for a moment, in order to thrill, relax, educate, engage, or challenge us. The more fully realized an alternative reality is, the more it engrosses its audience. Take science fiction, for example. People have always loved the genre, in print and on film, for its world-building and fantastical elements. But when Star Wars first hit cinemas, its cutting-edge special effects thrilled and amazed audiences in ways that made older movies seem instantly dated. Spaceships had never moved like that before! By leveraging a new computer-controlled motion camera system with unbridled artistry, George Lucas and company pushed cinema toward a new frontier, knocking down another wall between fictional worlds and our reality. 

So, what’s next? As technology continues to evolve beyond the service of our basic needs, artists are using it to push the boundaries of storytelling and the types of experiences we seek out. So, what does a generation growing up with miraculous technology dominating their daily lives expect from the future of entertainment? How will they expect to be entertained? Just as cinemascope was created to lure people back into theaters after the advent of home television sets, or as 3-D hoped to pull people back once home theaters evolved, the next big leap in entertainment is set around Extended Reality (XR) in one form or another. When we discuss XR, we’re essentially talking about three different formats: 

• Virtual Reality (VR) – Often incorrectly used to describe all other forms of XR, VR is specifically a fully immersive experience that tricks your senses into accepting a different world apart from the real world. VR often utilizes a head-mounted display or headset connected to controllers, a console, and/or a computer. Have you ever put on a headset to simulate being on a raging rollercoaster? That’s the VR experience in a nutshell.
  
• Augmented Reality (AR) – Uses technology that overlays digital information on real-world elements, maintaining the real world but enhancing it with new layers of perceived reality. It’s about technological supplementation. Remember the wild success of Pokémon Go, which merged reality with fantasy in a way that proved profitable (and occasionally dangerous)? AR is also being tested in the touring market, with holograms of deceased and unavailable musicians taking the stage in place of the real thing. 
  
• Mixed Reality (MR) – Combines real and digital elements, allowing the user to interact with and manipulate physical and virtual elements within a set environment with their own hands and without removing their headset. MR uses advanced sensing and imaging technology to let the user immerse themselves in this environment, bridging the gap between the real and digital worlds. While MR is often used in technical fields, look no further than Nintendo’s Mario Kart: Home Circuit to see it in action (the game uses radio-controlled cars that race around a player’s house, feeding streaming video from onboard cameras to the user). With MR, the idea is to break down the wall between the real and imaginary worlds. 

Each technology is striving for the same basic thing: immersion. Why watch a movie when you can be part of it? If the earth-shaking success of the video game market over the last thirty years has shown the entertainment industry anything, it’s that there is money to be made in the marriage of first-person experience and storytelling. Not just in games and movies, but in live events like concerts and football games. Imagine how exciting it will be to sit on your own couch, watching college football from the 50-yard line... 

But how will audiences and consumers take to wearing a headset or arm band while watching their favorite programs? As with most things, the promise of these new tools and techniques is boundless, but their individual successes will be based on execution. Look no further than Nintendo’s failed Virtual Boy in the mid-nineties—a gaming platform that promised an immersive experience only to deliver … well, mostly eyestrain and red lines. 

The pandemic certainly hastened adoption, with musical artists looking toward XR technologies to reach their audiences when traditional live venues were closed to them. As Steven Hancock, co-founder of MelodyVR said, "If [someone can] go to a show [they] always will, but for all of the limiting factors as to why people can't get to live shows, be it geographical restrictions, age restrictions, or cost ... we realized there might be an opportunity to smash all those barriers and truly globalize music." According to a 2019—2020 VRX XR Industry Insight Report, a lack of compelling content, as well as the price of head-mounted devices are two main factors inhibiting the widespread adoption of VR. Those elements are primed to change quickly. 

It can be hard to view the full potential of these technologies through the prism of what is available now. Behind the scenes, production crews are already using AR to light, plot, and plan their shots, saving time and money in the process (which you can read about in our Future of Work piece). But this begs questions like: “where does one place a camera in an open-ended and immersive narrative experience?” Fillmakers are already using XR to pick up where 3-D filmmaking left off, creating a whole new cinematic language—one that asks more involvement of its audience. (Learn more about the state of immersive storytelling by reading our discussion with Final Draft).

Three decades have certainly made a huge difference—not only in technology, but also in expectation. Today’s consumers hunger for new experiences, and AR, VR, and MR aim to deliver them in everything from scripted entertainment and professional sports to healthcare and the travel industry. The investment is there; primed players like Sony, Samsung, and Google are in the mix, and so is the potential for exponential market growth. In 2020, the VR market was estimated at $6.1 billion, and it’s expected to reach $20.9 billion by 2025 (with a CAGR of 27.93%). Similarly, the AR market looks to grow from $14.7 billion to $88.4 billion by 2026 (with a CAGR of 34.85%). According to eMarketer, 2026 will see consumers using 70 million VR headsets worldwide, a huge increase from 26 million in 2021 (21.91% annually). And when it comes to MR, the global market was valued at $376.1 million in 2020 and is expected to reach $3.9 billion by 2026 (with a CAGR of 47.76%). It sounds like science fiction, but these new realities are practical, promising, and completely within reach.