The exchange: Navigating XR
Posted on Mar 5, 2025 by Admin
XR is a term being bandied about a lot at the moment, but what exactly is it and what might its potential be in the world of filmmaking? A new white paper from Brainstorm – a company specialising in real-time 3D graphics and virtual set solutions for broadcast and feature films – explores exactly that. Nicola Foley digs into the findings with the team
Q: Firstly, XR is a broad and sometimes misunderstood term. How would you define its role specifically in the context of film and TV production?
A: Well, XR stands for extended reality, meaning the combination of real and virtual environments, actors or objects that merge to produce virtual content, so it’s as broad as it could be. As a form of virtual reality, it combines synthetic images (objects, backgrounds, graphics with data, etc) with real actors. However, for broadcast and film content creation, XR has been understood as the combination of virtual environments from an LED wall, and sometimes LED floors, with real characters on a stage and sometimes other graphic elements. This concept has lately evolved into the term virtual production, which is more understandable and includes chroma studio shooting; no longer just large LEDs.
Q: What motivated Brainstorm to create this white paper on XR? Were there particular misconceptions or challenges in the industry that you wanted to address?
A: Brainstorm has worked in virtual set and real-time 3D graphics creation for decades, and has pioneered different technologies, including real-time film previs back in the early 2000s with the movies I, Robot, AI Artificial Intelligence and many others. So this technology isn’t new for us, but we’ve recently seen how real-time, hyperrealistic virtual content creation has become more common.
With so many new people coming to use this technology, there comes misunderstanding and misinformation around the technology, its possibilities and results. Coupled with the fact that some sectors in the market believed real-time VP, or XR, was only possible using large LED volumes, this made us compile some information that ended up as a white paper. It details concepts, workflows, advantages and disadvantages of VP in general. In fact, as this technology has evolved and the industry is adopting it, we realised this ‘Understanding XR’ paper required updates, which is why we’re releasing a second edition now called ‘Understanding Virtual Production’.
Q: Having witnessed the evolution of XR, what have been some of the most significant milestones in its adoption within the film industry?
A: We must praise The Mandalorian and Industrial Light & Magic for creating a hit and putting this technology on the map. However, despite the initial hit, and the buzz in the industry that led some to believe that chroma was dead, once the hype cooled down the industry understood that XR was yet another technology allowing content creators to fulfil their dreams, improve quality of content, save time and reduce costs.
The fundamental difference is real-time production. The LED walls provide live, hyperrealistic 3D plates for the background, but real-time compositing can also be created in chroma sets, with the added value of recording the different layers for further post-production if required. Chroma is no longer constrained to VFX and post; it can be used for real-time production, as it normally is in broadcast. Now, VP is not only used in film, but in broadcast shows, drama and even live events.
Q: For smaller production outfits working with budget constraints, what practical steps or tools would you recommend to leverage XR effectively?
A: One of the main reasons why filmmakers and broadcasters of all sizes use virtual technology is because it’s possible to create, in real-time, virtual content that can’t be distinguished from reality and seamlessly includes actors. This opens the door to reducing the gaps between large and small facilities, increasing the quality and possibilities of smaller-scale projects. There’s no reason why VP can’t use chroma sets instead, or even be combined with LED volumes.
LED volumes don’t need to be huge, nor must they be used for entire projects. Tech like set extensions support the use of smaller volumes, with backgrounds digitally expanded. This allows LED-based productions to be more affordable – not to mention more studios incorporate LED volumes for rental, along with chroma sets, which gives greater flexibility.
Q: The white paper discusses challenges like colour calibration and video delays in multicamera XR set-ups. Can you share insights on how close the industry is to resolving these hurdles – and what innovations you see on the horizon?
A: The fact is that chroma is an affordable and established technology, while LED is relatively new, so many early adopters have been faced with challenges that required revising the initially optimistic assumptions regarding the outcome of these workflows. All the manufacturers involved, from software companies to camera vendors, work to facilitate the adoption of this technology as well as simplifying the workflows and technologies involved.
Now we can enjoy calibration tools that greatly simplify tracking and colour grading to ensure accuracy. With regards to multicamera productions, the adoption of multiplexing technologies such as Frame Remapping or GhostFrame reduce the complexity of the set-ups. These technologies allow for displaying different content on the LED wall simultaneously, which enables the various cameras to sync and each capture different images.
Q: Pre-production is crucial in XR workflows, as you’ve highlighted. What key considerations should filmmakers prioritise to ensure seamless integration between real and virtual elements?
A: As with any production, planning is crucial to ensure everything goes as smoothly as it can. This means it’s the final result, requirements and budget that will establish which technologies we should use. There will be some moments in which using an LED volume might be almost mandatory – like shooting scenes that require lots of reflections, in which having a real background or LED wall simplifies the shooting process. Others will require chroma sets. The best advice is to count on skilled and experienced staff who understand well the technology and its possibilities, so they can help schedule the production using the best tools for the job at any time, regardless of whether they have to go to outdoor, indoor or fully virtual shoots.
Q: As XR continues to evolve, what are the emerging trends or tech that you believe will drive its adoption and evolution over the next decade? And in what ways could it reshape storytelling?
A: Technology helps to facilitate production and provide us with new ways to tell stories. Using VP, we can shoot in impossible places, like extremely small spaces in which it might be difficult to fit all the necessary gear, or guarantee the perfect framing at any time, regardless of camera position.
Car scenes are another example, in which the interior of the car and close-up shots can be virtual, and not require long shooting hours and expensive permits. But we can also shoot in inaccessible areas, or where it’s impossible to grant permissions, such as remote locations, monuments etc, without requiring crews to build large props.
Last, but not least, VP allows us to spend hours shooting the perfect outdoor scene with the exact lighting and environment that the director and cinematographer want, without having to worry about the conditions changing, regardless of the time it takes to shoot the scene. We can make sunsets last forever! All of the above simplifies the production process and provides new tools and conditions to help filmmakers achieve their desired results.
Q: With XR becoming more embedded in film production, how do you see it influencing the skill sets and roles required in the industry? Would you say we’re heading towards an era of greater interdisciplinary collaboration?
A: Yes, definitely, but not to the point of excluding the current skill sets and expertise in the industry. On the contrary, XR and VP are simply extra tools we have in our hands to improve content and optimise production, but they of course require a good understanding of the technology, its downfalls and benefits, so we can extract the most out of it.
Lighting design crews will still be needed to illuminate scenes, but they might have to take into account the new baseline of shooting in chroma or LED; their skills will surely remain necessary for achieving the perfect lighting. DOPs will need to apply their understanding of how cameras, lenses, lights and the environment have to be combined for the results they want when using LEDs and chroma, but they will surely get there with some initial advice. Directors will also find they can create new framing, shots and looks by using this new tool.
There will even be some people, VP experts, that previously got their finished shots and had to do something with them, who will now be more involved in production, as their continued input is extremely valuable for the optimisation of productions. So, yes, this does open up a new era of interdisciplinary collaboration for achieving amazing results.
Read Brainstorms white paper at xrguide.brainstorm3d.com
This story appears in the February 2025 issue of Definition
