Bridging the Virtual Production Skills Gap
Posted on Mar 21, 2024 by Samara Husbands
Get the latest from industry experts
Finding VP talent with the right skills is a growing problem in our industry. Leading practitioners in the field weigh in on what we can do to solve it
The Panel
Addy Ghani, VP of virtual production at Disguise
Jonathan Brigden, Managing director at Distortion Studios
Eric Rigney, Global director of Mo-Sys virtual production academy, adjunct professor of virtual production at West Los Angeles College and Hollywood CPR. Member of SMPTE’s RIS virtual production education committee
Johnny Johnson, Senior creative technologist and
VP supervisor at StoryFutures Academy, CoStar National Lab and the NFTS
Lauren Rogers, Senior producer, Bild/MARS and lead tutor, MARS Academy
Joanna Alpe, CCO Bild Studios, head of academic strategy at MARS Academy
Marina Prak, Marketing manager at ROE Visual Europe
Jelle van Abbema, Technical support engineer at ROE Visual Europe
Definition (Def): From your perspective, what are the critical skills and competencies professionals in virtual production need today?
Lauren Rogers/Joanna Alpe (LR/JA): An understanding of virtual production workflows, communications pipelines and key technologies is critical, as well as a willingness to learn and agility to adapt.
The key technology pillars are real-time content, camera tracking, LED systems and media servers/playback workflows.
While these can be learnt in theory, nothing compares to having hands-on experience within a live and commercially working LED volume.
Often, we find a fear factor and lack of understanding might hold a production back from taking full advantage of the benefits virtual production (VP)can bring.
This was the driving force behind launching our MARS Academy, to share MARS Volume with professionals thirsty for gaining new critical skills and hands-on experience with the workflows.
Endorsed by ScreenSkills, we launched courses such as VFX Artists on Set, LED Systems for Film Professionals – all running out of a commercially working virtual production facility.
Eric Rigney (ER): Firstly, an overview of the overall VP ecosphere. You need an understanding of the different types of virtual production solutions – ICVFX (LED and projection), green screen, AR, XR – as well as their limitations and advantages.
Understanding when and which type of VP is appropriate for a given use case, budget and available skill sets is paramount. In some instances, traditional methods might be more appropriate.
Gaining hands-on experience or in-service hours is crucial too.
There must also be an appreciation of the importance of preparation – especially visualisation – and necessity of including all creatives within this process.
An experienced virtual production producer and VP supervisor are crucial players in the success of a VP experience, but are difficult to find. There’s a lot of ‘fake it ’til you make it’.
Johnny Johnson (JJ): The essence of VP remains making 2D narrative content, so the understanding and experience of collaborating with other filmmakers remains a core requirement of working in this area.
They should have an understanding of how departments work together and communicate on-set in an effective manner.
This extends to how the on-set VP team integrate, communicate and work together with the wider set of creatives.
Elements of the technology change often in subtle ways as we move forward, but in the main, a lot of the core technology has remained steady for some time now.
The ability to utilise Unreal Engine (or similar real-time rendering software) to make and manipulate digital assets as required is central to anyone looking to work in the technical area of VP.
Jonathan Brigden (JB): Virtual production is all about the creative use of technology.
A wide skill set of creative and technical curiosity is critical to the successful use of VP.
There is a small group of very experienced early adopter professionals, the majority of which are freelancers who can command a high day rate.
There is clearly a need for artist skills in Unreal Engine, but virtual production requires a very distinct skill set in what we would term visual engineering.
This is a combination of very high-level computer and media server skills, along with broadcast engineering and audio-visual skills.
A further skills gap is the complete colour workflow from capture to screen to ensure colour science of any shoot is clearly managed throughout the process.
Addy Ghani (AG): We need creative technologists with both an eye on the technology and understanding of scripted production.
For example, it is important to understand the advantages and limitations of an LED volume, and then use that knowledge to help guide a cinematographer to success on a virtual production shoot.
Marina Prak (MP): I think working in a virtual production space requires, of course, some technical knowledge.
But, it also demands the capability to effectively work in a team because everything in virtual production is integrated.
So, you must be a team player to make it all click.
Def: How have these skill requirements evolved with advancements in VP technologies?
JB: The technology surrounding virtual production is under regular development. Constant curiosity around the global advancements requires individuals to always keep their skills up to date.
For example, Disguise media server has been the only go-to for XR workflows. New entrants like PIXERA are arriving on the scene on a daily basis.
LR/JA: The technologies are always evolving. Shooting live action in front of LED screens running video content (2D playback) was essentially an evolution of back projection – think Audrey Hepburn in Roman Holiday zooming around on a scooter.
Then, real-time game engines entered the scene (3D playback), which added a new series of technologies to understand: game engine software (often Unreal Engine) and all its requirements for ensuring 25fps playback on huge LED screens.
Plus, camera tracking and the on-set requirements of this technology. The advanced media servers and playback workflows to ensure performance – suddenly a more in-depth knowledge of traditional AV systems is required.
Now, as AI works its way through the tech stack, a mix of 2D and 3D playback known as 2.5D is emerging as an efficient way of working.
By keeping up to date with these advancements, professionals ensure they are using the toolkit to best effect, and critically, not getting left behind.
Conversely, the virtual production team (oftentimes from a video systems and live event background) need to grow their skills and understanding in traditional filmmaking techniques.
This enables us to understand what the director/DOP wants to achieve and makes for a smoother on-set experience.
JJ: Over time, we are seeing more understanding from traditional creatives about the process of working with VP digital assets and pipelines, how the VAD team come in at an earlier stage in the process and increasingly how the production design and art department play into that collaboration.
On our VP course, there are increasing traditional filmmakers coming through, such as DOPs and production designers.
Previously, it was heavily orientated towards VFX technicians and artists.
Now, there is a more even mix.
MP: I think at the start it was about the LEDs syncing with the camera, the basics. What you see now – because DOPs evolve the technology of virtual production and the requirements become stricter – it is now going into the deeper technology of both colour accuracy and colour management.
Because virtual production is relatively new, we learn by doing.
So, with every step we make, the technology evolves, more things come into play and we discover new things that can be better developed and improved.
Jelle van Abbema (JVA): A lot of it is being better prepared in the pre-shoot. If you run colour management correctly, it’s less work in post – and making sure that workload is properly divided across your entire timeline.
ER: The skill sets are still new and developing, and craft unions can struggle to determine what aspects belong to which union/guild.
Hybrid solutions, like mixing green-screen VP with LED ICVFX VP, are becoming increasingly popular due to the cost savings and the greater creative capacity of the hybrid combination, taking advantage of the best of each.
AG: As the closest thing to a global standard, Disguise has paved the way for professionals to learn and leverage one single, cohesive tool for productions all around the world.
With the multiple resources available, including our Virtual Production Accelerator course, it’s now easier than ever to upskill yourself and get into virtual production so you can operate a platform like Disguise, or a content engine like Unreal.
Def: What challenges have you observed that indicate a skills gap in VP? Are there areas where professionals face difficulties?
MP: In all areas, to be honest.
LR/JA: In our experience, a simple lack of VP understanding can generate fear that becomes a psychological barrier to engagement.
It’s always rewarding for me to see attendees at our demo sessions or students at MARS Academy have that ‘penny drop’ moment experiencing the volume in real life.
It can be as simple as experiencing the volume, armed with a basic understanding which removes that fear and opens the door to getting involved.
Visualising and planning for VP has historically been a challenge we’ve observed.
This has improved greatly in recent times, however, with teams becoming more comfortable with the notion of investing more in the pre-production stage.
The old adage of ‘fix it in post’ is not as relevant here, as the investment put in upfront in pre-planning the shoot means the resultant footage moves much closer to final pixel, requiring little or no post.
To support productions shooting at MARS Volume, last year we launched our own internal previsualiser tool.
This effectively produces scale-accurate previs of our volume with client content applied to it. It’s a game changer, giving productions confidence that shots will be successfully captured on the volume.
JJ: Established filmmakers are often daunted by the scale and scope of digital software when they start.
Through training and practice, most take themselves to the level they wish to achieve to get their work done.
For example, DOPs learn how to pre-light basic scenes, and directors learn how to block out scene movements with simple previs.
They don’t necessarily need to know how to design physics simulations and programme interactive content.
The option, however, is always open for them to go further if they desire.
It helps there is no limitation to them other than time and determination to learn (they will need a recent laptop, Mac or PC to run the software).
However, what they do bring with them is the experience of collaborating on-set or in a production environment.
People coming from an interactive/games background generally have a solid background in the software, but lack the time spent on-set to learn how people communicate effectively in that world.
JB: The uptake of virtual production within an established, centuries-old filming workflow is extremely difficult to convince.
There have been a few bad actors in the business that have scared off the clients.
All the while, gaining best-practice experience is extremely difficult.
The speed of uptake versus the upskilling of the workforce is particularly tough to keep consistent so practitioners are getting enough experience on actual shoots on a regular basis to maintain continuity of experience.
AG: The basic skills gap is relatively easy to overcome with training.
However, we do need seasoned veterans who come with years of experience.
Because VP is still a newcomer, top-end experts are rare.
This will be solved once there are additional professionals with more time and experience on-set in an LED volume.
Our VP Accelerator course aims to solve this by offering that on-set experience and a chance to make your own short film in an LED volume.
Def: How well do you think current educational opportunities address the demands of virtual production? Are there aspects you feel are underrepresented or not adequately covered?
JJ: There are a range of courses available both from National Film & Television School as well as other institutions; they appear to be available across the UK in different areas.
We have been oversubscribed with applications to our certificate in virtual production on every occasion it has been run – so the demand to learn these tools remains strong.
Epic Games has also been a great supporter of educational efforts to use their software tools.
Our training is designed to suit a mix of filmmaking disciplines working together, which culminates in the creation of short VP films against an LED wall.
In the future, as the requirement for digital filmmaking grows, we expect to see an increase in more specialist training focused to specific disciplines such as production designers and cinematographers.
ER: Education is just starting to tackle the skills gap – and teachers are currently learning what they’re going to teach – but the process of building and funding a programme takes time.
Curricula are in the development and testing stages – it’s all occurring simultaneously.
We need a greater focus on hybrid solutions; it’s better to teach concepts with hands-on training to many students with separate workstations and large monitors, than spend the same amount of money to demonstrate concepts on a single large LED wall set-up.
If affordable, both would be best. Ideally, we want to be creating opportunities for students and professionals to practice VP, over and over again.
LR/JA: There are many great people involved in education within virtual production.
In the UK, ScreenSkills is a fantastic industry organisation that champions bridging critical skills gaps, and conducts crucial research to uncover where the gaps lie.
We’ve thoroughly enjoyed partnering with them to deliver courses.
We’ve also been privileged to work with various universities.
I’m always impressed with a forward-thinking faculty who seek us out to discuss their plans for ensuring their students are equipped for future skills in film and television.
University of the Arts London, University of Bath, Johnny Carson Center for Emerging Media Arts, Brunel University and Greenwich University are just some investing into facilities and curriculum development, incorporating virtual production skills – and we’re determined to keep supporting their efforts!
The area that is always the hardest to adequately cover is the hands-on, volume-based element of training.
It’s an expensive exercise to devote an LED volume to educational purposes.
Investing in skills training at a writers’ room level is also an area we are keen to develop and invest in further.
Once writers are confidently writing scenes with virtual production in mind, it will make for a more frictionless experience when bringing the shots to life on film.
JB: There are plenty of universities opening their own VP stages.
This will eventually ensure that students are better prepared for the virtual production needs of the future.
Animation courses must focus on encouraging students to have experience of real-time tools such as Notch, TouchDesigner, Unreal Engine as well as the more traditional software such as Maya and 3D Studio Max (now Autodesk 3ds Max).
Students also need to realise that employers need to see a wide range of skills to ensure they can give themselves the best opportunity to slot into a complicated workflow at different levels.
AG: Most education efforts at the moment focus on the technical aspects of VP, and very few focus on the application – which is much more powerful to boost one’s career. Disguise’s VP Accelerator programme strikes the balance of teaching technical skills as well as production application: how to correctly leverage the tech to create incredible content.
Def: In what ways can the industry collaborate with educational institutions to bridge the skills gap effectively? Are there successful models or partnerships you have seen?
ER: Create VP sandboxes, supervised by experienced virtual production professionals whenever possible.
If not, just do it by trial and error in a professionally safe environment. I don’t know of a programme I like yet – but I hope to build one soon!
JJ: It is always useful for people training to gain access to on-set time whenever possible.
This can be tricky due to IP protection and NDAs, but getting first-hand observations of how professional VP teams collaborate can help people understand the experience part of filmmaking, something which cannot be learnt online or in a book. We simulate this by shooting some VP films against an LED wall where the students take on a filmmaking or technology-based role suited to their background and interests.
JVA: The Disguise VP Accelerator is one of the things helping close that gap, but I think there should be more initiatives like it to better educate everyone.
JB: There are lots of collaborations between institutions and industry.
However, to fill the gap I have noticed plenty of practitioners offering their own courses.
Two years ago, we created a scholarship programme with our local University of the West of England (UWE), which allows students to gain studio experiences, see how our Unreal artist works and how a virtual production shoot comes together.
LR/JA: Open days, offering shadowing opportunities and developing internship programmes.
We are supporting the University of Bath as they build out world-class facilities to equip their students with industry-critical skills.
This partnership model, industry and university, is an important way to collaborate and work to strengths (academic rigour and learning pathways versus industry-relevant workflows and skills).
Being open as much as possible is key.
We host regular open days for educators; last year saw us welcome 16- to 17-year-olds from InvestIN at the volume for a MARS Academy bootcamp day.
They impressed us with swiftly picking up the VP toolkit and producing engaging content – all in one day.
Seeing behind the curtain and demystifying what is involved in virtual production is the aim of the game.
We have also worked at a high level with PhD students from UAL for their research projects.
A top-down, bottom-up approach ensures that we can inject industry relevance at all levels.
MP: A lot of technology suppliers offer – like we do with the ROE Academy – training sessions to get people into it, and then universities step in too.
I think it should be an official curriculum so people can get the right skills.
We work with the Academy of Live Technology, Netherlands Film Academy and Breda University, which focuses on gaming.
We see huge interest in the ROE Academy, and people also ask, ‘Apart from the basic skills being trained, can you also go into virtual production?’.
At the Netherlands Film Academy, one of the students said they wanted a degree in virtual production technology – he was the first student within the academy to do this, and had to do it all on his own.
The support wasn’t there, so he came to us.
We agreed because, for us, it’s important that the skill set is present.
We supplied them with a screen so the whole team of students could engage with the technology and learn from it.
This will deliver people to the market that already have the experience and have learnt from the process. I think that is where suppliers and educational institutions should meet.
Def: Is there a need for standardised certifications in virtual production – and how can the industry establish common standards for evaluating proficiency? How might these impact hiring practices?
AG: Yes, absolutely.
This sets a uniform bar for the newcomer to enter the world of virtual production.
At Disguise, our Virtual Production Accelerator graduates earn this certification, which indicates they are ready to begin at a competent skill level.
The expertise will come with lots of time and projects.
However, the consistency of a certification programme is needed in order to gauge professional skill level.
JVA: That might be useful, as there are so many different things – it’s very broad.
An indication of which parts of the entire chain you understand and can work with might help employers.
MP: I think, for employers, it would be great if they could evaluate the candidates they have based on a commonly accepted certificate.
For example, if somebody has this certificate, you can trust they have this understanding.
The knowledge needs also to be brand agnostic.
So, you can do Disguise training, but if you hire somebody to work with a media server, it’s best that they also have knowledge of PIXERA, Vizrt, Zero Density or something of a similar function. They need to be broad in their experience.
JB: The main need is experience, but some kind of certification or proof of experience would be fantastic.
Not dissimilar to flying hours for pilots.
We are also very keen to make sure that as many people as possible can get into the industry – so making courses accessible is essential.
LR/JA: Standardisation is something we are excited to see developing as the emerging VP industry matures.
The world of VP professionals is actually very small.
We tend to keep in touch with each other, sharing learnings and swapping experiences.
In this way, we’re all working towards aligning standards.
I find that people involved in the leading edge of virtual production tend to have things in common, like collaboration and striving for excellence.
We are quick to adopt best practices when they are revealed.
I can definitely see a future where key institutions have carved out reputations as generating excellent and industry-savvy grads.
The indicators of this are well underway with the investment in university facilities we see getting off the ground this year.
ER: SMPTE RIS is working it!
The On-Set Virtual Production Initiative (smpte.org).
Not easy, as the technology is changing rapidly.
There aren’t always such standards for traditional methods either.
Additionally, while virtual production’s success is heavily dependent on a facility’s digital infrastructure, facilities, integrators and VP engineers do not implement data industry standards in their designs, construction and maintenance.
When creating a certification of facilities, to what degree does their digital infrastructure meet data industry standards?
No one wants to dine in a restaurant that doesn’t meet health department standards.
What insurance company would want to insure a production willing to work in a highly digital environment that does not meet data industry standards?
Def: Looking ahead, what skills do you anticipate becoming increasingly important for professionals in virtual production?
JB: The skill set for virtual production is so wide: scene creation and animation; computer science; broadcast engineering; colour science; directing; producing; AI prompting; programming.
So many of the automated or functional skills will be taken over by machine learning and automation in the next two years, that we hopefully will be just making creative decisions around this incredibly complicated workflow.
LR/JA: Communication and leadership skills.
This is a highly collaborative area, and it can be make or break for a shoot if the various departments are not integrated, mutually supportive and respectful of each other.
Virtual production presents a new team, new technologies on set, which might be perceived as a black box to other departments.
The leadership to bring this together is mission-critical to a successful shoot.
Technologies will continue to develop, advanced understanding of 3D and 2.5D playback will continue to be important skills for professionals to develop.
JJ: The core skill is a specialisation in a filmmaking discipline, the same as it is in a traditional sense.
But, of course, learning how best to use Unreal Engine in various ways is currently key as this software is at the heart of real-time rendered filmmaking (at least for now).
For those looking to work in a technical area of VP, an increasing understanding of the hardware and how that plays into the process can be beneficial.
Ideally, an effective VP supervisor should be an expert in all the hardware and software tools used for a virtual production, along with a number of years in a senior creative filmmaking role so they can support and develop the skills of the rest of their team during production.
We are starting to see the beginnings of cross-media production, where the same digital assets are created and put into use for filmed content, game design and perhaps a VR or AR experience –potentially all at the same time.
So, the ability of technicians and artists to learn new skills across media formats will become increasingly useful as these areas and production pipelines overlap even further.
ER: Experienced VP producers and supervisors: knowing when and what flavour(s) are most relevant for a given production’s context.
Plus the ability to put a solid team together, setting realistic expectations and requirements for all players.
Data industry standards in the design, construction and maintenance of VP studios and services are also key.
AG: As the industry matures, we’ll shift away from generalists who do a few things on a production, to experts that specialise in a single subject.
This is the same journey that VFX professionals had – where if you want the best-quality digital human, for example, you can find specialists or companies that do just that extremely well.
MP: As the technologies evolve, your skill set needs to evolve.
Since it all evolves at a very rapid pace, everybody now engaged in VP can’t be sure on relying that what they know is what they should know.
I think everybody should continue to train and take on the new things that keep coming up, and that it will become more segmented because virtual production for film is a completely different process from virtual production for broadcast or live events.
I think that distinction needs to be made.
JVA: A lot of the knowledge does apply to both film production and broadcast, but the entire workflow might be completely different.
There are a wide variety of different ideas – about colour management, but also broadly – on how virtual production should be done.
There’s division within the sector on what should be standard practice.
Not everybody has the same idea of how something should be done.
Def: How can both educators and industry practitioners stay ahead of the curve and remain relevant?
MP: It’s very important that education centres stay close to the people engaged in this market.
Otherwise, what they teach will be out of date maybe in a year.
So, it must be the industry itself – as well as the educational institutes – that work together to help close these gaps in knowledge.
JB: There are some fantastic people (for example, Rob Chandler) who are pulling together the industry so that we can grow and be ahead of the curve.
Healthy competition through collaboration and communication among a very friendly set of creative technicians is a really enjoyable experience.
JJ: It is important for educators to keep up to date with technological advances.
This can be tricky, as areas of it change often.
With the increasing advancement of AI technologies, the speed of change is rapidly increasing.
Epic Games offers specialised training sessions, which can be useful to educators.
They also run an examination scheme to ensure authorised trainers remain up to date.
In general, practitioners in virtual production should remain keen on learning the new emerging technologies as and when
they come along.
AG: Always refresh and optimise the curriculum as the tech and the application evolves. VP is maturing fast; there are already new methodologies and schools of thought not discussed two years ago.
Just as one example, integrating AI-powered content engines into productions is something we’ll see in 2024.
ER: Participate in communities, organisations and committees, sharing what is learnt.
Practice as much as possible the skills that one plans to teach.
Provide demo stations/stages for students and professionals to practice skills over and over again.
LR/JA: Keep talking and spending quality time together!
The more we know each other and can share information, the better off both sides of the equation will be.
Being involved in forums, conferences and round tables.
We are looking forward to SXSW in March where we can again connect with both educators and expert practitioners from around the world – ensuring the future of virtual production skills training is well on track.
This Round Table was originally published in the March 2024 issue of Definition.
