Valley of the Dolls
Kevin Baillie, VFX supervisor on Welcome to Marwen, explains how he brought the dolls to life.
Words: Julian Mitchell / Pictures: Method Studios
Kevin Baillie works for Method Studios out of Santa Monica and was the VFX supervisor of Welcome to Marwen. He has also worked with director Robert Zemeckis on a number of his films but started his career as a pre-vis artist on Star Wars: Episode I – The Phantom Menace when he was just 18.
With the new movie, Kevin and his team have moved the motion capture world on from the uncanny valley inhabitants who repel us on a subconscious level.
“The original idea was to shoot actors on oversized stages and then augment them with digital doll joints. Robert is such a fan of his actors and their performances that he wanted to make sure that they came through at full fidelity in the final film. But the tests that we did to that effect looked really terrible, they were totally horrifying. So we were stuck as we couldn’t do that and we couldn’t do traditional motion capture because that technology isn’t good enough for the 46 minutes worth of human doll performance to the screen.“So we had that eureka moment of thinking that instead of putting digital doll parts on Steve [Carell] and the rest of the actors we could put Steve parts on the digital doll using his eyes and mouth. The process of getting to the realisation probably took about a year.”
The metamorphosis of actor Steve Carell from a nervous mocap artist to a fully formed Cap’n Hogie doll.
.
“It was so critical that we got the lighting right in that mocap stage.”
Success That Comes Through Failure
Kevin and his team realised through all their failed efforts to make the doll side of the movie work that this new way of achieving it was the only way and they had to make it work.
“It took a big leap of faith for Kim the DOP to believe he could light this big grey void that is the motion capture (mocap) stage as if it’s the final movie. That takes some imagination.
“To help him with that we created a real-time version of Marwen in the Unreal video game engine. The app allowed him to tweak sun direction and intensity with other custom light. He was actually able to work with Robert to understand what all the blocking and shooting orbits for every scene in the mocap world was going to be. He then spent about two weeks working in this real-time interactive Marwen, pre-lighting every single set-up we were going to be doing in the mocap world.
“By the time we got to the mocap stage we knew what the lighting was going to be. We had one monitor which showed us what Kim’s ARRI Alexa 65 was seeing on the mocap stage – which was actors in grey suits and blue screen – and then another monitor that showed us a real-time view of what that same camera was seeing in the Marwen universe.
“We could use that to compose and make sure that our lighting that we had set was actually going to look good in the final shot. That ended up being so important and I really didn’t know how important until we entered post-production. In post-production we found that if the angle of the light hitting the actor’s faces was even just a few degrees off in our digital renders compared to what it was in the real stage renders, it broke the entire illusion. It was so critical that we got the lighting right in that mocap stage because if we didn’t there was no room for error. It was kind of a ‘moon shot’ to get things right in this stage and this virtual production process really helped us to achieve that.”
Kevin and his team had to develop new motion capture techniques due to the amount of ‘doll’ time in the film, this included a tilt shift effect.
Meeting of Minds
Of course there are plenty of big movies that use mocap techniques and virtual production processes to achieve the shots but this movie more than others had to blend traditional cinematography techniques with virtual ones. “On this movie we were so constrained in terms of time, budget and resources that we had to get inventive; we had no choice but to really heavily lean on each other. I certainly couldn’t have lit the film in post without Kim’s lighting designs and Kim couldn’t have lit the movie on the mocap stage without our help.”
Having a film crew on a mocap stage isn’t a regular event but for this film it was the only way to shoot. “The unusual thing that happened on our motion capture stage was the intersection of every single normal live action component combined with every single normal motion capture component. There have been films dating back to even Avatar where James Cameron used what he called the Simulcam to see what the Avatar world looked like through a digital camera. But his actors were in these helmet cameras with these dots on their faces in flat lighting. You then have a movie like Jungle Book which was pre-lit, the lighting was pre-designed but the actor wasn’t being motion captured. So there’s all these movies out there which have used components of what we did and we just basically jammed them all together with the goal of bringing the actor’s souls to the dolls for 46 minutes of the movie without ever dipping into the uncanny valley. I think this is the first time that it has been achieved on this scale which I am particularly proud of.
“When the uncanny valley isn’t there then the audiences just focus on watching characters and the actors perform, that’s what I’ve heard from people who have watched this movie. I even heard that when people watch the doll scenes that they forget that they’re even dolls. It’s great that people aren’t talking about the thing that could have been the film’s biggest problem.”
Future Relationships
This film has moved the world of mocap on, something that hasn’t been done for decades. But what does the VFX world require from the purely photographic world to move it on even further?
“Cameras in general have been getting stunningly better at a crazy pace. We’re already getting all kinds of data off the cameras to help us. What I’m excited for in the future of cameras is products that not only give us data about what was done, how something was shot, but do things like actually sense the depth in a scene and give us real-time camera tracks right out of the camera using optical tracking and gyroscopic data and things like that.
“Certain technologies out there are getting close to being able to do that. If camera manufacturers can deliver a sense of what the camera is doing in the world in addition to the data they provide already, that would be amazingly beneficial. Also depth information; I would like to never
use a blue screen again, and there are companies working towards that with AI and machine learning to look at imagery and be able to rotoscope something out of the middle of a shot by just pointing at it. It senses that it’s an object and separates it from the background. An increased awareness not just of what a camera is doing but what is it seeing is very exciting.
