The Fast Frame Game
Posted on Nov 29, 2019 by Julian Mitchell
Not only has Gemini Man produced the first plausible virtual human, it has also resurrected the HFR experiment. But what’s different this time?
Words Julian Mitchell / Pictures PARAMOUNT
There were some motivational words for the crew on the set of Gemini Man, encouraging everyone to keep on keeping on with the technical innovation. The words were a version of JFK’s speech he used to galvanise America’s space race back in the 1962: “We choose to go to the moon. We choose to go to the moon and do the other things, not because they are easy, but because they are hard.”
Welcome to an Ang Lee film, where the technical envelope truly is pushed. Not only has Gemini Man brought back the experiment of high frame rates, but also introduced a fully digital young Will Smith – his face, anyway – which has been put front and centre in to the 3D action and, at last, this avatar is worth the cost of admission on its own. He looks great and believable, finally avoiding the dreaded eternity of the uncanny valley.
Goodbye Billy Lynn
But for this article we are revisiting HFR, a highly controversial cinema technique, one that has already been championed by some A-list directors like Lee, Peter Jackson and James Cameron. Around three years ago HFR withered (as we thought) into the shadows of cinema’s greatest technical follies. Ang Lee’s 2016 movie Billy Lynn’s Long Halftime Walk had a long title, but unfortunately a short life in the public eye. The HFR technical story was immense and featured groundbreaking techniques like frame blending and a 120fps 4K stereo capture. Perhaps the biggest misstep was that there were hardly any cinemas that could play back the movie as it was shot. Definition was lucky enough to watch the 120fps 4K, but only 12 minutes of it – we enthused about it at the time, but understood that, as a public performance, not many people would watch it this way.
There were hardly any cinemas that could play back the movie as it was shot in 120fps
Three years ago we described the experience in the following way: “So we saw it. Twelve minutes of Billy Lynn’s Long Halftime Walk in glorious 4K, 120fps 3D. The feeling was amazingly intense, especially for the battle scenes. Seeing everything doesn’t explain what 120 frames gives you, but one scene brought it home to me. An American soldier is firing his heavy machine gun at the enemy. While he is shooting, the vibration going through his body gives him an aura of dust surrounding his body shape. That is a new reality for me. You definitely felt like you could reach out and enter the scene, just walk right in there.”
The movie had a limited release as a 24fps movie, but soon disappeared from view after getting a mauling from the critics. Apparently wartime PTSD in vivid hyperreality wasn’t flavour of the month at the time. Now, you can buy a 60fps 3D Blu-ray, however and the comments on the Amazon webpage are full of praise for the look of the movie. There was plenty of blame for the demise of the movie put at the door of the distributors for their lack of marketing, but eventually the technical crew had to admit defeat and put HFR on the shelf marked ‘revisit this sometime’ and got back to ‘normal’ 24fps moviemaking.
Double trouble
Behind-the-scenes shots and stills from Gemini Man. © 2019 Paramount Pictures. All rights reserved. Photo credit: Ben Rothstein
Of course, technology marches on and if it’s not working for you now, then just wait a while and it’ll come around. Presumably this is what Lee, technical supervisor whiz kid Ben Gervais and stereographer Demetri Portelli have done with Gemini Man, because 120fps, 4K 3D is back! As is the public’s and critics’ mistrust of it, but let’s not worry about that and just applaud the effort. Portelli explains the technical difference between now and three years ago: “Last time when we were doing Billy Lynn, we were finishing it without a lab as we had built our own facility. Then it was ‘hopefully this will work’. When we were heading towards deadlines and distribution it was very nerve-racking. Billy Lynn was the proof of concept and it needed to happen, it was also a little bit sad what happened, as there was very little courage to actually release it in 3D.”
He continues: “With this project, Gemini Man, Paramount Pictures has been amazing by supporting the technology and the filmmaker and rolling out the film internationally as a high frame rate movie. They haven’t released the film as a 24fps movie for the audience, which is a wonderful success for us and a statement of confidence for HFR.”
At the same time, HFR has still been seen as a work in progress – mainly because seeing the full 120fps film is very difficult. “I live in Canada and there is nowhere here to see the 120fps version,” admits Portelli. “All we can do is carry on advocating for more people to see the movie as Ang (Lee) meant it to be seen.”
Cinemas need double laser projection to have a chance to show 120fps HFR. Dolby cinemas have that, but there aren’t that many of them around. It seems, then, that we’re in the same situation as three years ago; the filmmakers pleading with the public to seek out a 120fps version, then the almost inevitable disappointment at a fruitless search for a suitable cinema is followed by not going to see any version of the film, because of the let-down.
Portelli explains: “Ang said we had to careful with this film not to be so ‘tech head’ about seeing the 120fps version. You don’t have to and our message this time is: we have shot the thickest negative and the best possible amount of data that we can. That process allows us to deliver these other versions. If you want the 120fps, we give Dolby a very nice 2K version, and then we make a great 60fps version for the rest of the world. The 60fps really works and Paramount are branding it ‘3D Plus’. IMAX is 60fps as well, but its laser projection is at least bright, which tends to be good for 3D.”
Production for Gemini Man
For Billy Lynn, the production used Sony’s now old F65 cameras, which were unwieldy at the best of times. This time, they went back to using the Arri Alexa M – a favourite camera of director James Cameron for his 3D exploits.
“The M was gutted and modified as we needed to do 4K 120fps,” Portelli recalls. “The Alexa Mini doesn’t do that – we’d only get 60fps out of it, which is a shame because of its light carbon-fibre body. However, the M cameras allowed us to do quite a bit of Steadicam, as we had to be mobile. Gemini Man was quite a travel job. This was an action film, so you have to move the camera to achieve the visual aesthetic.
The Summicrons were on the Steadicam and the motorcycle to take the weight down as much as possible
“The feeling has been that you can’t do handheld 3D as that will induce motion sickness, but we moved the hell out of this camera. We built wire pulley systems so we could leave them hanging in the centre of the set and then run over and grab it, which made it workable.”
Portelli recalls that the hardest scene to film was the motorcycle chase in Cartegena, as they couldn’t use golf carts or have anything to do with any racing between cars over cobblestone streets. It had to be a motorcycle chasing a motorcycle.
He adds: “I think the most stressful day was the first day we started, but we had a lot of help. The flight head guys from Ukraine built us a small remote head, a mini Apex, and we were the guinea pigs for that. The whole reason for us using Leica glass was to get the weight down so we could have a mobile shooting rig.
“We had the Arri Ms and the Leica Summilux and Summicron lenses. The Summicrons were on the Steadicam and the motorcycle to take the weight down as much as possible to make it less dangerous. This is an ongoing conversation, especially when you use a two-camera mirror rig. How do we get the weight down and still make it indestructible?”
Portelli says getting that footage back in to post-production was the most work he’d ever done, as the crew had to analyse how the footage reacted over bumps and cobblestones, “especially when Weta [Digital, the visual effects company] had to comp in to a stereo image with a lot of vibration and things like that”. He adds: “But this is why we like to work with Ang. He challenges you, it’s like ‘this is what I want to do, you guys go and figure it out’.”
Working with Junior
Placing a CGI asset into a 3D action movie is not an easy proposition, but this was the challenge for Weta Digital. Portelli had to help enable the knit, as he explains. “It’s really exciting. Guys like me who shoot stereo, if you’re lazy and if you’re not careful and you don’t understand every shot, the VFX element can really make horrible 3D.
I like the way Ang has evolved his 3D now – it’s smooth and immersive
“I’ve got to run around and follow the first AD and the VFX supervisor to make sure there’s nothing happening I don’t know about, because I set the stereo based on everything in the frame and, most importantly, the closest thing. If there’s a CG character, gun or motorcycle handlebars, I need to shoot a reference and then set up the camera. So I need somebody to place a physical item like a fake handgun in the shot, because later they’ll put it in CG.”
Regis Harrington was the stunt driver on the motorbike in the Cartagena chase scene
The production used a body double for the young Will Smith character, or sometimes Will himself. Portelli describes the close-ups as “the greatest fun for me” – looking for truth in the human face with a character that is, ultimately, CG. “Add to that 4K 120fps, where there’s basically nowhere to hide, and you’re setting yourself up for quite a challenge. So those close-ups with the CG are very important to me – and where Will exists in theatrical space in part of the 3D movie experience, if you like. I like the way Ang has evolved his 3D now – it’s smooth and immersive.”
As for post-production, Portelli says he spent a lot of time adjusting and perfecting. “I do spend a lot of time in post adjusting the convergence point and saying things like, ‘This shot is really quick, so I think our eyes need to converge nearer his motorcycle, so there’s a balance to the 3D from cut to cut’. This film is the one where I’ve spent the most time in post-production trying to refine that.”
Portelli also has to liaise with the VFX supervisor and give the VFX department extra information, such as a reference geometry with notes on everything, which are delivered to a VFX vendor. “I said things like, ‘These two lenses are slightly different; the geometry needs to sit here, but it doesn’t need to be 100% perfect, because you can actually lose the stereo, so don’t worry about the edges’. Sometimes I had to fight to get flares back in that weren’t stereo correct in the original photography and VFX had taken out. Then the flares came back, but in the wrong place, and it went on and on like that. I also did a fair amount of colour matching and luminance matching between the stereo eyes, which is very important.”
Gemini Man hasn’t had the greatest critical response, but then punters are liking the ‘bright’ image as, perhaps, they haven’t got a reference for this new thing they’re seeing. But, in conclusion, we have to be thankful for pioneers like Ang Lee and his philosophy of always being out of your comfort zone. After all, that’s the place you’re working at your best.
