Toolkit: Sora
Posted on Apr 7, 2024 by Samara Husbands
Take a deep dive with Sora
Nicola Foley reports on the launch of OpenAI’s new text-to-video model
Was it only this time last year that we were laughing at weird AI clips of Will Smith eating spaghetti and creepy adverts for fake pizza restaurants?
Fast-forward to spring 2024, and AI-generated video content is no longer a joke. Sora, the new text-to-video model from OpenAI, has arrived like a tornado – and we’re not in the uncanny valley anymore, Toto.
This isn’t the first of its kind. The likes of Runway’s Gen-2, Stability AI’s Stable Diffusion and most recently Google’s Lumiere have been serving up AI-generated videos for some time.
But Sora is, seemingly, vastly more powerful and superior in some crucial ways.
Forming part of OpenAI – also responsible for ChatGPT – it’s also got one hell of a launch platform, with the company’s central website attracting some 1.5 billion visits per month.
All of which is to say, AI-generated video is poised to go mainstream in a big way.
Creating photorealistic, highly detailed videos (up to 60 seconds in length) in response to written commands, Sora represents a significant leap.
At present, it’s still in ‘red team’ phase (undergoing testing from authorised hackers) and not yet publicly available, but we got a chance to observe its capabilities when OpenAI CEO Sam Altman took to social media for a live demo.
He asked those watching along for video prompts, and the internet obliged with surreal gems such as: ‘A bicycle race on ocean with different animals as athletes riding the bicycles with drone camera view’.
The results, delivered minutes later, were notable for their quality: hyperrealism, multiple angles, a grasp of motion, even an apparent understanding of the nuances of cinematic grammar. We were seeing, in some instances, almost Hollywood-standard clips – a million miles from the glitchy AI videos doing the rounds just 12 months ago.
Not surprisingly, this quality of AI video synthesis has created unease within the industry.
In the most extreme reactionary move so far, actor and filmmaker Tyler Perry has halted the planned (reportedly $800 million) expansion of his own Atlanta studio after seeing Sora’s power. “That’s currently and indefinitely on hold because of Sora and what I’m seeing,” he told The Hollywood Reporter. “I’d gotten word over the last year or so that this was coming, but I had no idea until I recently saw the demonstrations of what it’s able to do. It’s shocking to me.”
At the Culture, Media and Sport Committee’s recent inquiry into British film and high-end TV, director James Hawes also expressed concern over this rapid development and its implications, stressing that protections need to be put in place now, stating: “We need to be meeting regularly and actively and angrily defending our creatives.”
For filmmakers, Hawes urged that harnessing and embracing the technology was essential. “We need directors to be trained and enabled to understand it. We need writers and other creatives to be doing similarly,” he argued.
Some are viewing Sora as more of an opportunity for creatives than a threat, arguing that it has the potential to enhance creativity and radically democratise filmmaking. “Sora will exponentially engage creative minds everywhere to tell more stories and make mind-blowing movies. It’s going to be a game changer,” director, writer and CEO of 90,000 Feet Studios Chris Nolan tells Definition magazine. “It will open more people’s minds to the possibilities that they can be filmmakers, beyond just the conceptual realms of today’s movies. Everyone now has the ability to make a movie and tell their story.
“I believe it will expand the art form and allow for more human expression. But we have to remember, it’s just a tool,” he continues. “The heart of film is the human drama, human truth and human actors. AI doesn’t experience life, that’s a human skill set we will always have,” he continues. “And as we’ve seen from movies with too much CGI, it falls flat. We don’t connect – it’s boring. We still want theatre, drama, a great human story.”
It’s also worth remembering that, for now, Sora can’t produce anything greater than a minute long, so we’re a while away from an AI-generated feature film yet. Where Nolan sees it being most useful in the near future is for prototyping and conceptualising scenes.
“Sora is also a great pre-production tool, so use it to plan shoots. There’s bound to be widespread adoption of Sora into the moviemaking process – it’s going to be everywhere – so be sure to get ahead of the curve,” he advises.
It’s worth pointing out that Sora, much like other text-to-video models before it, is not perfect. It seems to struggle with simple cause-and-effect – and hasn’t eradicated those creepy hands yet either, reminding us that there are fundamental flaws to be ironed out.
Cinematographer and virtual production consultant Quentin Jorquera told Definition that, while he was relatively impressed with what he had seen so far of Sora’s capabilities, ‘for now it’s all a lot of noise over nothing’.
“The same day as I discovered Sora’s images, I had planned to screen one of my favourite films to some film students: Sidney Lumet’s Fail Safe,” he shares. “I’m yet to see the kind of visceral reactions that movie got out of my students with any AI-generated content. Making pretty images is great – telling heartfelt stories that you receive like a punch in the face is another. All that said, the day I’ll be able to feed my script to an AI to help with my previs in order to convince a producer, I’d be happy with that!”
This feature was first published in the April 2024 issue of Definition.
