Camera Robotics
Posted on Jan 10, 2025 by Admin
Smart moves
Phil Rhodes takes a look at how robotic and AI systems are revolutionising broadcast
The word ‘robot’ covers a lot of ground, from sci-fi villains to automated car-factory welders. In film and TV – and particularly in the increasing crossover between them – there’s a growing world of automated devices and associated systems that let broadcasters do cleverer things than ever before. However, live production remains hard, and the techniques of single-camera drama often have to run to keep up with broadcasters’ needs.
Sergio Brighel is vice president for robotics and prompting technology at Videndum Production Solutions, a company with a deep background in camera support for broadcast. “Where the industry is leading at the moment,” Brighel begins, “is towards automation. Automation is being driven by the cost reduction happening across the industry – pressure from social media, YouTube, VOD platforms.”
Brighel’s ideas encompass much more than just hardware. He says that, with live production inevitably relying on a wide range of processes and systems, integration is key. “Orchestrators already exist, but they’re not that different from a washing machine program. Everything changes with a rigid schedule. What we want to do is introduce flexibility to account for the unexpected, so that presenters can deviate from the schedule and accommodate surprises during interviews, debates, talk shows or breaking news. AI is the only way to accomplish all of this.”
That would involve all the robotic equipment that companies like Vinten have long been building, but with much greater integration than has been the norm. “If it’s just me before the camera, it could record my camera, prompting and lighting settings, audio set-ups and all sort of other things,” Brighel says. “In the second phase, I would suggest machine learning – deep learning. The system could be trained to emulate a certain directing style and suggest the next moves. I think there’s a horizon of time between three and five years for that.”
The beginnings might already exist in the guise of Vinten’s VEGA platform. “The built-in foundations of VEGA allow that to happen in real time,” Brighel explains. “It’s a modular architecture, and one of these modules is an AI from a company called Seervision – now part of Q-SYS. With them, we developed the first techniques for automatically tracking people in front of the camera, and we started planting the first seeds of this digital assistant.”
Eventually, the decisions made by even the cleverest systems still need to be supported by hardware. Brighel foresees a meeting of the ways between traditional camera robotics and modern enthusiasm for integrated PTZ devices. “I just published a series of white papers: in one of them, I sketch out the possible future for PTZs. The Sony FR7 is the first of a completely new style of PTZ where you can replace the lens. We might even want to replace the entire camera head – full-frame with a 2/3 head – or move the same device into different physical domains without being constrained.
“It’s what I call modular robotics. In my view, it’s the future of this field of automation.” As fields mature, the user experience usually becomes less primitive, and it’s a process which Michael Geissler, founder and CEO of Mo-Sys Engineering, remembers well. “When I founded it, producers were swearing about motion control systems. There were a lot of frustrations, and I thought, ‘there must be a simpler way’. We designed a system where everyone on-set had their jobs back. The grip was pushing the dolly, the focus puller was focus pulling and it was recording. You say record, stop, play back or let’s do it again – but don’t fiddle around with spline curves. The idea was a film shoot where, if you don’t like it, you do it again.”
In the two-and-a-half decades since Geissler’s innovation, cameras have become at first smaller, then larger again, and always with that film-to-television crossover very much in evidence. “Our new remote head, the L20, has two different versions: one for film and one for broadcast, though each can be used for both. It’s one of the highest-precision, zero-backlash camera heads, and we’ve had this for years. We’re trying, for the first time, to combine the enormous precision we have in the L40 into a really small package, which is actually not much bigger than a high-end PTZ head.”
Decades of experience in hardware notwithstanding, Geissler quickly moves onto software and the sort of systems integration that separates live broadcast from high-end film and television drama. “A typical example is in VP, with LED walls. We have a product called MoViewer,” he continues. “With multiple cameras in front of an LED wall, you have to find a way to switch the background wall image out, but then the other, off-air cameras won’t have an image. The director has nothing to look at, which is weird for a camera you’re about to switch to.”
Pre-existing approaches might have involved timing multiple cameras to capture frames sequentially and displaying the appropriate background for each one at just the right instant. The problem is that the achievable precision of an LED video wall, using pulse-width modulation, depends on how long each frame is displayed for. The more cameras, the shorter the display time for each. “For eight cameras, you have to timeslice each frame into slices that are one-eighth of the exposure time. You need eight times more light, and to use a global shutter camera that no broadcaster has.
“We found a solution,” Geissler says, “to have AI-assisted generation to give a multiview preview for off-air cameras. Disney, through its Star network in India, has been using this to broadcast the Cricket World Cup with 15 million viewers. And it’s not ridiculously expensive either; you can do it with any LED controller and any camera.”
Camera movement has long combined precision engineering and electronics, though that alliance has become much closer as other systems start to require positioning data. Ian Speed, owner and founder of Camera Revolution, points out that “all this camera movement was doable 20 or 30 years ago, but it was all analogue. Whether it’s a camera crane that has encoded axes so you can generate graphics, or a crane for when the VFX guys want 20 passes of a shot, it’s all now doable in standard grip equipment.”
The pressure to create ever more sophisticated broadcasts has attracted technology from other areas and has provoked the development of entirely new features. “It’s being driven by the requests from the broadcasters,” Speed continues. “How else can we generate an augmented-reality signal? That’s why we’re looking at the automated dolly system AGITO, and we just got a Towercam. This is the tech we’re talking about – for the football, the rugby, the flying wire cameras, the Spidercam systems. Being digitised means you can put graphics over the top of them. We’ve never really utilised Technocranes before, but we’ve been asked for a completely encoded Technocrane.”
Making these things possible is one challenge; making them creatively usable is another entirely. “For the AGITO remote dolly, rather than having a fixed track down the side of a cycling or running track, it instead runs on a magnetic track – you just have to run a mag strip. The gallery can programme shots into the dolly, while the new telescopic cranes we’ve got have the ability to output encoded data.
“Technocrane now has the Technodolly, and it’s possible not only to record a job but also get the set-up to repeat the shot. The grip and operator can do the first pass – the hero pass – then the system can take over and do seven more passes. We can now do it with a wire system that’s all digitised.”
Crew might worry about reduced opportunities, though Ryan Turner, equipment development technician for Camera Revolution, is an experienced crane technician who has found his niche in a world of increasingly ambitious broadcasts. The job, as he puts it, “without sounding too showbiz, is to spice it up a bit. Not too much – you don’t want to upset the blokes down the pub watching the football – but especially with augmented reality, we’re trying to put something different on the screen. Especially in what we’ve been doing on the rugby, which is heavily presenter-based – the presenters now interact with the virtual world. Things like players popping up out of the ground!”
