The big squeeze
Posted on Jan 14, 2022 by Alex Fice
We look at the capabilities of different codecs, and explain how underlying technology affects the performance, usability and compatibility of its hardware
Words Phil Rhodes
One day, memory cards will be big enough to store all our high-resolution, high-frame-rate and high-dynamic-range images, uncompressed. That day hasn’t yet arrived – at least not for most of us.
Codec design is a wonderful example of engineering compromise, balancing computer processing power, storage space and image quality. Performance improves continuously, but now just one piece of mathematics does the lion’s share of video compression. Its proper name is Advanced Video Coding, but most of us have encountered it as H.264 – part ten of the MPEG-4 standard – in every form, from ultra low- bandwidth videoconferencing to high-level cinema origination.
It comes in various guises. AVC-Intra and AVC-Ultra are Panasonic’s configurations, appearing in the VariCam series and AU-EVA1 – handling 4K, 4:4:4 material at 12 bits per pixel. Exactly how AVC data is handled depends on the application, and the standard gives manufacturers options to achieve a useful balance of quality, workload and file size. Turning off the more complex encoding techniques makes the material easier to encode and decode, and AVC will handle it down to low bit rates, as in Panasonic’s portable AVC-Proxy.
The standards describe a series of profiles and levels. Sony calls its high-end version XAVC, which has been used in professional cameras since the F5 series – and is often described in terms of its bit rate. The top-end Class 480 operates at 480Mbps at 24, 25 or 30fps; or 960Mbps at 50fps or more. XAVC uses AVC’s High Profile and level 5.2, which allows DCI 4K at 60fps, up to 12 bits deep. By comparison, the more compact AVCHD – developed jointly by Sony and Panasonic – uses High Profile at level 4.2, limiting resolution to 2K at 60fps. AVC provides a huge variety of options, although all of them should be understood by a well-written decoder intended to handle MPEG-4 part ten material.
Codec design is a wonderful example of engineering compromise, balancing computer processing power, storage space and image quality
With AVC having been incumbent for a while, we might expect to see a sequel. One does exist: High Efficiency Video Coding or, with great originality, H.265. HEVC achieves perhaps twice the quality for a given file size, in exchange for extra mathematical effort. It’s found in cameras such as Panasonic’s compact HC-X2000, which packs 4K, 10-bit pictures at 60 frames into a package that weighs under a kilo. That’s impressive. Cramming in the hardware to handle that, and conventional AVC for video streaming, is even more astounding.
To say that AVC and its brethren have revolutionised both acquisition and distribution of digital video barely hints at the extent of its influence. Although, it was not solely intended to behave as a mezzanine codec like ProRes. By switching off the inter-frame compression, we can make AVC behave more like ProRes, but Apple’s codec will probably always be a little easier on the silicon.
The mathematics underlying ProRes are well established. It’s a fantastic illustration that simply having a standard is more important than it being hugely advanced – especially if we want computers to decode many streams simultaneously. It is technologically related to JPEG still images, and thus the Motion JPEG moving image format, as well as the original MiniDV, DVCPro and DNxHD formats. All of these (in addition to AVC, in part) use a technique called discrete cosine transform, where the picture is broken up into blocks and the pixel values of those blocks are expressed as a waveform – a wiggly line. That line can then be approximated as a sum of various sine waves.
RAW CODECS
It’s a reasonably safe assumption that much of the data-reduction mathematics in the more recent ProRes Raw is similar to non-Raw options, so electronic devices can more easily implement both in the same chips. Atomos has made an art out of very full-featured recorder-monitors capable of recording ProRes Raw; a feat requiring not only appropriate recorder engineering, but also detailed technical liaison with camera manufacturers. The Ninja Cast has it both ways, providing recording in ProRes Raw and streaming to more conventional, AVC-based distribution.
Compressed Raw codecs, despite their recent popularity, actually go all the way back to Cineform Raw in 2002. Now open source, Cineform is used by Kinefinity, among others. The underlying approach is broadly similar to JPEG and ProRes, although built around wavelets – more complex wave shapes than sine waves, allowing for better compression, again, at the cost of computer workload.
The most uncompromising approach is to record uncompressed images, as was common in early stages of digital cinematography
Red’s Redcode uses similar techniques. With the company’s stalwart dedication to sheer resolution, it has been challenging for some users to meet the high computing power demands. Red has shipped plug-in boards specifically intended to accelerate the process. Modern GPU-based workstations have made the extra hardware unnecessary. But the success of Red’s cameras, as well as the company’s contribution to the Panavision DXL series, illustrate the world’s willingness to handle this workload.
The most uncompromising approach, of course, is to record uncompressed images, as was common during the earliest stages of digital cinematography. Workstations faced with uncompressed material – perhaps from one of Arri’s cameras, via the company’s collaboration with capture specialists Codex – might still need software described as a codec. Although its function is more organisation and filing than processing. Shot uncompressed, the company’s most capable models – the Alexa 65 and hugely popular LF – can be among the hungriest consumers of data space around. That is until we start thinking about high-speed cameras from the likes of Vision Research. A 4K Super 35 Alexa, purportedly coming soon, might bring similar considerations to smaller sensors, too.
Read the full article here.
