The Future Of The TV Pixel
Posted on Jun 4, 2015 by Julian Mitchell
More pixels or better pixels? Of course we want both.
Dolby has been working for six years to revolutionise broadcast television. Unsurprisingly they call it their Dolby Vision. The idea was to simulate the dynamic range of modern cameras and try and present as near to that as possible to the television viewer. With new Dolby Vision encoded UHDTV sets arriving this year they are already ahead of their competitors.
More pixels? Or better pixels? Of course we want both, and Dolby are leading the way with Dolby Vision. They are showing modified reference LCD monitors capable of up to 4000 cd/m2 – which make a standard monitor look drab. More importantly, they are developing standards and tools to support high dynamic range transmission and display.
Current cameras are capable of capturing a much higher dynamic range that our transmission systems allow. Dolby Vision aims to release this constraint, starting with grading systems, with support from most of the big players from Baselight to Resolve.
The intention is that the material is graded to the HDR standard using Dolby’s reference monitors – getting the most from the capture medium. Dolby showed Baselight automatically outputting a second ‘standard’ dynamic range image, calculated automatically from the HDR grade using Dolby’s algorithms. The colour timer, director and DoP can tweak this signal independently from the HDR.
Transmission uses a dual level CODEC – a base layer (with the ‘standard’ dynamic range signal) and enhancement layer, with metadata and additional image data to reproduce the HDR signal.
At the consumer end, Dolby have developed chipsets with silicon vendors for TVs and set top boxes, and – one of Dolby’s strengths – they are working with the standards groups to ratify the CODECs etc.
Selling 4k to consumers is probably pretty easy – 4k is a bigger number than 1920, so it must be better, right? The demonstrations of Dolby Vision really show the advantage of better pixels too – if it can be brought to market at a reasonable price, I can’t believe that consumers wouldn’t want the rich and vibrant images HDR allows.
But who is signed up to the Dolby Vision way of producing HDR? Blackmagic Design has already put their hat in the ring with an announcement at IBC but if you start asking other interested parties like Digital Vision or SGO there is less to talk about mainly because of a 23 page NDA agreement in existence.
The Blackmagic’s DaVinci Resolve press release says; ‘DaVinci Resolve 11.1 adds new support for Dolby Vision image processing. Customers will be able to work with 40x more brightness than a conventional television, a vastly improved, fuller colour palette, and up to 1,000 times more contrast to reveal depth and detail that makes images look astonishingly real.”
The basic Dolby HDR demo – in the flesh it’s much more impressive!
Dolby themselves announced no more news at IBC but had an HDR demo running on their stand and behind closed doors for press and interested parties. ARRI has been working with Dolby at major trade shows to help showcase Dolby Vision display technology. This has contributed to spreading awareness in the industry that dynamic range is an even more important factor than spatial resolution in creating a superior visual experience. You can understand Arri making this stand as their own camera’s HDR capabilities are currently persuading many high-end movies to use them.
But Dolby are not alone in the race to put HDR silicon in the next generation of UHDTV sets. Other companies such as Sony, Thomson, Philips and the BBC are working on their own HDR solutions. But creating standards and pushing them through is what Dolby are good at and they have been working on Dolby Vision for at least six years.
Example of the Dolby Vision demo, the still image is sourced from the company. The demo however is impressive.This year there will be Dolby Vision UHDTV sets from companies like TCL (A Chinese manufacturer) Vizio and Sharp with service providers like Netflix and XBOX. There will also be more than 15 Dolby Vision launch titles with totally re-mastered and re-graded content. The race is on to get HDR capable TV sets to market and Dolby seem to be ahead of the game at the moment.
Dolby’s Ian Lowe explains the concept, “Concept is, today’s cameras, Arri Alexa or whatever, are already high dynamic range. If I view them on a regular kind of rec709 monitor now – maybe one with not very good blacks – I might be seeing five to six stops of exposure, that’s it. If you are using a Dolby PRM-4220 monitor, maybe I’ll see nine or ten stops – lets say nine. Put the PRM in 600 nit mode we know because we’ve worked with Arri and we’ve seen the test chart they have and we display all 14 stops of an Arri Alexa with one of their test charts on a screen. So we know at 600 we can display 14 stops, you can do more than that so we have a prototype display (The Pulsar) that theoretically does about 19 stops should there be a camera to display that.
“The cameras have already been shooting HDR it’s just we never see it because we see it in 709. The output from the Baselight system or a number of other manufacturers, Blackmagic Design has announced that they will have Resolve 11 later this year with Dolby Vision in it. We have what’s known as a PQ curve which is a container that can go up to 10,000 nits. Our Pulsar works at 4,000. This curve has just been implemented by SMPTE under SMPTE 2084 as a way of doing High Dynamic Range display. So basically we put the grading system in to PQ mode, we put it on to a 4,000 nit display and we grade in exactly the same way, the tool set is the same. The colourist’s way of working is the same, the difference is instead of him using tons of power windows and everything else, he is seeing it. So typically to play in the highlights you’d put in a power window and fiddle around in that space. Then you’d do the same again and then look at the overall picture to try and understand how and where that fits.
“What we do is have a plug-in part within the FilmLight software (for instance), to do some analysis and through that and a content mapping box that we have we can then derive a rec709 version. So the difference between the 4,000 and 100 nit generates a bunch of metadata and that gets tagged with the 709.
“Arri already have PQ curves for their Alexa and Amira cameras so on-set with a Dolby Vision display we can already see HDR. We’re working with Digital Vision , SGO, FilmLight, Nuke, various other people to get all those bits in play.”
House Of Cards 3 was re-mastered in Dolby Vision by Encore Post in LA on a BaseLight system for a 4K release on Netflix. They re-graded and re-mastered the original .r3D files. The next series will also be encoded in Dolby Vision.
The Encoding Process
Martin Tlaskal, a lead developer at FilmLight was happy to talk about the theory of how HDR is mastered with Dolby Vision but also with an example of the recent House Of Cards re-mastering in Dolby Vision. “Essentially what we’ve done is to incorporate some of the Dolby algorithms in to BaseLight natively. So what that means is the colourist will load a Raw sequence in to a Baselight timeline in to what’s called PQ (Perceptual Quantisation) space. This has a very large dynamic range in terms of gamma and the amount of brightness information it can contain, then that’s rendered to the screen.
“When they want to transpose that colour correction or grade to a low dynamic range screen you can’t just wash all the colours down because it just looks incredibly flat and boring, so what Dolby has done is to make an algorithm like a tone-mapping algorithm which will analyse on a special Dolby Vision operator inside BaseLight. Image processing then works out the specific dynamic range requirements for each frame and then works out a curve which it thinks does the best job of rendering that high dynamic range content in to a low dynamic range display.
“So you get the intent if not the exact stream, highlights and shadows you get in Dolby Vision. Doing it naively and just squashing the highlights in to range, the shadows in to range and the mid-tones and everything else will make it look extremely flat and boring. It’s their algorithm that makes the high dynamic range stuff to look good even on a normal monitor.
“The whole Dolby Vision process is about analysing the master, the best quality image and then that Metadata of that analysis is sent along with the Master. If you go to the IBC Dolby stand there is a reference monitor which is 4,000 cd/m2, the other monitor which is their prototype of what a consumer television would be like is half that dynamic range around 2,000 cd/m2. But since they have the master and the Metadata, their algorithm can run in the two monitors and say ‘Alright I’m going to roll off the highlights a little bit so you don’t get any clipping but it’s still going to look nearly as good as the reference monitor’. So they have the analysis which allows them to run maths in the TV to tone-map and make the stuff look good on any display.”
FilmLight and possibly the other vendors like Blackmagic Design, Digital Vision and SGO have integrated their algorithms to make the post pipeline work so directly in the grading suite the colourist can master for extreme high dynamic range and also master for current TVs and make them look good. The grader will have two screens and check they both look good and then will press render and you have a master that can be deployed in either way.
“When it comes off our systems we have nothing to do with it. It is a Dolby Vision encoded product and their maths runs on their chips inside TVs or in their reference monitor.”
The question is how far away is mass production and how far away are Dolby’s competitors from releasing their own new HDR TV sets?
Most post productions systems who sign up to the Dolby Vision system will carry the APIs and start HDR mastering. Whether their contracts with Dolby includes an exclusive clause is down to negotiation but the clever post companies will leave that clause out. “Why do we need a proprietary tech?” commented Steve Owen of Quantel, “We’re already working with HDR content”.
At the recent SMPTE Tech Conference, Sony demonstrated its next-generation professional broadcast monitor, the BVM-X300, a 30-inch, UHD OLED display with 10-bit colour and the ability to reproduce peak white up to 1000 nits, 10 times more than the 100 nits that all consumer content conforms to now. The footage shown was shot during Carnival in Brazil using a Sony F65 camera at 60 fps with 16-bit colour and about 14 stops of dynamic range. That content was then graded for 1000 nits peak white with 10-bit colour and shown on the BVM-X300 at 60 fps.
