Z8 HLG / SDR Tone Mode RAW File comparison

[Steve]

Below is a BCG only video is just shot (sorry, it's just off the cuff and not my usual quality). There's been a lot of discussion about Tone Mode and if / how it affects RAW files. Yesterday, Nikon updated NX Studio for Z8 compatibility - including HLG. So, I shot a quick video as I played with a couple of contrasty RAW files taken in both modes.

However, keep in mind I'm still learning this too and I'm posting this video as must to get your option and open discussion than anything else. PLEASE do not take this as the last word on the topic - it's more like the first tentative baby steps

hlg-sdr

This is "hlg-sdr" by Steve Perry on Vimeo, the home for high quality videos and the people who love them.

vimeo.com

 

[dupcak]

Very interesting - definitely would not have guessed the shadow performance implications. Without getting a chance to play around with this yet, I'm still leaning towards thinking this is a gimmick more than a real feature - but I'm sure someone will find the right application.

 

[John Navitsky]

thanks Steve. as you noted in the video, i expected, based on the published HLG/SDR curves, that the shadows would be the same and that would have been useful to me (better highlights, same shadows). but given the shadow difference, i think my conclusion is the same as yours.

 

[Marco74]

Dear Steve,
Can you be so kind as to share the two sample raw files?
I understand your consideration, but I would like to see if there is a big difference in my Eizo monitor.
If I set the viewer profile BT.2020, all the photos look nicer with a more extensive dynamic range, but until now, nobody of my images has the HLG tone mode.
I'm curious to see the difference with the new format.
Thanks,

 

[Steve]

Dear Steve,
Can you be so kind as to share the two sample raw files?
I understand your consideration, but I would like to see if there is a big difference in my Eizo monitor.
If I set the viewer profile BT.2020, all the photos look nicer with a more extensive dynamic range, but until now, nobody of my images has the HLG tone mode.
I'm curious to see the difference with the new format.
Thanks,

Here ya go:

sdr-hlg-raws.zip

Shared with Dropbox

www.dropbox.com

However, I don't think the monitor will change the way the files respond to adjustment (i.e., the noise at the end of the video)

 

[John Navitsky]

i suppose there still might be situations where HLG could be useful, if you're worried about highlights and you know you aren't going to be pushing shadows

 

[Marco74]

Here ya go:

sdr-hlg-raws.zip

Shared with Dropbox

www.dropbox.com

However, I don't think the monitor will change the way the files respond to adjustment (i.e., the noise at the end of the video)

Thanks, Steve.

Looking at your files, I noticed the noise difference you highlighted in your video.
From my perception, the photo with HLG looks with more details.
I take a screenshot of my screen; sorry, I'm not able as you to do this type of task.
On the left is the standard JPG, and on the right is the HLG version (100% view)

You can only see EXIF info for this image if you are logged in.

 

[Andy Miller Photo UK]

Well said Steve and having read all the footnotes, books and manuals/guides now - I cannot currently see a reason to shoot HEIF stills yet.

For video perhaps the storey is different - but if going a long way on an expensive trip I am going to want to shoot footage in the highest format/codex I can. Gerald Undone recommends using ProRes 422 HQ N-Log and he should know -- but I still prefer N-RAW which provides more format and fps choices, grading in DVR18 does not take long. But one needs to not overshoot -- so don't use 60p or 8.3k if you only need 4.1k 30p and don't use HQ when the much smaller Normal/Standard Quality will do - the key is 12-bit RAW not limiting oneself to "just" 10-bit or 8-bit.

Like just shooting RAW it simply gives one more choices when I get back home.

I do understand there are many folk who don't have the time or inclination to process RAW files OR they need to transmit images in real time -- JPG is their format.

I do not yet understand where HEIF fits in -- Nikon is on a curve by adding HEIF and other formats -- they are clearly not alone. Out if interest Apple has just come out with Ventura 13.4 and released a whole new bunch of Pro Video Formats that came along with the same update -- I have not heard of any other them.

I suspect that these tech firms are investing in HEIF and other new formats that will be developed and refined and "they hope" will provide better results that are as accessible as JPG. AND that is great for those folk who just want to take a shot and just the results without the "pain" or time to process a RAW file.


FYI here are the release notes -- Apple The Pro Video Formats package provides support for the following codecs that are used in professional video workflows:
• Apple ProRes RAW and ProRes RAW HQ*
• Apple Intermediate Codec
• Avid DNxHD® / DNxHR® decoder
• AVC-Intra 50 / 100 / 200 / 4:4:4 / LT
• AVC-LongG
• XAVC
• XF-AVC
• XF-HEVC
• DVCPRO HD
• HDV
• XDCAM EX / HD / HD422
• MPEG IMX
• Uncompressed 4:2:2

 

[Steve]

Thanks, Steve.

Looking at your files, I noticed the noise difference you highlighted in your video.
From my perception, the photo with HLG looks with more details.
I take a screenshot of my screen; sorry, I'm not able as you to do this type of task.
On the left is the standard JPG, and on the right is the HLG version (100% view)
View attachment 61896

I think it's just the flatter looking profile. I'm not familiar enough with Studio NX to really put that this to the test, but I'm fairly confident that I can get the same amount of detail with some careful processing. Although, for me, those shadows are tough to get past (and being locked down to 400 ISO minimum is tough too).

I'm hoping Adobe will update Lightroom to support HLG Z8 files and then I can really play. NX is just so painfully slow and clunky on my Mac (and it's a m1 studio), that I can't stand using it for more than a few minutes at a time.

 

[Marco74]

I think it's just the flatter looking profile. I'm not familiar enough with Studio NX to really put that this to the test, but I'm fairly confident that I can get the same amount of detail with some careful processing. Although, for me, those shadows are tough to get past (and being locked down to 400 ISO minimum is tough too).

I'm hoping Adobe will update Lightroom to support HLG Z8 files and then I can really play. NX is just so painfully slow and clunky on my Mac (and it's a m1 studio), that I can't stand using it for more than a few minutes at a time.

I have the M1 Studio too, Nx Studio is slow and painful, but I find it helpful in analyzing the photos by looking at both the exif data and the point of focus.
I also agree that, most likely, the feeling of sharpness depends on the difference in contrast, but I'm still curious to see a future evolution of the format HLG.
We will probably discover the real potential of the format in a few months when it will be supported by various software.

 

[Steve]

I have the M1 Studio too, Nx Studio is slow and painful, but I find it helpful in analyzing the photos by looking at both the exif data and the point of focus.
I also agree that, most likely, the feeling of sharpness depends on the difference in contrast, but I'm still curious to see a future evolution of the format HLG.
We will probably discover the real potential of the format in a few months when it will be supported by various software.

I agree - it should be a better way to go, however, at the moment the limitations in hardware, software, and, frankly knowledge is going to hold us back for a bit.

 

[John Navitsky]

i kinda suspect that basically it’s optimized for human skin

 

[JoelKlein]

What does BT.2020 stand for?

 

[Marco74]

What does BT.2020 stand for?

https://www.eizoglobal.com/library/management/ins-and-outs-of-hdr/index1.html

A simple explanation.

 

[MotoPixel]

To fully take advantage of HLG files, you should be viewing on an HDR capable display with BT.2020 color space (DCI-P3), HLG gamma curve and brightness of >400 nits, typical 1000 nits. Just as in shooting HDR video with an HLG gamma curve, the images require a proper monitor setup to display properly.

With stills, iPhones record in DCI-P3 color space, but use a 2.6 gamma curve and with video, I believe, phones from iPhone 12 on can record 10-bit HDR in DCI-P3 and HLG. As you've probably noticed, video shot and displayed on the latest iPhones is incredibly vibrant and bright. I'm thinking Nikon is laying the groundwork for the eventual move towards a more standardized wider color gamut and broader dynamic range in displays...moving on from sRGB and SDR.

What are the implications of this for those who produce mostly for printing? Is the wider color space of value for final output? I can certainly see the advantages for electronic display, but only on a proper HDR display.

This is really complicated by the fact that there are few reasonably priced monitors that support HDR10, HDR10+ or Dolby Vision and most require metadata in the image data stream to cause the display to switch to HDR mode. I'm trying to find something in a consumer <50" range that can serve as a editing display for HDR video. I'm curious to hear what others are thinking in this area.

 

[Patrick M]

To fully take advantage of HLG files, you should be viewing on an HDR capable display with BT.2020 color space (DCI-P3), HLG gamma curve and brightness of >400 nits, typical 1000 nits. Just as in shooting HDR video with an HLG gamma curve, the images require a proper monitor setup to display properly.

With stills, iPhones record in DCI-P3 color space, but use a 2.6 gamma curve and with video, I believe, phones from iPhone 12 on can record 10-bit HDR in DCI-P3 and HLG. As you've probably noticed, video shot and displayed on the latest iPhones is incredibly vibrant and bright. I'm thinking Nikon is laying the groundwork for the eventual move towards a more standardized wider color gamut and broader dynamic range in displays...moving on from sRGB and SDR.

What are the implications of this for those who produce mostly for printing? Is the wider color space of value for final output? I can certainly see the advantages for electronic display, but only on a proper HDR display.

This is really complicated by the fact that there are few reasonably priced monitors that support HDR10, HDR10+ or Dolby Vision and most require metadata in the image data stream to cause the display to switch to HDR mode. I'm trying to find something in a consumer <50" range that can serve as an editing display for HDR video. I'm curious to hear what others are thinking in this area.

Perhaps a Smart TV. My Samsung has a terrific screen, QLED and also HDR. I throw images and videos to it from my iPad using screen mirroring. Quick and my guests are always impressed. I’ve not thought about using it as a monitor for my Mac as it’s an iMac 27” 5k.
Worth a look, so to speak

 

[Andy Miller Photo UK]

To fully take advantage of HLG files, you should be viewing on an HDR capable display with BT.2020 color space (DCI-P3), HLG gamma curve and brightness of >400 nits, typical 1000 nits. Just as in shooting HDR video with an HLG gamma curve, the images require a proper monitor setup to display properly.
With stills, iPhones record in DCI-P3 color space, but use a 2.6 gamma curve and with video, I believe, phones from iPhone 12 on can record 10-bit HDR in DCI-P3 and HLG. As you've probably noticed, video shot and displayed on the latest iPhones is incredibly vibrant and bright. I'm thinking Nikon is laying the groundwork for the eventual move towards a more standardized wider color gamut and broader dynamic range in displays...moving on from sRGB and SDR.
What are the implications of this for those who produce mostly for printing? Is the wider color space of value for final output? I can certainly see the advantages for electronic display, but only on a proper HDR display.
This is really complicated by the fact that there are few reasonably priced monitors that support HDR10, HDR10+ or Dolby Vision and most require metadata in the image data stream to cause the display to switch to HDR mode. I'm trying to find something in a consumer <50" range that can serve as a editing display for HDR video. I'm curious to hear what others are thinking in this area.

While I am sure you are correct -- I always edit my Raw files with the Working Space set to ProPhoto RGB (PS/LRC/others) or Hasselblad RGB L* (Phocus) my Monitors and Printers are all colour calibrated and NONE have a colour gamut as wide as these working settings.

I happen to use Apple Mac OS with two Apple monitors (both Pro Display XDR) and these are colour calibrated but start with the P3-1600 nits preset.
In summary -- folk should work in the widest possible colour space to ensure there is no data loss; calibrate their monitors and printers recognising that they will not see the full gamut of the colour space. THEN - if you are at all concerned look at proofing the output with the Colour Gamut of the Output will be displayed or used (sRGB or Adobe RGB or P3 or...) and "fix" any significant colour issues.

For most of us who are not delivering product or make-up work colour accuracy is not a huge issues. For those who it is then they should already know what to do. If not there are lots who will help them - for a fee.

 

[Andy Miller Photo UK]

Below is a BCG only video is just shot (sorry, it's just off the cuff and not my usual quality). There's been a lot of discussion about Tone Mode and if / how it affects RAW files. Yesterday, Nikon updated NX Studio for Z8 compatibility - including HLG. So, I shot a quick video as I played with a couple of contrasty RAW files taken in both modes.

However, keep in mind I'm still learning this too and I'm posting this video as must to get your option and open discussion than anything else. PLEASE do not take this as the last word on the topic - it's more like the first tentative baby steps

hlg-sdr

This is "hlg-sdr" by Steve Perry on Vimeo, the home for high quality videos and the people who love them.

vimeo.com

@Steve - Re-shooting Lossless RAW 20fps in SDR or HLG
— something odd — As shown below across 4 of my higher spec cards - I appear to have different burst/buffer capacities when shooting Lossless RAW with the Tone Mode set to SDR and HLG and I simply cannot understand why.
Looking at the histograms in Raw Digger there seems to be no different to the data in RAW files. Same when looking the images in LightRoom Classic - no difference that I could spot.

 

[Steve]

@Steve - Re-shooting Lossless RAW 20fps in SDR or HLG
— something odd — As shown below across 4 of my higher spec cards - I appear to have different burst/buffer capacities when shooting Lossless RAW with the Tone Mode set to SDR and HLG and I simply cannot understand why.
Looking at the histograms in Raw Digger there seems to be no different to the data in RAW files. Same when looking the images in LightRoom Classic - no difference that I could spot.

View attachment 62000

I think it's because the HLG files are smaller, it gives them an edge with the buffer.

 

[Andy Miller Photo UK]

I think it's because the HLG files are smaller, it gives them an edge with the buffer.

WHY? I am shooting Lossless RAW not JPG/HEIF -- is the embedded JPG now a HEIF?

Ricci confirmed on the Nikon UK School what I heard you say which is one sees a more accurate histogram when HLG is set AND this could impact exposure when shooting at the limits.

I asked him the same question via email today.

 

[MotoPixel]

Relating to the new HEIF HLG format...when I went to download the codec for the latest NX Studio version that supports the new format, I noticed that there was an option to select Z8 and Z9...a bit of a giveaway that we'll see a new firmware release for the Z9 in the not too distant future!

 

[Andy Miller Photo UK]

We can but hope and pray

 

[Steve]

WHY? I am shooting Lossless RAW not JPG/HEIF -- is the embedded JPG now a HEIF?

Ricci confirmed on the Nikon UK School what I heard you say which is one sees a more accurate histogram when HLG is set AND this could impact exposure when shooting at the limits.

I asked him the same question via email today.

That's what I suspect - that they are using an HEIF as the preview. However, the difference is enough to make me scratch my head a bit - it seems too make more of a size difference than I'd expect.

 

[Andy Miller Photo UK]

That's what I suspect - that they are using an HEIF as the preview. However, the difference is enough to make me scratch my head a bit - it seems too make more of a size difference than I'd expect.

You are correct - I took two identical shots for comparison both in Lossless RAW and the only difference was the Tone Control selection :
-- the RAW file I shot with Tone Control set to SDR is 48.62 MB
-- the RAW file I shot with Tone Control set to HLG is 43.78 MB
This is crazy -- so I repeated the test in different lighting SDR 52.46MB vs HLG 45.05MB

The RAW file shot with HLG is always smaller and some times much smaller.

As a test I went on to take 60 identical shots in SDR (total file size 3.929GB) and then 60 identical shot of the same target in HLG all Lossless RAW (total file size 3.022GB) = in this test the average size of HLG files is 77% of the average size of size of SDR files same format and file type of the same scene nothing else changed !!!

Why when shooting Lossless RAW would one not always use HLG?

Faster more shots in the buffer and no change to the RAW data in the file. As a Mac user I simply have no idea why not to use HLG.

I am with others come on Nikon give us the firmware update for the Z9 that levels it up with these new options.

 

[John Navitsky]

@Andy Miller Photo UK

Why when shooting Lossless RAW would one ever not use HLG? SMaller files, so bigger buffer and no change to the RAW data in the file. As a Mac user I simply have no idea.

if you look at the video Steve posted you'll see the shadow areas were significantly impacted in an adverse way.