Home made test comparison between D5, Z6 and Z6II at high ISO

[Marco74]

I hope that this is the right place to post this thread. Otherwise, Steve, please move it in the right place or advise me and I will move it.

These days I spent some time analysing the noise at high ISO of the D5, Z6 and Z6ii.
After reading the ZII reviews on Dpreview and after finding some differences between the files of my two Z6, version one and two, I came up with this idea.

Basically, I took photos starting from an ISO value of 6400 up to 32000 in 12 bit and 14 bit.
Same light same subject, lens, position and diaphragms leaving the camera body to vary the time of exposure to obtain the proper exposition.
Matrix metering and self-timer set to 5 seconds.

I manage the raw files in this way:

I opened them with Capture NX. Removed noise reduction and saved in uncompressed 16-bit TIF. The same thing pumping the esp. + 2EV.
Then I opened each file with Photoshop without making any corrections.
Through the plugin Topaz Denoise Ai, the last release, I marked on an Excel sheet, Noise and Sharpening's values suggest automatically by the program.
I could probably do the same job using the Topaz program without using Photoshop and the plug-ins, but I think that when I finished.

The noise reduction values suggested by the program increase as the photo noise increases, so I think a valid method to have a non-subjective evaluation of the files, considering what I have available. As indicated in the title, it is a home test without any particular claim.

My personal conclusions are as follows.

In the Z6II the noise is lower than both the Z6 and the D5 but .... the dynamic range has been penalized. In fact, in the + 2EV files the noise is higher than in the Z6.

12-bit files are always noisier than 14-bit files.

What do you think?

You can only see EXIF info for this image if you are logged in.

You can only see EXIF info for this image if you are logged in.

Kind regards, Ciao
Marco

 

[EricBowles]

I like your approach. It's a simple, straight forward approach that is consistent with expectations, but provides some quantitative perspective.

Another approach you might consider is to simply shoot lossless compressed RAW of exactly the same scene or large JPEG files of that scene. While you can't compare across different cameras, differences in file size are largely based on noise in the embedded JPEG files. High ISO files are much larger than low ISO files. Be sure to turn off in camera NR with this approach.

 

[Marco74]

Eric,
I'm not an expert in this specific sector.
I didn't plan the test in detail, I just did it.
I agree with you, that thinking about it better, I could make everything easier.
I used a home approach for personal purposes, then the result seemed interesting to me, and I thought I'd share it.

If anyone wants to take a more accurate and reliable test, I would be happy to read it, as long as it is concise and not too boring. I'm not very fanatic of these things.

In my type of photography, I often go up a bit with ISOs value, so having an idea of how my Nikon bodies work in these particular conditions allows me to have a better result.
Not necessarily a better photo, unfortunately, I have my limits.

Thanks for the suggestions, next time I will do as you wrote, or better, if I have another strange idea, first I write to you and ask you how to do it.

 

[Pistnbroke]

There are two ways to compare dependent on.....Are you turning up the ISO to raise shutter speed or are you raising ISO because its getting dark? Most of the tests are the first so you blast away at a well lit object and look at the noise. Thats for the birds. For the second the best way is a dark room and light the subject with a variable output led light. Turn the light down to give yourself the test iso via auto iso . Thats for the 13th century church wedding in winter with 5w low energy bulbs in the light fittings.

 

[Marco74]

Interesting reflection.
Unfortunately, I don't have a precisely dimmable light source otherwise I'd be curious to try. In my case, I illuminated a subject with a spotlight from above, leaving about 30/40% of the image with dark areas so that I could open the shadow later and analyze the noise in those points. In a dark room, I raised the ISO manually.

A dear friend pointed out to me that my charts are not easy to read. The noise part is quite obvious, while the sharpening value is not. Indeed, it is difficult to understand the logic behind the algorithm used by Topaz to suggest an adequate sharpness value. What do you think?

 

[Pistnbroke]

You dont need a precicely dimable light..you slowly turn it down and watch the iso rise in the viewfinder readout taking shots at 800 1600 3200 etc.... Or auto iso manual and close the iris to raise the iso.

 

[Marco74]

Sorry, but I struggle to understand the difference between varying the aperture and letting the machine automatically calculate the ISO or, as in my case, leave the aperture fixed and change the ISO while the device calculates the time needed to obtain the correct exposure. What am I missing along the way?

In addition to this, I am interested in interpreting the data or even a possible denial of the results.
In my opinion, what I now understand is that the changes made to the management of the Z6II sensor are not necessarily to be interpreted as positive. The high ISO noise of the files always scares us, however, if to improve this parameter I have to give up the pose latitude and the micro-contrast, I'm not sure it's a price I would like to pay.

 

[EricBowles]

Interesting reflection.
Unfortunately, I don't have a precisely dimmable light source otherwise I'd be curious to try. In my case, I illuminated a subject with a spotlight from above, leaving about 30/40% of the image with dark areas so that I could open the shadow later and analyze the noise in those points. In a dark room, I raised the ISO manually.

A dear friend pointed out to me that my charts are not easy to read. The noise part is quite obvious, while the sharpening value is not. Indeed, it is difficult to understand the logic behind the algorithm used by Topaz to suggest an adequate sharpness value. What do you think?

With most noise reduction techniques there is a balance between noise reduction - which softens - and sharpening to overcome the effects of noise reduction. I would assume Topaz is striking that balance. That's why NR is often an iterative process of both NR and Sharpening.

 

[Marco74]

Thanks, Eric,
Personally, I don't really like the sharpening mask applied by Topaz. Noise reduction sometimes works great, others less so. In the photos I recently posted on this blog, I preferred to use Neat Image over Topaz.
In some cases, the even better effect is achieved using Topaz Sharpening Ai than with Denoise.

In my graphs, I have reported the program's sharpening value for completeness of work but thinking about it. It is not very useful and significant.

Post scriptum:
I took a look at your blog and finally decided to put it in my favourites list because I realized that I still have a lot to learn and there is a lot of useful information on your blog.

 

[Calson]

I test my cameras at ISO 6400 which is as high as I need for 99% of my photography. Even shooting wedding receptions outside at night with only votive candles for light have I needed ISO 4000 and with an f/2.8 zoom lens that was all I needed to get the shots for my clients. If I was shooting sports at night or in poorly lit venues then I can see the need for higher ISO settings.

 

[kimball]

What do you think?

I think it's an interesting test!

But I also don't think one can make any significant camera comparison conclusions from it. Note that this is not a measurement of noise, but a measurement of how the Topaz algorithm responds to the contents of different files from different cameras with different sensors and different resolutions. Granted, you've gone to great length to eliminate many variables! But the Topaz algorithm, as an unknowable black box (especially if it's truly based on machine learning like the "AI" marketing suggests), is a largely uncontrolled variable that renders any conclusion moot, IMHO.

My personal conclusions are as follows.

In the Z6II the noise is lower than both the Z6 and the D5 but .... the dynamic range has been penalized. In fact, in the + 2EV files the noise is higher than in the Z6.

Have you checked out Bill Claff's photonstophotos.net website? His data shows the Z6II has essentially the same DR as the Z6.

https://photonstophotos.net/Charts/PDR.htm#Nikon D5,Nikon Z 6,Nikon Z 6II

Also, you might find opening the files in RawDigger interesting. It reports some basic statistics, for example the root-mean-square deviation of pixel values, which is essentially a measurement of noise if you are sampling from a portion of the image that is evenly lit and the same solid color (for example, any color patch from a ColorChecker where the lens has been slightly defocused to avoid photographing the texture of the patch).

 

[Marco74]

I think it's an interesting test!

But I also don't think one can make any significant camera comparison conclusions from it. Note that this is not a measurement of noise, but a measurement of how the Topaz algorithm responds to the contents of different files from different cameras with different sensors and different resolutions. Granted, you've gone to great length to eliminate many variables! But the Topaz algorithm, as an unknowable black box (especially if it's truly based on machine learning like the "AI" marketing suggests), is a largely uncontrolled variable that renders any conclusion moot, IMHO.



Have you checked out Bill Claff's photonstophotos.net website? His data shows the Z6II has essentially the same DR as the Z6.

https://photonstophotos.net/Charts/PDR.htm#Nikon D5,Nikon Z 6,Nikon Z 6II

Also, you might find opening the files in RawDigger interesting. It reports some basic statistics, for example the root-mean-square deviation of pixel values, which is essentially a measurement of noise if you are sampling from a portion of the image that is evenly lit and the same solid color (for example, any color patch from a ColorChecker where the lens has been slightly defocused to avoid photographing the texture of the patch).

You have a right, and my test is not as valid as a laboratory test.
That we don't know how the Topaz algorithm works is another fair consideration.

When I got the idea, I didn't want to rely only on a subjective aspect. It is complicated to visually assess the difference in noise between one photo and another, and then it is a personal value.

I had seen Bill Claff's graph some time ago, but there is no comparison with the 12-bit raw, which I still think is interesting, at least for me.
Then I imagine that even in the cameras' sensors and the various components there are production tolerances. Maybe my camera bodies may not even fully reflect the graph.