Why do mirrorless cameras struggle with red?

[Bennvdm]

Not 100% related to the topic at hand, but hereis an flash AF assist llight article that also appears to confirm that Canon dual pixel AF has pdaf sensors on all pixels whilst nikon and sony is only on green & blue

The essential pro feature that no mirrorless camera offers

The inability of mirrorless cameras to fire flash AF illuminators means they can't compete with DSLRs for critical shooting in low light. Manufacturers need to find a solution.

www.dpreview.com

edit - add 2nd Neil van Niekerk link with A1 vs D5 low light event Comments.

Finding a solution for poor low-light auto-focus with mirrorless cameras - Tangents

Finding a solution for poor low-light auto-focus with mirrorless cameras As much as I love all the advantages mirrorless cameras offer, at this point there is still a serious flaw in my experience – the low-light autofocus response just isn’t as solid as the top-end DSLRs. The autofocus...

neilvn.com

Even with dual pixel af Canon R5 also has issue with flash Af assist light, as dslr had no ir cut filter on af module sensor, only on sensor, but as AF is on image sensor on mirrorless, this is after IR filter. Majority of AF assist light is IR, so therefor no visible to human eye and mirrorless.

 

[JP Onstwedder]

There is a good explanation of the way phase detect AF works on the Reikan
FoCal blog (https://blog.reikanfocal.com/2023/05/how-it-works-on-sensor-phase-detect-autofocus/)

It seems there is no fundamental reason why colour (wavelength) makes a difference, in the sense that detecting the phase shift is possible for any colour. But it is also fairly clear that it is possible for an AF system to be less sensitive to certain colours. That’s because the phase detect sensors are pixels on the main imaging sensor, and those pixels have a colour filter and a microlens. It’s the colour filter that is modified to use a pixel for AF by blanking off half the pixel. The difference in images that are produced by pairs of these half-blanked pixels is what allows the computation of focus for the AF system. So if all these pairs of phase detect pixels were have a red filter, which means no red light hits the pixel because the filter stops it, then the AF system would have no information to work with.

But that seems so obvious that I struggle to think Sony and Nikon engineers decided the consequences would be acceptable, whereas only Canon engineers ‘got it’? I suspect it’s more subtle than this.

 

[Wes Peterson]

Thank you -- since the same type of sensor is in the A1 and the Z9 (well as far as I know) then one could reasonably expect to see the same results in a Z9 as an A1 with RED subjects.
Surely Sony and Nikon know what they are doing and so one may enquire does Canon follow a similar system in their (R3/R5 bodies?)

I've heard that Nikon uses color info for subject detection. Is it possible that the Sony doesnt?

 

[jeffnles1]

It would be interesting to print a focus target on dark red paper and do a little testing.

Interesting idea. I think a scenario would be put a dark red object several feet in front of a leaf green backdrop and try it since that is what seems to be the scenario that both Steven, my wife, and I experienced. Interestingly he is using Sony, my wife is using Sony and I'm using Canon.

Honestly, it was only with the Summer Tanager our cameras (wife and I) were having trouble picking up among a springtime leaf backdrop. In another week, the leaf canopy here will be so thick getting any photo of one of these birds will be somewhere between impossible and blind dumb luck since they tend to live and forage among the tops of the trees in deep forest.

In the case of the photo I posted above, it may have been something about contrast between red and green, it may have been all the leaves moving in the breeze behind the bird "confusing" the camera's AF, it could have been the photographer (that would be me) or some combination of all these elements. I honestly didn't think a lot about it afterward until I read Steven's post and saw it reflected my experiences.

This has been a good conversation.

Jeff

 

[sh1209]

Interesting idea. I think a scenario would be put a dark red object several feet in front of a leaf green backdrop and try it since that is what seems to be the scenario that both Steven, my wife, and I experienced. Interestingly he is using Sony, my wife is using Sony and I'm using Canon.

Honestly, it was only with the Summer Tanager our cameras (wife and I) were having trouble picking up among a springtime leaf backdrop. In another week, the leaf canopy here will be so thick getting any photo of one of these birds will be somewhere between impossible and blind dumb luck since they tend to live and forage among the tops of the trees in deep forest.

In the case of the photo I posted above, it may have been something about contrast between red and green, it may have been all the leaves moving in the breeze behind the bird "confusing" the camera's AF, it could have been the photographer (that would be me) or some combination of all these elements. I honestly didn't think a lot about it afterward until I read Steven's post and saw it reflected my experiences.

This has been a good conversation.

Jeff

I agree and I do want to do some further testing myself because it is annoying whenever it happens.

 

[sh1209]

Thank you -- since the same type of sensor is in the A1 and the Z9 (well as far as I know) then one could reasonably expect to see the same results in a Z9 as an A1 with RED subjects.
Surely Sony and Nikon know what they are doing and so one may enquire does Canon follow a similar system in their (R3/R5 bodies?)

I only owned the Nikon Z9 for about four months, but it did do this with the 500 PF several times as did both versions of the Z6 and Z7. It is nearly every time you have a dark red subject with green foliage all around it. I don’t know if it just confuses the sensor or it just can’t process the contrast between those two or what. I don’t know how the conditions are where you live this time of year, but just humor me and try to find a red subject with green surroundings and you will certainly see this happen unless the firmware updates in the Z9 have corrected issues from the original version. I think my Z9 had maybe one update before I sold it.

 

[sh1209]

This is a great example right here to where the camera absolutely would not focus on this bird. I was no more than 20 to 30 feet away. Nothing obstructing the view but whenever they are staring directly at you, the camera will not focus unless you help it.

You can only see EXIF info for this image if you are logged in.

 

[DRwyoming]

I thought the camera uses the same feed for color or BW. The BW conversion is just a profile/picture control.

The initial data off the individual photosites is pure grayscale data. It’s only after demosaicing that the color pixels emerge. Hard to say when selected parts of that data are fed to the AF processing but for speed reasons I would guess that the AF sensor data is routed to AF processing prior to demosaicing.

Again pure speculation on my part but prior to demosaicing it’s all grayscale data that just happens to come from photosites sitting behind the individual colored Bayer matrix filters. It’s certainly possible that the image is rendered as demosaiced color pixels before the selected pixels are routed to AF processing but from a data volume and processing standpoint that seems like it would dramatically slow down AF speed compared to routing the raw photosite amplitude data from select AF sensor photosites directly to AF processing.

 

[sh1209]

What is even more strange is, I can shoot a warbler 30 yards away that is completely neutral in color in almost dark conditions, and every mirrorless camera I have ever used will pick that bird up instantly. This just makes absolutely no sense to me. A neutral bird seems to me like would be where my camera would struggle, but it doesn’t.

 

[sh1209]

The initial data off the individual photosites is pure grayscale data. It’s only after demosaicing that the color pixels emerge. Hard to say when selected parts of that data are fed to the AF processing but for speed reasons I would guess that the AF sensor data is routed to AF processing prior to demosaicing.

Again pure speculation on my part but prior to demosaicing it’s all grayscale data that just happens to come from photosites sitting behind the individual colored Bayer matrix filters. It’s certainly possible that the image is rendered as demosaiced color pixels before the selected pixels are routed to AF processing but from a data volume and processing standpoint that seems like it would dramatically slow down AF speed compared to routing the raw photosite amplitude data from select AF sensor photosites directly to AF processing.

You could be onto something there. I really wish we had a sensor engineer that could chime in on this. Lol.

 

[sh1209]

Something else I would like to try to switch the camera to a black-and-white color profile and shoot JPEG‘s to see if that makes any difference. If that same bird I just posted the picture of were sitting in the open during winter with no green around, it would pick it up instantly or in the snow or any other situation other than green.

 

[boviscopophobic]

If you're in a situation where the camera is refusing to acquire autofocus, it might be worthwhile to take a photo in that unfocused state, then look at the individual color channels in RawDigger.

 

[bleirer]

Thank you -- since the same type of sensor is in the A1 and the Z9 (well as far as I know) then one could reasonably expect to see the same results in a Z9 as an A1 with RED subjects.
Surely Sony and Nikon know what they are doing and so one may enquire does Canon follow a similar system in their (R3/R5 bodies?)

Good question. I know the Canon cameras use this dual pixel AF, using phase detection where there is a left and right diode on each pixel. This cut and paste from a Canon website. It is saying every pixel is used, not just blue and green ones:

Each pixel on the Dual Pixel CMOS sensor has two independent photodiodes (the parts of the sensor that record light intensity or brightness). The camera's processor compares the signals from the two photodiodes, and if they match, it knows that this area of the image is in focus. If there is any deviation between them, it looks at pairs of photodiodes across a group of pixels, and can then calculate which direction the lens needs to be adjusted to achieve sharp focus, and how much focus adjustment is required. In this way, Dual Pixel CMOS AF phase-detection focusing usually requires less trial-and-error and is more effective than contrast detection.

What's more, where other AF systems use only a limited number of dedicated individual pixels for phase-detect AF, Dual Pixel CMOS AF uses every pixel on the imaging sensor, which means that the active AF area covers in effect the entire image frame. It also gives the camera a significant advantage for tracking a subject around the frame, because there are no gaps between the AF points.

 

[bleirer]

Perhaps there are some pros and cons to CMOS vs. stacked/bsi, could be some tradeoffs involved to achieve the speed where Canon is slower but without these particular tradeoffs. Just speculating.

 

[Cameron T]

Speaking strictly for my z9, though I believe all af systems work the same way, the initial pass seems to be af working with no colors (just shades of gray), with machine learning second.

The image of the tanager shows very little contrast on the bird itself, and little contrast between the bird and the surrounding foliage. This seems to indicate to me that the luminance values are what are used to put a given gray value for the af system to work, though I don't know the secret sauce since I don't work for nikon (and may not be exactly the same for Sony or Canon either).

That being said, I haven't had an issue focusing on male cardinals, females are slightly more of an issue depending on facing direction and what's covered by the foliage/branches/etc.

The last image of a cardinal I'd expect to nail focus on, because even in grayscale the cardinal has contrast between the black and red feathers, as well as the black feathers and the beak.

The machine learning/ai should have been able to catch the tanager and cardinals easily enough based on the images in the thread, based on my experience.

I believe Thom hogan talks about this in his book on the z9, and Steve may have referenced it (but I don't have the ability to review them in detail at the moment).

 

[sh1209]

Speaking strictly for my z9, though I believe all af systems work the same way, the initial pass seems to be af working with no colors (just shades of gray), with machine learning second.

The image of the tanager shows very little contrast on the bird itself, and little contrast between the bird and the surrounding foliage. This seems to indicate to me that the luminance values are what are used to put a given gray value for the af system to work, though I don't know the secret sauce since I don't work for nikon (and may not be exactly the same for Sony or Canon either).

That being said, I haven't had an issue focusing on male cardinals, females are slightly more of an issue depending on facing direction and what's covered by the foliage/branches/etc.

The machine learning/ai should have been able to catch the tanager and cardinals easily enough based on the images in the thread, based on my experience.

I believe Thom hogan talks about this in his book on the z9, and Steve may have referenced it (but I don't have the ability to review them in detail at the moment).

I think you’re onto something with the contrasting colors because if that tanager were in the open it would lock on right away. From my experience, the more red the bird and more dense the green foliage the worse it becomes.

 

[Cameron T]

I think you’re onto something with the contrasting colors because if that tanager were in the open it would lock on right away. From my experience, the more red the bird and more dense the green foliage the worse it becomes.

I still think it should have caught it, but I haven't had an experience with tanagers here first hand to confirm.

There could have been enough to get it to focus in a smaller af area, especially with the eye visible.

As with everything, testing is the bane of existence here

 

[sh1209]

I still think it should have caught it, but I haven't had an experience with tanagers here first hand to confirm.

There could have been enough to get it to focus in a smaller af area, especially with the eye visible.

As with everything, testing is the bane of existence here

It’s not only birds as well, it happens if you’re shooting roses out in nature, that are surrounded with a lot of green foliage even on a tripod trying to do macro shots. It’s hard for the camera to acquire focus on that dark red rose, but if that rose is in a vase in the middle of the room, it will lock on right away. One of the members on here Eric Bowles is discussing a similar issue to this with someone on DP review. I was looking at yesterday, but I’m not sure what they ever came to as far as a conclusion.

 

[bleirer]

You can switch Lightroom temporarily to LAB to compare the L of the green foliage with the L of the bird, see if they are close to the same. Just spitballing at this point. You make the choice top right in the histogram module, don't recall exactly where.

 

[sh1209]

You can switch Lightroom temporarily to LAB to compare the L of the green foliage with the L of the bird, see if they are close to the same. Just spitballing at this point. You make the choice top right in the histogram module, don't recall exactly where.

It’s like all the nikon dslrs I had over the last 20 years, never once not a single time Did this ever happen. It definitely seems to elude to the fact that it’s something to do with the mirror sensor. I wish Eric Bowles, would chime in here because I’m curious what they ended up determining on that DP review thread. I got interrupted yesterday while trying to read the thread and haven’t had a chance to get back on there this morning.

 

[bronaldo]

I've never had trouble with autofocus on red birds, but I definitely have trouble with post-processing. It takes a lot of time to get the colour right in LR.

 

[arbitrage]

Interesting thread.
It seems out on the west coast we really don't have any red birds.
I've shot some red birds in the tropics but that was back in DSLR days.
So I can't relate my experience but I'm enjoying the read.

 

[Doug Herr]

Hasn't been a problem for me

You can only see EXIF info for this image if you are logged in.

 

[Doug Herr]

Interesting thread.
It seems out on the west coast we really don't have any red birds.
I've shot some red birds in the tropics but that was back in DSLR days.
So I can't relate my experience but I'm enjoying the read.

I have not had tis problem with either Summer Tanager (a Sacramento vagrant) or Northern Cardinal (Green Valley AZ) both of which are quite red.

 

[Maljo]

Have not noticed any problem with focusing on red birds or red anything, actually.