Log shooting

[Alistair]

Yes I do think it is worth it. I am surprised that the free version takes them as they are 10 bit but must be there is an exception. It would be good to know the color space and gamma of the transcoded file. Does it look washed out after transcoding like a log file?

 

[Alistair]

Alistair,

So I did some digging, the transcoding utility I used uses ffmpeg and it converts a Canon C-log3 file into an Apple ProRes 4:2:2 10-bit file. I checked in Davinci and indeed this is the file format of all my transcoded C-log3 files that I upload and edit in the free version.

So my question is are these file worth editing in DR or should I just go back to my h.264 4:2:0 files and forget about shooting in log for now? I did a comparison last Fall of colourful male Wood Ducks with regular 8-bit vs log files transcoded as per above and the transcoded log files had noticeably richer colours.

Here's the info I could find about Apple ProRes 4:2:2 HQ files....

Apple ProRes 4:2:2 is a codec that's used in video post-production. It's known for preserving the quality of HD video.
The 10-bit encoding in ProRes 4:2:2 gives better color information.
ProRes 4:2:2 is visually lossless, meaning it retains its quality even after multiple rounds of decoding and reencoding.
ProRes 4:2:2 HQ is a version of ProRes that offers the same high quality as ProRes 4444, but for 4:2:2 image sources.

What do you think, is it worth it to continue?

Rudy

FWIW I just posted a collection of clips using various 8k codecs. The opening sunrise scenes are 8 bit and have not survived the YouTube compression as witnessed by posterization in the sky. The remaining clips are 10bit log and there is one clip in 12 bit raw. Can anyone tell which is the 12 bit raw clip ?! Ensure the video playback quality is set to 4k or 8k if it has been processed by YouTube.

Shore Bird Project

This is a collection of clips from a project I am working on to encourage community engagement in the protection of shorebird species in urban areas in Auckland, NZ. Kindly ensure the video playback quality is set to 4k (or even 8k if it has been processed by YouTube).

bcgforums.com

 

[sh1209]

I assume so - but I've never used FCP.

So I did purchase the Phantom pack last night and look forward to trying it today. Everyone I’ve talked to about them, all seem to really like them.

 

[Rudy Pohl]

Yes I do think it is worth it. I am surprised that the free version takes them as they are 10 bit but must be there is an exception. It would be good to know the color space and gamma of the transcoded file. Does it look washed out after transcoding like a log file?

Alistair,

Yes the transcoded ProRes 422 HQ file if very washed out, just like a typical log file.

I found the instructional video that I used 2.5 years ago to build a utility that converts Canon, Sony, Panasonic and other brand log files into ProRes 422 HQ files, which are 10-bit video files supported by Davinci Resolve free version (see below). At the 4:15 minute mark he talks about the ProRes 422 HQ codec.

Info below is taken from Apple's documentation about their ProRes codec...
Apple ProRes 422 HQ
Apple ProRes 422 HQ is a higher-data-rate version of Apple ProRes 422 that preserves visual quality at the same high level as Apple ProRes 4444 but for 4:2:2 image sources. With widespread adoption across the video post-production industry, Apple ProRes 422 HQ offers visually lossless preservation of the highest-quality professional HD video that a single-link HD-SDI signal can carry. This codec supports full-width, 4:2:2 video sources at 10-bit pixel depths, while remaining visually lossless through many generations of decoding and reencoding. The target data rate is approximately 220 Mbps at 1920x1080 and 29.97 fps.

-Rudy

 

[PhilcTx]

Hopefully not to muddy the water too much more - but there are alternatives to Apple ProRes that may be worth considering and likely supported in free DR. - A candidate is DNxHR -

The difference between DNxHR and ProRes codecs

T he Avid DNxHR and Apple Prores codec families are designed to meet the needs of modern, streamlined post-production workflows. These days we capture source material on a variety of cameras- action cams, smart phones, drones and high-resolution cameras, and codecs makes it easy to work with any ...

lowepost.com

I've used ffmpeg utilities in the past for transcoding to ProRes. There are some issues with this on a Windows system for pro use:

"The open-source program FFmpeg offers a reverse-engineered ProRes compatible export format on Windows. As an unauthorized implementation, it should not be relied upon in a professional environment."

A guide to ProRes and ProRes RAW

Though ProRes and ProRes RAW are both high-quality codecs, you have to use them appropriate to reap their workflow benefits.

www.videomaker.com

 

[sh1209]

Hopefully not to muddy the water too much more - but there are alternatives to Apple ProRes that may be worth considering and likely supported in free DR. - A candidate is DNxHR -

The difference between DNxHR and ProRes codecs

T he Avid DNxHR and Apple Prores codec families are designed to meet the needs of modern, streamlined post-production workflows. These days we capture source material on a variety of cameras- action cams, smart phones, drones and high-resolution cameras, and codecs makes it easy to work with any ...

lowepost.com

I've used ffmpeg utilities in the past for transcoding to ProRes. There are some issues with this on a Windows system for pro use:

"The open-source program FFmpeg offers a reverse-engineered ProRes compatible export format on Windows. As an unauthorized implementation, it should not be relied upon in a professional environment."

A guide to ProRes and ProRes RAW

Though ProRes and ProRes RAW are both high-quality codecs, you have to use them appropriate to reap their workflow benefits.

www.videomaker.com

You were not kidding on the Phantom luts. I have tried them on a few clips this morning and I am amazed how well they work and you can literally drop them on the footage and you're done. Thank you for the recommendation.

 

[Alistair]

Alistair,

Yes the transcoded ProRes 422 HQ file if very washed out, just like a typical log file.

I found the instructional video that I used 2.5 years ago to build a utility that converts Canon, Sony, Panasonic and other brand log files into ProRes 422 HQ files, which are 10-bit video files supported by Davinci Resolve free version (see below). At the 4:15 minute mark he talks about the ProRes 422 HQ codec.

Info below is taken from Apple's documentation about their ProRes codec...
Apple ProRes 422 HQ
Apple ProRes 422 HQ is a higher-data-rate version of Apple ProRes 422 that preserves visual quality at the same high level as Apple ProRes 4444 but for 4:2:2 image sources. With widespread adoption across the video post-production industry, Apple ProRes 422 HQ offers visually lossless preservation of the highest-quality professional HD video that a single-link HD-SDI signal can carry. This codec supports full-width, 4:2:2 video sources at 10-bit pixel depths, while remaining visually lossless through many generations of decoding and reencoding. The target data rate is approximately 220 Mbps at 1920x1080 and 29.97 fps.

-Rudy

That is a great hack Rudy. I did not realise the free version processed ProRes 10 bit, although thinking about it more I guess it has to as there probably is not any 8 bit ProRes! It is great too that, by the sounds of it, the transcoding has preserved the log gamma and probably also the Canon Cinema colour space.

But I suspect your ProRes file has not encoded the camera data that DVR relies on to identify footage source and apply the appropriate colour management. However if you use the 3 nodes and the colour management settings I described in a reply in this thread yesterday, it will apply the correct colour management settings and give you the full colour and DR potential of the file.

 

[Rudy Pohl]

That is a great hack Rudy. I did not realise the free version processed ProRes 10 bit, although thinking about it more I guess it has to as there probably is not any 8 bit ProRes! It is great too that, by the sounds of it, the transcoding has preserved the log gamma and probably also the Canon Cinema colour space.

But I suspect your ProRes file has not encoded the camera data that DVR relies on to identify footage source and apply the appropriate colour management. However if you use the 3 nodes and the colour management settings I described in a reply in this thread yesterday, it will apply the correct colour management settings and give you the full colour and DR potential of the file.

Alistair,
I used the 3-node colour management set up you described above and it worked wonderfully! Stunning colours with no saturation applied at all. Thanks for all your time and help, I really appreciate it. Rudy is very happy.
Rudy

 

[sh1209]

Alistair,
I used the 3-node colour management set up you described above and it worked wonderfully! Stunning colours with no saturation applied at all. Thanks for all your time and help, I really appreciate it. Rudy is very happy.
Rudy

I’m really happy now as well since I have finally figured out how to go about doing this correctly. I bought the Phantom LUTs last night for the Sony footage and they do a tremendous job.

 

[Alistair]

Just to satisfy my own curiosity regarding the original tenet of this post (that there appears to be no IQ advantage to shooting log and some disadvantage in the extra work required to edit it - hopefully that is a faithful summary! ) I test-shot an indoor scene, once in standard and then again in log. The key aspects of the test were:
1. All footage-8k/30 10 bit 4:2:0 Sensor ISO 100 / non Log footage-Rec.709 / log footage-Rec.2020
2. I used the same exposure for both clips (i.e. same shutter angle, same aperture) and clipped no highlights.
3. I processed two copies of the non-log footage; one copy edited directly in the input colour space of Rec.709 (clip a) and the other copy transformed to DaVinci Resolve Wide Gamut and then transformed back to Rec.709 for output (clip b).
4. The log footage I used two transforms - one from the input space of Rec.2020 to DaVinci Resolve Wide Gamut for editing and then from DaVinci Resolve Wide Gamut to Rec.709 for output (clip c)
5. No editing was done to the log footage other than to drop it into a timeline template pre-configured in DaVinci Resolve with the colour space transforms set out in (4) above. The other two clips were edited in DaVinci Resolve to try to resemble as closely as possible the log footage.

This is the scene.
Clip a:

Clip b:

Clip c:

And a crop of the shadow area where the focus is also directed:
Clip a:

Clip b:

Clip c:

What can we then say about this? I think the following:

1. Slightly more colour information is available in the shadows of the log footage. This may be due to the wider colour space the log footage is encoded to combined with log footage shadows being encoded with the higher bit depth usually associated with the mid-tones. The benefit is subtle but is there.
2. The log footage, once placed in the correct colour managed environment, looks materially better than non-log clips before editing. After editing two non-log clips, they still do not look as good as the unedited log clip.
3. Editing the non-log clip in the intermediate colour space does appear to confer some benefit.
4. Much more work is required to make any proper conclusions, work I am not inclined to do! But this work should include comparisons made at higher ISO, different Codecs and different Chroma sampling rates. Suffice to say that for my current workflow and equipment and for the subtle IQ benefit and significantly less editing work required, I am satisfied to use Log as my "go-to" for the time being until further work is done to persuade me otherwise!

 

[vbnut]

Thanks Steve, for starting this thread. It is very timely for me as I'm just starting my video journey, currently with a Canon R3, but I just finished a rental test drive of the R5II and R1 and will purchase one of them. Thanks to everyone for asking great questions and providing such informative answers. This is one of the most information-rich threads I've ever seen, so I'm going to have to reread it slowly and carefully multiple times, to put together a workflow.

I have one question. I was considering shooting 8K RAW (or 6K RAW on the R1) due to the purported ability to process the RAW in Davinci Resolve (I might have to purchase the Studio version) and then export 33 MB stills (18 MB from 6K Raw) stills. Can anyone discuss the differences between processing RAW and Log footage, and/or point me at some references for how to process RAW footage?

 

[MotoPixel]

Thanks for taking the time to make this comparison.

IMHO your methodology is sound and bears out my own experience, in particular the better shadow detail/color and the advantages of editing in DWG Intermediate. And that's something that provides further gains if you happen to be shooting in NRaw, though you're pretty much forced into using NR if shooting an scene at night or lower level interiors.

Cheers!

 

[sh1209]

Just to satisfy my own curiosity regarding the original tenet of this post (that there appears to be no IQ advantage to shooting log and some disadvantage in the extra work required to edit it - hopefully that is a faithful summary! ) I test-shot an indoor scene, once in standard and then again in log. The key aspects of the test were:
1. All footage-8k/30 10 bit 4:2:0 Sensor ISO 100 / non Log footage-Rec.709 / log footage-Rec.2020
2. I used the same exposure for both clips (i.e. same shutter angle, same aperture) and clipped no highlights.
3. I processed two copies of the non-log footage; one copy edited directly in the input colour space of Rec.709 (clip a) and the other copy transformed to DaVinci Resolve Wide Gamut and then transformed back to Rec.709 for output (clip b).
4. The log footage I used two transforms - one from the input space of Rec.2020 to DaVinci Resolve Wide Gamut for editing and then from DaVinci Resolve Wide Gamut to Rec.709 for output (clip c)
5. No editing was done to the log footage other than to drop it into a timeline template pre-configured in DaVinci Resolve with the colour space transforms set out in (4) above. The other two clips were edited in DaVinci Resolve to try to resemble as closely as possible the log footage.

This is the scene.
Clip a:
View attachment 105370

Clip b:
View attachment 105371

Clip c:
View attachment 105372

And a crop of the shadow area where the focus is also directed:
Clip a:
View attachment 105373

Clip b:
View attachment 105374

Clip c:
View attachment 105375


What can we then say about this? I think the following:

1. Slightly more colour information is available in the shadows of the log footage. This may be due to the wider colour space the log footage is encoded to combined with log footage shadows being encoded with the higher bit depth usually associated with the mid-tones. The benefit is subtle but is there.
2. The log footage, once placed in the correct colour managed environment, looks materially better than non-log clips before editing. After editing two non-log clips, they still do not look as good as the unedited log clip.
3. Editing the non-log clip in the intermediate colour space does appear to confer some benefit.
4. Much more work is required to make any proper conclusions, work I am not inclined to do! But this work should include comparisons made at higher ISO, different Codecs and different Chroma sampling rates. Suffice to say that for my current workflow and equipment and for the subtle IQ benefit and significantly less editing work required, I am satisfied to use Log as my "go-to" for the time being until further work is done to persuade me otherwise!

Thanks for the testing. I think c looks the best to my eye.

 

[Alistair]

Thanks Steve, for starting this thread. It is very timely for me as I'm just starting my video journey, currently with a Canon R3, but I just finished a rental test drive of the R5II and R1 and will purchase one of them. Thanks to everyone for asking great questions and providing such informative answers. This is one of the most information-rich threads I've ever seen, so I'm going to have to reread it slowly and carefully multiple times, to put together a workflow.

I have one question. I was considering shooting 8K RAW (or 6K RAW on the R1) due to the purported ability to process the RAW in Davinci Resolve (I might have to purchase the Studio version) and then export 33 MB stills (18 MB from 6K Raw) stills. Can anyone discuss the differences between processing RAW and Log footage, and/or point me at some references for how to process RAW footage?

Editing RAW is very easy in DVR! Controls resemble most RAW photo editors:

Shooting in log is not necessary - it is simply recorded as a metadata. But you can apply a log gamma to the non-log footage and export to 10bit to save on storage space is you want to go that way. Extracting stills from Raw is feasible but note that the sensor readout speed is usually slower in video than stills (at least it is in Nikon-world), there is a slight crop to 16:9 and, unless shooting with a mind to extracting stills, your shutter speed will be rather slow.

 

[shardul]

I recently attempted N-raw with N-log on Nikon Z8. I found the following 2 videos quite helpful for Davinci Resolve. Both provide about the same level of editing control and output. From what I understood, neither of them are using LUTs (unless internally DR is using them). I found these very easy to setup and use. I feel (not scientifically compared) the edited raw/nlog videos are looking better than the once shot in H.264/H.265 in the past.

Nikon Asia:

David Fuller:

 

[sh1209]

This is with one step of adding the utopia DJI D-log LUT from Phantom. I think it looks great as is without any other corrections. I have been super impressed by these LUTS.

You can only see EXIF info for this image if you are logged in.

 

[shardul]

I recently attempted N-raw with N-log on Nikon Z8. I found the following 2 videos quite helpful for Davinci Resolve. Both provide about the same level of editing control and output. From what I understood, neither of them are using LUTs (unless internally DR is using them). I found these very easy to setup and use. I feel (not scientifically compared) the edited raw/nlog videos are looking better than the once shot in H.264/H.265 in the past.

Nikon Asia:

David Fuller:

Hello Alistair,

Hope you can guide me with this. Out of these 2 methods, which one do you suggest to use?

Also in the free version of DR, since the input codec is limited to 8 bits, what happens when you load the N-raw 12 bit NEV file? It does seem to be allowing editing, but is it limiting only 8 bit of data under the hood?

 

[Alistair]

Hello Alistair,

Hope you can guide me with this. Out of these 2 methods, which one do you suggest to use?

Also in the free version of DR, since the input codec is limited to 8 bits, what happens when you load the N-raw 12 bit NEV file? It does seem to be allowing editing, but is it limiting only 8 bit of data under the hood?

Hi Shardul,

Both these approaches are actually doing the same thing; they are recognising the correct colour space and gamma in which the footage has been shot, then transforming this colour space/gamma to a large colour space/gamma optimal to undertake editing in (known as an intermediate colour space, in this case davinci wide gamut/davinci intermediate) and then finally, once editing is done, transforming the colour space/gamma to one appropriate for output (usually Rec.709/gamma 2.4 for internet-accessed digital content).

The first of the video tutorials you reference achieves this at the clip level in the colour tab of DVR, the second achieves it at a project level using project settings. Both methods are perfectly valid. My preference is for the approach shown in the first video (clip level colour management) because I tend to combine a number of different clip types in my projects. I may have some raw clips, some Nlog, some SDR, some from my phone or even a different camera if I am collaborating with others on a project.

I also find reassuring the visual logic of working with sequential nodes in the colour tab. I can tell at a glance that I have the correct colour space/gamma transforms for each clip and that I have applied edits in the correct colour space/gamma before transforming to the constraints of Rec.709 output space. It is easy too to turn on and off the effect of each node (ctrl/d) to check the effect.

I like the 3 node approach:
First one to transform from the colour space the footage is shot in to intermediate editing space:

Second one to apply edits:

Third one to transform from editing colour space/gamma to output space after editing:

You can these use the most basic settings at the project level as you are doing all colour management at the clip level:

If you are using a LUT, I suggest inserting a fourth node between nodes 2 and 3 shown above and apply the LUT on this node. It may not be clear how colour management is handled in the LUT, but usually it contains the transform to Rec.709 done by node 3 above (this node will now be node 4 once you insert a node for the LUT). It is then trivial to toggle node 4 on and off to confirm this. You can also make any changes required to node 1 if the LUT requires a different intermediate space or assumes a different input (scene/capture) space. The LUT also contains grading which may make your node 2 redundant or requiring further work and this all becomes easy to do by toggling the node on/off, making adjustments etc.

Below I have added a fourth node between nodes 2 and 3 shown above and have applied a LUT to this node. If you are not sure how the LUT is handling colour management, it is a trivial task to toggle nodes 1 and 4 on/off to determine this. I tend to "expose to the right" when shooting so using node 2 to bring exposure down to the correct level before the LUT is applied. Also I use node 2 to apply a little sharpening and any other colour or contrast adjustment I want before the LUT is applied.

It can seem a bit overwhelming but once you use it a few times, it all becomes clear.

If you are only using one type of clip in your project and are not using LUTS, colour management at the project level makes sense. The settings below are for Nikon nlog and then you can just use adjustment nodes in the colour tab.

Good luck!

 

[Rudy Pohl]

Hi Shardul,
....
Good luck!

Hi Alistair,

I've been following along in your discussion with Shardul and I'm delighted to report that I have understood every word you've written. And the reason is because you took the time and made the same selfless effort to explain these things to Steve and I a couple of weeks ago. I feel that I still have not thanked you enough for the incredible help and encouragement you've been to me here in this forum, especially as it concerns working with log profiles. So once again, a very big and grateful thanks!

You're in good hands Shardul and have come to the right place!

Cheers,
Rudy

 

[Alistair]

Hi Alistair,

I've been following along in your discussion with Shardul and I'm delighted to report that I have understood every word you've written. And the reason is because you took the time and made the same selfless effort to explain these things to Steve and I a couple of weeks ago. I feel that I still have not thanked you enough for the incredible help and encouragement you've been to me here in this forum, especially as it concerns working with log profiles. So once again, a very big and grateful thanks!

You're in good hands Shardul and have come to the right place!

Cheers,
Rudy

That is most generous of you Rudy, thanks. No need to thank me though, I am no expert but once I finally figured this stuff out for myself I am very happy to share it and try to save others some time !

 

[shardul]

Hi Alistair,

I've been following along in your discussion with Shardul and I'm delighted to report that I have understood every word you've written. And the reason is because you took the time and made the same selfless effort to explain these things to Steve and I a couple of weeks ago. I feel that I still have not thanked you enough for the incredible help and encouragement you've been to me here in this forum, especially as it concerns working with log profiles. So once again, a very big and grateful thanks!

You're in good hands Shardul and have come to the right place!

Cheers,
Rudy

I second that. I also really appreciate you taking time to explain this in detail and reinforcing the understanding. So happy to be here on this forum and learning new things.

I would really appreciate if you can also answer the second part of the question. How does the free version of DR handle the NEV file with N-raw 12 bit (since it only supports up to 8 bit). I am contemplating buying the Studio license, but only if necessary as I am not a professional to justify the cost

 

[sh1209]

I’d like to put my two cents in here as well. I have shot video for a long time but never really started diving deep into editing until about 6 years ago or so. Having said that, most of my videos before then were just done on iMovie since I’m a Mac user. I for one think we have a great group of talented folks on here that have helped me a lot. I stayed after Steve for a while trying to get the video section added and I’m very glad he did. For me Nimi, Garfield, Allistair, Motopixel, Rudy and PhilTx have helped me immensely. I am very grateful for all the pointers along the way and I feel I have a deeper understanding of editing video because of this forum. I hope it continues to grow because bringing in knowledgable members sure makes things better.

 

[Rudy Pohl]

I’d like to put my two cents in here as well. I have shot video for a long time but never really started diving deep into editing until about 6 years ago or so. Having said that, most of my videos before then were just done on iMovie since I’m a Mac user. I for one think we have a great group of talented folks on here that have helped me a lot. I stayed after Steve for a while trying to get the video section added and I’m very glad he did. For me Nimi, Garfield, Allistair, Motopixel, Rudy and PhilTx have helped me immensely. I am very grateful for all the pointers along the way and I feel I have a deeper understanding of editing video because of this forum. I hope it continues to grow because bringing in knowledgable members sure makes things better.

Steve - I had know idea that you were involved in getting Steve Perry to get this video forum onto the Back Country website. Way to go and many thanks!
Rudy

 

[sh1209]

Steve - I had know idea that you were involved in getting Steve Perry to get this video forum onto the Back Country website. Way to go and many thanks!
Rudy

You’re welcome. I asked about it for a while but I’m sure glad he added it. I feel like a lot of good information has been shared on here. I know I’ve learned a lot. I think there were a few of us at the end that were asking for it. I can’t remember exactly who it was but I’m so glad it’s on here now.

 

[MotoPixel]

Steve, thanks for the mention and also being instrumental in getting this forum section going. And also to Alistair, thanks for taking the time to provide an excellent write-up on the methodology in setting up DVR color management. Hopefully, this site can save those just now getting their feet wet in video editing the time and frustration of having to ferret out this information from the minimal or scattered resources available prior.