It happened to me.....data loss! Lightroom questions....

If you would like to post, you'll need to register. Note that if you have a BCG store account, you'll need a new, separate account here (we keep the two sites separate for security purposes).

Member's gallery
Steve said:
DNG can be opened with a variety of software, although not Nikon as far as I know (just tried). However, the edits are instruction sets, so although a non-adobe program may be able to open the DNG, that doesn't mean the edits will follow. The only way to insure you have the edits is with Adobe software.

You can convert them to DNG via the catalog. Just select all (CMD/CRL + A) > Library > Convert Photo To DNG. You'll get some options, make sure "fast load data" is checked. You can embed your original RAW files in the DNG (huge files though), leave them alone, or delete them after conversion. If you want a set of images that are easily accessed when you're gone, full quality JEPGS are the best bet. They are the most universal. Plus, what are the chances someone would want to come in and edit your RAW files after you go? I certainly don't see it happening with my stuff. I think my family would just want the processed images, if they wanted any of it.

Also, keep in mind that when you edit your RAW file in Lightroom, you're basically working on a DNG version of the file. So, it's not like you're losing any data.

All that said, I still can't bring myself to convert. I get tempted like this from time to time, but I also have a bunch of other Tiffs and PSDs in the mix too (some that have minor Lightroom tweaks). So, I'll probably just keep relying on my numerous backups.
Click to expand...

For me, the huge downside to DNG is that if you have to turn on "automatically write changes to XMP" in Lightroom Classic to get the benefit of storing your up-to-date editing and other metadata information with the file. However Lightroom won't actually write XMP sidecar files for DNGs. It will insert the XMP data into the DNG file itself every single time you make an edit. This may sound obvious and exactly like what you want.

However, unfortunately it instantly makes common backup solutions that maintain historical copies of files, like macOS' Time Machine, Carbon Copy Cloner, SuperDuper, etc all absolutely worthless for your image library because they will backup the whole file again, even if just a few bytes have changed. So literally every time you fiddle with a development slider or any library metadata they will see the file has changed and want to write an ENTIRE NEW COPY to your backups. Every single time! Lol this kills me.

Example: a D850 compressed raw is roughly... 50-60 MB. Let's say you finish a couple of days of shooting and have 1000 images. You import them into Lightroom and walk away for the evening. All your images get backed up. That's 55 GB of data. The next day you don't have a lot of time to edit but you do some quick keywording of those images. Guess what? All 1000 will get backed up again. 55 GB of additional data added to your backups, just for a KB or two of keywords. Then the next day you apply a develop preset to all the images. 55 GB of backups... again!

Of course there are other, more sophisticated backup tools that only backup differences in files. Both Backblaze and Crashplan do this to save bandwidth. But I don't know of any local disk-based backup solutions that operate like this.
 
Member's gallery
Perhaps it's my age but I can't bring myself to trust database programs that store stuff in files that can't be read without using the program. I keep my raw files and sidecars (*.nef) in separate folders by camera name and edited files containing all layers in project folders . I save versions of edited files as well. I save the edited files in *.tif format with information about them in the file names. In a past life I was responsible for managing a large medical research database so my backup consists of 2 drives attached to the computer, one working, the other a time machine backup, a separate backup drive in the safe and one more in a safe deposit box off site. There are downsides, not least the cost of four 20 terabyte drives, but no single drive failure can cost me images. I've played with Lightroom and use Capture One for some edits but not for the catalogs. I do use Premiere for video but that's for another post.
 
Niels.hr.Hansen said:
I thought LR kept edits in side car files, so you would be able to import RAW and side car in a new database and have exact same edits as in the old. Maybe it's an option.
Darktabel offers this as a method, I use it both for an extra level of backup of the database and when bringing edits from my laptop to the desktop.
Click to expand...
Yes, but it is a choice in preferences to do it automatically, I don't know what the default is. Otherwise you can choose to do it manually via the menus.
 
Member's gallery
kimball said:
For me, the huge downside to DNG is that if you have to turn on "automatically write changes to XMP" in Lightroom Classic to get the benefit of storing your up-to-date editing and other metadata information with the file. However Lightroom won't actually write XMP sidecar files for DNGs. It will insert the XMP data into the DNG file itself every single time you make an edit. This may sound obvious and exactly like what you want.

However, unfortunately it instantly makes common backup solutions that maintain historical copies of files, like macOS' Time Machine, Carbon Copy Cloner, SuperDuper, etc all absolutely worthless for your image library because they will backup the whole file again, even if just a few bytes have changed. So literally every time you fiddle with a development slider or any library metadata they will see the file has changed and want to write an ENTIRE NEW COPY to your backups. Every single time! Lol this kills me.

Example: a D850 compressed raw is roughly... 50-60 MB. Let's say you finish a couple of days of shooting and have 1000 images. You import them into Lightroom and walk away for the evening. All your images get backed up. That's 55 GB of data. The next day you don't have a lot of time to edit but you do some quick keywording of those images. Guess what? All 1000 will get backed up again. 55 GB of additional data added to your backups, just for a KB or two of keywords. Then the next day you apply a develop preset to all the images. 55 GB of backups... again!

Of course there are other, more sophisticated backup tools that only backup differences in files. Both Backblaze and Crashplan do this to save bandwidth. But I don't know of any local disk-based backup solutions that operate like this.
Click to expand...
That's a good really good point. Your scenario would consume a lot of data for sure!
 
Member's gallery
Steve said:
That's a good really good point. Your scenario would consume a lot of data for sure!
Click to expand...
There are pros and cons to using DNG files or using sidecar files. The choice should be made with respect to one's workflow. Unless you are starting from a blank slate and really want a feature of one or the other, it is not always without issue to switch out mid-stream, but it can be done.

--Ken
 
Replytoken said:
The default is that edits are kept in the LR database.

--Ken
Click to expand...

That is too bad in this case, it could have helped the original poster. The edits also stay in the catalog even if you also put them in the sidecar, so the only downside is a little delay while it writes.
 
Member's gallery
In reference to CCC. It will only backup files that are altered. For example, if you have 10 photos in a file folder, and only one that has changed, it will only update that one photo not the entire folder. During the copy routine you can watch what is copied and what is not. It only copies from the source file what has changed.
 
Member's gallery
Ralph Bruno said:
In reference to CCC. It will only backup files that are altered. For example, if you have 10 photos in a file folder, and only one that has changed, it will only update that one photo not the entire folder. During the copy routine you can watch what is copied and what is not. It only copies from the source file what has changed.
Click to expand...
Yes, only files that have changed should be updated, but isn't the update during the backup re-writing the entire file? If a DNG file is from a high resolution camera and contains the original raw file it can be quite large.

--Ken
 
Member's gallery
Replytoken said:
Yes, only files that have changed should be updated, but isn't the update during the backup re-writing the entire file? If a DNG file is from a high resolution camera and contains the original raw file it can be quite large.
Click to expand...

You are both correct. The issue is that, when "Automatically write changes to XMP" is turned on, my example of merely keyword tagging 1000 images from a day's shooting will cause that keyword to be written to the XMP data inside the DNGs, thus triggering CCC to backup the entire 1000 DNGs, again.

That's not an issue if you've set CCC to not save history. However, then it's not a full backup of historical data. It's just a current clone/snapshot, which has its purposes in some scenarios but is nowhere near as valuable for a photo collection.
 
Member's gallery
Isn't this where incremental backups come in. Only the changed data is backed up and linked to the last full backup of an image. When restoring both sets of data need to be collected by the system.
 
Member's gallery
gbodave said:
Isn't this where incremental backups come in. Only the changed data is backed up and linked to the last full backup of an image. When restoring both sets of data need to be collected by the system.
Click to expand...
There's incremental in terms of "backup only changed files" and there's incremental in terms of "backup only the changed data of the changed files." I don't know of any backup programs that do the latter to local disks. I do know that CrashPlan and BackBlaze do that for their cloud backups.
 
Member's gallery
kimball said:
There's incremental in terms of "backup only changed files" and there's incremental in terms of "backup only the changed data of the changed files." I don't know of any backup programs that do the latter to local disks. I do know that CrashPlan and BackBlaze do that for their cloud backups.
Click to expand...
You gave a great example. I do not know of any backup programs that save just the changed data to a specific file (i.e. an increment) while keeping the prior whole backup version of it. So, if you update 1000 DNG's from a D850, for example, that have raw files embedded in them, that is a lot of data to be backed up. My answers to this issue would be to either back up the catalog frequently, or use sidecar files. If the time to back up this many files is not an issue, then one could proceed ahead.

--Ken
 
Member's gallery
Steve said:
DNG can be opened with a variety of software, although not Nikon as far as I know (just tried). However, the edits are instruction sets, so although a non-adobe program may be able to open the DNG, that doesn't mean the edits will follow. The only way to insure you have the edits is with Adobe software.

You can convert them to DNG via the catalog. Just select all (CMD/CRL + A) > Library > Convert Photo To DNG. You'll get some options, make sure "fast load data" is checked. You can embed your original RAW files in the DNG (huge files though), leave them alone, or delete them after conversion. If you want a set of images that are easily accessed when you're gone, full quality JEPGS are the best bet. They are the most universal. Plus, what are the chances someone would want to come in and edit your RAW files after you go? I certainly don't see it happening with my stuff. I think my family would just want the processed images, if they wanted any of it.

Also, keep in mind that when you edit your RAW file in Lightroom, you're basically working on a DNG version of the file. So, it's not like you're losing any data.

All that said, I still can't bring myself to convert. I get tempted like this from time to time, but I also have a bunch of other Tiffs and PSDs in the mix too (some that have minor Lightroom tweaks). So, I'll probably just keep relying on my numerous backups.
Click to expand...
makes since .... after all I send a high quality jpg to my pro lab to get really big metal prints :)
 
You must log in or register to reply here.
Back
Top