Backup Strategy for Remote Shoots

If you would like to post, you'll need to register. Note that if you have a BCG store account, you'll need a new, separate account here (we keep the two sites separate for security purposes).

J

JPaul_Johnson

New member
Member's gallery
Here's my situation... I'm in Florida and I've just spent the day at Black Point where I shot over 6000 pictures with my Z8/Z9. The day was pretty long and I don't have time to parse the pictures before tomorrow's shoot. It's taken about 3 hours to transfer the shots to an external HDD (USB 3.1 reader, USB3 HDD). I don't think my HDD will be able to hold the entire trip if this pace stays constant. I try to keep at least 2 backups, but this is the first time I've had so many shots. I have well over 300GB of pictures. My 5TB external HDD would last around 15 days at this pace, and I could get a second one for the redundancy, but it takes a bit of time and effort to make these backups too.

Other than just not taking so many shots, there must be a better way. Does anyone have a cost effective strategy that works well for a busy shoot?
 
Member's gallery
Paul…

My suggestion is to continue your present policy (copying all files to an external 5TB spinning hard drive?), but to purchase at least one 4 TB SSD external drive in order to reduce the transfer time and, presumably, provide safer storage.

Perhaps, budget US$100 per terabyte for 'inexpensive' SSD drives. Others will be able to answer from experience.

… David
 
Last edited:
Member's gallery
 
Member's gallery
Obviously you cannot reuse your cards until you have copied the images to another hard drive and formatted the cards.

Depending on your individual file sizes, you may have taken around 250 GB of images – which as you note takes time to transfer to older hard drives.

Buying a faster hard drive as already suggested is likely to be much more cost-effective than buying extra cards.

You might perhaps consider the number of shots you take. If you take a modest average of just one minute to rate, delete some, edit and catalogue 6000 images, that is 6000 minutes or 100 hours of your time.
 
Member's gallery
1. Buy two really good T7 Samsung SSD drives of 2TB
2. Make sure you have fast CF-Express B cards
3. If not have it yet do get a proper CF-Express B card reader such as the Sony MRW-G1
4. Check if your laptop does have a USB 3.1 USB-C type port
5. Make indeed your backups in the evening. When I do this during a trip do I start my laptop immediately after arriving in my hotel and copy the content to my laptop. During the copying do I take a shower and whatever needs to happen while swapping cards etc.
When all is on the laptop is it time to connect the SSDs, both of them, to the laptop and copy the pictures from the laptop to the SSDs. You might have to do it 1 by 1 or both of them at the same time.
6. Delete the folder on your laptop after a successful copy
7. When arriving home do I import & copy the pictures onto my NAS and in my LR catalog and start culling.
8. After culling the pictures do I copy the pictures from my NAS to those same SSDs after I have deleted the backup folder. I do this one drive at the time

Hope this can help.
 
Member's gallery
I sympathize with your dilemma.

As a test I copied 2504 ~ 106GB files to and from a Sony Tough CFExpress card in a no name USB C 3.2 Gen 2 CFExpress reader to my desktop computer that has 2 X USB C 3.2 Gen 2 capable interfaces as well as an internal NVME drive and standard internal HHD drive connected to a 6BG/s SATA interface.

From desktop HDD to CFExpress card took approx 8minutes 35 seconds.
From CFExpress to desktop HDD took 10 minutes 30 seconds.
From CFExpress to desktop NVME drive took 2 minutes 32 seconds.

These tests show your transfer time could come down from about 3 hours to about 30 minutes at the slowest rate and about 9 minutes at the fastest rate to transfer your ~6000 images from a CFExpress card.

I expect these times or better would be achievable on my laptop which is now 18 months old, it is fitted with 2 X USB C 3.2 Gen 2 capable interfaces as well as a NVME drive.

The use of a computer with a USB C 3.2 Gen 2 capable interface and an internal NVME drive would be a good selection.

External drives are available as described above which will work at high speed and should match or exceed these times.

The other thing to consider is a CFExpress USB C 3.2 Gen 2 capable reader as well as a USB-C to USB-C cable rated to at least 20Gbps.

Investing in capable computing and peripherals is very worthwhile as your images and time are precious to you.
 
Member's gallery
I had similar issues in Botswana with images from my DSLRs. Evaluate your download system to make sure the fastest speeds are available to you as detailed in the two posts just above mine: Fast cards, fast card reader, fast ports, fast laptop drive, fasy external hard drives.
 
Member's gallery
BrianSmall said:
Dumb question but why don't you just edit what you have?
Click to expand...
Thanks for the response. Frankly, I had a 15 hour day just shooting, and hopefully, another one tomorrow, and on and on. It's going to take me days to cull these shots for only this shooting day, and I want to do this every day I'm here. The copy process to just one external took me until 1AM, and I want to repeat this process every day! I don't get out so much that I can blow an opportunity. And I need some sleep.
 
Member's gallery
David Berry said:
Paul…

My suggestion is to continue your present policy (copying all files to an external 5TB spinning hard drive?), but to purchase at least one 4 TB SSD external drive in order to reduce the transfer time and, presumably, provide safer storage.

Perhaps, budget US$100 per terabyte for 'inexpensive' SSD drives. Others will be able to answer from experience.

… David
Click to expand...
Thanks David. I've done this now. Just got a 4TB Samsung T9 from Fedex. This should help.
 
Member's gallery
StarTracker50 said:
You spent 3 hours backing up. Could you not have spent that time cleaning up, or are you like me, hate to delete anything? Also, look into this. Blazing fast!
Click to expand...
Thanks for the links StarTracker50. I ordered a 4TB T9 yesterday. Just got it. And, Yes to the "hate to delete," at least for now. I'll cull them when I have the time to do a good job.
 
Member's gallery
I just buy a ton of CF Express cards, and don't use a laptop.

I have over 20TB of storage cards, and I'll never fill that on a trip. I fill one card (which I know when it happens since I have my bodies set to overflow), and then I'll remove that card, flip it upside down in the card holder, put in a new one and keep shooting infinitely.

Alternatively, get SSD's or faster HDDs if you want to continue that route. Or do as others have said and maximize your time transferring, by doing other things at night like showering, eating, sleeping, etc.
 
Member's gallery
Len Shepherd said:
Obviously you cannot reuse your cards until you have copied the images to another hard drive and formatted the cards.

Depending on your individual file sizes, you may have taken around 250 GB of images – which as you note takes time to transfer to older hard drives.

Buying a faster hard drive as already suggested is likely to be much more cost-effective than buying extra cards.

You might perhaps consider the number of shots you take. If you take a modest average of just one minute to rate, delete some, edit and catalogue 6000 images, that is 6000 minutes or 100 hours of your time.
Click to expand...
Astute observations. I actually shot 345GB and 7259 files. My workflow will let me cull them to "keepers" at a rate of perhaps around 30 shots per minute, and I expect to reduce the number to under 150. Then I'll do some mild editing on these before I use them. Spending that time during a shoot is not a good idea for me. I have no deadlines and can do a better job later. I agree that a faster drive will help, thus the T9. I should likely just get 2 of these, but there are other issues that affect the speed of copying.
 
Member's gallery
Vincent_VdB said:
1. Buy two really good T7 Samsung SSD drives of 2TB
2. Make sure you have fast CF-Express B cards
3. If not have it yet do get a proper CF-Express B card reader such as the Sony MRW-G1
4. Check if your laptop does have a USB 3.1 USB-C type port
5. Make indeed your backups in the evening. When I do this during a trip do I start my laptop immediately after arriving in my hotel and copy the content to my laptop. During the copying do I take a shower and whatever needs to happen while swapping cards etc.
When all is on the laptop is it time to connect the SSDs, both of them, to the laptop and copy the pictures from the laptop to the SSDs. You might have to do it 1 by 1 or both of them at the same time.
6. Delete the folder on your laptop after a successful copy
7. When arriving home do I import & copy the pictures onto my NAS and in my LR catalog and start culling.
8. After culling the pictures do I copy the pictures from my NAS to those same SSDs after I have deleted the backup folder. I do this one drive at the time

Hope this can help.
Click to expand...
Thanks Vincent. This is very sensible. I try to do this to the extent I can. My laptop, a 17" LG Gram, can't hold this much data so I have to go from the reader (ProGrade CFexpress USB 3.1) to the WD Passport (USB3.0) directly. That's my hardware bottleneck I believe. It slows the copy to around 25MB/sec, even though the laptop has USB3.1.

To really make it as quick as possible I think I might need a new laptop that will support the latest USB 4 standard, and a reader to match. And I would ensure I had a large enough NVMe to temporarily store the data, much like you're doing....

Unless I could quickly go from the reader to the external SSD. Then I would do it that way to save the time copying to the laptop, and I would save the wear and tear on the laptop NVMe. I've actually worn them out with too much TBW, and I don't have any control on what LG puts in them (TBW rating). Wish I could specify the NVMe drive for a build.

Like you, I'm not going to waste my shooting time culling shots because time is so precious during a shoot away. It's nice to think that I might be doing the things I need to do. Thanks.
 
Member's gallery
nmerc_photos said:
I just buy a ton of CF Express cards, and don't use a laptop.

I have over 20TB of storage cards, and I'll never fill that on a trip. I fill one card (which I know when it happens since I have my bodies set to overflow), and then I'll remove that card, flip it upside down in the card holder, put in a new one and keep shooting infinitely.

Alternatively, get SSD's or faster HDDs if you want to continue that route. Or do as others have said and maximize your time transferring, by doing other things at night like showering, eating, sleeping, etc.
Click to expand...
That's brilliant, and expensive. I shoot 1TB Lexar Silver Professional now. The card is now available for about $200 I think, and I'm not shooting 8K video, so it should be fine. At this heavy rate, a card would take maybe 2 days to fill. It's a 21 day trip, with perhaps 14 shoot days, so 7 cards plus backup cards (copy setting), for the Z9. So 14 CFE cards, or about $2800. Then the Z8, but it's not as busy. Perhaps another 4 CFexpress cards and 4 SDHX cards.

This actually is pretty sensible. A new laptop that meets the USB4 standard and has a 4TB NVMe could cost more than the cards. Aaah, but the card isn't as much fun outside the camera, and I can't cull shots on a down day very well without the PC. Still, it's worth considering. Thanks.
 
Member's gallery
JPaul_Johnson said:
What about using the ethernet port? Has anyone tried that direct to a PC?
Click to expand...
Yes. I bought a 10Gbps pci card for my pc when I upgraded my network. Great for home use, I also git the 10Gbps T4 Ethernet adaptors for my Mac…my nas is all 10Gbps

But in the field, I copy Z8 to iPad to external SSD Via usb-c cable. It’s a data cable, and gets me 1Gbps speeds….tnats ok in the field. I dump every shoot daily. Takes minutes, not hours! I’m talking under 1k images and no video.

 
Member's gallery
Patrick M said:
Yes. I bought a 10Gbps pci card for my pc when I upgraded my network. Great for home use, I also git the 10Gbps T4 Ethernet adaptors for my Mac…my nas is all 10Gbps

But in the field, I copy Z8 to iPad to external SSD Via usb-c cable. It’s a data cable, and gets me 1Gbps speeds….tnats ok in the field. I dump every shoot daily. Takes minutes, not hours! I’m talking under 1k images and no video.

Click to expand...
Brilliant. I think I need to see how to connect the Z9 directly to my laptop and test the bandwidth. Thanks for the info.
 
Member's gallery
Random USB comments:

1) Shoot for having all devices (computer, reader, external drive(s)) to have 10Gb/s USB (USB 3.2 Gen2 aka 3.1 Gen2). Basically look for the 10Gb/s annotation in the specs. This is the fastest speed that is ubiquitous. This works out to be about 1000MB/s transfer rate in practice. This is "fast enough" to easily copy cards to drives. Trying to eek out more performance with things like 20Gb/s USB makes your life complicated, the options few and the returns diminishing.

2) USB4 is a really good thing, but it is not commonly available, and not really necessary for this use case. It's certainly a good thing to look for when purchasing new equipment like laptops, hard drives. That said, I wouldn't bother from a card reader perspective.

USB4 drives are emerging and I'm keen on making sure all new drives I get have it. Here's an example:

eshop.macsales.com

OWC Express 1M2

Ultra-fast, compatible, and reliable portable USB4 NVMe SSD with speeds up to 3151MB/s. Build your own or choose ready-to-run solutions for your Mac or PC.
eshop.macsales.com eshop.macsales.com
 
Last edited:
Member's gallery
John Navitsky said:
Random USB comments:

1) Shoot for having all devices (reader, external drive(s)) to have 10Gb/s USB (USB 3.2 Gen2 aka 3.1 Gen2). Basically look for the 10Gb/s annotation in the specs. This is the fastest speed that is ubiquitous. This works out to be about 1000MB/s transfer rate in practice. This is "fast enough" to easily copy cards to drives. Trying to eek out more performance with things like 20Gb/s USB makes your life complicated, the options few and the returns diminishing.

2) USB4 is a really good thing, but it is not commonly available, and not really necessary for this use case. It's certainly a good thing to look for when purchasing new equipment like laptops, hard drives. That said, I wouldn't bother from a card reader perspective.

USB4 drives are emerging and I'm keen on making sure all new drives I get have it. Here's an example:

eshop.macsales.com

OWC Express 1M2

Ultra-fast, compatible, and reliable portable USB4 NVMe SSD with speeds up to 3151MB/s. Build your own or choose ready-to-run solutions for your Mac or PC.
eshop.macsales.com eshop.macsales.com
Click to expand...
Thanks. I agree completely and chose the Samsung T9 for this very reason. But wouldn't it be great if we could just run a LAN cable from the Z9 to the PC and then use the Explorer to find the Z9 and copy the files. I hope it will let me do this. Research is hard.
 
Member's gallery
Congratulations on your back up solution. That's a good step. I have had a 2 TB portable solid state drive for several years. Just be sure to use fast cables or cables capable of high speed data transfer.

Given the volume you are photographing each day, I would give some thought about how to triage your work and delete 80-90% or more of the images. For wildlife, unless it is a rare subject the image needs to be excellent to have any meaningful use. For example, you don't want to apply presets in Lightroom or Camera Raw if you are deleting 90% of the images - and that can speed up your workflow dramatically.

Arthur Morris goes through all of his images on a daily basis after any shoot - regardless of how many images he makes. He selects 5-10 images or less from the day that are "Selects" and discards the rest. Only the Selects are backed up. Certainly you could make an exception if you have a particular sequence that is exceptionally good or unique, but for a typical burst every photo or every photo but one or two would be discards. None of this involves studying the images or dwelling on your decision - it's a discard unless it's good enough to hang on your wall or publish.

Burst shooting is a clear place to work on discarding more images. If you have a clear idea of what you are trying to accomplish, a borderline image or less than perfect image is an easy discard. You can discard an entire sequence based on head position or wing position. High frame rates should not mean you keep any more images - it means you have a better chance of an ideally timed image, but most of the images are not keepers.
 
Member's gallery
Perhaps in addition to increasing your storage capacity, you can also evaluate your shooting approach. I have found that as my skills - and portfolio - have evolved, I tend to be much more selective on what and when I shoot. I no longer need to rip of 20 fps for a long time, over and over again of the same subject sitting on a perch, standing in the water, etc. If the light is not great, if the pose or background are not good, I simply pass on taking the shot(s).
 
Member's gallery
This is just my opinion, but I don't delete anything except out of focus or missed shots in the field. Like you, I also don't have time after shooting on something like a May trip to Yellowstone to go through all the images. I get on the road at 4M, get back after dark exhausted and still have to eat dinner, shower, get ready for the next day, etc... Not going to be sitting there for a couple hours editing.
I bought a couple Samsung T7 4TB SSD USB C drives for about $250 each and a USB C reader for my CF Express A cards and use that in conjunction with my laptop and a small USB C hub. With the hub, I can copy from the card reader directly to the SDD drive and backing up for even a busy day rarely takes more than an hour. I have about 2 TBs worth of CFE-A cards which has always proven enough for a day.
 
Member's gallery
Butlerkid said:
Perhaps in addition to increasing your storage capacity, you can also evaluate your shooting approach. I have found that as my skills - and portfolio - have evolved, I tend to be much more selective on what and when I shoot. I no longer need to rip of 20 fps for a long time, over and over again of the same subject sitting on a perch, standing in the water, etc. If the light is not great, if the pose or background are not good, I simply pass on taking the shot(s).
Click to expand...
That's a very good point. With most wildlife - there are a lot of photos that are not taken, and everything that is taken has a specific purpose and can be culled aggressively. I was out for three hours yesterday for birding and bird photography. I identified 45 birds - migration is going strong. But I only made 200 photos with my Z8 and only kept 10 frames. I only downloaded 10 frames to my computer - my review and choice of selects was made using the card - not my hard drive.
 
Member's gallery
EricBowles said:
That's a very good point. With most wildlife - there are a lot of photos that are not taken, and everything that is taken has a specific purpose and can be culled aggressively. I was out for three hours yesterday for birding and bird photography. I identified 45 birds - migration is going strong. But I only made 200 photos with my Z8 and only kept 10 frames. I only downloaded 10 frames to my computer - my review and choice of selects was made using the card - not my hard drive.
Click to expand...


That's really a philosophical question for each individual photographer rather than a tenet that should be universally accepted, though. I would never pass up a shot because it wasn't perfect. You never know what you're going to catch. Maybe if you're a professional who has a very specific shot in mind, that would work, but how many people here are full-time professional photographers? What percentage of wildlife photographers actually make money at it?
 
You must log in or register to reply here.
Back
Top