Storage question for the brain trust

If you would like to post, you'll need to register. Note that if you have a BCG store account, you'll need a new, separate account here (we keep the two sites separate for security purposes).

mrt

Well-known member
Supporting Member
Marketplace
I am running out of room on my 5tb WD Passport portable drive that I use only for photos. I use a 2021 Mac Powerbook. My wife and I spend a fair amount of time camping and birding each year, and during the evening I cull the days catch, while everything gets stored on the WD drive. When I get home, sometimes weeks later, I back up to the cloud using BackBlaze and also to a desktop Toshiba hard drive, currently also 5tb. I use Lightroom and currently everything is under one catalogue. It's time to do a hard drive size upgrade or come up with a new strategy for storage in the field and transferring to a larger hard drive at home. There aren't too many portable drives with more than 5tb of storage. Desktop drives are 3 1/2" and so have more memory, but require a separate power supply and are bigger and heavier. I'm wondering what others do in the field and at home considering the use of Lightroom and frequent travel. Thanks for your help.
 
I carry three Samsung T7 2TB drives when I travel, along with my laptop. After every day‘s shoot, I load the images onto my laptop, then copy them onto each of the three SSDs. One drive I keep on my person, another I keep at the hotel/inn/lodge, and the third stays in my car (rental or my own). This ensures that I always have a full library of trip images, in case there’s a hardware failure or theft. Once home, everything gets backed up to my two WD 5 TB drives, then deleted from my laptop. At some point I’ll upgrade my external drives to larger OWC or Synology units, and/or do some serious culling from my library. I’m no Tom Mangelsen or Art Wolfe, so I figure the world won’t end if some of my images die with me. 🥴
 
I think there are two different issues that need to be dealt with. One is for travel and the other is for home storage. For home storage I think you should go with large capacity, 3&1/2 inch HD drives. They are fast and durable and cheap. Not as fast as SSDs but for storage they work great. Since you have already max out a 5 tb drive I would look into something like a 12 tb drive. I would get an enclosure that holds at least 2 drives. OWC has many enclosure choices and you can configure them as separate HDs or as a RAD. OWC specializes in Mac stuff.
For travel the 2Tb SSDs would be my choice as well and there are many options out there. Samsung has some great portable SSDs.
 
For travel I would encourage you to consider SSD vs HDD. SSD are a lot more robust and some like the Samsung T7 shield have an extra rubber bumper for added protection against small drops. While HDD are [considerably] cheaper they are (1) slower and (2) more likely to fail, especially if dropped. Samsung T7 2TB shields were slightly less than $200 apiece at B&H (short time sale). IMO these are a very safe investment.

Over the last few weeks prices have dropped but capacity in the mainstream selection has more or less been static.
 
You have a couple of options as I see it. If you want to have a main catalog that stays at home, then you could create a travel catalog for rod use and then merge it with the main catalog when you get back home. Rinse and repeat as often as you travel.

If you want access to your main catalog while traveling, but do not want to bring all of your original image files with you, then you could put your main catalog on a small portable (SSD) drive and bring it with you everywhere. Make sure you save Smart Previews of the images that you import so you can view and use them as a kind of proxy file if you want to do editing. This strategy involves a separate drive from the drive with the catalog that will store your original image files and remain at home. The issue with this approach is how you move and store your newly acquired images from a trip when you get back home. Assuming that you imported them into the catalog, you need to let LR Classic know that you moved their location from the travel drive to the permanent "home" drive with all of the other original image files that are been imported in the catalog.

There are variations on these two approaches if they have any appeal to you. But in either event, if you are using LR Classic, you need to make sure you understand how LR imports images and how to move them so LR does not "lose" their locations if you move them. This is how people get into trouble with LR Classic.

Good luck,

--Ken
 
Every year or so export the photos you took as a catalog on a large external drive (or drives) at home that is backed up offsite (backblaze). Your working catalog will remain a manageable size to ensure you can use an SSD for speed, While making sure only a small percentage of your data is at risk in the field at any one time.
 
Here is a good video on managing a travel catalog. (I can't remember exactly where I found it!)

 
My primary LR catalog is at home (currently close to 8TB). The photos are spread across two drives (internal HD and External HD), I keep the current year's photos on the internal and all previous years on the external. The actual catalog files live on my internal drive.

When I travel I bring a laptop and use LR as well. I'll process interesting photos while traveling and when I get home I export all the photos "export as catalog" than import that back into my home LR which merges everything together. I do backup when traveling as well, keeping photos on multiple external USB drives in case something happens.

Not sure if that helps, hope it does.



John
 
I keep one lightroom catalog for all photos on the computer drive, the ssd one so far. I store raw files one year per folder and keep only the current year on my regular computer drive. I keep edited "keepers" in layered tiffs in the same folder as the original raw and sync the folder in Lightroom once in a while to ensure it appears in the library next to the raw. Most of the time Lightroom makes this step unnecessary. Previous years get moved off the main drive but remain linked to the lightroom catalog , so storage is basically infinite. I backup everything of course.
 
When traveling, as I was recently, each evening I download to a 5T LACIE external drive. This becomes my backup as I, fortunately, have never had to erase a card to reuse. Sometimes I do process images, and I use LrC, as with my recent trip to Africa, and I do this using what I call a "travel" catalog. When I return home, if all images have been downloaded to the external drive, I use that to import the images into LrC. The way I use the export as catalog/import as catalog function is: I download all images first to the desktop computer LrC catalog, then I go back to the travel catalog, filter the images that were processed or marked, and download that metadata only. I import the metadata to the desktop computer catalog and it automatically syncs up with the already loaded images. This will only work if you do not change any file names. At home I have several hard drives, 8T or 10T to store images. When I've rechecked each card using the desktop catalog, to ensure that I have not forgotten to download something, I then erase all images from the travel drive and erase the travel catalog, thus a clean start for the next trip.You have to find a system that works for you personally. Having a set system and using it each time will keep you organized.
 
Thank you all for sharing your strategies. It seems like the way to go is to have a travel drive for that trip's photos and then import into the main LR catalogue when I get home. On trips, I do culling, but not much editing, since my laptop screen even though 16", is too small for details for these old eyes. I've been avoiding the importing step and possibly screwing up my main catalogue by carrying my primary storage drive, and on longer months-long trips a backup drive with me, but that risks too much. I've been lucky so far. I think I'll get a 10tb desktop drive for home and repurpose my 5tb WD or an older and smaller LaCie for travel. If you see any holes in this thinking please let me know. Again thank you. It's much appreciated.
 
When my 6TB internal 3.5" started getting full after only a year or two of filling up the older 4TB, I knew that I had to stop just buying a bigger hard drive (an 8TB would just fill up in a year or two).

I moved to a NAS. My Synology is now 8 x 8TB drives in Raid 6. The ~35TB capacity (1 drive is a hot spare for those doing the math) is currently about ~20TB full.

You could start with just 4 x 8TB. RAID 6 would use 2 of them for parity (so you would have 16TB of capacity to start), but that's it—all drives you add after that goes towards your capacity.

Backblaze would back that up, and you could also pick up an external USB 16TB for not too much for local backup.

Chris

Edit for clarity: I forgot to mention the main benefit for those unfamiliar with NASs: when your NAS fills up, you simply add another drive.

For example, if you buy a 6-bay NAS and put 4 x 8TB (as mentioned above, in RAID 6), when the 16TB capacity starts to get full, you simply add another 8TB drive and your capacity expands to 24TB automatically (with a little bit of time for the NAS to do some maintenance). You can add one more later for 32TB.
 
Last edited:
Thanks for your thoughts. It took me 6 years to fill up the 5tb. I’m pretty ruthless in the culling department.
 
Thanks for your thoughts. It took me 6 years to fill up the 5tb. I’m pretty ruthless in the culling department.
Good point, but how confident are you that you will use up storage at the same rate? What were the file sizes 6 years ago?

But it's also true that with storage being very large, you could get a single 12-14TB for probably less than you paid for the 5TB 6 years ago and be good with that for a very long time. I would make it a 3.5" internal drive for reasons of performance.

Chris
 
Now that I’m shooting a Z9 at sometimes 20 fps not confident at all. I’m determined to be even more ruthless. I’m probably going to go to a 10-12T. The manner of storage and/or the ports will probably change by then anyway.
 
I take enough memory cards to last for the duration of the trip. When I get back home I upload the files to a NAS 5-drive in RAID level 6 configuration that has a 20TB capacity. Once a month I backup the primary NAS to a backup NAS.

An alternative to buying more memory cards for me would be buying a RAID drive from LaCie


Setting this up for RAID 1 all the data is written to both drives (so only 2TB available) and if one drive fails all the data is intact on the second internal drive. At a price of $250 it would be my first alternative choice.

I prefer not to spend my nights when traveling hunched over my laptop and moving data from cards to an external storage device. It also requires a laptop with two USB C ports which means a larger Windows laptop as my MacPro and Window netbooks do not have two such ports. With XQD and CFexpress memory cards I need a port to attach an external card reader.

This LaCie also has a SD card slot and so for those using SD cards it is an added convenience as many laptops lack a built-in reader.
 
Last edited:
For the road, I go without laptop and use the Western Digital Passport Pro Wireless SSD drives. Just set it copying when you're at base and go eat dinner / to sleep.

These batteries on these drives will last 8 hours (well, all day since you don't turn them on until you want to copy), so you can backup when you fill up cards in the middle of the day (still without laptop). I use 2 so I have redundant copies and can then immediately format the card in the camera.

With a second set of cards, you can continue to shoot while the copy happens. And since it's SSD, it can happen with a bit of jostling going on (driving down the road).

Chris

PS: unfortunately, WD discontinued these, so I will baby mine. You might still find some, but they'll be pricey. Hopefully, someone will come out with another like product in the future, or maybe there is (SSD drive, internal battery, built in OS, copies as soon as you plug the storage card in).
 
First of all, do you/why do you need to keep that many images? I have 3 Samsung 2TB T7s, one is for the current catalog, one for "keepers" that operates on one in - one out, and one for travel (that doesn't have to be 2TB but more than you need beats less than you need every time).

Out of curiosity, in the original post you say you have a 2021 Mac Powerbook ? ? ?
 
I have downloaded the free 'Blackmagicdesign Disk Speed Test'. Here are some read/write results. I used a OWC Thunderbolt 4 cable (it does matter).
Macbook Pro M1 13" 1TB 2920
OWC Envoy Pro SX 2TB 950/1800
OWC Envoy Pro EX 1TB 450/807
OWC Mercury Elite 12TB 333
LaCie hard disk 4TB 128

I first took the LaCie 4TB portable hard disk on a trip with the 13" laptop. When I tried to backup from my camera to this disk, my computer said it would take 3+ hours! :( I eventually bought the EX then the SX. In retrospect, I should bought the laptop with 2TB SSD even though it was extremely expensive. You get what you pay for.

On a trip I like to copy pics from my camera to the M1 SSD, then back them to the SX. I might have to format the cards for the next day.

When I get home I copy the files to the (fastest) M1 SSD and cull/edit them (Lightroom). When finished I move them to the 12TB hard disk array.
 
Last edited:
For those of us running macOS…DropBox has apparently declared bankruptcy and is no longer going to support either the upcoming Ventura or future versions…good thing I don’t have a Drobo.
 
I carry three Samsung T7 2TB drives when I travel, along with my laptop. After every day‘s shoot, I load the images onto my laptop, then copy them onto each of the three SSDs. One drive I keep on my person, another I keep at the hotel/inn/lodge, and the third stays in my car (rental or my own). This ensures that I always have a full library of trip images, in case there’s a hardware failure or theft. Once home, everything gets backed up to my two WD 5 TB drives, then deleted from my laptop. At some point I’ll upgrade my external drives to larger OWC or Synology units, and/or do some serious culling from my library. I’m no Tom Mangelsen or Art Wolfe, so I figure the world won’t end if some of my images die with me. 🥴
I seldom travel far from home anymore but when I did or do the 3 samsung ssd's and using carbon copy cloner to make and exact copy from one to the other two is what I do. I must keep far fewer images than many. I have an OWC Envoy Pro SX w/thunderbolt OWCTB3ENVPSX04 4TB and only have 1.21 TB on it :) So so far my 4 back up drives that I clone to do not have to be more than 2TB but moving larger 1 is now 4TB.
 
We all have different needs, workflows, and philosophies.

I sometimes shoot brackets or stacks (usually 5 frames per final image, but sometimes more with astro). I shoot time-lapse photography (hundreds or even thousands of images that produce no more than a minute or two of time-lapse video). I do high resolution, multi-row panos. Some of us shoot actual video.

I do an initial cull of obvious bad shots (poor focus, lens cap on, unintentional triggers, bad sequences, etc.). After, I mark keepers as part of my workflow, which I can filter on later so I'm only looking at those. But I don't trash the non-keepers beyond the initial cull. I have revisited previous shoots and found something I didn't see before.

Then there's the work images (Photoshop documents). I don't keep intermediates or social media JPGs, but the hard post work is saved in a PSD/PSB files and many of those get quite large (in the GB range).

I now shoot with a 100MP medium format camera in addition to the FF gear.

Storage is cheap compared to when I first started shooting digital in the previous millennium, when I could squeeze it all on a 120GB drive. And it's now much cheaper per GB/TB than it was just a few years ago.

Chris
 
We all have different needs, workflows, and philosophies.

I sometimes shoot brackets or stacks (usually 5 frames per final image, but sometimes more with astro). I shoot time-lapse photography (hundreds or even thousands of images that produce no more than a minute or two of time-lapse video). I do high resolution, multi-row panos. Some of us shoot actual video.

I do an initial cull of obvious bad shots (poor focus, lens cap on, unintentional triggers, bad sequences, etc.). After, I mark keepers as part of my workflow, which I can filter on later so I'm only looking at those. But I don't trash the non-keepers beyond the initial cull. I have revisited previous shoots and found something I didn't see before.

Then there's the work images (Photoshop documents). I don't keep intermediates or social media JPGs, but the hard post work is saved in a PSD/PSB files and many of those get quite large (in the GB range).

I now shoot with a 100MP medium format camera in addition to the FF gear.

Storage is cheap compared to when I first started shooting digital in the previous millennium, when I could squeeze it all on a 120GB drive. And it's now much cheaper per GB/TB than it was just a few years ago.

Chris
Yup it is cool how broad a range of interests/preferences photography encompasses and I get to enjoy seeing the results from many different photographers.

Since I do not use PS and do not do time lapse, almost no video, no stitched panos etc. that keeps my storage needs much smaller.

I have a friend who now lives in the Canyon Lands in Utah and his photography is amazing he shoots a lot of night skies, panos, composites etc. etc. and his storage needs dwarf mine and he has far more patience than I do.
 
Back
Top