Cloud storage - thoughts on

If you would like to post, you'll need to register. Note that if you have a BCG store account, you'll need a new, separate account here (we keep the two sites separate for security purposes).

P

Pam Jensen

Member
Member's gallery
To all you photographers with amazing beautiful photos: What are your thoughts and experiences with cloud storage companies (specifically for large RAW files)?
-Ease of storage,
-ease of retrieval,
-cost, etc….
 
Member's gallery
Although I use a cloud file sharing service I would never store my images on the cloud but use external hard drives instead. Once an image is on the cloud you lose control over it -- if there's a fire at the server farm, or sabotage, or ... there go your photographs.
 
Member's gallery
I have a NAS at home that backs up to a cloud storage - quite sure my home and the cloud don't burn at same time.
As I have 1 Gbit/s internet connection transfer speed to the cloud could be fine, but I'm sure latency will kill the fun, if I was to work directly on the cloud storage with images. For my office job I use One-drive it works reasonable, but sometimes presentations runs to 100 Mb and then they load slowly.
 
Member's gallery
I researched this a while back, but never moved forward with it. Thought of uploading around 4 TB of images seemed a bit daunting. I did find a provider with reasonable rates and good reviews, BackBlaze . May revisit this in the future. Currently run an OWC dual drive backup, and weekly back up my pics to two other "MyBook" style drives which I keep elsewhere.
 
Member's gallery
If you care about your photos you should have two or more backups. One of those should be in a different location be it cloud or otherwise.

I use my website for storage. The cost is a little higher than BackBlaze, but it hosts my website, databases, mailing lists, and mail in addition to general storage. The primary limitation is the speed and data caps on my internet connection. So only limited files get uploaded.

I keep three backups on external drives. One drive is stored in a locked cabinet at work. The one at work is swapped periodically with one of the drives at home. It is possible a major disaster could wipe out both my home office and my work office, but the odds are low enough I'm not worried about it.
 
Member's gallery
BackBlaze is a good provider and I have several friends using their cloud service. BackBlaze would not fully replace external storage. Cloud access to individual files is okay, but for folders and large volumes of big files, it's a bit slow. But it's getting a lot better, and I think it is at the point where it's a reasonable part of a solution.

I am currently using onsite external drives for my primary storage as well as backup storage. I have a third copy stored offsite. Collectively, each copy is about 11 TB of images stored on a pair of 8 TB drives.

I have one additional copy of "hero" images stored on the cloud with Amazon Photos - which is free.

I have a large set of business images in large JPEG format stored at my Zenfolio website. These can be downloaded if needed, but are not the raw files.

I would not count on a large volume of free cloud storage for my images. There is a history of free cloud sites changing terms and giving a very short time to download images. Because cloud storage can be slow, it's hard to exit a cloud provider within 30 days. But if you pay for storage, it's a case of getting what you pay for. As you increase the amount you are willing to pay, speed improves and the business is more viable long term.

As we look forward, I expect to be using cloud storage in the future. It's inherently cheaper and safer if configured properly. Amazon Web Services is the ultimate provider for a number of cloud providers. Adobe is a bit expensive as a storage provider, but the ability to put images, edits and your catalog in the cloud is a nice solution. Adobe is certainly heading that direction.
 
Member's gallery
I use a combination of external hard drives and cloud storage.

  • Primary storage (for my images) is an OWC thunderbay drive w 10 TB drives run as a raid system using Softraid in Raid 5 configuation.
  • I have a 2nd copy of the above unplugged in the house. I connect it to my computer weekly (or more frequently) to backup using Carbon Copy Cloner. Quick backup, always available but at risk if the something happens to the house (fire, theft).
  • I keep a third copy set of disks in the bank vault which, until Covid, I backed up monthly or after every trip. Post Covid backup has been less frequent but recently I have been to the bank more often. Easy to restore, not 100% up to date but generally okay.
  • I have a final copy in Backblaze which is automatically uploaded. Has everything, images and other files. So far I have only needed to restore a few other files. But for $60/70 a year for close to 20TB, a great deal.

3-2-1 strategy (actually I have 4-3/2-1 strategy).

1 Primary copy
2 formats (cloud and Raid 5 system)
3 locations - home, bank vault, cloud
4 copies - 2 at home (1 primary and 1 quickly accessible), 1 in bank vault, 1 in the cloud

I SWAN (Sleep Well At Night) and don't worry about the safety of images or files.
 
Member's gallery
Pam Jensen said:
To all you photographers with amazing beautiful photos: What are your thoughts and experiences with cloud storage companies (specifically for large RAW files)?
-Ease of storage,
-ease of retrieval,
-cost, etc….
Click to expand...
I don't use cloud storage, except occasionally when I want to share casual photos with others via Google Drive. There are a number of good, dependable cloud storage vendors available, and they have built-in redundancy and security, which protect your assets. The primary reason I don't use cloud storage myself is because of my internet speed - while it's fine for web use and streaming movies, it just doesn't cut it for uploading massive image files. Instead, I use external hard drives for backups.
 
Member's gallery
I've been using iDrive backup services for years for all my personal files and upgraded it now to a 5TB, and been backing up my photos also.
Since they are a backup service, not necessarily a sharing service, prices are friendly.

Always have all my files backed up twice, on an external drive and cloud.
There's a saying we should have files backed up three times being the third one also an external drive but offsite.
 
Member's gallery
One of the big issues with cloud backup services is related to files or drives that are not regularly connected. For example, Backblaze requires external drives to be connected regularly for at least four hours. If the external drive is not connected for 30 consecutive days, those files are automatically deleted. It's assumed you deleted the files on purpose. You can retain files longer for an additional fee, but that's the kind of risk that you need to be be very careful to avoid. Backblaze is also just a backup - you still need to maintain files on your computer, external drive or server. This is true for files on your computer or other places as well.

Other cloud providers may have similar provisions, but some providers simply keep all files indefinitely. You need both long term retention and regular backups.
 
Member's gallery
The biggest issue with cloud storage is really the size of the files you will be uploading. Most ISPs have a much slower upload speed than download speed so it can be a time consuming process. For example, a common internet speed in the US is 100Mbps download 5Mbps upload. This would mean it would, if you were able to utilize full speed, it would take you 20x longer to upload than download each file. You will also want to check the privacy policy of chosen provider, see what you can find out about their backup process, downtime, and any other information that would be important to you. Cloud is a good way to keep a copy of photos offsite in case something unfortunate happens to your main storage location.

I store photos on Amazon Photos which is included in Prime, but very reasonably priced outside of Prime. It works well and is easy to retrieve photos. I keep a copy on a local drive and one backed up to the cloud.
 
Member's gallery
Yeah, I looked at Amazon photos also, but does not accept raw files :(
It would be good for a second cloud backup for "free" since it's included in Amazon Prime.


abc123brian said:
I store photos on Amazon Photos which is included in Prime, but very reasonably priced outside of Prime. It works well and is easy to retrieve photos. I keep a copy on a local drive and one backed up to the cloud.
Click to expand...
 
Member's gallery
Cloud storage works well as near-line backup, but not at all as working storage, due mostly to the bandwidth/latency limitations of the internet. There are various strategies to deal with this, but most bulk cloud storage providers have a cost differential between upload and download (your data uploads but it a lot slower out) that's separate from the speeds you get from your ISP.

Basically it's a good part of a data protection strategy. But photo processing is highly sensitive to storage speeds and even an M2 SSD is sometimes not fast enough. There's a hierarchy of storage speeds. 1) M2 2) onboard 3) offboard thunderbolt/usb-c 4) off board SATA 5) local network storage 6) internet/cloud. Generally you want the fastest for working and then a schedule/protocol for what gets backed up where and when. E.g. work storage backed up close daily, backup then backed up elsewhere weekly, and then that backup backed up somewhere else bi-weekly/monthly or what you feel comfortable with. The last one might be off site. At least, this triple backup is what I feel comfortabnle with.

As much as Adobe wants all your photos on their cloud (Lightroom CC) it has serious limits. :)
 
Member's gallery
EricBowles said:
One of the big issues with cloud backup services is related to files or drives that are not regularly connected. For example, Backblaze requires external drives to be connected regularly for at least four hours. If the external drive is not connected for 30 consecutive days, those files are automatically deleted. It's assumed you deleted the files on purpose. You can retain files longer for an additional fee, but that's the kind of risk that you need to be be very careful to avoid. Backblaze is also just a backup - you still need to maintain files on your computer, external drive or server. This is true for files on your computer or other places as well.

Other cloud providers may have similar provisions, but some providers simply keep all files indefinitely. You need both long term retention and regular backups.
Click to expand...
True. I keep my photo raid system connected all the time. My wife, who has her own backblaze license, received frequent warnings to backblaze that her disk not connected.

Backblaze is my backup of last resort. Just in case all else fails ...
 
Member's gallery
Yes, you guys are right, I can upload NEF files.
I figured out what happened the first time I tested this. In the middle of the NEF files, there was a PSD file, that one was the first to upload and since failed, all other files ended up failing also, and I assumed only JPEGS were accepted since they were the only ones uploaded.

Thanks. Now I can have a second cloud backup for my photos and this one is basically free, which is always welcomed these days.

abc123brian said:
I have mostly RAW files uploaded. It should work for you.
Click to expand...
 
Member's gallery
Sorry, started this several days ago but didn't have time to finish, others have probably provided the same information.

The biggest limiting factor for cloud storage is internet speed, both upload and download speeds. I live in a rural area and have 300Mbs of download speed but only 20Mbs of upload speed. If you have a fiber connection you could possibly have 1Gbs of both upload and download speed. The speed limitations generally prevent you from experiencing the same file access speeds you would with a locally attached drive. To upload 100GB at 20Mbs takes about 12 hours, at 1Gbs it takes about 15 minutes. If you have a 1Gbs network connection in your home you are only sharing speed with your home users, but with a 1Gbs internet connection it is difficult to maintain that level of throughput because of the number of bandwidth users.

I use BackBlaze as one of my backup options and have over 5TB stored in the cloud. I do like having a copy of my images and documents stored remotely, not just for backup security but I can access them remotely if necessary. The BackBlaze backup utility runs in the background and I can configure when it runs and how much of my internet bandwidth it will use. If I upload a large number of images and videos onto my computer from a trip it may take several days to a week for all of them to get uploaded. Designed as a backup service it will retain files that you delete from your backed up local drive for 30 days to a year depending on your plan. It is not designed to be used as a storage or archive service. It is also computer specific, so each computer needs to have a BackBlaze account. It is something I configure and monitor and on rare occasion downloaded a file I deleted or needed remotely but it is basically just a backup. My unlimited BackBlaze storage with a 1 year retention is about $80 a year.

I use Microsoft OneDrive for some of my personal files and work files. OneDrive offers 1TB of storage and acts as a syncing service and will sync the files stored on you local OneDrive folder to the cloud. OneDrive folders can be shared with other OneDrive users. I use it for all of my work files so I can work from home and at my office seamlessly. If I open a file at work and change it the file is saved locally and then synced to the cloud, when I access the file at home if the file on my computer is older than the one in the cloud it will download it and the changes will be saved locally and then synced back to the cloud. It also have versioning so you can access previous versions of files. I do use it for a lot of Photoshop work files but not any of my Photoshop image files. If you have Office 365 you have 1TB of storage free.

I believe that most all of the cloud based storage solutions do behave like OneDrive. They create a folder that resides on your local drive and then syncs to the cloud. This allow access to the files stored locally when internet access is unavailable but provides security of a copy in the cloud and the ability to access the file remotely. They usually have some type of application that is installed and that runs in the background to manage the file upload.

I am not aware of any cloud storage options that act like a network accessible drive that is just mapped as a local drive on your computer and the files reside only on the network storage device but they may be out there.
 
Member's gallery
There are options in Dropbox (which is what I use) to set it to not store images locally. Everything gets pushed up to the cloud. Some of my laptops don't have the local storage to shadow a couple of terabytes. I would presume many of the others have the same option buried somewhere.
 
Member's gallery
eaj101 said:
There are options in Dropbox (which is what I use) to set it to not store images locally. Everything gets pushed up to the cloud. Some of my laptops don't have the local storage to shadow a couple of terabytes. I would presume many of the others have the same option buried somewhere.
Click to expand...

Yes, you are correct. OneDrive and iCloud drive both have options to optimize local disk space and you can also choose to have certain files stored locally. I just notice that when opening a file on a cloud drive the process is a background download locally, open, then save locally, and background upload to the cloud. It is just a different process than using a network attached drive.
 
Member's gallery
Patrick M said:
Given the cost and size limitations of cloud storage is one reason I went with Synology. its essentially my own cloud as I can access it from anywhere if I have internet connectivity.
the drives are hot swappable, so I can easily get bigger if need be, and I can target other web based cloud storage like google, should I feel the need to back up off line. As the Synology disc station is in my home network, I have instant access. In ten years I have two disc stations, since my first just got out of date, but it serves as a weekly back up non the less. Synology continues to update their DSM (Disk Management System) and their latest relest, version 7 is a dream to use.
Click to expand...
Another recommendation for Synology if you decide to go with your own NAS. Well built, good software.
 
Member's gallery
In Australia the internet mostly only covers a couple of hundred kilometer from the cities.
So I dont use it unless I really have to.
At least 3 backups in the field and a NAS back in the studio...
 
Member's gallery
Abinoone said:
I don't use cloud storage, except occasionally when I want to share casual photos with others via Google Drive. There are a number of good, dependable cloud storage vendors available, and they have built-in redundancy and security, which protect your assets. The primary reason I don't use cloud storage myself is because of my internet speed - while it's fine for web use and streaming movies, it just doesn't cut it for uploading massive image files. Instead, I use external hard drives for backups.
Click to expand...


I have a couple of the Big Silver Mac Pro desk top computers, each computer has a direct hard drive slot to the mother board, 4 in each so I can store 8 hard drives of what ever size or type I want be it 1 TB or 6 TB each, I use enterprise server grade only.
Each hard drive is formatted to X fat so it can be plugged in and play on any computer PC or Mac lap top or anything that has a USB port, basically they are all like giant USB sticks.
I down load the selected keeper only original files, with in the folder there is another folder with edited files. Back up is easy, I load the files to two different hard drives so I have a back up, its unlikely a pair of hard drives will fail at the same time.
They say what about a Fire, well mate I can tell you the last thing on my mind will be photos'.
What about a break in, who wants to take two out of date Monster Desk tops worth about $250 each.

The most valued thing in my house the mattress I sleep on, its packed with $100 bills LOL.

For a full time professional business its a different story...........

You can buy a little desktop hard drive reader, you can take any hard drive and slot it into the reader like a Disk, instant viewing......

The Mac pros are simply a storage device, if I want to work on some images I simply take a copy and do with what I wish.

Cheap super reliable super simple, I can just pull a drive pop in a new one copy and paste done.
Filing is simple I use the blue folders, Birds are all in bird file with sub files, dogs in a dog file with a sub file etc etc so simple and I am not defendant on systems or software or updates.

I can store up to 24 TB per tower directly slotted onto the mother board...........if a tower dies yet to ever happen, $300 gets you another one where X fat works.
You can use any spare PC tower but I find the mac Pros are bullet proof.

Only an opinion

OZ down Under


Mac Pro.jpg
You can only see EXIF info for this image if you are logged in.
 
Member's gallery
I have 1TB with Adobe, I absolutely love it. Easily expandable. Backup that instantly works in the background is important to me.

I always shoot JPG + RAW, and I'm ruthless when it comes to culling RAW images in LR - this approach helps to not tax my cloud storage space. That said, my RAW workflow has a caveat - before I even import RAW images into LR to start culling, I copy the entire session's images to an external hard drive, dedicated only to archiving RAW images that I might want to pull later (and frequently do). In LR I end up keeping more JPG's (casual family snapshots & similar) than RAW.

I also back up my LR catalog weekly (or immediately after an important shoot) on multiple hard drives, rotated in multiple locations.

It sounds like a lot of work here in writing, but honestly it's a really simple process that has me well-covered.
 
  • Like
Reactions: Roy
You must log in or register to reply here.
Back
Top