Do you use RAID for the disks in your Thunderbay? Or do you simply use them as independent disks in an enclosure?
Back-up suggestions
- Thread starter rlitschgy
- Start date
If you would like to post, you'll need to register. Note that if you have a BCG store account, you'll need a new, separate account here (we keep the two sites separate for security purposes).
It's setup for RAID.
Thanks.
SMB
Well-known member
Have used a Drobo for over ten years through several iterations and have never had a problem upgrading. It is true that they have a proprietary system that would be a problem if they went out of business but at the moment that doesn’t seem likely. Besides one should always have at least a second backup set so if you’re that concerned it can be another brand. I will say their service is excellent especially their tech folks (although getting through can be an issue). I recommend their warranty program.
I have had two 5D models and now an 8D and have moved drives without issue.
Niels.hr.Hansen
Well-known member
You're quite right. If disaster strikes.
I as a hobbyist lose my photos, I can curse, swear and cry. It's about that. You might have other data on your system that will cost you some money, but likely you'll survive.
As a pro making a living from your photos, you will have a severe financial hit and even go bankrupt.
So if your a pro, be a pro with your data. And here it's just like the difference between aps-c and FF. Being a Pro requires bigger guns.
You are very welcome. After I getting so much input here in the forum myself, it is nothing but fair to - at least try - to give a bit of it back. Why should we all reinvent the wheel for ourselves if others have found a dozen ways to solve your problem and you just have to choose one, tweak it for your needs and make it no. 13 for the next person having the same problem.
Not A Speck Of Cereal
Well-known member
The trick is to expect drives to fail and put that as a requirement in your backup scheme, whatever it is. "The backup system should be robust enough to experience drive failure and recover well, quickly, with NO data loss".
Getting a different external hard drive that you believe is more durable than the WD drive you have now is not that. Because it's still a drive that can fail. (Edit: I didn't know what a Drobo was when I typed that--it could be more robust than a single WD drive.)
For me, that means a Synology NAS in RAID6, using a file system that recovers from bad sectors, bit rot, and the Synology OS that allows rebuilding of data on drive failure. I keep a hot swap drive in the box so it will recover immediately, but it will also just single out the bad drive and tell you to replace it. No data loss. RAID5 allows 1 drive failure at once, RAID6 allows 2 at once. The number and size of the drives depends on your needs, but if you buy a box that has more drive bays than you need right now, then you will be able to expand in the future. (These also allow plug-in expansion boxes as well.)
But this same philosophy (the requirement above) could be applied to much simpler systems, no RAID, including a bunch of external USB drives (or mix of internal and external). They're more trouble and less robust than a good NAS, but if you have enough of them (main drive plus TWO back up drives of the same size), then that will do. And they're cheap. When (not if) one goes tits-up on you, you can get another one cheaply to take it's place in a day or two. (It takes longer to get it replaced under warranty, but of course, do that too.)
If you're a recreational shooter, do the best you can. But if this is your livelihood (or you want the best), you should probably look into a good RAID system.
Chris
Getting a different external hard drive that you believe is more durable than the WD drive you have now is not that. Because it's still a drive that can fail. (Edit: I didn't know what a Drobo was when I typed that--it could be more robust than a single WD drive.)
For me, that means a Synology NAS in RAID6, using a file system that recovers from bad sectors, bit rot, and the Synology OS that allows rebuilding of data on drive failure. I keep a hot swap drive in the box so it will recover immediately, but it will also just single out the bad drive and tell you to replace it. No data loss. RAID5 allows 1 drive failure at once, RAID6 allows 2 at once. The number and size of the drives depends on your needs, but if you buy a box that has more drive bays than you need right now, then you will be able to expand in the future. (These also allow plug-in expansion boxes as well.)
But this same philosophy (the requirement above) could be applied to much simpler systems, no RAID, including a bunch of external USB drives (or mix of internal and external). They're more trouble and less robust than a good NAS, but if you have enough of them (main drive plus TWO back up drives of the same size), then that will do. And they're cheap. When (not if) one goes tits-up on you, you can get another one cheaply to take it's place in a day or two. (It takes longer to get it replaced under warranty, but of course, do that too.)
If you're a recreational shooter, do the best you can. But if this is your livelihood (or you want the best), you should probably look into a good RAID system.
Chris
JANuser
Active member
You can achieve the same thing with DAS, in RAID, and not have to deal with the network overhead of a NAS. Unless you have multiple computers that need to access those files a NAS is needlessly expensive.
Not A Speck Of Cereal
Well-known member
Yeah, and it could be a lot faster than a NAS. I ended up having to install a 10gbe network to make my NAS fast, but a DAS would be fast at the get-go. I sometimes wish I had gone that route.
Chris
JANuser
Active member
I have a DAS and it works well, but it is older an limited to 3TB (x2) drives, so that will be a problem going forward. Another solution I'm looking at is turning an old computer into a file server, in my case an older Mac Mini with two drive bays. The only downside to that solution is that it is limited to 2.5" drives, but if you have an older tower that can store 2+ 3.5" drives it is worth considering.
Of course the advantage of a NAS or DAS can be that the make less noise and use less power, but if you already have an older machine that can accept multiple drives, it's one possible solution.
Of course the advantage of a NAS or DAS can be that the make less noise and use less power, but if you already have an older machine that can accept multiple drives, it's one possible solution.
Not A Speck Of Cereal
Well-known member
Other advantages: a) having an NAS/DAS operating that is dedicated / designed to the task and does nothing else; b) hardware RAID over software (I learned my lessons on software RAID); c) if you're thinking of using an old computer for this with no RAID, then of course that's a huge difference (disadvantage).
Calson
Well-known member
All drives, hard discs and SSD, will fail at some point in time. The bare minimum is to have a drive enclosure that holds 2 drives and the data is mirrored and identical on both drives. If one fails you still have the other drive to use and when the failed drive is replace the NAS will copy data from the working drive to the new one. With drives so cheap it makes no sense not to do at least this for storing image files.
I have a primary NAS and back it up monthly to a second NAS and that NAS is kept at a neighbor's house. If a fire burns my house to the ground or thieves break into my house while we are away I do not lose everthing. Backing up to a remote server makes sense except that I have seen many of these companies bite the dust and did so with less than 48 hours notice to customers and Google and Amazon have changed their cloud storage practices as well with very little notice to customer and it would take a very long time to retrieve everything from an existing cloud server and then upload it to a different one.
Internet access for friends and family means internet access for hackers and it is not something I would chance to gain some convenience. We have two managed servers and will at times FTP files to folder that has limited permissions so that another person can download the files from that single folder. And we can kill that user ID after they have the files.
I have a primary NAS and back it up monthly to a second NAS and that NAS is kept at a neighbor's house. If a fire burns my house to the ground or thieves break into my house while we are away I do not lose everthing. Backing up to a remote server makes sense except that I have seen many of these companies bite the dust and did so with less than 48 hours notice to customers and Google and Amazon have changed their cloud storage practices as well with very little notice to customer and it would take a very long time to retrieve everything from an existing cloud server and then upload it to a different one.
Internet access for friends and family means internet access for hackers and it is not something I would chance to gain some convenience. We have two managed servers and will at times FTP files to folder that has limited permissions so that another person can download the files from that single folder. And we can kill that user ID after they have the files.
Published information indicates SSDs in typical use can be expected to last over 300 years. That's more than long enough for my use.
JANuser
Active member
SSD's can still fail in far less time than that. Sure the physical silicon might not degrade, but that does not mean it will work as a unit. The physical electrical connections can fail, and worse yet the controller chip, or a flash chip that make the drive work can fail. If the controller on the drive fails you will have no way to gain access to the data, since the data is divided between numerous flash chips (2-8 depending on the drive size typically) that make up the drive. This would be an even worse scenario on a drive that has built in 256bit encryption (which most non-budget drives have), since each controller will provide a unique key. If that controller dies, you are hooped. Sure you could try to decrypt it, but it might take a few hundred years.
Another factor, SSD's are not good for long term data storage, since data starts to degrade quickly if the drive is not powered on. Data rot on an SSD starts within 6 months, or less, of the loss of power. An HDD is a much better option for archiving and long term backup, since data rot doesn't start for years typically.
Another factor, SSD's are not good for long term data storage, since data starts to degrade quickly if the drive is not powered on. Data rot on an SSD starts within 6 months, or less, of the loss of power. An HDD is a much better option for archiving and long term backup, since data rot doesn't start for years typically.
Not A Speck Of Cereal
Well-known member
Also, I wouldn't hang on that 300 year number. A quick google serves up the following: "Current estimates put the age limit for SSDs around 10 years, though the average SSD lifespan is shorter".
I suspect the 300 year number was an early published estimate.
Chris
I suspect the 300 year number was an early published estimate.
Chris
Anything can/will fail eventually. But based on past HDD history and previous professional test lab experience I would not trust an electro-mechanical device such as an HDD over a solid-state device once the SSD is past typical component infant mortality hours.
JANuser
Active member
Which is why you use a RAID array, never trust one device, of any kind, for your most important data. Another factor is cost. An 8TB HDD is a few hundred dollars, an 8TB SSD is a few thousand.
Not A Speck Of Cereal
Well-known member
SSDs will last longer than spinning discs, true enough. But how much longer? It looks like about twice. That's good, but not earth shattering great.
The main reason to use SSDs, for me, is the much great write/read performance.
But my NAS box has ~50TB worth of 8TB drives. It would be way too expensive for SSDs at this point (even the lowest grade SSD, which I wouldn't want).
So I use SSD for boot, scratch, LR catalog, and local edit, then store all the source files and edited documents on the NAS with spinning discs. And I have robust data protection to cover failed drives, bit rot, and bad sectors; and I back it all up, so I'm not concerned with losing data.
But at the end of the day, it's just a personal choice. There's nothing wrong with sticking with either or both. We should worry about comparisons if they all work for us.
Chris
The main reason to use SSDs, for me, is the much great write/read performance.
But my NAS box has ~50TB worth of 8TB drives. It would be way too expensive for SSDs at this point (even the lowest grade SSD, which I wouldn't want).
So I use SSD for boot, scratch, LR catalog, and local edit, then store all the source files and edited documents on the NAS with spinning discs. And I have robust data protection to cover failed drives, bit rot, and bad sectors; and I back it all up, so I'm not concerned with losing data.
But at the end of the day, it's just a personal choice. There's nothing wrong with sticking with either or both. We should worry about comparisons if they all work for us.
Chris
JANuser
Active member
Indeed. I figured since we were talking about backups, I would cover that subject matter. I also use plenty of SSD's, but never for backups.
We have a small 2-HDD RAID array NAS for data & image backups of 2 computers that has worked well for several years.
But as a hobbyist/enthusiast photographer I have no real need for dozens or hundreds of terabytes of image backup capacity that need expensive large SSDs or cheaper HDDs. YMMV. For our computer's storage needs I don't plan to ever purchase another HDD. If either/both of the 2 NAS drives fail that would be the only candidates for replacement with good quality HDD. Replaced a failing 1TB spinner in my PC recently with a Samsung 1TB SSD that was $10 more than a new 1TB HDD- a no brainer. Smaller SSD's can be affordable
This goes back to the difference in requirements between professional and hobbyist backups.
We've sent images for years to a couple of stock agencies and a POD site - but certainly am not a professional. If our image backups disappeared it would be disappointing but no real major loss to our lives. Never needed to go back into our image backups for anything so far. In fact probably need to go thru and delete a lot of old images. Most of the image files on our computers and backups will never see the light of day.
Pros that reply on their images to pay the mortgage and put food on the table have greatly different backup requirements than hobbyists/enthusiasts.
But as a hobbyist/enthusiast photographer I have no real need for dozens or hundreds of terabytes of image backup capacity that need expensive large SSDs or cheaper HDDs. YMMV. For our computer's storage needs I don't plan to ever purchase another HDD. If either/both of the 2 NAS drives fail that would be the only candidates for replacement with good quality HDD. Replaced a failing 1TB spinner in my PC recently with a Samsung 1TB SSD that was $10 more than a new 1TB HDD- a no brainer. Smaller SSD's can be affordable
This goes back to the difference in requirements between professional and hobbyist backups.
We've sent images for years to a couple of stock agencies and a POD site - but certainly am not a professional. If our image backups disappeared it would be disappointing but no real major loss to our lives. Never needed to go back into our image backups for anything so far. In fact probably need to go thru and delete a lot of old images. Most of the image files on our computers and backups will never see the light of day.
Pros that reply on their images to pay the mortgage and put food on the table have greatly different backup requirements than hobbyists/enthusiasts.
JANuser
Active member
I'm not a pro by any means, but the photos take up the amount of space they take up. What are you backing up? JPEGS or RAW files? Modern cameras have large files, so even if you only take a few thousand shots a year that can fill a 1TB drive quickly. Even more so if you shoot any video.
I backup both the RAW files and edits locally, I already have close to 3TB (that alone shows I'm not a pro), going back to 2004. I do that because as times goes on the editors get better and I often revisit some older photos.
I backup both the RAW files and edits locally, I already have close to 3TB (that alone shows I'm not a pro), going back to 2004. I do that because as times goes on the editors get better and I often revisit some older photos.
Not A Speck Of Cereal
Well-known member
Yes, hobbyists can own cameras with large sensors and take vast amounts of captures for time-lapse, multi-row panoramas, astro / focus / bracketing stacks, etc. A single 2 hour time-lapse sequence can be anywhere between 200-800 frames (50MB each), depending on the interval. (I've done larger.)
But it's probably also true many don't have such large storage requirements.
Chris
But it's probably also true many don't have such large storage requirements.
Chris
Niels.hr.Hansen
Well-known member
I backup my RAW and sidecars, JPEGs can be recreated from the 2 first.
rlitschgy
Supporting Member
Well here is the direction that I am considering. The ThunderBay 4 56 TB 4 Bay Thunderbolt 3 Raid 5 Array from OWC as my main storage for my Library & Computer. Short of a major disaster I hope this will keep my system active safe and handle my collection of images for years two come. If I lose my computer the greatest lose would be my programs which I could re-apply to the suppliers using my license to re-install them and both Lightroom & Photoshop are a monthly membership. The work that would be on the OWC unit although I would hate to lose it would not be my best work which is why I would only be relying upon the Raid array to keep it safe but don't have a back up plan for a disaster. As back up I have two 4 TB Western Digital My Book portable hard drives that I will use to back up my Edited Images, best work, and all my portfolio work which will be kept off-site. I'm considering OWC 14-Port Thunderbolt 3 Dock as I will always have two 4 TB WD My Book portable drives (which are also backed up on the same type of drives off-site), with my older work , connected to my computer using this dock as well for a disc reader/writer, a USB Lexar 3.0 Compact Flash card reader and a Spyder 5 Pro calibration unit when needed. I decided not to go ahead with the Drobo unit for reasons suggested by some members, and I did decide to go with the OWC unit for simplicity and believed that with the number of members who are presently using these units, that if or when I get into trouble someone may be kind enough to walk me through my issues. I hope it is reasonably simple to set-up and get it running, BUT experience has taught me (Like hard drives expect them to fail) I too expect things not to go as smoothly as I hope it will, it's just one of those things. I really am thankful for the help everyone has given me, if you could only see the notes I had to take which listed all the terms mentioned ( thank God for Google), I looked everything up. When I first started reading the response from all of you it was like you were all speaking a different language that I needed to learn. Although my education is far from complete on this issue I will say I have learnt much more than I ever thought I would need. Thank you all.
I've had a computer just die on me once and on another occasion had to get a new computer without any notice, so had to reinstall my programmes and it is a pain, but it can be done. After the first time I now keep a list of my programmes and have a software folder in my email to keep the various messages from the software house when I buy software in a subfolder - one for each software - and the license key will be in there so easy to C&P. I always add my emails in at an early stage when working with a new machine so when I start installing software I've got all of the info I need to hand. The folders in the software folder are also a back up for my software list.