Backup Photo Storage

[mikew09]

I remember this question being asked a few yrs back, but technology being what it is that is like a life time.

I am still living relatively in the past and using two external USB 3 x 2 Tb drives of which I manually sync once a day. I used to have a off site storage (internet) too but with 1.5 Tb of data this can become hard to achieve.

I am about to upgrade my external storage looking to something more along the lines of Mirrored / RAID storage, possible hot swap.

So, what are others doing for storage / backup of their photo data?



Cheers,

Mike

[Gazza]

I remember this question being asked a few yrs back, but technology being what it is that is like a life time.

I am still living relatively in the past and using two external USB 3 x 2 Tb drives of which I manually sync once a day. I used to have a off site storage (internet) too but with 1.5 Tb of data this can become hard to achieve.

I am about to upgrade my external storage looking to something more along the lines of Mirrored / RAID storage, possible hot swap.

So, what are others doing for storage / backup of their photo data?



Cheers,

Mike
Very timely. I'm in the process of having a big CULL myself. I'll also be very interested in others thoughts.

I'm using windows 'Backup' to an external drive (Just have my pic folder selected) which I assume is being backed-up on daily basis in the background? :crossed:
(no problems yet to test it out...(touch wood)) :grinning01:

[Tannin]

using two external USB 3 x 2 Tb drives[/b

External drives are still the best: by far the cheapest, and generally the most reliable - provided, of course, that you have multiple units.

[b]of which I manually sync once a day

Ouch! This is way, way too often. Very dangerous. Many data threats take a long time to become obvious. Suppose you have one of those data-encryption infections (which are very common now). What you need is an OLD backup, pre-infection. Aim to have a year-old backup, one a month or two old, and a recent one. Old backups are precious in many ways and can protect you against a number of different threats, including your own stupidity. (Are you stupid? Well, do you own a computer? Yes? Then you've done something stupid! Chances are, you will do some other stupid thing soon enough. We all do. It goes with the territory. Good backups are the only defence.)


I used to have a off site storage (internet) too but with 1.5 Tb of data this can become hard to achieve.

Yes. It gets very expensive. Plus your provider may not be around when you need them. Plus they read your data and sell it. (Read the fine print!)

I am about to upgrade my external storage looking to something more along the lines of Mirrored / RAID storage, possible hot swap.

Don't even think about it. Can of worms.

Your best answer, by far, is simply to buy more external drives. Plenty more! No such thing as too many backups.

[Geoff79]

I have a bunch of external drives in various states of working order and updated content.

I have one external drive I update quite frequently when I have, say, a few weeks of recently processed photos ready to move on my laptop. Then I have a second external drive which I probably only get to update every few months.

I do have other drives but they’re mostly full and a bit of an unorganised mess. I actually just got a really good deal on a 4tb drive, which still sits unwrapped. I must get to it and organise everything.

I hope that by the end of it I have all my processed photos and videos on all my still working drives.


Sent from my iPhone using Tapatalk

[Brendo09]

I've just uploaded all of my photos (and it's only about 80gb) to our personal Onedrive and copy that to my Google drive account (120gb). So there's two identical cloud storage sites. I also have the same on a 2tb harddrive.

Time will tell if it's worthwhile... I have portable drives, I just forget to update them properly and make backups.

[Steve Axford]

I would suggest that the concept of archive is very important here. Backup takes am image copy of what you have. If you (or someone else) changes what you have, then your backup also changes. It may take a year, but if you accidentally delete something and don't notice for a year - it's gone - forever. Archive stores every photo you have put on it - forever. Backup and archive are quite different.

[Tannin]

^ That was my point, Steve, except that I only hinted at it and you explained it concisely and clearly.

[Brian500au]

I have two 3tb hard drives. I use a back up program which only adds files that have changed - and never deletes files from the target. I alternate these back two drives every month.

I also have a copy of all the files on my computer hard drive and a second copy on a NAS raid which copies only changed files from my computer hard drive. I can access these files from other computers in the house (laptops etc).

[Steve Axford]

It is worth remembering that most data loss is due to - user error. Not malicious intent, not software or hardware error, no it is when you accidentally delete something that you need. If you don't notice the error then backup will just replicate that error. I don't backup any of my photos. But I archive all of them to two disks, which never get overwritten. I can add to them, but I never delete from them and I only have one online at a time. My current data I keep on RAID disk to make hardware failure unlikely.

[Tannin]

^Hmmm .... yes. But given the notorious weirdnesses and incompatibility gotchas of RAID controllers, that's maybe better in theory than in practice. Yes, there ARE good RAID setups, but they are horrendously expensive and far from simple. But given your comprehensive archiving, you have a fallback, so that's fine. (I rely on a single front-line drive, but it's backed up very, very thoroughly. Actually, "archived" might be a better term, my method is more akin to the latter rather than the former.)

[Steve Axford]

RAID 1 is generally ok as it is very simple. It is expensive but it reduces the risk of data loss due to hardware failure. Nothing will ever reduce the chances of data loss to zero, after all the world could end and we don't have the option of off-planet archive just yet (or the universe could end).

[bricat]

I also burn discs and keep a copy offsite. Doesn’t require much effort

[farmmax]

I'm with Steve. I just copy photos onto external drives in a fairly organised archive. They are never overwritten. I always keep the original unaltered raw files. It is interesting to look back and see what I've done in the way of PP over the years, and still have the original raw file to have another play with now. It is also amazing how a photo I thought useless years ago, turns out to contain something I need later.

[mikew09]

using two external USB 3 x 2 Tb drives[/b

External drives are still the best: by far the cheapest, and generally the most reliable - provided, of course, that you have multiple units.

[B]of which I manually sync once a day

Ouch! This is way, way too often. Very dangerous. Many data threats take a long time to become obvious. Suppose you have one of those data-encryption infections (which are very common now). What you need is an OLD backup, pre-infection. Aim to have a year-old backup, one a month or two old, and a recent one. Old backups are precious in many ways and can protect you against a number of different threats, including your own stupidity. (Are you stupid? Well, do you own a computer? Yes? Then you've done something stupid! Chances are, you will do some other stupid thing soon enough. We all do. It goes with the territory. Good backups are the only defence.)


I used to have a off site storage (internet) too but with 1.5 Tb of data this can become hard to achieve.

Yes. It gets very expensive. Plus your provider may not be around when you need them. Plus they read your data and sell it. (Read the fine print!)

I am about to upgrade my external storage looking to something more along the lines of Mirrored / RAID storage, possible hot swap.

Don't even think about it. Can of worms.

Your best answer, by far, is simply to buy more external drives. Plenty more! No such thing as too many backups.

Very interesting feedback Tony. Thanx for the tip on to often mirror sync - good point.

- - - Updated - - -


I would suggest that the concept of archive is very important here. Backup takes am image copy of what you have. If you (or someone else) changes what you have, then your backup also changes. It may take a year, but if you accidentally delete something and don't notice for a year - it's gone - forever. Archive stores every photo you have put on it - forever. Backup and archive are quite different.

Thats something I was not aware of Steve - is there a specific tool you use?

- - - Updated - - -


I have two 3tb hard drives. I use a back up program which only adds files that have changed - and never deletes files from the target. I alternate these back two drives every month.

I also have a copy of all the files on my computer hard drive and a second copy on a NAS raid which copies only changed files from my computer hard drive. I can access these files from other computers in the house (laptops etc).

Nice setup by the sounds - what program do you use Brian?

- - - Updated - - -


It is worth remembering that most data loss is due to - user error. Not malicious intent, not software or hardware error, no it is when you accidentally delete something that you need. If you don't notice the error then backup will just replicate that error. I don't backup any of my photos. But I archive all of them to two disks, which never get overwritten. I can add to them, but I never delete from them and I only have one online at a time. My current data I keep on RAID disk to make hardware failure unlikely.

What RAID set platform do you use Steve and are you happy with it. Starting to sound like a mix of the two may be worth considering as the way to go

- - - Updated - - -


I also burn discs and keep a copy offsite. Doesn’t require much effort[/QUO

I used to do the same but I aint going to get 3 TB on a couple of disks - LOL

To be honest I used to find burning to DVD effort intensive and had a very poor success rate with the burns

- - - Updated - - -

[QUOTE=farmmax;1432732]I'm with Steve. I just copy photos onto external drives in a fairly organised archive. They are never overwritten. I always keep the original unaltered raw files. It is interesting to look back and see what I've done in the way of PP over the years, and still have the original raw file to have another play with now. It is also amazing how a photo I thought useless years ago, turns out to contain something I need later.

I would not say I am unhappy with importing to an external drive, doing my culling and edits and then syncing with a second drive - I would like a more efficient process for syncing the two drives - it sounds like there may be suitable software to cover a backup / archive solution so I am keen to hear what you guys recommend.

Prior to Windows 10 with less than 1Tb of data, I tried the windows backup tooling at the time but on two occasions I had corrupted data so have not been there since. There has to be better software I assume.

[Steve Axford]

I don't know of any archive software, but it's not difficult to do. Basically you want to store all your photos as taken. If you lose some of the post processing, then that isn't all that critical. I periodically copy my RAW input files to archive (duplicate disks), then delete the originals from my current disk. I do this about monthly, so I have the last month only on my active disk (if I have some critical photos, eg from a trip, I will archive them immediately (but probably not delete them from my active disk). The active disk is a Lacie RAID array, which reduces the chance of a hardware failure. I will also periodically copy my processed files to archive, but this is much less often. This doesn't protect me against all possible failures, but it eliminates the most likely failures and is easy to manage. I frequently have to access my archive when people want to by some photos from me. I then use the file name from my website and do a disk search for the original file - bingo, there it is in all its RAW glory. I think anything like this takes some thought of what is important to you.

[John King]

Agree with Tannin and Steve.

Syncback (https://www.2brightsparks.com) is your friend. It's even worth getting the paid version IMO. I've been using it personally and professionally for many years. It works.

[mikew09]

Agree with Tannin and Steve.

Syncback (https://www.2brightsparks.com) is your friend. It's even worth getting the paid version IMO. I've been using it personally and professionally for many years. It works.

I have downloaded and am testing Syncback now. Looks to be a great tool and does make the backup of my other files, mail data etc a fair bit easier and less prone to human error.

I could never figure out why my 2 x 2Tb drives never had the same available space - a simulated run of this with Mirror settings showed me a load of hidden photo files I had removed from one drive and somehow they remained hidden away on the second drive. It can be hard to find miss matches when there is 1.8 Tb of data to try and compare. Thanx for the tip on the software team

[richtbw]

I use two 2TB USB external hard drives and shoot RAW. One of the drives is the "primary" in that I always use that specific one to download/move the images from memory card. Editing is done using the primary drive. SyncToy from MS is used at least once a month, to keep the two drives synchronised. Very few if any images are stored on the PC or Laptop.

[mikew09]

I use two 2TB USB external hard drives and shoot RAW. One of the drives is the "primary" in that I always use that specific one to download/move the images from memory card. Editing is done using the primary drive. SyncToy from MS is used at least once a month, to keep the two drives synchronised. Very few if any images are stored on the PC or Laptop.

That is the same as my process albeit I was using a tool to compare but needed manual intervention to sync up. Wasnt ideal for a mirror setup and could lead to human error as it did. I just started trialing Syncback that by comment in this thread are my experience so far seems to be very good and did identify some orphan files that had got past my other compare tool - quote a few Gb's actually.
The two drives are now in a mirror config and the automation of is working really well. I also work off a primary and only use the second drive as the redundancy copy.

It is quite a hands off approach but also allow me to run a test mirror run to see what the changes are before committing a run - I have it set to a weekly schedule at this point.

This may breath some like back into using the two external drive setup and I may just go to two 4TB drives with the upgrade as syncback allows me very easily to have a third drive for off site holding.

[mikew09]

Just an update:
I am moving to 2 x 3Tb drives USB 3 external drives (2 x TB drives almost full). I syncback tool suggested is proving to be excellent.
I have setup scheduled tasks to cover off:
1. Scheduled Mirror - documents etc as I store all tax needed documents, warranties etc electronically.
2. Scheduled Backup - Mail data store (only caveat here is I need to ensure I close my mail when done )
3. Scheduled Mirror - What I called archived photo - these are my old photos included scanned photos from over the yrs from old photo albums - rarely if ever change
4. Scheduled Mirror - All my photography set since I started dabbling in photography
5. Manual Mirror - a manual task I use to do the first copy for mirror of newly uploaded photo's after cull and edit etc
6. A cloud storage device to auto sync to my cloud acct which will have a schedule backup done from the slected folder on my mirror set - storage device undecided as yet but most vendors have their own solution

After doing an initial analysis I am considering an upload of my must keep photos to cloud stoage. These will be a groups of photos from each photo set (possible only one or two in cases) of photos that are absolute keepers. The photos I have edited and printed etc that I must absolutely keep a copy of including the RAW original. I am sure we all store a whole load of photos at each shoot that never get edited. I am pretty aggressive with my culling to keep numbers down but if I lost my photos (house firs - theft etc) looking a lot of these would be less painful as long as I had my absolute keepers and original RAW from each set.

Taking away the cost of new mirror drives that I would purchase anyways I dont see the update to this new model being that expensive albeit I havent priced the cloud storage solution yet.

Happy to hear thoughts on this setup model

[John King]

The best and fastest "cloud storage" is a separate drive kept at a local bank or friend's house. Rotate periodically with an active drive. Do not leave active drives plugged into the computer! Keep one in a safe, or similar, where it cannot easily be found and stolen by the five minute burglar.

The single biggest problem with cloud storage is that you can never see the financials of the company. That is, your cloud storage could evaporate at the press of a button by a liquidator ...

[mikew09]

The best and fastest "cloud storage" is a separate drive kept at a local bank or friend's house. Rotate periodically with an active drive. Do not leave active drives plugged into the computer! Keep one in a safe, or similar, where it cannot easily be found and stolen by the five minute burglar.

The single biggest problem with cloud storage is that you can never see the financials of the company. That is, your cloud storage could evaporate at the press of a button by a liquidator ...

In the past (at one time) I used to keep a back up in the office and will probably revert to that for my old scanned family photos etc. Anytime we go away for a period, I put a mirror drive away. We live in the country and I work from home so the house is rarely without some one around. Having said that, the burbs have been getting progressing closer over the yrs and talking to the local coppers the crime rate is getting much worse in town. Crikey, you can even order a home delivery pizza these days
:lol:

Good point about the cloud

[Wayno]

I use Synctoy to back up my photos to an internal hard drive within my PC.
Recently I bought a WD My Cloud Home 8T external device that plugs straight into my router. Very handy (wished I had bought it earlier) as I can look/download files off it from my laptop and mobile phone.
I also have Crashplan which is an online back up service. I figured that I need an back up off site.

:th3: