Data Backup Strategy

[Xenedis]

I learned this morning that one of my seascaping comrades has recently suffered a complete data loss due to a corrupted disk, which unfortunately had the only copies of all of his images on it.

In light of that catastrophic event, it's a timely reminder to discuss data backup strategies.

I take backing up quite seriously, and I have adopted an approach that provides disk-based and geographical redundancy.

My PC is a Apple MacBook Pro (17") with a 750GB hard disk. All of my image (and other) data is stored on it. In my home office I keep two LaCie d2 Quadra 1TB external hard disks, which have a FireWire 800 port offering faster data transfer rates.

I keep a third external hard disk off-site to ensure geographical redundancy in case my local external disks suffer loss, theft or damage. My off-site disk is a 1TB Western Digital My Passport Essential SE.

My approach is to manually back up my data to my two on-site drives, generally every week or two. This method might sound cumbersome, but it works for me, and I employ a very structured filing system, not only for my image data, but for all of my data.

My off-site disk is also backed up weekly, and I use Time Machine to back up my entire system.

The end result is that I have four copies of all of my data, some of which goes back 17 years. (The current value of that 17-year-old data is questionable, but I am sentimental.)

In the past I used CD media, and later DVD media, but as my volume of data increased, and as image file sizes and the byte count of individual photoshoots also increased, optical media quickly became cumbersome and insufficient, and that was when I invested in external hard disks.

The other issue is that I don't trust optical media. Sure, hard disks can fail, too, but the key is not to rely on one disk. Hard disks are much more convenient to use, a lot faster, and they store a lot more data.

My key advice on data backups is:


back up regularly (weekly or fortnightly is good);
back up to multiple storage devices, never relying on one alone;
keep your data in at least two geographically separate locations; and
use the most effective and reliable storage media of the day, and update when needed.



In conclusion, remember this:

There are two kinds of people in the world: those who have had a hard disk fail, and those who are going to have a hard disk fail.

I cannot stress enough the importance of making multiple backups on a regular basis. Never rely on one disk alone; always keep multiple copies, and preferably one off-site.

[JM Tran]

you are doing pretty much what I have been doing too for the last few years.

apart from the 2 main external back ups and 1 offsite back up, for location work and overseas work I have a Lacie Rugged external drive which can take a lot more beatings than the usual drives, 1 copy on there, 1 copy on MBP's SSD and another on a memory card - using dual card slots whenever applicable but not always possible.

[I @ M]

Isn't there a law about this, written by some bloke called Murphy from memory, that goes something like " those who follow a thorough back up regime are the least likely to have a hard drive fail". :rolleyes:

[ricktas]

Whilst I feel for your comrade, the issue of data integrity and backups is not new. I reckon every photography forum on the net has had discussions about it, repeatedly over the years.

If people in 2012 insist on not having backups, then they really do deserve what they get. Harsh maybe, but I am sick of hearing these hard-luck stories cause someone thought they didn't need to do what everyone else does.

[Kym]

The cloud is now affordable, eg. http://www.crashplan.com/ :th3:

[JM Tran]

thats very cheap Kym, will look into that site more now.

[Xenedis]

you are doing pretty much what I have been doing too for the last few years.

apart from the 2 main external back ups and 1 offsite back up, for location work and overseas work I have a Lacie Rugged external drive which can take a lot more beatings than the usual drives, 1 copy on there, 1 copy on MBP's SSD and another on a memory card - using dual card slots whenever applicable but not always possible.

Yeah, I've seen those LaCie rugged drives... built like a tank.

If you're on the road a lot andf the disk is getting jjostled around, one of those would be a better choice than the standard, non-ruggedised cases.


Isn't there a law about this, written by some bloke called Murphy from memory, that goes something like " those who follow a thorough back up regime are the least likely to have a hard drive fail". :rolleyes:

It's a bit like carrying a large umbrella: If you do, it won't rain. :-)


Whilst I feel for your comrade, the issue of data integrity and backups is not new.

Definitely not new, but a very timely reminder.

If I can save one person some pain by harping on about this stuff, than that'll be a good contribution in my book.


I reckon every photography forum on the net has had discussions about it, repeatedly over the years.

Indeed. It's also interesting to hear about the approaches others take.

We were discussing this stuff around the breakfast table. One guy has a RAID-based NAS and backs up from that. For some that might be a bit extreme, but for others it might not be enoug


If people in 2012 insist on not having backups, then they really do deserve what they get. Harsh maybe, but I am sick of hearing these hard-luck stories cause someone thought they didn't need to do what everyone else does.

I think my comrade has surely learned.

His disk is with some data recovery mob, but depending on the difficulty, I'm told, the cost of recovery can get into the $3K bracket.

As far as I'm concerned, while we can all discuss aperture, shutter speeds, processing techniques and whatnot, data management (including backup policies) is yet another critical category of information all photographers should possess, so the sooner those in the know inform those who don't, the better.


The cloud is now affordable, eg. http://www.crashplan.com/ :th3:

Cloud storage also came up this morning.

For some people it's an option worth pursuing. Personally, apart from the initial pain of uploading a shed-load of data, I'm really not all that keen on my personal data (esp. the financial stuff) being stored on someone else's disks. I suppose one can be selective about what gets pushed into the cloud, but at any rate I consider managing and protecting my data, my responsibility.

[mini696]

The end result is that I have four copies of all of my data, some of which goes back 17 years. (The current value of that 17-year-old data is questionable, but I am sentimental.)

Of all the things I have on my computer, the hardest (impossible) things to replace are the photos... You cant ask your three year old to reverse a few years so you can retake that photo of him smacking the cat... Or whatever.

Games, programs, favorite websites can all be replaced fairly easially.

Anyway. I have a Raid5 external setup over 4 hard drives (one can fail and I wont lose anything). I also backup onto two seperate 1tb hard drives, one kept at home, one at my parents.

Its all done by manually copying files every couple of weeks.

[Wayno]

Cheers for the Crashplan website. Not expensive for peace of mind.
:beer_mug:

[rodw]

Putting a NAS on your home network is another way. Fortunately, I moved all my data and 30 gb of photos onto one before my HDD Died. Currently running 2x 3 TB drives mirrored and a backup client that auto syncs files on the PC to the NAS.

[ameerat42]

How bad is it, Xenedis. Years ago I had a (then) mate to whom this happened. He borrowed an identical 8 GB HDD from me, swapped the controller board and retrieved all his data. He then swapped it back and gave me back the drive. I haven't needed to do it since, but surely if you could get a similar drive...
Am.

[Xenedis]

How bad is it, Xenedis.

His drive is an external USB-powered disk (Western Digital My Book).

He said the data was corrupted, but I'm not aware of the specifics.

[ameerat42]

Just a thought: they'd still be an HDD unit in some controlling circuitry.

[Xenedis]

Just a thought: they'd still be an HDD unit in some controlling circuitry.

Yep, they're regular HDDs.

As above, I don't know the specifics of the problem; but he has it with a data recovery company.

[Xebadir]

My levels of paranoia...well might exceed many of you - but I blame my career.

As a PhD student one of the first things I was told was the horror stories of the old floppy disk backup and losing your PhD entirely (and it happened to people). My approach for that is:
Macbook 320 gb Harddrive which has all my PhD stuff
Backup 1 - 750 gb portable (find they are more reliable and tougher) with all my data (the stuff that took the past year to produce) plus full Mac Backup at home
Backup 2 - 500 gb portable with duplicate of the above minus a few less critical original datasets stored at office.
Backup 3 - 500 gb portable as above, updated monthly stored at parents farm in NE Vic ;).

My backup procedure is weekly, additionally all backup materials for source data, code, important musings are stored on the university server. I always have either the home drive or the Mac with me any time I am away from home.

So you can imagine it gets complex...but wait - theres more.

500GB PC laptop harddrive - past 3 months photography or working files.
1TB desktop drive (my only of this type, otherwise portables) - General storage - contains website copy, music files, plus all my other digital media.
500 GB drive - Photo Drive 1 - All photos archived.
Photo Drive 2 - As above, full backup of all photos, stored in office.
Video Drive 1 - Storage for storm Video, 500GB drive.
Video Drive 2 - Backup for the above and overflow, stored in office.

DVDs - The top tier shot archives (A grades) and any job archives are put onto DVD as additional backup and for quick reference.

Grand Total - ~5.5 TB storage give or take. Given my uni work original data space numbers 5-10 TB...not too bad ;).

And yes - I realise it probably sounds paranoid - but think about spending 3 and a half years on something only to lose it in the final stretch. The same could be said for my storm images - I would be heart-broken. One EMP and we are all stuffed though :P.

[Anne Sh]

I have been tossing and turning over this for a while. I am equally paranoid as others above me.

I have a laptop and desk Macs which are backed up with regularly through Time Machine. All my photos are backed up in the house on a Drobo and also on what I call the "shed book" - which is just an external hard drive that lives in my shed.

Recently I found that my 'shed book' is full and I am now considering my options.

I am considering going to two new external hard drives one to live in the shed and the other to live offsite.

I have heard that it is important to use the external drives regularly or at least each 12 months. Is this true or should they be used more often?


When I travel I take my laptop and two portable external hard drives and back up on to both on a daily basis depending on how much I have shot.

I am also amazed that people don't back up their computers!

[Ventureoverland]

My levels of paranoia...well might exceed many of you - but I blame my career.

As a PhD student one of the first things I was told was the horror stories of the old floppy disk backup and losing your PhD entirely (and it happened to people). My approach for that is:
Macbook 320 gb Harddrive which has all my PhD stuff
Backup 1 - 750 gb portable (find they are more reliable and tougher) with all my data (the stuff that took the past year to produce) plus full Mac Backup at home
Backup 2 - 500 gb portable with duplicate of the above minus a few less critical original datasets stored at office.
Backup 3 - 500 gb portable as above, updated monthly stored at parents farm in NE Vic ;).

My backup procedure is weekly, additionally all backup materials for source data, code, important musings are stored on the university server. I always have either the home drive or the Mac with me any time I am away from home.

So you can imagine it gets complex...but wait - theres more.

500GB PC laptop harddrive - past 3 months photography or working files.
1TB desktop drive (my only of this type, otherwise portables) - General storage - contains website copy, music files, plus all my other digital media.
500 GB drive - Photo Drive 1 - All photos archived.
Photo Drive 2 - As above, full backup of all photos, stored in office.
Video Drive 1 - Storage for storm Video, 500GB drive.
Video Drive 2 - Backup for the above and overflow, stored in office.

DVDs - The top tier shot archives (A grades) and any job archives are put onto DVD as additional backup and for quick reference.

Grand Total - ~5.5 TB storage give or take. Given my uni work original data space numbers 5-10 TB...not too bad ;).

And yes - I realise it probably sounds paranoid - but think about spending 3 and a half years on something only to lose it in the final stretch. The same could be said for my storm images - I would be heart-broken. One EMP and we are all stuffed though :P.

Dood, that's more backup backups than Cupertino:)

Sent using Forum Runner

[Epicaricacy]

I do cloud backup as well as multi hard disks....

1 - smug mug, although this doesn't do my raw files, but at least I will still have the processed images. unlimited, and great for sharing.
2 - sky drive.... Four free 25gb accounts covers my photos at the moment. Howe er, I believe they have cut the amount down for new account holders to 10gb or so.

Google drive doesn't offer enough free storage yet, nor Dropbox, although I do have 9gb free on Dropbox.

- - - Updated - - -

Only just saw how old this topic is. But is never old in a way...