Storage and Backups of image ideas?

[hdn177]

Hi All, Well I wanted to see what ideas and how your currently organizing and storing all your images. I'm only a hobby photographer so I'm sure my small amount of pictures (250gig worth) isn't all that much in comparison to what others need capacity for. Now that i'm mainly shoot in RAW mode, I'm building up multiples of each image also due to PP.

Also now that having a larger mega pixel camera, my images are now alot larger in size and with the capability of many more images available in burst mode at the one shoot, I'm now getting many more than I'm use too. I can see this all filling up and taking over more HDD space quickly.

Currently, I have one of my 2 internal HDD partitioned (500gig) dedicated to only images. I then backup to a portable USB drive(usually kept elsewhere in case something ever happened, robbery/natural disasters/ hardware failure etc) and the laptop gets a copy as well. Fairly simple and I think effective if anything ever goes wrong. This is the same for any other data I want to keep, as well as having the images in lower res uploaded to Flicker, PhotoBucket, FaceBook's etc, there is a few separate backup locations.

The other issue for me is that the family and I are moving to the UK next year and I want to make sure I setup now to move the backups efficiently and can take any important data with me easily and safely.

I'm thinking the best thing to do for now will be to install another internal HDD and get a larger USB backup (without a power supply). We are hoping to sell almsot everything we have (not much :eek:) to reduce the cost of the move. I really only want to take the HDD with me to install into the new pc's when we get there.

After reading a few threads and the AP library, there is plenty of good information already about. So anything to help make this more efficient for the move to another country will help greatly.

Thanks again.

[ricktas]

Hi,

I would make sure that you have at least 3 copies of your files that you take with you, what if your luggage is lost (not unheard of). If you have 3 copies (3 hard disks, or two hard disks and DVD backups) placed in differing bags, then you are secure. You may say, but I am taking the hard disk with me as carry-on luggage and it will not be out of my sight, but what happens at the x-ray machine if your bag gets dropped on the ground, or you accidentally drop it from the overhead locker? Hard Drives do not like being dropped!

At home, I have my files in three locations, on an internal hard drive (current year only), on an external hard drive (previous years only), everything on another external hard drive (that lives at a friend's place), and a 12TB RAID 5 array, that has backups of everything on it.

There is another solution here for your move, that may suit and be easier and take the pressure off you, no need to buy additional hard drives. That solution is online storage. Buy some GB online and store your photos in a secure online location. You can do this yourself, by buying a domain and hosting that offers storage above what you need, and then upload all your photos to your site. However this can be time consuming to setup from scratch and there are companies out there who offer online storage facilities.

Storage at home, is a bit different to storage when moving countries, and you need to consider keeping weight down etc for travel/relocation, whereas at home that doesn't enter the equation, so maybe online storage is the way to go, for you, leaving valuable space and weight in your luggage for something else.

EDITED TO ADD:

https://www.sugarsync.com/ offers 500 GB at a very reasonable rate

[kiwi]

As our esteemed leader suggests don't put all your eggs in one basket or have all your pics at a single location. HDDs are so cheap I'd get three copies, post one ahead of time, keep one with you, and keep one here

Macrium free is a good way to keep an image of your pc or laptop

[mercho]

Yep agree with what has been said already, at least 3 different drives, keep one in Aus somewhere. Also look at getting a small pelican case to carry on with you with the HDD...

[kiwi]

pelican cases are great..and they are made out of farm bred pelicans not the wild ones so will not get you into trouble at customs fyi

[Xenedis]

I wanted to see what ideas and how your currently organizing and storing all your images.

From my 2010-written blog entry (http://xenedis.wordpress.com/2010/07/23/my-approach-to-image-storage-and-backups/) on this very topic:


My Approach to Image Storage and Backups

In a digital age, and especially with photography being so prolific these days, an important issue for photographers to consider is their approach to file storage and backups.

I have been photographing digitally since 2002, and in my early days I found a method of filesystem organisation which works well for me.

My backup strategy also works well for me, and as I'll explain further into this article, provides me with disk-level redundancy as well as geographical redundancy.


1. Filesystem Organisation

At the time of writing, I have 32,969 image files (ie, with a filename extension of .bmp, .cr2, .gif, .jpg, .png, .psd and .tif). Another 1,075 files are .xmp files, which contain metadata written by Adobe Camera Raw during the raw conversion phase.

That's a lot of files!

It would be impossible to manage without a good directory structure.

The system upon which I decided in the early days was to have one directory for each year, and within a given year, a directory for each shoot, named after the shoot. Also a part of the directory name is the date of the shoot, in YYYY-MM-DD format.

Here is a slightly dated screen capture of my directory structure:

http://farm5.static.flickr.com/4118/4807641693_ab5ed523ba_o.png

As can be seen, each shoot has its own aptly-named directory within the year in which the shoot took place.

Now, some people place the date first, but my mind is wired to remember the location or event rather than the date it took place. To my mind, "I shot at Long Reef at some stage this year" holds more meaning than "I shot somewhere on the 21st of March".

Other people break down their directory structure further, with a subdirectory for each month. Again, the specific date on which I did a photoshoot doesn't rank anywhere near as important as what the subject was. Being able to pinpoint a shoot to a specific year by memory is as low-level I need to get with dates.

As for image filenames, I don't rename the files the camera produces.

If I shoot an image and the camera calls it IMG_0001.CR2, and I decide to process it, I will end up with at least three more files:


IMG_0001.XMP (the metadata containing the non-destructive adjustments applied in Adobe Camera Raw);
IMG_0001 Processed.PSD (the 16-bit, multi-layered, Photoshop-processed version of my image); and
IMG_0001 Processed.JPG (the 8-bit JPG exported from the PSD file).

I may produce smaller versions of my images for specific purposes, or black and white versions, and in those cases the exported JPGs would be called "IMG_0001 Processed 1024 x 683.JPG" or "IMG_0001 Processed Black and White.JPG" respectively.

I don't keep any lower-level directories for processed versions, 'keepers' or any other category; all files from a single shoot are in a single directory.

Naturally, the image filename counter on a camera will reset once it reaches a certain frame count, and as it stands, I have three IMG_0001 files in some cases, but filename duplication is not a cause for concern, as a directory containing a single photoshoot's images will never have two identically-named files. On the few occasions when I have used two cameras to cover an event (such as a band), I have kept each camera's images in its own subdirectory.

One organisational strategy I don't employ is image tagging. My existing filing strategy has been in use for quite a number of years, and I haven't found a need to search for an image's contents based on something I remember being in the image, or any other criteria for which tagging would be useful. I'm able to find what I'm after by recalling the year and the event/shoot, based on the naming convention I use.

2. Backup Strategy

I take backing up quite seriously, and I have adopted an approach that provides disk-based and geographical redundancy.

My PC is a Apple MacBook Pro (17") with a 500GB hard disk. All of my image (and other) data is stored on it. In my home office I keep two LaCie d2 Quadra 1TB external hard disks, which have a FireWire 800 port offering faster data transfer rates.

I keep a third external hard disk off-site to ensure geographical redundancy in case my local external disks suffer loss, theft or damage. My off-site disk is a 500GB Western Digital My Passport Essential.

My approach is to manually back up my data to these drives, generally every week or two. This method might sound cumbersome, but it works for me, and I employ a very structured filing system, not only for my image data as outlined above, but for all of my data.

At some stage I should investigate a more automated method, but this manual approach has worked for a number of years.

The end result is that I have four copies of all of my data, some of which goes back 16 years. (The current value of that 16-year-old data is questionable, but I am sentimental.)

In the past I used CD media, and later DVD media, but as my volume of data increased, and as image file sizes and the byte count of individual photoshoots also increased, optical media quickly became cumbersome and insufficient, and that was when I invested in external hard disks.

The other issue is that I don't trust optical media. Sure, hard disks can fail, too, but the key is not to rely on one disk. Hard disks are much more convenient to use, a lot faster, and they store a lot more data.

To conclude, my key advice on data backups is:


back up regularly (weekly or fortnightly is good);
back up to multiple storage devices, never relying on one alone;
keep your data in at least two geographically separate locations; and
use the most effective and reliable storage media of the day, and update when needed.



Since I first wrote that article, I've switched to using Apple's Time Machine backup software for my off-site drive.

[ameerat42]

Maybe get a smaller sized external HDD that you can carry in your on-board luggage. The WD Passport series, for example.
They are a bit dearer/unit storage space than the Elements and MyBook ones, but...
Am.

[Xenedis]

Oh, and for transportation, definitely invest in Pelican cases.

I use a Pelican case for transportation of an expensive item which requires both protection and security. These cases are great, and will protect whatever you put in them.

Some other advice I'd offer about hard disks:

Firstly, partitions are nice from an organisational viewpoint, but they don't give you protection against corruption, disk failures or power surges.

Secondly, your backup disks should be external so as to avoid accidental data loss from surges or user errors. Don't keep them connected all the time. Mine are completely isolated from power, and I connect them only when I need to run a backup.

[Analog6]

Considering 2Tb HDDs on eBay range from under $100 (I couldn't quite believe it either) and for good brands like Western Digital, Samsung and Iomega, then as others have said, that is obviously the way to go.

[hdn177]

Thank you everyone for the replies. Has been a great help. I ran out and purchased a 2TB HDD for my 1st step. Next will be a external drive too match and start making it a regular occurring part of backup and preparing for our move closer to the time. Thanks for the links as well rick, they will defintly be what I look into now. I also never considered posting over before leaving. :th3: