Photo De-duplication, Organisation and Backup

[landyvlad]

OK so this is a 3part question.

De-Duplication
I have a bunch of photos (many of which will be duplicates) in a LOT of folders spread across 3 hard drives, 2 SSD's and a NAS. (But we'll leave the NAS out of this for the time being mostly because its broken...)

What I need to do is:
-Identify and delete (or quarantine) duplicates, ensuring that I keep the highest res versions in each case.
I need to identify duplicates based on content not just filename .
I suspect I have:
- a lot of different photos with the same filename, and
- a lot of the same photos with different filenames.
Also if an image has say a RAW and a JPG version - keep both.

Advice welcome.


Organisation

Once I've de-duplicated I would like to organise the images in some way. Now obviously YYYYMMDD from the metadata would be easiest to accomplish BUT
- some files don't have metadata attached with date info.
- date info is often wrong if camera didn't have date set correctly.
- simple numeric systems like that make it hard to identify (from the folder structure) what the contents are.

I'd prefer to avoid, if possible, being tied into a particular piece of software (I'm looking at you Adobe Lightroom) for tagging and library management. Something more generic and ideally FREE would be preferred, if such a thing exists.

Advice welcome.


Backup

Once I have the above taken care of, I'd like to have a means of backing up my photos regularly (eg ONE-WAY sync to an external drive)

How are other people managing this?


Thank you :)

[ameerat42]

What an encyclopedia...:D

About point 1: I've taken it on n:umm:tice, but the next point might possibly cover it too.

On point 2: Have a look at Faststone Image Viewer (https://www.faststone.org/) (organiser, bit of an editor, and general good bloke).
DON'T get the other three products, as they're not free, whereas FIV is both free and quite independent.

There may be others that fit the bill (free and independent) but others can tell you about any.

On point 3: Well, what I do is not Ph.D material, but it seems to suit my needs:
- On original and worked-upon files, make separate backups to 3 external HDDs. These are typically
1 or 2TB HDDs, and not SSDs*. But they are USB3 and I back up over a USB3 port. Anything less is
real steam age speed. I usually get 100-120 MB/sec. Yes I get 300+ to SSDs in the same way.

- This is typically done from the PC to already made-up folders on the HDDs, and so they're usually
incremental backups. The MAIN PROGRAM I use to coordinate this sort of transfer is WINMERGE (https://winmerge.org/?lang=en).
Don't be scared of the web page, and I usually use only "folder compare" method. It's probably
also capable of some of the organising functions of FIV, but habits, you know :cool: As well, there
are other programs like Winmerge but I can't think of them. Winmerge (and the other I can't think
of) are also free and independent.

Anyway, have a good look for a start.



* A different discussion.

[landyvlad]

Thanks I got as response elsewhere which recommended the following too:

- Irfanview
- cccleaner (for identifying dupes only. Don't be tempted to use it on your system cleaning)
- Alldups
- Rename Master
- Awesome Duplicate Photo Finder

[landyvlad]

Just bumping this - anyone else care to contribute or has used the programs in my list and can offer comment?

Cheers

[Brigitte]

I'm with AM, both have suited my purposes. Faststone allows you to bring up 4 images at once to compare, very useful. I organize all my pics by date, then number and in some cases some other detail.

[landyvlad]

Might as well post here again. I am still looking for a good duplicate photo finder. I've come across a few and they find heaps of duplicates etc and then... don't let you do anything except they say to upgrade to the paid version to manage duplicates."

Before I shell out a single $ I need to know if the program will actually function the way I want it to.
So frustrating.