FreeFile Sync

[Brigitte]

Has any one used FreeFile Sync program to transfer their images to an external hard drive? Trying to make my transfers a bit easier and fool proof if that is possible.

[ameerat42]

Brigitte.

1) Do you just want to back up your files regularly? - Like once a day/week?
What is your operating system? If Win 10, just use that backup system. Some new
external HDDs come with backup software, but it's hardly worth installing when you
already have one.

2) Or do you just want to transfer files ad hoc to the external drive. Well, Win
File Manager can be a bit limiting but it works. For this sort of thing (actually
transferring image files between drives) I use FastStone Image Viewer* (https://www.faststone.org/), a free
program. The good thing about it is that it can be used to view files on your
camera card w/o removing it. It is visual, intuitive to use, and has handy bulk
file xfer capability among other things.

3) If you want to make (fairly) identical backups of your images then perhaps
Winmerge (https://sourceforge.net/projects/winmerge/)is what would help.
If you could provide a bit more info on your needs and on your OS it would be good.

* NB: The first one only on that page.

- - - Updated - - -

Actually, your Freefile Sync looks to be an equivalent of Winmerge. From its
description it looks manageable.

[Brigitte]

Thanks for the reply. I am still on Windows 7, most of the time I'm just adding the current processed files. Occasionally I will back up a whole folder. Backing up is done on a needs basis rather then a regular schedule. I do have and use FastStone as well PS6. From what I have seen of FreeFile sync it may be of benefit, nor sure but as it is free it may be worth a go. I also thought it looked managable.

[ameerat42]

:nod: I will await your verdict after you try it :th3:

[arthurking83]

Yes.
Been using it for a good 5 years or more.
Does a bit more than just transfer to external drives too.
I have a NAS(network Attached Storage) where my main backups are held, as well as external USB connected drives.

Once you learn all of it's intricacies, it's quite useful .. more useful than simply using a backup program(eg. Time Machine on Apple), and whatever file sync program came with your manufacturer based external HDD came with.

Something that happened to me many years back was that I'd bought a docking station for various reasons. Mainly to help when I sorted others PC issues, so have easy HDD docking/connectivity solution, but also that I had a reserve backup of files.
Primarily images but not just so, old business accounts, and various other files and stuff.
This way, I could pop in a HDD, do a file sync/transfer, remove that 'popped in' HDD, and once removed, stored safely in a drawer. Don't assume that just because it's on a separate drive that those files are safe.
The stupid docking station design was faulty. Not my specific device faulty, the moron company that made the internal chips simply didn't know how to make those chips properly.
Of course I didn't know this, so started transferring > 1Tb of all manner of files from the old HDD to a new one, and it continually stalled .. locked up the file transfer. The other thing I didn't realise(at the time) was at the stalled moment of the transfer it was corrupting those files it was trying to transfer. I have well over 100K files, and to view each one isn't a viable proposition(unless you have too), so thought nothing of it.
Anyhow a while later when viewing some other files, I noticed that some thumbnails weren't generating on some of the images. Reason was that stupid docking station had corrupted the files back then.
I stupidly(unwittingly) relied on the 'backed up' files as my primary source, deleted the original files that should have been fine, and now my backups were of a few corrupted files. ie. files gone!
Having looked into it, while there weren't badly captured images, they weren't anything I could live without .. but it raised the concern that your backups may not be backups if you're not careful about them.

This is why I had to find some sort of solution to file transfers, file transfer confirmation, and integrity of that backup process.

Another thing that I experienced at about the same time, via my sister, was that ransomware was on the rise. She has a business that relied on two PCs, they received many emails a day, and as she was sorting through them clicked on one that had a PDF that rear invoice.
Seemed innocuous enough and she didn't thing anything of it .. and of course it was ransomware! :eek:
I dunno if you understand how this works, but the one she received propagated over the network, looked for all common file types, that most computers will store.
Both their PCs had almost all of the files on them encrypted, and the :violent10: bastards wanted $200 for the encryption key.
Anyhow, the PCs were pretty much useless, I built them at least one to get them going, and she was lucky that the cash register/purchasing/EFT files used some arcane file type that the ransomware didn't affect.

While I'm not seeing much news recently of huge numbers of ransomware attacks now, don't get complacent and presume that some new method for extorting the general public isn't around the corner.
So, what happens in a ransomeware attack is that any device across the network(most folks will have some form of connected network in their places of work or residence) .. it crawls across the entire network and tries to find most file types and encrypt them.
Who knows what abomination antisocial application is just around the corner in the near future.

So the trick is to separate the storage device from the network in everyday life, and only do the backup once reconnected and sure it's safe to do so.

USB HDDs are good, but limited as to how much storage space you can reasonably amass. From memory, I think I have just a tad over 8Tb of stored files.
Photos are approx 2.5Tb, I have many videos saved, much of that is old saved video footage of the kids when young.
Having > 8Tb worth of external USB HDDs on desk-near desk can be annoying. Enter NAS! :th3:

NAS is network connected, each drive is separately controllable, and I have two dedicated for photos(one for videos, one for other very very important files).
One of the photos drives is always on. My easy backup. No brainer, in that when the time comes and I do a backup/sync I don't need to do anything other than hit the sync button on the the program.
The other photo drive is set to the OFF state. This is my 'removed' drive. If it all goes pear shaped, this drive will not get affected .. as it's not there. I need to go into the browser, connect to NAS, find the settings, turn on the 'OFF' HDD. Then apply the sync to that drive too, and turn it off again.

Another thing I found handy was external HDDs with on off(power) switches. Not all do, makes life easier rather than having to pull USB cables to disconnect drives off the entire system/network.

FreeFileSync helps with managing this process a lot easier than I found with any other backing up/file transferring programs.
The other handy feature I found using FFS was it's ability to easily and clearly show differences between files .. important so that you DON'T rely on a backed up version of a file that may well be corrupted.

Son was shooting with the D70s one day. I think, may have been the D300, but anyhow two images got corrupted in camera. No idea why, and they are a weird corruption. The raw file itself is corrupted and shows it when viewed as raw, but the raw file looks fine if set to display the speedier jpg file.
Anyhow, the corruption got worse, and I only picked it up when viewing the differences between source and target files.
That file hadn't been touched in many years(yeah, it's still saved!) yet one file was different to the other when doing a compare in FFS. File size was a byte or two different between the two files, even tho the time stamp were both the same.
Had a look into it, using various other image viewing processes, and none could now open the raw file that would otherwise have been saved to the backup location. That particular file isn't important(kept for the curiosity factor) but the ability to find minute differences and avoid accidental/unwitting errors is handy ... as long as you know what to look for.

So yes .. definitely worth the install factor, depending on your requirements tho. I use it tho not really for the photos factor, I use it more so for the other files on the PC, as said, primary concern are old business accounts files I need to keep for a couple more years. But then there's many personal files, legal files .. etc. stuff that may be important to have backed up.
Always works, is very simple once you got your head around a new program.
I have it setup now where it's a couple of clicks affairs to do backups(saved backup routines), but one thing I never do are automated backups.

[ameerat42]

Yes.
...
I have it setup now where it's a couple of clicks affairs to do backups(saved backup routines), but one thing I never do are automated backups.

An interesting story, AK. Would it be OK to say what brand and approximate year model that device was?
I'm wondering if it is the same as "external drive dock" (not dry-dock). I have two of those and they have
never played up.

And, could you expand on the last clause cited above?

- - - Updated - - -

PS: Now you two have got me interested in this FFS as an alternative to Winmerge. :confused013

[Jaded62]

I've been using MS Synctoy for years for both work (sync between my laptop and server) and home for backups. Simple and easy to use.

[Brigitte]

Many thanks for your very detailed reply. There is a lot of in formation there for me to digest and look into. I had figured out that the external back up should not stay connected to the system and work with 2 backups one of which only gets connected once in a while to bring it up to date. I do back ups on a needs basis as they are not permanently connected. I will do some more reading on FreeFile Sync on how it works and then install it. At the moment we are all supposed to have lots of free time but currently the garden has priority. Once Melbourne's lovely winter weather sets in I will turn my full attention to this.
Once again many thanks for sharing your wisdom. Keep safe. Brigitte

[arthurking83]

.... There is a lot of in formation there for me to digest and look into. .....

Most of it is superfluous blathering about by me!
Just a general rundown on what and why, ignoring it all other than FFS is a handy program to use to do backups. .. more specifically syncs.

Your method of having 'offline' backup to your backup is pretty much an ideal method.
If the volume of files isn't enormous and a couple of external drives fits your needs, my rantings wouldn't apply.
Just that in my situation total file sizes are close to 8Tb and this much worth of external USB drives wasn't an ideal way to store it all .. hence the NAS.
Mine is a 4 bay, easy to upsize as needed, etc. The other specific need I wanted to have was a media player, so I got a NAS that also had that function.
Kids (and self) used to watch their movies etc ... one box basically covered multiple needs.

Warning, more ranting and useless info below: :p

Am: the specific docking station I had trouble with was a Vantech Nexstar twin port USB3 docking station(NST-D400S3).
White box with a large blue coloured circular power button on it.
The manufacturer themselves wasn't the problem, read good things about their products overall, thinking that they're worth a shot.
What the issue was, the specific chipset they used in the product(from a company called JMicron) This chip had issues over USB3(from SATA), couldn't maintain the high speed without issues.
Using it over USB2 wasn't a problem, but that's not why you purchase a USB3 product .. to use it over USB2 .. is it?
Anyhow they did issue a firmware update, but the problem persisted.
The Vantec product I had tho is now discontinued, so no longer available for sale(thankfully) .. and I have no idea what other products use that chipset.
ie. the same issue could apply to any USB3 docking station from any manufacturer if they use that same JMicron chipset, when used over USB3.
In the end I tried far and wide to find more specific data on what chipsets each product uses, and not much was forthcoming
I eventually found Volans(brand), who specifically had in their specs the chispet they use in their products(in that case was a Asmedia chipset).
The bonus with Volans is that they're 'made in Aus too.
Never any issue with many drives and transfer with that docking station.

At a guess, I reckon the Vantec saga was approx 2014, maybe 2013 .. can't recall exactly but about that time frame.

What I meant with a couple of click backup routine was that with the NAS, I don't need to physically touch any switch/cable/port/whatever to activate the off line HDD.
It's simply hidden, whilst still on. So on the network you can't see it, but it is there.
So in any browser, for my QNAP NAS, they seem to have coded it to work better with Chrome(which I don't particularly like) in that it works more fully(than say FF or MS's Edge).
But haven't tried Edge again now that they're using the Chrome browser engine tho.

So go to NAS, enable the disabled drive, and it mounts on the network(I have it set as Y:\) once activated, in FFS, I have a saved routine that I sync between the normal, always on X:\ drive on the NAS, and it syncs the two two drives on the NAS.
I used to do what Brigitte does and have two USB enclosures, both with power switches, and turn one on/off as my offline .. etc.

Bit of a handy aspect of the NAS, is the power save function, it uses about 1w for the majority of the time, ie. it hibernates all four drives. As it's over the network, the slight delay in power up again isn't an issue. Basically by the time the browser sends the command, it's all up and running again.
Always on USB drives use approx 5w(or more) each. Not a massive amount of power, but the clutter + the power saving .. + the future expandabilty .. it all made sense to invest in the NAS as an overall solution.

As for other file sync programs. The QNAP has it's own, but not as much control/flexibility compared to FFS. I've tried MS Synctoy, OK, not as flexible again. Haven't tried winmerge that I recall.
But for many many years I used RichCopy(Hoffman utility) I liked the flexibility to allow multi threaded transfers, made faster transfers and all.
But then settled on the simple and easy to use FastCopy program. FastCopy was the only program that could easily maintain between 80-100MB/s over USB3.
Problem is that it doesn't give you the easy to use interface that FFS does, where as said before, you can quickly see if there is a file discrepancy somewhere .. and that you don't overwrite a good file with a suddenly corrupted bad file!
I know this is rare, and it's only ever (almost) happened to me once with that corrupted D70s file that my son somehow captured.

As I remember them, and note I can't say for sure as it's been a long time between drinks .. but the major advantage with FFS over most is the ability to compare initially, review what you want synced(if you're inclined to do so) then hit the sync button to begin the sync.

Reason I blathered on so much is this is what caught me out with the corrupted files. These programs don't know what a corrupted file is, and just blindly copy stuff over.
So now, with FFS, I do the Compare routine, it lists what will be transferred, if I see files to be transferred that I haven't touched in a long time, I get suspicious as to why .. as happened with the D70s file.
I didn't touch it at all, so file size shouldn't have been affected, realistically neither should time stamp(what most sync programs will be set to compare).
But on seeing the partially corrupted file(remember corrupted on the camera), now when opened again it got corrupted even more, where it wouldn't display at all. File size changed by a very small byte or two. When I saw that this was a file to be synced, instead of syncing I went back into the viewer(ViewNX2) and noticed it now won't display.
I deleted that file, and when I did the sync, all files except this one were transferred over to the backup. I set FFS at the same time to transfer the backup copy into the location where the now totally corrupted file had been deleted off.

That's another handy quick/easy file transfer feature in FFS. You'd only understand it once you use the program, but it displays two windows of the directories you're working with(source and destination) between the two you see red or green arrows indicating which way the transfer will happen. If you see an anomaly, you click the arrow and set it to do nothing, go one way, or the other.

There are some caveats with this ability to review the sync before you activate it(and hence possibly do more harm than good). If you have thousands and thousands of files(as you may with photos and such), this is a tedious process.
But with a bit of use, and experience you don't really need to review every file. What I do is just speed scroll down the list and look for key indicators that could possibly overwrite good files with bad(as the corrupted D70s file).
All I look for are the things like reverse syncs(question why if the target directory hasn't been modified in any way), and then also what files in previous old folders are being synced. It used to take maybe 10 mins to carefully click page down till the final file showed up in the list, but now I just hold Page down and it flies through it all, and as it flashes by I can quickly see any anomaly. This takes about 30 sec to 1 min to flick through.

[ameerat42]

Ta, AK. I asked because that time period was when I got my XSOnic dock (2-bay), only USB2, but (still) great.
Abt 2 yrs ago I got a USB3 Volans (also 2-bay) and it has been fantastic as well. I sometimes have had to use
both docks at once. You're right about the On/Off switch being handy. Also, the Volans has a computerless disk
clone feature - which I have never used.

- - - Updated - - -

- Oh, and: I dnloaded FFS, and will compare it with Winmerge soon.

[Brigitte]

No such thing as useless info, it all helps to broaden my knowledge base even if it leaves somewhat confused:D Once again thanks and I'm sure this will help others down the track.

[ameerat42]

(The whole thread has already helped me :D)

[arthurking83]

No such thing as useless info, it all helps to broaden my knowledge base even if it leaves somewhat confused ....

:th3:

Yeah, confusing .. can be because most of the info I posted doesn't happen in every day life.
The reason for the long post is that sometimes 'stuff' happens, and that happened to me. (the file corruption during transfer saga).

There are different types of file corruption. In the one instance files got corrupted during failed transers. Those files are gone. Could do nothing to restore them.
Sometimes tho, you get files corrupted 'in camera'. Whether that means the cameras file writing operation failed in some strange manner, or that the card had a hissy fit while the data was being written to it from the cameras inner workings.
I've heard of both, experienced the latter only. That experience was when my son took the D70s on a school excursion and two of about 300-400 images somehow got corrupted while written to the card.

143549
This is what the image looks like in most software I use, primarily Fastone and ViewNX2.
Both those software can be set to view the raw file(it is saved/stored as an NEF) in one of two ways. Either as the quicker loading jpg file, or in higher quality raw mode.
The programs setting uses one of two methods to display the file.

Quick bit of background info: raw files contain basically a whole bunch of numerical data that gets coded to produce an image on your screen. That's what a raw file viewer/converter does. Part of that data tho also contains embedded jpg images, most raw files contain 3 different jpg preview files for the various purposes like displaying thumbnails or quicker previews of the raw file.

In in the software, setting to use quick JPG mode it uses the embedded jpg image in the raw data, the largest file of the jpg files in the raw file. This is what your camera displays on the rear review screen too. Makes everything run quickly.
When set to raw mode, it then runs differently, it uses the raw data does it's magical software gobbeldygookiness in the background and shows you what the image will look like at higher quality settings.

In ViewNX2, there is a quick handy way to switch between the two modes. In Fastone tho you need to go into settings and change this. There is no quick easy way to switch between the two modes.

So, in ViewNX2 if I hit the [RAW] mode button, the images changes from the above photo looking image to this:

143550

Once I do this in VNX2, I can't then revert back to the proper looking non corrupted version of the image. Something to do with caching thumbnails. I have to go delete cached thumbnails and so forth.
No way I can convert the raw file of this photo into a proper looking photo using any raw file viewer/converter. Trust me I've tried.
Only way to get the image converted from NEF to a jpg, is to extract it from the embedded jpg preview file, because while the NEF file data is corrupted, the embedded jpg isn't.
For that I use a program called InstantJpgFromRaw(IJFR), handy little program to have .. very easy to use.

side note: I think the other file that got corrupted on the same day for him looked even weirder than those examples, where the top 1/3 of the image renders fine, and the bottom 2/3rds renders similarly to the corrupted image above.

And this is why I posted all this additional info, because its so easy to backup corrupted files and not know it.

One thing I do know tho, is that using Adobe software for corrupted files in the above manner, you may only ever see the proper image only on initial import into say LightRoom/ACR, but only if you're quick.
I've tested this a few times, and what happens is that initially LR/ACR shows you the embedded jpg files of the raws, but begins a process where it then renders it's own preview files. Once that process reaches corrupted raws as above, no way you can recover it in Adobe software.
And lets say you also have Fastone, and you open that same folder of images, you will see the proper version of that corrupted file as it's default setting is to view the raws in jpg mode(ie. the embedded jpg preview file).

So it can be a complicated and confusing issue in some situations.

After much to-ing and fro-ing and trying to work out how to insure against backing up corrupted files, I found only 1 usable solution. It's a little(very little) tedious, but for me has ensured that I don't backup corrupted files over files that aren't.
That solution is an image(more of a media) viewer called XnViewMP(free).
Actually there are two, but the second one is not worth mentioning as it's a lot more tedious and very occasionally hit and miss.
But XnView is workable. What it does, as well as being a very reasonable viewer like Fastone, it allows you to view all specific image files types at the same time, in the same window, and that filters can be set to display only certain file types, like NEF, or CR2, or whatever your main raw type is.
Most of us store out images in structured folders, date/subject/location/etc. To use a image viewer and have to browse through all your folder and then view each image is not a workable proposition unless your dealing with very low numbers.
Get into the high hundreds or thousands ... and :eek: .. no way.
So XnView allows you to set viewing 'recursively' which means that you open the main folder where all your images are stored(eg. E:\Photos\backups\) it will then takes it's time to build up a database of all the thumbnails from scratch. Care must be taken to be sure that any previous thumbnail database has been deleted too.
Once thumbnails have been created, set the program to view your important file types, otherwise you see all manner of garbage that is useless and creates clutter. All you want to see are the important files .. for me NEF. So that's my filter.
Then in the main window, click on one image and use page down to speed scroll through the entire collection of images.
75K images takes me about 2-3 mins. I don't look at each image specifically, I just look for thumbnails that don't show any actual image as they whizz past.

I do this pretty much once a year, when I do my final backup to the hidden drive on the NAS box .. the one I have to manually turn on before backing up too.

The files that got corrupted during the USB dock saga couldn't be saved in any manner, not IJFR extraction, and I tried a small NEF repair tool I found online ... so sometimes nothing can be done. The only reason I was alerted to the fact that they were corrupted at all, was that no program could display the thumbnails for them, and got me curious as to why no thumbnails .. just that generic cartoony thumbnail that some programs display .. no photo preview.

[tandeejay]

I used FreeFileSync for a while when I was trying to sync photos between the main computer and the laptop.

One thing I found with the download, was that you had to be VERY careful not to click the check box that would install "bloat ware" it was sometimes very trickily worded to make you think that by not checking the box you wouldn't install anything.

I'm currently using Acronis True Image to backup to an external drive.