Distribute.IT Disaster - lessons to be learned

[Kym]

This thread is about learning lessons not bitching about Distribute.IT

Background:

Distribute.IT an Australian hosting company were cyber attacked and and as a result 4,800 sites were wiped out.
The gory details are in the following two links.

http://www.distributeit.com.au/
http://www.smartcompany.com.au/internet/20110622-five-lessons-from-the-distribute-it-hosting-disaster.html

So what should I be doing about my hosted sites/blogs etc.?

In a word backup!
I don't just mean relying on your hosting providers backup, but also implementing your own secondary backup.
FYI AP runs a daily backup and that is also copied daily to another site on the other side of the planet from our hosting provider.

It is also worth noting that I don't actually care about backups...
I ONLY care about restore!
So make sure you can restore from what ever backup you use. Test it regularly.

If you have a gallery or other site, you should think about implementing some form of backup that you have control of.
This is especially true if you run any form of online commerce site.

[kiwi]

Ouch

Kym, I have a host and that host also provides the website config tools etc - I wouldnt have the foggierst idea how to back that up anywhere. Perhaps you could provide some guidance on how you might go about that ?

[Kym]

How AP does it is as follows (on a Linux server)


Around 3am we take AP offline and backup the database and compress (zip) the backup files
This is done using a scheduled (cron) job


During the day the site (including the zipped database backups) are incrementally copied to another server (mine in fact)


My server gets backed up nightly, which contains, among other things, the 12 GB that makes up AP
I also do off site backups of my server from time to time (every few weeks)


AP's hosting provider also makes weekly backups


The above strategy gives AP multi level protection.

Technically it's not that hard, but you do need to have some Admin tech skills which means the non-tech user will need assistance.

[Kym]

Another one...


Many Accounts (Cpanels have been hacked) by "amhdosa Hacker" today, alot of accounts affected www.osmosis.net.au (currently down) 4.08pm est , people are in panic stages with lost websites

Email from Osmosis.


+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
We had a major incident last night on our web hosting server. Nearly all of our websites on this server were effected.

One of our clients did not secure their website and this allowed hackers to gain access. This is very difficult to monitor as we cannot easily find these "holes" on our servers.

This also effected our recent backups and we were unable to restore from a recent backup – this is why the server has been unaailable most of the day.

We were able to get a backup from mid may 2011 and restore this – this will have effected any sites that have had changes to their source code or databases since this time.

If you have a more recent database backup and source code then please load these onto your websites so that you can bring them back up to date.

If we can help in any way then please contact us.

I understand that this will cause problems for you or your clients and we apologise for this.

We will make changes to our backup strategy to prevent this from happening again as we take daily and weekly backups.

I would also encourage you to take a backups of your database and source code from time to time and to copy them to a place off of our server. This can then act as a emergency copy as well.

If you are using open source software then make sure that it has the latest security patches – hackers look for these sites and break into them.

regards
David Richardson

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++