Ubuntu auto-backup system

If you’ve ever been bitten by data loss or partition wipe on system upgrade, namely Ubuntu, read on. Or just get the (not yet complete) script from my GitHub.

I won’t explain why backup or custom install more here.

Backup means…

Way often I hear “backup” meaning something else entirely, so:

*Synchronization* – is when you have a mirrored copy. Anytime file A is changed, it’s copy gets changed. That’s NOT backup. If you delete A by accident, you lose it.

*Snapshot* – is an incredibly nice and meaningful word which contains lots of meanings. None means backup, however.

Backup means:

  1. data you want safe can be safely restored in a timely (enough for you) manner,
  2. restoring all data or (if you choose to) just selected parts of it,
  3. restoring either most recent or older versions, for as many versions as you have,
  4. automated solution.

There’s obviously more to backup, like local vs remote, or image vs differential, but the gist is in points above.

Design restore, not backup

Have you restored it? If not, suck it. Plan for restoring the data, that’s why you do it. You want restore to be painless and full.

When planning what will be backed-up and where, take into consideration:

Why? – Why are you protecting yourself against disaster? Does it matter if you lose data? What losses will you suffer ($$$)?

What? – What are you going to backup? Your entire hard drive or just some of the data?

When? – When is the best time to backup your system? How often will you perform a backup? When will you use full backups and incremental backups.

Where? – Where will the backups be stored? On-site? Off-Site? Cloud?

Medium? – Attached storage (usb stick, usb hard drive, tape drive), backup server?

Recovery time objective (RTO): How fast should data be recovered? Can you continue to operate if data recovery is not recovered for a day, a week, etc?

Recovery point objective (RPO): How much data can be lost. Can you lose two hours, two days or two weeks of data?

– quoted from Ubuntu backup manual: https://help.ubuntu.com/community/BackupYourSystem

Ubuntu?

So, system-specifics. And they’re not just Ubuntu’s, other distros are rather similar. Can’t be more specific which distros cause there’s many of them. Still, backup the following:

1. /home (Users’ personal files and settings)
2. /etc (System-wide configuration settings)
3. /var/spool/cron/crontabs (cron goodies, auto-runs)
4. The backup/restore script
5. List of installed software
6. Hardware and other info

For first three see tools, below. Script – put in home, then backing it up backs it up along. You want this to be fully automated. Read on for more.

Exclude what you don’t need:

/home/*/.cache
/home/*/.dbus
/home/*/.dropbox
/home/*/.gvfs
/home/*/.local/share/Trash
/home/*/.mozilla/firefox/*.default/Cache/**
/home/*/.qt
/home/*/.thumbnails
/home/*/Examples
/home/.ecryptfs
/home/lost+found

Installed software

I went with apt-clone over dpkg --get-selections for it preserves all repositories information, keeps track of what packages were automatically installed, allows to repack locally installed DEB files and does all that in one command.

Beyond files and data: hardware and disk partitions

Quoting OldFred super-moderator from UbuntuForums:

Various hardware listings/configurations
lshw -html > ~/hardware_info.html
udevadm info --export-db > file.txt
dmidecode > bios.txt

Now go to http://sourceforge.net/projects/bootinfoscript/ and download the script there.
sudo bash ~/path-to-extracted-script/boot_info_script055
This creates RESULTS.txt with MBR/Grub/partition information.

Tools

DejaDup works out of the box (and since version 24 each backup has integrity test made), but if you need more, see threads in sources for more than 40 tools you can use. I shall name only a few that deserved highlight:

Duplicity

The power behind the DejaDup frontend. Run DEJA_DUP_DEBUG=1 deja-dup --backup and see for yourself. Ideal for scripting and battle-hardened.

Etckeeper

sudo apt-get install etckeeper and with Bazaar installed that’s it, without it, go to /etc/etckeeper/etckeeper.conf and uncomment your VCS and comment bzr. Then run etckeeper init; etckeeper commit "Etc dir as of now" and there you go: now your etc is under version control of your choice. Just copy the .git or .hg or whatever to backup and there you have it. You may need to tweak rights though, it’s root:root.

Additional love points for being able to call VCS commands with etckeeper vcs commandGoesHere and the hooks on apt it has, each apt (un)install is clearly recorded.

Storebackup

http://www.nongnu.org/storebackup/en/node2.html

It has unique features and is a re-thought backup tool. For example, it can detect duplicate files (even on different computers), can split large images (TrueCrypt, Xen, KVM, …), supports isolated backups during travel to later integrate them with master backup, restore even without Storebackup etc.

I was bought by the premise: max comfort for the user.

Cloud?

I found two things of interest here, though both just synchronize data. Third obvious choice is Dropbox (30 days of history only though!). Of course, there’s more, simple search for _online backup solutions_ will find them.

Symform

http://www.symform.com/

10 GB for free, possible to get more if you either pay or contribute your own unused HDD space. Nice GNU/Linux install for most popular distros (.deb file). Somewhat slow UI though and can’t work with nested directories (error message said “can’t apply changes” so go figure what’s wrong).

Cyphertite

https://www.cyphertite.com/

8 GB for free, encrypted, lose the key === lose the data. I’ve played with it less than with Symform for now.

Updated on 20th june:

Cyphertite just announced closure. August 18th they will close the service, it wasn’t profitable enough. They recommend Tarsnap instead, but it ain’t as free as Cyphertite was. It’s closely¬†secure though, perhaps even more.

Sources

What I used to write this up.

General:
http://www.theregister.co.uk/2013/07/12/storagebod_monomyth/
https://help.ubuntu.com/community/BackupYourSystem
http://ubuntuforums.org/showthread.php?t=2244774
http://ubuntuforums.org/showthread.php?t=1748541

Tools:
http://sourceforge.net/projects/bootinfoscript/
http://www.nongnu.org/storebackup/en/
https://www.cyphertite.com/
http://www.symform.com/

Recovery:
https://help.ubuntu.com/community/DataRecovery
https://wiki.ubuntu.com/RecoveryMode
http://askubuntu.com/questions/9135/how-to-backup-settings-and-list-of-installed-packages
http://askubuntu.com/questions/99739/how-can-i-test-the-integrity-of-a-backup-made-with-backup-deja-dup

Advertisements

One thought on “Ubuntu auto-backup system

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s