Gentoo Linux Backup HowTo

Различные виды бэкапа в Gentoo Linux при использовании встроенных утилит

A very simple but useful way to do a remote backup is using dd and netcat.

Netcat is available in two flavors:

emerge gnu-netcat

or

emerge netcat

 nc -l -p 10000 > image.gz

On your machine run dd to read the partition, gzip to compress the content and netcat to transfer it over to the other machine:

 dd if=/dev/hda1 | gzip | nc -w 5 remote_ip 10000

See How to clone a Linux box using netcat for additional information.

For an extremely lean backup system (~30 lines of bash scripting... but even more compressible had I not wanted to make it user-friendly) that relies on the most basic tools (bash, tar, xargs, etc.), take a look at my script: backup.cron Gentoo Forum Thread.

You maintain a collection of backup lists that are effectively arguments to tar and the script will keep n many backups of that data on some hdd. From here, you can do whatever you want with that tarball, like scp it over to other boxes.... redundancy is nice in the backup business. people in the thread have offered extensions like iso images, splitting, etc.

This was designed to be a starting component in your larger backup system. I haven't looked into incremental backups with this, but it is possible using tar... maybe when i have some more freetime...

Flexbackup is a great little perl tool to backup important directories. Like the name implies, flexbackup is very flexible.

# emerge flexbackup

This is what the important part of my /etc/flexbackup.conf file looks like:

$set{'research'} = "/home/david/research /var/cvsroot/python /var/cvsroot/latex";
$set{'mail'} = "/home/david/.thunderbird";
$set{'etc'} = "/etc /home/david /var/www/davidgrant.ca/htdocs";

$prune{'/home/david'} = ".jpi_cache konserve-backup .cxoffice .wine .mozilla .kde3.1 .thunderbird";
$prune{'/home/david/.thunderbird'} = "Junk News";

$compress = 'gzip'; # one of false/gzip/bzip2/lzop/zip/compress/hardware
$compr_level = '6'; 

$device = '/mnt/sata/backup';

The "research", "mail", and "etc" names are just convenient names which you use at the command line when telling flexbackup what to backup. Each directory within "research" will get backed up to its own tarball, however, as "home-david-research.0.tar.gz", "var-cvsroot-python.0.tar.gz", and "var-cvsroot-latex.0.tar.gz".

$prune$ is a useful feature which allows you to mask out certain directories which you don't want to be included in the tarballs.

I set $compress to 'gzip' but you can use bzip2 if you want smaller tarballs. bzip2 is a bit slower to pack and unpack, however. $compr_level=6 also makes things a bit quicker, yet still makes the tarballs quite compact.

I use the following crontab to create my backups:

0 3 1-7 * * flexbackup -set all -full -w 7
0 3 * * 6 flexbackup -set all -differential
0 3 * * 1-5 flexbackup -set all -incremental

This will perform a full backup on the first Sunday of every month, a differential backup every Saturday, and an incremental backup every day of the week. All backups are performed at 3am.

[Flexbackup home page]

Please note that if you have noexec on your /etc/fstab for /tmp,
you must remove it or change flexbackup.conf to point to another directory.  

There is an excellent tutorial regarding incremental backups available here:

[Easy Automated Snapshot-Style Backups with Linux and Rsync]

Using incremental backups, you can create multiple snapshots of your data while conserving disk space and saving processing time. It's a very fast and cheap way of creating snapshot-style backups of your data.

The script is fairly easy to use, even for a non-expert. I just had to make a few simple changes to the backup script on the web site to get it working on my system.

If you need more functionality than Mike Rubel's script can give you, or if you're uncomfortable editing a bash script, then you may be interested in one of these programs:

• rsnapshot

• You want to mirror your disk/backup a partition.
• I assume you have created a partition on another drive and mounted it (in this example /mnt/usbharddrivemain), you can:

Duplicate a partition (Good for a once off):

# rsync --progress --stats -avxzl \
--exclude "/mnt/usbharddrivemain/" --exclude "/mnt/usbharddriveboot/" \
--exclude "/usr/portage/" --exclude "/proc/" \
--exclude "/root/.ccache/" --exclude "/var/log/" \
--exclude "/sys" --exclude "/dev" \
--exclude "tmp/" /* /mnt/usbharddrivemain

Duplicate the partition and delete outdated files (which have since been deleted on the primary partition):

# rsync --progress --stats --delete -avxzl \
--exclude "/mnt/usbharddrivemain/" \
--exclude "/mnt/usbharddriveboot/" \
--exclude "/usr/portage/"
--exclude "/proc/" --exclude "/root/.ccache/" \
--exclude "/var/log/" --exclude "/sys" \
--exclude "/dev" \
--exclude "tmp/" /* /mnt/usbharddrivemain

For my boot (another partition), I use the following (respectively):

# rsync --progress --stats -avxzl /boot /mnt/usbharddriveboot
# rsync --progress ---avxzl --stats --delete /boot /mnt/usbharddriveboot

To restore, one can either boot off the secondary disk OR one can use a Gentoo Live CD. Repeat the above commands (with respect to new mount locations), but alter the source and destination parameters, e.g. /mnt/usbharddrivemain /mnt/driveToRestoreTo not the other way around.

rdiff-backup is a simple to use, yet powerful, backup utility. It can be used to build a mirror of a local source directory on a remote machine. The program automatically keeps an archive of differences with respect to previous mirrored versions so that old files or old version of still existent files can be restored. All the underlying network traffic is handled by ssh. See the rdiff-backup Main page for a detailed description.

rdiff-backup is present on the gentoo portage tree, to install it simply do

emerge rdiff-backup

In order to be able to perform a backup of local files on a remote machine, rdiff-backup must be installed both locally and on the remote machine. In principle, no root privilege is necessary since the program is run with user privilege both locally and remotely.

Let us assume that you properly installed rdiff-backup on the remote machine remotehost.remotedomain. Then the use of this utility can be as simple as

rdiff-backup ~/mydir remoteuser@remotehost.remotedomain::mydir-backup

for this to work, diff-backup must be installed both locally and on <remotehost.remotedomain>.

In this way, a new subdirectory named mydir-backup is created in the HOME directory of user remoteuser at remotehost.remotedomain. If this directory was already present on the remote machine, it is updated to reflect the present content of mydir, but the differences with respect to the previous version are also stored. Examples on how to manage the remote mirror and, in particular, how to recover past version of the whole directory or part of it can be found here.

Depending on the nature of the ssh user authentication scheme in use on the remote machine, the previous command can stop and ask the user for a password or a passphrase. The interactive behavior of the program can of course be avoided if an authentication agent (ssh-agent) is properly configured on the local machine. However, if one wants to implement the backup activity in an automatic, unattended way, for instance as a cron job, it is quite possible that such an agent is not available to the program when it is lunched.

Let us see how this problem can be easily solved.

First of all you need a passphrase-less key pair to be used specifically with rdiff-backup. You can create it with

ssh-keygen -t dsa

when asked, choose .ssh/backup_dsa as the file where to save the new key and press enter when asked for a passphrase.

Now copy the newly generated public key to the remote host

scp .ssh/backup_dsa.pub remoteuser@remotehost.remotedomain:

then log in remotely and add the new key to the list of authorized keys

ssh remoteuser@remotehost
cat backup_dsa.pub >> .ssh/authorized_keys
rm backup_dsa.pub

The last line is not necessary, but it is always safer to remove the unnecessary files. Now you have to tell the remote machine that the new key can only be used for specific tasks. For this purpose, edit the file .ssh/authorized_keys with you preferred editor and add a command specification in front of the newly added key (it will be the last line) to obtain something like

 command="/usr/bin/rdiff-backup --server",no-pty,no-port-forwarding ssh-dss [...meaningless characters follow...] 

Now it's time to leave the remote host and configure the local machine. You have to be sure than when rdiff-backup is started, it makes use of the just generated, passphrase-less, key pair. You can do that using the powerful ssh aliasing feature. Edit .ssh/config and add

Host remote-backup
   Hostname remotehost.remodomain
   IdentityFile ~/.ssh/backup_dsa
   IdentitiesOnly yes

in this way, when you use remote-backup as the host identifier in a ssh or scp command, the host actually contacted will be remotehost.remodomain but the key stored in /.ssh/backup_dsa will be used, instead of the default one (whatever, if any, it would be).

The previous configuration allows for a straightforward use of rdiff-backup as a cron job. If you want an automatic backup, one minute after 1 AM, of ~/mydir add the following line to your cron table

01 1 * * *      rdiff-backup /home/localuser/mydir remoteuser@remote-backup::mydir-backup

notice that the alias remote-backup is used instead of the true host name remotehost.remodomain.

Sometimes you want to backup things that aren't 'ready' to be backed up. Things like /boot on some systems where it isn't auto-mounted, or maybe a running mysql daemon. It would be nice to be able to mount /boot, or put mysql into read-only mode for the duration of the backup. That is where wrapper scripts come into play.

 
#!/bin/sh

# mount /boot for backup
mount /boot

# MySQL Read Only 
# NOTICE: the following method is an EXAMPLE!
# you might not want your passwd in plain-text in a wrapper script.
mysql -u root --password='yourpass' -e "SET GLOBAL read_only=1;"

# Run flexbackup, and pass shell args
flexbackup $*

# give the system 10 seconds to clean up
sleep 10

# MySQL Read/Write
mysql -u root --password='yourpass' -e "SET GLOBAL read_only=0;"

# umount /boot
umount /boot

The basic idea is to do preparation works in your wrapper, run the actual backup, and then set your system back to operationg parameters.

The above mentioned crontab for flexbackup would then become:

0 3 1-7 * * /etc/scripts/flexwrapper.sh -set all -full -w 7
0 3 * * 6 /etc/scripts/flexwrapper.sh -set all -differential
0 3 * * 1-5 /etc/scripts/flexwrapper.sh -set all -incremental 

Источник: http://gentoo-wiki.com/HOWTO_Backup

Комментарии пользователей

Anonymous

Thanks Ответить

Логин: Anonymous [Авторизоваться]

Текст сообщения