Lets make a killer backup tool

[dedanna1029]

There is something missing.

Once everything is backed up, have it zip or tar.gz the backed up files, then delete the originals.

Else you will be eating hard drive space amundo.

[viking60]

I cannot see that we are missing that. This script will delete anything that has been backed up that is not on the main box anymore. So the backup will be a replica of your home (and every other directory you choose to back up) with the exclusion you would like to do in ex.txt. It is not ziped or tared because it is so incredibly easy to put back the files you need when you have fubared.
You just navigate to the file you need and put it back. The exclude list is the instrument to keep the backup as small as possible. Of course you can zip or tar it as you like but that will disturb the synchronization.

[dedanna1029]

Oh okay. So there is compression already. Sorry, wasn't seeing it, at least not blatantly.

[viking60]

There is no compression. As I see it this is not that kind of backup.

[dedanna1029]

Okay, that's why I mentioned that above. It will eat hdd space, no, if there isn't any?

[viking60]

Of course. a backup has to be stored. If you do not have any space, you could put in on dropbox. If you dare

[viking60]

I have had the backup script as a cron job and since I am not a trusting guy I needed a log to make sure that the backup has been made.
So I added a functionality where the script creates backup.log and adds the date and time of the backup (when the backup ended).
Every new backup will be added as a new line in the backup.log. You can delete it whenever you want the script will create a new backup.log and add the info on the next backup.
So the script looks like this now:

Code: Select all

#!/bin/bash
cd /home/thomas
for i in `cat viking.backup`;
do rsync -azrv --delete --delete-excluded  --exclude-from 'ex.txt'  $i /media/My?Book/thomas-pc_backup/;
echo -n 'last backup is at the bottom of this list:'>>backup.log|date>>backup.log
done

The content of backup.log will look like this:

last backup is at the bottom of this list: ma. 14. feb. 21:03:25 +0100 2011
last backup is at the bottom of this list: ma. 14. feb. 21:03:31 +0100 2011

All I have to figure out now is why it writes two lines per backup to backup log???
Could it be something in the timing/date?
However this is not critical if the lines are there the backup has been made,

[viking60]

Ok moving on..
The script had the obvious problem that you manually had to create ex.txt (for exclusions) and viking.backup (for content to back up).
So I have made a check if the files do exist and if not they will be created and populated with data.
This way it is possible for a beginner to start using the script without editing viking.backup and ex.txt.
The huge advantage of selecting what to back up and what to exclude in separate files can then be explored later.

The manual labor is now down to providing the home path on top of the script and providing the path to the backup medium (towards the end of the script).
Here we go - suggestions are welcome:

Code: Select all

#!/bin/bash
cd /home/thomas   #You must provide your home directory here
# checking if ex.txt exists
if [ -e 'ex.txt' ]
then
echo "ex.txt exists"
else
#Creating ex.txt and populating it with /home/<my_name>/.gvfs to avoid errors when running as root
echo '#Here you can put the all the files and directories you do not want to backup. Like the example below:'>>ex.txt
echo '*/.gvfs'>>ex.txt
echo "ex.txt has been created"
fi
#Creating viking.backup and populating it with my home directory and /etc (must run as root!)
if [ -e 'viking.backup' ]
then
echo "viking.backup eksisterer"
else
echo '/home/thomas'>>viking.backup #provide your home path inside the ''
echo '/etc'>>viking.backup
fi
#The Magic happens!
for i in `cat viking.backup`;
do rsync -azrv --delete --delete-excluded  --exclude-from 'ex.txt'  $i /media/My?Book/thomas-pc_backup/; #provide the path to where your backup is to be stored here
#Creating a log (backup.log) to se when the last backup was run - nice when checking cron jobs
echo -n 'Last backup at the bottom:'>>backup.log|date +'%d-%m-%Y at %H:%M'>>backup.log
done

We are now down to one file that requires some input data.
Please test it
(you can do that by editing viking.backup in your home directory, and pointing it to a small directory or file. That way you will not back up your entire home directory when testing)

[viking60]

OK I get it. It was to complicated for you so I have made a Dummies version

Code: Select all

#!/bin/bash
#Setting up some colors I might use later here:
red='\e[0;31m'
RED='\e[1;31m'
blue='\e[0;34m'
BLUE='\e[1;34m'
cyan='\e[0;36m'
CYAN='\e[1;36m'
NC='\e[0m' # No Color
echo -e "Hi, please type $CYAN/home/<your name>$NC here (your home directory): \c "
read  hjem
echo -e "Hi, please type the $CYAN/full/path/to_the target/$NC here (where your backup will be stored): \c "
read maal
cd $hjem   
# checking if ex.txt exists
if [ -e 'ex.txt' ]
then
echo "ex.txt exists"
else
#Creating ex.txt and populating it with /home/<my_name>/.gvfs to avoid errors when running as root
echo '#Here you can put the all the files and directories you do not want to backup. Like the example below:'>>ex.txt
echo '*/.gvfs'>>ex.txt
echo "ex.txt has been created"
fi
#Creating viking.backup and populating it with the home directory and /etc (must run as root!)
if [ -e 'viking.backup' ]
then
echo "viking.backup exists"
else
echo $hjem>>viking.backup #Creates viking.backup and putting </home/your_name> in there
echo '/etc'>>viking.backup #adding the /etc directory to be backed up
fi
#The Magic happens!
for i in `cat viking.backup`;
do rsync -azrv --delete --delete-excluded  --exclude-from 'ex.txt'  $i $maal;
#Creating a log (backup.log) to se when the last backup was run - nice when checking cron jobs
echo -n 'Last backup at the bottom:'>>backup.log|date +'%d-%m-%Y klokken %H:%M'>>backup.log
done
#Viking says roar!!!

Just start the script and follow the prompts and the magic will happen
The script is preset to back up /home/<your_name> and /etc. You can run it as your normal user but then it will not back up /etc and give you error messages.
The next time you run it as root it will be correctly synced though.
To backup somthing different than your home directory you can create viking.backup manually (in your home directory) and put in what you would like to back up.
like this:

/home/me/Documents/my_document.ods
/var

nothing but the paths - one per line must be put in there.
(Or you could brake the script after a few seconds with CTRL+Z and edit the created viking.backup in your home directory to back up a smaller directory)
Edit ex.txt in the same way to exclude what you do not want to be backed up.

'
It works!
I think we shall call this the berserk backup script 1.0_Beta

[rolf]

It was to complicated for you so I have made a Dummies version

Seriously, congratulations! I've been just too jammed to do any beta testing. Maybe this weekend....

[viking60]

I need to brush up the comments and make it all english. Also if you have strange foreign dates you need to modify the date.

Code: Select all

date +'%d-%m-%Y

might be

Code: Select all

date +'%m-%d-%Y

or something more exotic.
And if you have a viking.backup in your home directory already it will not be affected by this script. It will back up what is ordered there.
The same goes for ex.txt - It will exclude what ever is in there too.

[rolf]

Ok, I woke up early, so I could do a little test with your script. To do so, I created a viking/ directory on my backup partition. This is on a Seagate sata disk in an external enclosure, connected by firewire. My theory is the disk will last longer if I only turn it on to do some backup operation, idk. The script seems to work fine. I made a little mistake not entering the full $HOME path but it did not matter, as I had made the viking.backup and ex.txt files.

I noticed, after beginning, my choice for the backup test was quite large, ~13G, so I stopped the test. It seems to work fine. As my system does not back up the entire home directory and I do not feel a need to save /etc, I think I will incorporate your two configuration files and backup command into my situation. Thanks for the work!