Lets make a killer backup tool

What do you have and what do you want?

Moderators: b1o, jkerr82508

User avatar
viking60
Über-Berserk
Posts: 9351
Joined: 14 Mar 2010, 16:34

Re: Lets make a killer backup tool

Postby viking60 » 25 Jan 2011, 14:45

OK here is something that goes without error messages:

Code: Select all

#!/bin/bash
for i in `cat viking.backup`;
do rsync -azrvv --delete --delete-excluded  --exclude-from 'ex.txt'  $i /media/My?Book/thomas-pc_backup/;
done

If you wonder about the ? in My?Book it is necessary because omitting will not work in rsync for HD names with spaces.
The files viking.backup and ex.txt must be in the same directory as the script. Preferably at /home/<your_name>/

In viking.backup is where I put what I want to backup:

Code: Select all

/home/thomas
/etc

As you can see I want to backup the entire <your_name> directory
I also want a complete backup of the etc directory
Since I want that, I have to run the backup as root.
Now that will present an error because the file /home/<your_name>/.gvfs is spesifically denied too root so we have to exclude it from the backup if we want to run it as root. So it will get an honorable mentioning in my ex.txt file where I keep all the files and directories that should be excluded:

Code: Select all

*/tmp     #excludes the entire tmp directory
*/.config/gsmartcontrol/*        #excludes the content of your /home/<your_name>/.config/gsmartcontrol directory
*/.cache/*     #based on the above; what do you think this does :)
*/.gvfs

I use the * instead of <your_name> which is more elegant.

This works! The excluding works. BUT the deletion of excluded, already existing files, on the backup does not happen.
Depending om my mood I will now try to sort out the delete problem or refine functionality of the script a bit further....
Edit: I solved the delete problem and edited the code above :dance:
Now the functionality is perfect. Going on to play with the script now....
Manjaro 64bit on the main box -Intel(R) Core(TM) i7 CPU 920 @ 2.67GHz and nVidia Corporation GT200b [GeForce GTX 275] (rev a1. + Centos on the server - Arch on the laptop.
"There are no stupid questions - Only stupid answers!"

User avatar
rolf
Guru-Berserk
Posts: 1107
Joined: 16 Mar 2010, 16:07

Re: Lets make a killer backup tool

Postby rolf » 25 Jan 2011, 15:11

Good onya! :cheers

I've eyed that clean and efficient --excludes text file option but did not want to put the effort into learning how to implement it. Now, my evil scheme is finally beginning to pay off. :twisted:

On --exclude erasing already copied files, I had settled that it was not intended to work that way but, maybe, your Norse-land reindeer herder work ethic will bring me more fruits, there. Keep it up!
Image

User avatar
viking60
Über-Berserk
Posts: 9351
Joined: 14 Mar 2010, 16:34

Re: Lets make a killer backup tool

Postby viking60 » 25 Jan 2011, 15:21

Yes I edited above and after two Fluesopp and several Fleinsopp I solved it (Historical fact: The Berserks eat that - thats why they went Berserk).
You can now exclude all you want in ex.txt or whatever you want to call it! :dance1 And it will be deleted in your existing backup.
It looks like this:
:A
Manjaro 64bit on the main box -Intel(R) Core(TM) i7 CPU 920 @ 2.67GHz and nVidia Corporation GT200b [GeForce GTX 275] (rev a1. + Centos on the server - Arch on the laptop.
"There are no stupid questions - Only stupid answers!"

User avatar
rolf
Guru-Berserk
Posts: 1107
Joined: 16 Mar 2010, 16:07

Re: Lets make a killer backup tool

Postby rolf » 25 Jan 2011, 16:03

Good job! :s
If I ever run out of pannekakers, I might have a look at what are Fluesopp and Fleinsopp :gimme
Thanks. :coffee_smile:

Oh, after looking at the berserker wikipedia, I see that Fleinsopp is something I once picked in Oregon's cow pastures, then into a blender for a milkshake. That was long ago.
Image

I'll stick with pannekakers! Image

p.s. After seeing what looked like a suckling duck, I had to go to Youtube to see your desktop movie. However, even at fullscreen, here, I'm afraid the resolution is not high enough for it to be practical to me :( Maybe the imaginary part works well... :think:

User avatar
viking60
Über-Berserk
Posts: 9351
Joined: 14 Mar 2010, 16:34

Re: Lets make a killer backup tool

Postby viking60 » 25 Jan 2011, 21:45

I'll make another one later.
Now I need an expert to tell me how to execute backup.sh as a cron job?
Manjaro 64bit on the main box -Intel(R) Core(TM) i7 CPU 920 @ 2.67GHz and nVidia Corporation GT200b [GeForce GTX 275] (rev a1. + Centos on the server - Arch on the laptop.
"There are no stupid questions - Only stupid answers!"

jkerr82508
Guru-Berserk
Posts: 211
Joined: 16 Oct 2010, 21:45
Location: Fife, Scotland

Re: Lets make a killer backup tool

Postby jkerr82508 » 25 Jan 2011, 23:28

The easiest way to set up cron jobs is probably to enter 'crontab -e' in a terminal and enter the necessary info. (There are several alternative ways.) These pages may be helpful in understanding the format of the crontab file:

http://www.linuxhelp.net/guides/cron/
http://www.scrounge.org/linux/cron.html

I think that there is a GUI for KDE, probably called kcron or something like that, but I've never used it.

Jim

User avatar
viking60
Über-Berserk
Posts: 9351
Joined: 14 Mar 2010, 16:34

Re: Lets make a killer backup tool

Postby viking60 » 25 Jan 2011, 23:45

Yes I am familiar with the format but the bash script will not run corectly.

Code: Select all

00 12 * * * /bin/sh /home/thomas/backup.sh

Does not run :confused
Manjaro 64bit on the main box -Intel(R) Core(TM) i7 CPU 920 @ 2.67GHz and nVidia Corporation GT200b [GeForce GTX 275] (rev a1. + Centos on the server - Arch on the laptop.
"There are no stupid questions - Only stupid answers!"

User avatar
viking60
Über-Berserk
Posts: 9351
Joined: 14 Mar 2010, 16:34

Re: Lets make a killer backup tool

Postby viking60 » 26 Jan 2011, 00:21

New video as promised:
:A

It all works dam good now.
Well except for that cronjob. It probably is something obvious - as usual.
Manjaro 64bit on the main box -Intel(R) Core(TM) i7 CPU 920 @ 2.67GHz and nVidia Corporation GT200b [GeForce GTX 275] (rev a1. + Centos on the server - Arch on the laptop.
"There are no stupid questions - Only stupid answers!"

jkerr82508
Guru-Berserk
Posts: 211
Joined: 16 Oct 2010, 21:45
Location: Fife, Scotland

Re: Lets make a killer backup tool

Postby jkerr82508 » 26 Jan 2011, 02:10

I would have thought that

Code: Select all

00 12 * * * /home/thomas/backup.sh

would suffice. My scripts are all (very simple) bash scripts, which I make executable and they run OK. For example:

Code: Select all

05 * * * * /home/jim/bin/quick-bu

but I'm no expert on cron, or much else these days. :)

Jim

User avatar
rolf
Guru-Berserk
Posts: 1107
Joined: 16 Mar 2010, 16:07

Re: Lets make a killer backup tool

Postby rolf » 26 Jan 2011, 02:22

Maybe cron isn't running as if in your ~/ directory? Maybe the script needs a cd /home/thomas or full paths for the arguments? idk Make executable comes to mind, also but idk.

User avatar
viking60
Über-Berserk
Posts: 9351
Joined: 14 Mar 2010, 16:34

Re: Lets make a killer backup tool

Postby viking60 » 26 Jan 2011, 08:35

That is the full path to the script.
jkerr82508 wrote:
but I'm no expert on cron, or much else these days. :)

Jim

To modest there - fore sure :!:

I tried to provide the path to rsync too so it looks like this:

Code: Select all

28 08 * * * /bin/sh /bin/rsync /home/thomas/backup.sh

:confused
I am going to check my installation now must be something I have not installed...
Manjaro 64bit on the main box -Intel(R) Core(TM) i7 CPU 920 @ 2.67GHz and nVidia Corporation GT200b [GeForce GTX 275] (rev a1. + Centos on the server - Arch on the laptop.
"There are no stupid questions - Only stupid answers!"

User avatar
rolf
Guru-Berserk
Posts: 1107
Joined: 16 Mar 2010, 16:07

Re: Lets make a killer backup tool

Postby rolf » 26 Jan 2011, 09:14

I was thinking along the lines of putting

cd /home/thomas

at the beginning of the script or using `cat /home/thomas/viking.backup` and '/home/thomas/ex.txt'
in case it's a matter of cron not knowing the working directory, or like that. :confused


Return to “Software”