VectorLinux
July 23, 2014, 03:20:53 am *
Welcome, Guest. Please login or register.
Did you miss your activation email?

Login with username, password and session length
News: Visit our home page for VL info. To search the old message board go to http://vectorlinux.com/forum1. The first VL forum is temporarily offline until we can find a host for it. Thanks for your patience.
 
Now powered by KnowledgeDex.
   Home   Help Search Login Register  
Please support VectorLinux!
Pages: [1]
  Print  
Author Topic: In praise of rsync.  (Read 1574 times)
nightflier
Administrator
Vectorian
*****
Posts: 4018



« on: July 01, 2009, 06:12:41 am »

A few years back, Bigpaws pointed me in the direction of rsync for performing data backup. Now I use this little gem all the time. Local file transfers, between machines on my LAN, through an SSH tunnel over the Internet, rsync does it all quickly, efficiently and reliably.

Latest success story I'd like to share: Got a computer in for repair. XP refusing to load. Tried a non-destructive reinstall, but the installer didn't even recognize old windows installation. A fresh install usually fixes stuff, but at the cost of having to re-install all your applications again. The box had MS Office and some other proprietary software, and of course the original install media/unlock codes were not with the machine. Besides, there were a bunch of customizations, bookmarks, the usual.

I booted a Linux Live CD and took a look. NTFS partition was there, files readable. I rsync'd everything from C: to my file server.

Next I ran the built-in, destructive recovery feature. Three hours later I had a new default OEM install of Windows. Another 45 minutes and it was cleaned up.

Then, I booted the Live CD again and reversed the copy process, overwriting the existing files on drive C:

I will admit my expectations were low, but windows loaded, with all the old settings intact. It ran slow, and I found that the filesystem had gotten very fragmented. I ran defrag, it was still running when I went to bed two hours later. Next morning it was done and the machine ran fine.

Open source empowers users.
Logged
sledgehammer
Vectorian
****
Posts: 1419



« Reply #1 on: July 01, 2009, 07:09:49 pm »

Nightflier,

Sometime ago, Bigpaws, I believe, suggested I consider rsync as an alternative to simplelinuxbackup.  It took a while, but now I use the Grsync root backup tool (on the 6.0 menu) instead of simplelinuxbackup.  I use Grsync to do a complete backup of the data partition on my laptop at least once a week. 

When (if) you get time, could you give a couple of examples of how you use rsync to transfer files between machines on a LAN?  I'm going to look into that as it could be very quite helpful. Right now, we copy to a usb stick or email.  Rather inconvenient and probably not all that secure.

John
Logged

VL7.0 xfce4 Samsung RF511
hata_ph
Packager
Vectorian
****
Posts: 3256


-- Just being myself --


« Reply #2 on: July 01, 2009, 11:13:16 pm »

Ya, I also am interested to learn about rsync... Smiley
Logged
nightflier
Administrator
Vectorian
*****
Posts: 4018



« Reply #3 on: July 02, 2009, 06:36:56 am »

For quick local operations, the basic syntax is:
Code:
rsync -av /path/to/source/ /path/to/destination
The "a" option specifies "archive" mode, which preserves permissions, "v" gives more feedback.
Adding a trailing slash to "source" avoids creation of a "source" sub-dir in "destination".
In the above described rescue operation I first mounted my file server using cifs.

rsync is best for directories, not individual files. To copy a single file, use scp:
Code:
scp /path/to/sourcefile username@remote-macine:/path/to/destination
or the other way:
Code:
scp username@remote-macine:/path/to/sourcefile /path/to/destination

My desktop is a little non-standard: http://forum.vectorlinux.com/index.php?topic=4564.0 I use my central file server a lot. Big files go over the network mountpoints to the server, and scripts sync my home dir with a copy that resides on a server drive.

This example illustrates how I use rsync to back up my home dir:
Code:
#!/bin/sh
# Backs up home dir to server. Runs every hour from crontab
rsync -auz -e "ssh -i /home/username/.ssh/username-rsync-key" --exclude-from /home/username/scripts/exclude --delete /home/username/  192.168.1.X:/home/username

- This archives all files with permissions intact (a), updates newer files (u), uses compression (z),
- connects using ssh and a key without a password * (see below),
- excludes mount points that map to the server, using a simple text file:
Code:
Music
Storage
- deletes extraneous files on server.

Another script shuts the machine down:
Code:
#!/bin/sh
# Opens a terminal, runs backup, then shuts down
konsole -e rsync -auz -e "ssh -i /home/username/.ssh/username-rsync-key" --exclude-from /home/username/scripts/exclude --delete /home/username/  192.168.1.X:/home/username

sudo /sbin/shutdown -h now

Adding the "konsole -e" gives me a terminal window so I can see the progress.

* For automated network operations, using ssh without first mounting the network resources, you first need to configure your machines for access using password-less public/private key pairs. I relied heavily upon this guide:
http://troy.jdmz.net/rsync/index.html
For backup through the internet I implemented the restrictions script, but not for my local network.

Logged
sledgehammer
Vectorian
****
Posts: 1419



« Reply #4 on: July 02, 2009, 08:56:11 am »

Thanks!

I have bookmarked this page and will return to it often.

John
Logged

VL7.0 xfce4 Samsung RF511
Windozer
Vectorite
***
Posts: 386


Have Vector Linux, Will Travel.


« Reply #5 on: July 03, 2009, 10:55:00 am »

Hi Nightflier,

do you think this would work for backing up temporarily mounted NTFS partitions?  I've got a mish-mash of XP and Linux boxes. At this point, the XP stuff is automated in the usual way w/NTbackups to a server box. But for the Linux stuff I just let cron kick off a simple copy of what I need ... it would be nice to be able to have one thing do both.  Here's what concerns me:

1. having to jump through more SAMBA hoops
2. Would the NTFS stuff restore properly after the image was saved on say Reiserfs?

I've put ghosted XP drive images on and off of Reiser before, but it seemed weird somehow... it worked...

Am I choosing a more difficult route trying to us rsync? 

thanks
- Howard
PS: I think your non-standard tempfs system is really cool!
Logged

483,617th Registered Linux Snoozer
bigpaws
Vectorian
****
Posts: 1843


« Reply #6 on: July 03, 2009, 11:50:59 am »

It would work with any mounted filesystems. You would just need to inform
rsync when the drives were mounted.

I use rsync in cygwin as well as on Linux. My clients have off site and on site backups
being preformed with rsync. The only time I have had a failure is when there is no
connection.

Thanks for the praise guys.

Bigpaws
Logged
Pages: [1]
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.19 | SMF © 2013, Simple Machines Valid XHTML 1.0! Valid CSS!