Pinterest Stumbleupon Whatsapp
Advertisement

Rsync is fairly simple: it’s a tool that’s specialized in copying files. For us, this means that rsync removes many inconveniences involved in manual backups. This results in a more seamless backup process, compared to using the file manipulation commands native to the Linux terminal How to Manage Files in the Linux Terminal and Desktop How to Manage Files in the Linux Terminal and Desktop Once you make the change to Linux, you'll find a sensible desktop that is easy to use, from installing new apps and launching them to organizing your data. Read More .

For example, rsync recognizes unchanged files from the last transfer, and saves time by not overwriting them. Other things like the ability to compress your files also make your backups more speed efficient. While these sorts of things could be theoretically done by hand, rsync puts all of these tasks into one convenient tool 10 Easy Ways to Restore Your Linux System 10 Easy Ways to Restore Your Linux System Windows' System Restore feature is a good way of making and maintaining entire system backups. If only Linux had a similar feature... oh wait, it does - in fact, we've got 10 options to choose... Read More .

We’ve already covered Grsync Grsync - A Simple GUI to Help You Use 'rsync' Easily [Linux] Grsync - A Simple GUI to Help You Use 'rsync' Easily [Linux] Read More in the past, but knowing how to use the tool powering it (that is, rsync) can prove to be a valuable asset. Hopefully, this article will demonstrate to you that using rsync without a graphical back-end is a fairly simple task.

Rsync Basics

All rsync commands are fundamentally the same, so it should be easy enough to pick up. Here’s the most basic command which simply copies the contents of one folder to another:

rsync -r -u -v ~/Source-Folder/ ~/Copy-Folder

rsync basic

The -r option stands for “recursive“. Put simply, without this option, rsync ignores files which are stored inside folders, meaning it won’t copy everything. We add the -u option (for “update“) to ensure that your transferred files won’t overwrite files in the target folder which are more up to date. For example, if you edited a file in the transfer folder, but didn’t from the original folder.

Advertisement

The -v option (for “verbose“) lets you see what rsync has done, which is good for monitoring its behaviour and actions. It’s not strictly necessary, but you might end up appreciating the extra information — without it, rsync is much more silent.

For more advanced forms of backup, all we have to do is to add extra options (i.e. -[letter]) to rsync. You can actually put all these letters together in one single, big option (e.g. -ruv) if you want to save space. Just remember to put them in before specifying your folders!

Choosing Your Backup Directories

As you saw above, you first select the folder which you’d like to copy files from, and then select where you’d like them copied to. Also take note of the trailing / at the end of the source folder. Doing this ensures that you’re only copying the contents of the folder, rather than the folder itself. You can leave out this forward-slash if you’d rather bring the folder along.

Helpful tip: the ~/ symbol represents your home folder What Are Those Folders in Your Linux Root Directory? What Are Those Folders in Your Linux Root Directory? Open a file manager on your Linux box and select Computer in the sidebar to display your system folders. But do you have any idea what each of them hold? Let's take a look! Read More (where your Documents, Desktop, Downloads, Music, etc. folders are stored). This is much faster than simply typing out your full source folder location, and is username agnostic.

Excluding Files and Folders With rsync

Sometimes, you don’t want to back up entire folders worth of data, and rsync can handle that as well. Apart from just choosing more specific folders to copy, you can also use the –exclude option to skip them. This lets you tell rsync to ignore a selected folder, file, or pattern.

rsync -ruv --exclude 'Subfolder' ~/Source-Folder/ ~/Copy-Folder

rsync folder exclusion

As you can see above, the Subfolder directory was not transferred. You can also exclude files with this: just type its name down in quotes.

To stop multiple similar files/folders from being transferred, use the * symbol with the –exclude option. This acts as a substitute for any other file name.

rsync -ruv --exclude '*.txt' ~/Source-Folder/ ~/Copy-Folder

rsync pattern exclusion

This command meant that rsync ignored all files that ended with .TXT and only copied along a folder. The * symbol acts as a wildcard — it represents all the potential words and letters you could, in this example, name a TXT file. This is a basic exclusion pattern for rsync.

There is something important you should know about exclusions: they are located relative to your copy source! Put simply, you need to tell rsync the position of the files you are excluding in relation to where you chose to copy your files. Here’s an example of this in play:

rsync specific exclusion

Since we are copying data from the Source-Folder directory, we don’t need to specify where exactly the Subfolder directory is located. It’s right inside it. However, if we then want to exclude File-1.txt from inside that folder, we need to state its location, with the ‘Source-Folder’ directory as its root. Keep this in mind if you find your exclusions failing!

Making Backups Faster With rsync

As previously stated, rsync has the ability to compress the files it copies, then decompress them at the other end. This is meant to reduce the amount of data transfer required to copy a file, trading time for the CPU usage needed for compression. So if you’re on a laptop and want to save some battery life Easily Increase Your Battery Life With TLP for Linux Easily Increase Your Battery Life With TLP for Linux Linux tends to guzzle up more battery life than Windows, even though most Linux installations are lighter than Windows on system resources. Why is that? Read More , you may not want to use this.

rsync -ruv -z ~/Source-Folder/ ~/Copy-Folder

rsync compression

All we’re doing here is adding the -z option to rsync: this represents the compression option. It’s short for zlib, which is the software rsync uses to do this. Essentially (pun intended) it zips your files from one place to another.

To see the improvements in transfer speeds, simply look at rsync’s output. More specifically, the “speedup is [x]” (measured in seconds). Use this as a gauge to whether or not compressing your file backups is worth it to you. Every computer is different!

Testing the Waters

Before leaping in the deep end and using rsync proper, it’s always good to make a dry run first. Doing this allows you to see exactly what rsync will copy and where, before the data is backed up. All you need is to add the -n option (short for “no changes made“) to your command to do a test run:

rsync -ruv -n ~/Source-Folder/ ~/Copy-Folder

rsync dry run

As the above image indicates, no files are actually transferred. However, you still get to see what would have happened if you left out the -n option. Because of this, a dry run in rsync is an extremely fast and easy precautionary step to take, especially if you’re using a lot of options chained together.

Going Further

Hopefully, this article has provided you with the know-how necessary to back up your data quickly and efficiently from the command-line. However, rsync is an extremely versatile tool, so if you find your backup needs exceed that of this guide How to Make Data Backups on Ubuntu & Other Distros How to Make Data Backups on Ubuntu & Other Distros How much sensitive data would you lose if your disk drive died? Naturally, you need a backup solution, but making backups in Linux can be tricky if you don't know what you're doing... Read More , don’t be afraid to enter this command:

man rsync

rsync manual

The document you see goes through all the options covered here in much more detail, along with many others. For example, it explains how to exclude files by their size, useful for filtering out blank or redundant files.

Do you prefer using the command line to back up your data? Why or why not?

Leave a Reply

Your email address will not be published. Required fields are marked *

  1. Pratish Patel
    December 22, 2016 at 4:15 pm

    -z is not needed if you are not transferring files over Wan.
    Eg copying from local to USB drive or other internal drive or on a NAS in a same home network don't use -z as it's not much effective due to CPU taxing.

    Yes if you copy over Internet from your local host -z will speed up transfer.

  2. Erik Mathis
    December 22, 2016 at 2:42 am

    Rsync is not a backup tool. Sorry.. but it's ment for efficiently transfering files around. Yes having your file somewhere eles is a backup, but in that sense only.

    Try to restore a file you changed three times over a week, and going back to the original one. You can't because rsync has no point in time recovery concept. Each time rsync copies down that changed file it just updates that file. You lose the ability to restore the older versions of that file. Try Rdiff-backup, it makes use of the efficiency of rsync, but maintains point in time recovery.

    • Austin Luong
      December 22, 2016 at 3:31 am

      Thank-you for pointing this out - I agree with you 100% on this matter. As I stated in my first sentence of the article: "Rsync is fairly simple: it’s a tool that’s specialized in copying files."

      If one's backup needs exceed that of Rsync's then they are free to explore other alternatives.

  3. Falx
    December 21, 2016 at 3:19 pm

    Thanks for posting the article. I've only recently been using rsync as my go-to backup utility on both my Mac and Linux machines. However, I've never fully explored the other options and command switches available with this utility.

    • Austin Luong
      December 22, 2016 at 4:04 am

      Glad you found it useful!