Order files transferred by rsync

Question

We are currently using rsync to run the backup job. This work takes several hours. Since some parts of the backup are more important than others, we would like to back them up before everything else (i.e. what rsync transfers these files in the first place). This should be easy to do with rsync --files-from .

Unfortunately, the files we need for prioritization are located next to all other files in the same directory (flat hierarchy, boo!), They just have a certain extension for their identification. Thus, we would like to get a list of files that rsync will transfer, sort it locally, and then transfer it to rsync using rsync --files-from .

Using rsync -v --dry-run gives us most of the way (I think), but displays directory changes and changes to file properties, which means that we really don't get a list that can be used right away.

So my question is this: how can I get a list of files to use from rsync?

Answer 1

Will rsync work twice? for example, run it once for high priority files and once for the rest?

Answer 2

I would comment on @alberge's answer, but I don't have enough reputation: - (

@Daniel, if you use separate rsync commands, I can make a suggestion to use a for loop:

 for pp in "*.png" "*.jpg" "*.bmp" "*.gif" do rsync ... $pp done 

only one rsync command to change if you want to make any changes and a good managed list of things to sync. The order in the for statement is the order in which they are used.

Answer 3

You can use rdiff-backup first or just replace rsync with rdiff-backup. For example, you can get a list of files with:

 rdiff-backup --compare in-dir user@host ::out-dir 

However, I would go with another suggestion to try two passes. Also, did you try to enable compression?