Is it possible to implement an option to run rsync in parallel?

[edoardovicendone]

Hello,

First of all thanks for your effort on maintaining rsync.

I am opening this discussion because I have noticed that when running rsync (I am on RHEL 7) they are spawn 3 processes and it usually copy at a speed around 50MB/s per process. This seems to have been observed also here:

https://stackoverflow.com/questions/24058544/speed-up-rsync-with-simultaneous-concurrent-file-transfers

I know there are "workarounds" using xargs or parallel commands to obtain more process running in parallel, but could it be took into consideration having an option, for example --parallel, to decide how many processes spawn?

Thanks a lot,
Edoardo

[anttiryt]

I think this needs some more information. What is the 50 MB limit from. Is it disk read/write, network, ssh compression or what.

[michaelmess-de]

Maybe running rsync in parallel might speed up rsync over the internet when bandwith or ping time is an issue.
But when there are hard disks (no SSDs) involved, running multiple rsync processes in parallel would kill performance by performing random access to disk, causing continuous head movements instead of continuously reading the data from disk.

Maybe the proposal at #770 could significantly improve performance?