Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision | Next revisionBoth sides next revision | ||
| parallel_rsync [20.05.2020 19:32] – [the code] Pascal Suter | parallel_rsync [20.05.2020 19:32] – [known issues] Pascal Suter | ||
|---|---|---|---|
| Line 21: | Line 21: | ||
| how many jobs should run in parallel and how many directories deep you want to parallellize your jobs really depends on your sepcific situation. if you have several terabytes of data and you do a complete sync it makes sense to dive deeper into the structure than when you just want to update an already existing copy of the same data, in that case it might be faster to only dive 1 to 2 levels deep into your structure or even not use this script at all, when most of the time is spend by " | how many jobs should run in parallel and how many directories deep you want to parallellize your jobs really depends on your sepcific situation. if you have several terabytes of data and you do a complete sync it makes sense to dive deeper into the structure than when you just want to update an already existing copy of the same data, in that case it might be faster to only dive 1 to 2 levels deep into your structure or even not use this script at all, when most of the time is spend by " | ||
| - | ==== known issues ==== | + | |
| - | * when i wrote the script i used excessive escaping to make sure that more complex file names could also be copied. however i do not escape the source directory name or the target directory name. i sadly realized only after wrinting all this escaping madness, that i could have used the -0 option of xargs and corresponding null options to the other tools so that i could have avoided the need for all this escaping madness in the first place | + | |
| ==== the code ==== | ==== the code ==== | ||