Dec 05 2008

Move files into an existing directory structure, on Linux

Published by at 2:50 pm under Technology   

I recently needed to move a large number of files (millions) in a deep directory structure, into another similar directory structure, “merging” the contents of each directory and creating any missing directories. This task is easily (though slowly) performed on Windows with Control-C Control-V in Explorer, but I could find no obvious way to do it on Linux.

There is quite a bit of discussion about this on the web, including:

  • Suggestions to do with with tar; this is a poor idea because it copies all the file data, taking an enormously long time.
  • Suggestions to do it with “mv -r”… but as far as I can tell, mv does not have a -r option.

After a little thought I came up with the script below. I’d love to have a Linux/Bash guru out there point out how awful it and and send me something better!

A critical feature for me is that it does not overwrite files; if a source file name/path overlaps a destinate file, the source file is left alone, untouched. This can be changed easily to overwrite instead: remove the [[-f]] test.

If you found this post useful, please link to it from your web site, mention it online, or mention it to a colleague.

2 responses so far

2 Responses to “Move files into an existing directory structure, on Linux”

  1. I think konqueror behaves the same way as the windows explorer in that case.

  2. John says:

    Won’t rsync do the job for you?

    Something like

    rsync -avh /dir1 /dir2

    Granted, this will overwrite older files with newer files. You can add “–ignore-existing” to the options to override that behavior.

    rsync –ignore-existing -avh /dir1 /dir2