Problems issuing cp command using Popen in Python

Question

I am trying to copy files from one place to another using the subprocess library and the Popen method. When I run after the script, I get the error cp: cannot stat /some/dev_path/* . I was told that * does not apply to file names and that there is a problem. Also in some other posts, people suggested using a call instead of Popen, but the call will not return stderr as far as I know.

 devPath = '/some/dev_path/' productionPath = '/some/prod_path/' p = subprocess.Popen(['cp', '-r', devPath + '*', productionPath], stdout = subprocess.PIPE, stderr = subprocess.PIPE) pout, perr = p.communicate() if perr != '': sys.exit('Error: ' + perr) 

Answer 1

The * extension (globbing) is a function of your shell, for example bash. Therefore, you will have to use the shell=True keyword argument in your subprocess.Popen call.

However, for this case, I highly recommend using shutil.copytree instead.

(First of all, because it is much simpler (see Zen of Python ) and less error prone. Error handling is much cleaner, you get wonderful exceptions, including a list of errors (for operations with multiple files, like yours), and you don’t you need to deal with the spawning of the subprocess and communication with it. Secondly, it is an unnecessary waste of resources for branching the child process, if you do not need. Other problems include quoting / escaping and, possibly, introducing security vulnerabilities into your code if you couldn’t properly sanitized user input.)

For example:

 from shutil import copytree from shutil import Error try: copytree('dir_a', 'dir_b') except (Error, OSError), e: print "Attempt to copy failed: %s" % e 

In addition, you should not create file system paths by concatenating strings together, but use os.path.join () instead. This will use the correct directory separator ( os.sep ) for the current OS and allow you to easily write portable code.

Example:

 >>> import os >>> os.path.join('/usr/lib', 'python2.7') '/usr/lib/python2.7' 

Note: os.path.join still only works with (smart) string manipulation - it doesn't care if this path is accessible or even exists.