Play Unix cat command in Python

Question

I am currently reproducing the following Unix command:

cat command.info fort.13 > command.fort.13 

in Python with the following:

 with open('command.fort.13', 'w') as outFile: with open('fort.13', 'r') as fort13, open('command.info', 'r') as com: for line in com.read().split('\n'): if line.strip() != '': print >>outFile, line for line in fort13.read().split('\n'): if line.strip() != '': print >>outFile, line 

which works, but there should be a better way. Any suggestions?

Edit (2016):

This issue again began to attract attention after four years. I wrote some thoughts on a longer Jupyter laptop here .

The essence of the problem is that my question was related to the (unexpected by me) behavior of readlines . The answer I was aiming for could be better asked, and read().splitlines() better to answer this question.

Answer 1

The easiest way is to simply forget about the lines and just read the whole file and then write it to the output:

 with open('command.fort.13', 'wb') as outFile: with open('command.info', 'rb') as com, open('fort.13', 'rb') as fort13: outFile.write(com.read()) outFile.write(fort13.read()) 

As pointed out in a comment, this can lead to high memory usage if one of the inputs is large (since it first copies the entire file to memory). If this could be a problem, the following will work just as well (by copying the input files in chunks):

 import shutil with open('command.fort.13', 'wb') as outFile: with open('command.info', 'rb') as com, open('fort.13', 'rb') as fort13: shutil.copyfileobj(com, outFile) shutil.copyfileobj(fort13, outFile) 

Answer 2

 def cat(outfilename, *infilenames): with open(outfilename, 'w') as outfile: for infilename in infilenames: with open(infilename) as infile: for line in infile: if line.strip(): outfile.write(line) cat('command.fort.13', 'fort.13', 'command.info') 

Answer 3

 #!/usr/bin/env python import fileinput for line in fileinput.input(): print line, 

Using:

 $ python cat.py command.info fort.13 > command.fort.13 

Or allow arbitrary large strings:

 #!/usr/bin/env python import sys from shutil import copyfileobj as copy for filename in sys.argv[1:] or ["-"]: if filename == "-": copy(sys.stdin, sys.stdout) else: with open(filename, 'rb') as file: copy(file, sys.stdout) 

Usage is the same.

Or in Python 3.3 using os.sendfile() :

 #!/usr/bin/env python3.3 import os import sys output_fd = sys.stdout.buffer.fileno() for filename in sys.argv[1:]: with open(filename, 'rb') as file: while os.sendfile(output_fd, file.fileno(), None, 1 << 30) != 0: pass 

The above sendfile() call is written for Linux> 2.6.33. In principle, sendfile() may be more efficient than the read / write combination used by other approaches.

Answer 4

Iterating over the file gives the lines.

 for line in infile: outfile.write(line) 

Answer 5

You can simplify this in several ways:

 with open('command.fort.13', 'w') as outFile: with open('fort.13', 'r') as fort13, open('command.info', 'r') as com: for line in com: if line.strip(): print >>outFile, line for line in fort13: if line.strip(): print >>outFile, line 

More importantly, the shutil module has a copyfileobj function:

 with open('command.fort.13', 'w') as outFile: with open('fort.13', 'r') as fort13: shutil.copyfileobj(com, outFile) with open('command.info', 'r') as com: shutil.copyfileobj(fort13, outFile) 

This does not skip empty lines, but the cat does not do this either, so I'm not sure if you really want to.

Answer 6

The list of concepts are awesome for things like this:

 with open('command.fort.13', 'w') as output: for f in ['fort.13', 'command.info']: output.write(''.join([line for line in open(f).readlines() if line.strip()]))