Python ftplib file corruption?

Question

I upload files to Python using ftplib, and until recently, everything seemed to work fine. I upload files as such:

ftpSession = ftplib.FTP(host,username,password) ftpSession.cwd('rlmfiles') ftpFileList = filter(lambda x: 'PEDI' in x, ftpSession.nlst()) ftpFileList.sort() for f in ftpFileList: tempFile = open(os.path.join(localDirectory,f),'wb') ftpSession.retrbinary('RETR '+f,tempFile.write) tempFile.close() ftpSession.quit() sys.exit(0) 

Until recently, I downloaded files that I needed were just fine, as expected. Now, however, my files that I upload are corrupted and contain long lines of ASCII garbage. I know that these are not files sent to FTP from which I pull them out, because I also have a Perl script that does this successfully from the same FTP.

If this is any additional information, here is what the debugger displays on the command line when downloading the file:

Has anyone encountered problems with corrupted file contents using retrbinary() in Python ftplib?

I am really stuck / upset and have not come across anything related to possible corruption here. Any help is appreciated.

Answer 1

I just ran into this question yesterday when I was trying to download text files. Not sure if this is what you did, but since you say it has ASCII garbage in it, I assume you opened it in a text editor because it was supposed to be text.

If so, the problem is that the file is a text file and you are trying to download it in binary mode.

Instead, you want to receive the file in ASCII transfer mode.

 tempFile = open(os.path.join(localDirectory,f),'w') # Changed 'wb' to 'w' ftpSession.retrlines('RETR '+f,tempFile.write) # Changed retrbinary to retrlines 

Unfortunately, this removes all newlines from the file. Ugh!

So, you need to add the extra newline characters again:

 tempFile = open(os.path.join(localDirectory,f),'w') textLines = [] ftpSession.retrlines('RETR '+f,textLines.append) tempFile.write('\n'.join(textLines)) 

This should work, but it doesn’t look as good as it could. So a little cleaning work will bring us:

 temporaryFile = open(os.path.join(localDirectory, currentFile), 'w') textLines = [] retrieveCommand = 'RETR ' ftpSession.retrlines(retrieveCommand + currentFile, textLines.append) temporaryFile.write('\n'.join(textLines))