How to upload a large file in python via ftp (with monitoring and reconnecting)?

UPDATE # 1

The code in the question works pretty well for a stable connection (like a local network or intranet).

UPDATE # 2

I have implemented a class FTPClientwith ftplib that can:

  • track download progress.
  • reconnect in case of timeout or disconnect
  • performs several file upload attempts
  • shows the current download speed.

After reconnecting, the download process from the disconnect point continues (if the FTP server supports it). See my answer below for more details.


Question

python, (0.3-1.5Gb * 200-300 ) FTP, . ftplib. . , KEEPALIVE, .

with closing(ftplib.FTP()) as ftp:
    try:
        ftp.connect(self.host, self.port, 30*60) #30 mins timeout
        # print ftp.getwelcome()
        ftp.login(self.login, self.passwd)
        ftp.set_pasv(True)
        ftp.sock.setsockopt(socket.SOL_SOCKET, socket.SO_KEEPALIVE, 1)
        ftp.sock.setsockopt(socket.IPPROTO_TCP, socket.TCP_KEEPINTVL, 75)
        ftp.sock.setsockopt(socket.IPPROTO_TCP, socket.TCP_KEEPIDLE, 60)
        with open(local_filename, 'w+b') as f:
            res = ftp.retrbinary('RETR %s' % orig_filename, f.write)

            if not res.startswith('226 Transfer complete'):
                logging.error('Downloaded of file {0} is not compile.'.format(orig_filename))
                os.remove(local_filename)
                return None

        os.rename(local_filename, self.storage + filename + file_ext)
        ftp.rename(orig_filename, orig_filename + '.copied')

        return filename + file_ext

    except:
            logging.exception('Error during download from FTP')

  • 7-15 .
  • FTP- , , . , .

  • , - ?
  • ,
+3
1

, . Stackoverflow , . GitHub (pyFTPclient) - (~ 120 ).

( 3G ), . , , .

. .

+8

All Articles