Download a file-like object with Paramiko?

Question

I have a bunch of code that looks like this:

with tempfile.NamedTemporaryFile() as tmpfile: tmpfile.write(fileobj.read()) # fileobj is some file-like object tmpfile.flush() try: self.sftp.put(tmpfile.name, path) except IOError: # error handling removed for ease of reading pass 

Is it possible to make such a download without having to write the file somewhere?

Answer 1

Update . With Paramiko 1.10 you can use putfo :

 self.sftp.putfo(fileobj, path) 

---

Instead of using paramiko.SFTPClient.put you can use paramiko.SFTPClient.open , which opens the file object. You can write about it. Something like that:

 f = self.sftp.open(path, 'wb') f.write(fileobj.read()) f.close() 

Please note that it may be useful to download paramiko data in 32 KiB chunks, since the largest block containing the SSH protocol can process without breaking it into several packets.

Answer 2

Is StringIO what you are looking for? ( doc page )

SFTPClient get() and put() functions take paths, not files, which makes things a little awkward.

You can write a wrapper for paramiko.SFTPClient to provide it with the necessary functionality.

Here is my best unverified attempt:

 from paramiko import SFTPClient class SFTPClient2(SFTPClient): def put(self, local_file, remotepath, callback=None, confirm=True): fl = source_file file_size = os.fstat(fl.fileno()).st_size try: fr = self.file(remotepath, 'wb') fr.set_pipelined(True) size = 0 try: while True: data = fl.read(32768) if len(data) == 0: break fr.write(data) size += len(data) if callback is not None: callback(size, file_size) finally: fr.close() finally: fl.close() if confirm: s = self.stat(remotepath) if s.st_size != size: raise IOError('size mismatch in put! %d != %d' % (s.st_size, size)) else: s = SFTPAttributes() return s def get(self, remotepath, local_file, callback=None): fr = self.file(remotepath, 'rb') file_size = self.stat(remotepath).st_size fr.prefetch() try: fl = local_file try: size = 0 while True: data = fr.read(32768) if len(data) == 0: break fl.write(data) size += len(data) if callback is not None: callback(size, file_size) finally: fl.close() finally: fr.close() s = os.fstat(fl.fileno()) if s.st_size != size: raise IOError('size mismatch in get! %d != %d' % (s.st_size, size)) 

If this works, the get and put functions should now accept local file descriptors, not paths.

All I had to do was get rid of the code that opens the file from the path and change the code that gets the file size to use os.fstat instead of os.stat .