ZipExtFile in Django

Question

I am wondering if there is a way to upload the zip file to the django web server and put the zip files in the django database WITHOUT accessing the actual file system in the process (e.g. extracting the files in the zip to the tmp directory and then upload them)

Django provides a function to convert a python file to a Django file, so if there is a way to convert a ZipExtFile to a python file, that should be fine.

thanks for the help!

Django Model:

from django.db import models class Foo: file = models.FileField(upload_to='somewhere') 

Using:

 from zipfile import ZipFile from django.core.exceptions import ValidationError from django.core.files import File from io import BytesIO z = ZipFile('zipFile') istream = z.open('subfile') ostream = BytesIO(istream.read()) tmp = Foo(file=File(ostream)) try: tmp.full_clean() except Validation, e: print e 

Output:

 {'file': [u'This field cannot be blank.']} 

[SOLUTION] Solution using an ugly hack:

As Don Quest correctly points out, file classes such as StringIO or BytesIO should represent the data as a virtual file. However, the Django File constructor accepts only the type of the embedded file and nothing else, although classes like the file would also do the job. The hack is to manually set the variables in Django :: File:

 buf = bytesarray(OPENED_ZIP_OBJECT.read(FILE_NAME)) tmp_file = BytesIO(buf) dummy_file = File(tmp_file) # this line actually fails dummy_file.name = SOME_RANDOM_NAME dummy_file.size = len(buf) dummy_file.file = tmp_file # dummy file is now valid 

Please continue to comment if you have a better solution (except for a special repository)

Answer 1

Unaware of Django, I can tell you about the io package. You can do something like:

 from zipfile import ZipFile from io import StringIO zname,zipextfile = 'zipcontainer.zip', 'file_in_archive' istream = ZipFile(zname).open(zipextfile) ostream = StringIO(istream.read()) 

And then do what you would like to do with your “virtual” stream stream / file.

Answer 2

There is an easier way to do this:

 from django.core.files.base import ContentFile uploaded_zip = zipfile.ZipFile(uploaded_file, 'r') # ZipFile for filename in uploaded_zip.namelist(): with uploaded_zip.open(filename) as f: # ZipExtFile my_django_file = ContentFile(f.read()) 

Using this, you can convert a file that has been loaded into memory directly into a django file. For a more complete example, suppose you want to load a series of image files inside zip into a file system:

 # some_app/models.py class Photo(models.Model): image = models.ImageField(upload_to='some/upload/path') ... # Upload code from some_app.models import Photo for filename in uploaded_zip.namelist(): with uploaded_zip.open(filename) as f: # ZipExtFile new_photo = Photo() new_photo.image.save(filename, ContentFile(f.read(), save=True) 

Answer 3

I used the following django file class to avoid having to read ZipExtFile into another data structure (StingIO or BytesIO), while at the same time correctly importing what Django needs to save the file directly.

 from django.core.files.base import File class DjangoZipExtFile(File): def __init__(self, zipextfile, zipinfo): self.file = zipextfile self.zipinfo = zipinfo self.mode = 'r' self.name = zipinfo.filename self._size = zipinfo.file_size def seek(self, position): if position != 0: #this will raise an unsupported operation return self.file.seek(position) #TODO if we have already done a read, reopen file zipextfile = archive.open(path, 'r') zipinfo = archive.getinfo(path) djangofile = DjangoZipExtFile(zipextfile, zipinfo) storage = DefaultStorage() result = storage.save(djangofile.name, djangofile)