What is the best way to store large files in Plone 3?

Question

I want to use a large number of large files on the Plone website. By large files, I mean about 5 MB (music) and many of them. I already did it right on ZODB, this is not a good idea. I am running Plone 3.1.1 and Zope 2.10.6.

Answer 1

Zodb blob support is the best, most integrated way to work with large files. Large files are stored transparently in the file, not in the zodb object database. "Transparent" in this case means that you will not notice it in your actual programming job after the initial setup.

The blob functionality was transferred to the current versions (halfway 2008) of zope and can be used in plone 3. Use plone.app.blob in your project for this: http://plone.org/products/plone.app.blob .

Answer 2

Yes, you should not use anything other than ZODB BLOB support at this point. It works great with 3.x releases.

Additional information in ticket number 6805

- Alexander Limi, co-founder of Plone

Answer 3

Explaining to my knowledge:

• from various candidate technologies for PLIP (Plone Immovervement Proposal), plone.app.blob is a leading competitor with broad support

  - for exceptional use cases, we sometimes find something other than the recommended BLOB

• 4.0 is currently the most likely milestone for plone.app.blob to become a product in the Plone core

• meanwhile plone.app.blob is the recommended add-on for current versions 3.x Plone

  - for use cases that offer BLOB-like technologies.

Answer 4

As you already know, a long-term solution for this is to support ZODB BLOB. Ticket 6805 is probably the most authoritative source on this. Unfortunately, this milestone is set to 4.0, and running it on an earlier version may not be a good solution.

Historically, there were many Plone products for storing files from the outside, storing only metadata in ZODB. I tried several of them, and in my experience there is not one that works well with the current releases of Plone / Zope. Do not believe me in this, however, I have not tried any products of this type last year or so.

Personally, I would like to find a solution that is as simple as possible and does not include Plone more than necessary. Saving music files to disk, serving them directly from apache / of any web server that you use, storing only metadata in Plone - in a product that you write yourself, will give you a reliable solution with good performance. That is, your product should create links to a path on your web server where music files are available.

If you need authorization to download music files and it is assumed that you are running lighthttpd or apache before your Zope, viewing an X-sendfile based solution is probably the best option. With the X-sendfile, you store files on disk and add a header (X-sendfile) in response when the music file should be sent to the client browser. The web server will pick this header and send the file to the client without Plone.

Some pointers:

• http://tn123.ath.cx/mod_xsendfile/ (apache module)

• http://john.guen.in/past/2007/4/17/send_files_faster_with_xsendfile/ (Ruby example)

Answer 5

I have plone.app.blob installed on some sites with low traffic and installable (ready to catalog, if you like) for my more busy production sites in the same instance.

There is 4.0, but I will definitely review (and probably click the install button for plone.app.blob on my production sites) about 3.4 times.

A few links:

http://n2.nabble.com/PLIPs-I%27d-love-to-see-for-Plone-3.3-tp1123218p1130015.html

http://dev.plone.org/plone/ticket/8629#comment:2 highlight

... 3.4, when we are likely to have the specification blob filestorage support added to plone.recipe.zeoserver and zope2instance. What will give us a standard location for any owner / permission, installers must do.

In context: I play roughly with plone.app.blob and a very mixed package of other add-ons with versions 3.1.7 and 3.2a1 Plone based on standard and experimental installers. In these environments, without me considering things with gloves, Plone sies behave remarkably well, and when (as expected) experiments lead to strangenesses, community support steps and is right.