What is the Python egg cache (PYTHON_EGG_CACHE)?

Question

I just upgraded from Python 2.6.1 to 2.6.4 on my development machine and after running the python script, the following message was presented:

Cannot extract file to cache cache

The following error occurred: trying to extract a file in the Python Egg Cache:

[Errno 13] Permission denied: '/var/www/.python-eggs'

The Python Egg Cache Directory is currently set to:

/var/www/.python-eggs

Perhaps your account does not have write access to this directory? You can change the cache directory to set the PYTHON_EGG_CACHE environment variable to indicate an available directory.

Nothing exists in python docs, so I’ll lose a little information about best practices, where the directory is and what it was used for.

Can someone explain what Python egg cache is?

Also can you explain why / how it differs from the site-packages directory that Python uses to store eggs (as I understand it)?

Answer 1

From my research, it turns out that some eggs are packaged as zip files and stored as such in the Python site-packages directory.

These laid eggs must be unpacked before they can be executed, so they will be expanded into the PYTHON_EGG_CACHE directory, which by default is ~/.python-eggs (located in the user's home directory). If this does not exist, it causes problems when trying to launch applications.

There are a number of fixes:

• Create the .python-eggs directory in the user's home directory and make it writable to the user.
• Create a global directory for unpacking (for example, /tmp/python-eggs ) and set the environment variable PYTHON_EGG_CACHE to this directory.
• Use the -Z switch when using easy_install to unzip the package during installation.

Answer 2

The python egg cache is just a directory used by setuptools to store installed packages that meet the specification . You can read setuptools here .

In addition, as indicated in the error message, you can specify a different egg cache directory in your environment by setting PYTHON_EGG_CACHE = / some / other / dir. The easiest way to do this is to set it to ~ / .bash_profile (assuming you are using bash), for example:

 export PYTHON_EGG_CACHE=/some/other/dir 

You may need to install it in your Apache environment if you are using a web application.

Answer 3

This is a dark side effect of using an otherwise beautiful egg mechanism.

Eggs are packages (a directory full of files) packaged in a single .egg file to simplify depolation.

They are stored in the /site-packages/ directory.

As long as the files stored in the egg are .py files, it works great. Import Python can import objects from any file object, like a regular file.

But when something like .so happens, python cannot explain to the underlying OS that it wants to load a library that does not have a physical name. And the only workarounds that, according to the authors, were unpacked into a temporary directory. Naturally, this is not /site-packages/ , since /site-packages/ not available to regular users.

So you can

• set PYTHON_EGG_DIR to /tmp or

• grant www user write permission /var/www/.python-eggs
  (so that files are not unpacked every time you clean / tmp) or better than

• unpack the egg as suggested by @ shalley303
  (and avoid unzipping the eggs at runtime).

Answer 4

Python eggs are zip-compressed packages containing both Python modules and metadata. The egg cache is where the extracted egg contents are stored so that the Python modules inside it are useful.

Answer 5

You can also disable the use of .egg after installing it. You need to go into the directory of package sites, extract .egg, and then move it to a hidden file (or delete it or something else).

Here is an example of what I did to disable the MySQLdb module .egg file that caused this error when the python script was launched from Zabbix.

 cd /usr/local/lib/python2.7/site-packages
 unzip MySQL_python-1.2.3-py2.7-linux-x86_64.egg
 mv MySQL_python-1.2.3-py2.7-linux-x86_64.egg .MySQL_python-1.2.3-py2.7-linux-x86_64.egg

Answer 6

Phillip B Oldham is right. You can add these lines to your code:

 import os os.environ['PYTHON_EGG_CACHE'] = '/tmp' # a writable directory