Installing my sdist from PyPI puts files in unexpected places

Question

My problem is that when I download my Python package in PyPI and then install it there using pip, my application breaks because it installs my files in completely different locations than when I just install the same package from local sdist .

Installing from a local sdist puts the files into my system as follows:

/Python27/ Lib/ site-packages/ gloopy-0.1.alpha-py2.7.egg/ (egg and install info files) data/ (images and shader source) doc/ (html) examples/ (.py scripts that use the library) gloopy/ (source) 

This is the way I expected, and works great (for example, my source can find my data directory because they are next to each other, just like in development.)

If I load the same sdist into PyPI and then install it there using pip, then everything looks very different:

 /Python27/ data/ (images and shader source) doc/ (html) Lib/ site-packages/ gloopy-0.1.alpha-py2.7.egg/ (egg and install info files) gloopy/ (source files) examples/ (.py scripts that use the library) 

This does not work at all - my application cannot find its data files, and obviously this is a mess, polluting the top level directory / python 27 with all my junk file.

What am I doing wrong? How do I install a package like installing sdist locally? Is this what I should try to achieve?

More details

I have setuptools installed and also distributed, and I call distribute_setup.use_setuptools ()

WindowsXP, Python2.7.

The catalog of my development is as follows:

 /gloopy /data (image files and GLSL shader souce read at runtime) /doc (html files) /examples (some scripts to show off the library) /gloopy (the library itself) 

My MANIFEST.in mentions all the files that I want to include in sdist, including everything in the data, example, and doc directories:

 recursive-include data *.* recursive-include examples *.py recursive-include doc/html *.html *.css *.js *.png include LICENSE.txt include TODO.txt 

My setup.py is pretty verbose, but I think it's best to include it here, right? I also include duplicate links to the same data / doc / examples directories, as indicated in MANIFEST.in, because I understand that this is necessary so that these files are copied from sdist to the system during installation.

 NAME = 'gloopy' VERSION= __import__(NAME).VERSION RELEASE = __import__(NAME).RELEASE SCRIPT = None CONSOLE = False def main(): import sys from pprint import pprint from setup_utils import distribute_setup from setup_utils.sdist_setup import get_sdist_config distribute_setup.use_setuptools() from setuptools import setup description, long_description = read_description() config = dict( name=name, version=version, description=description, long_description=long_description, keywords='', packages=find_packages(), data_files=[ ('examples', glob('examples/*.py')), ('data/shaders', glob('data/shaders/*.*')), ('doc', glob('doc/html/*.*')), ('doc/_images', glob('doc/html/_images/*.*')), ('doc/_modules', glob('doc/html/_modules/*.*')), ('doc/_modules/gloopy', glob('doc/html/_modules/gloopy/*.*')), ('doc/_modules/gloopy/geom', glob('doc/html/_modules/gloopy/geom/*.*')), ('doc/_modules/gloopy/move', glob('doc/html/_modules/gloopy/move/*.*')), ('doc/_modules/gloopy/shapes', glob('doc/html/_modules/gloopy/shapes/*.*')), ('doc/_modules/gloopy/util', glob('doc/html/_modules/gloopy/util/*.*')), ('doc/_modules/gloopy/view', glob('doc/html/_modules/gloopy/view/*.*')), ('doc/_static', glob('doc/html/_static/*.*')), ('doc/_api', glob('doc/html/_api/*.*')), ], classifiers=[ 'Development Status :: 1 - Planning', 'Intended Audience :: Developers', 'License :: OSI Approved :: BSD License', 'Operating System :: Microsoft :: Windows', 'Programming Language :: Python :: 2.7', ], # see classifiers http://pypi.python.org/pypi?:action=list_classifiers ) config.update(dict( author='Jonathan Hartley', author_email='[e-mail]', url='http://bitbucket.org/tartley/gloopy', license='New BSD', ) ) if '--verbose' in sys.argv: pprint(config) setup(**config) if __name__ == '__main__': main() 

Answer 1

The data_files parameter data_files intended for data files that are not included in the package. You should probably use package_data instead.

See https://docs.python.org/3/distutils/setupscript.html#installing-package-data

This will not install the data in site-packages / data, but in my opinion, in any case, it should not be installed where it is. You will not know what package it is in. It should be installed in site-packages//gloopy-0.1.alpha-py2.7.egg/[data|doc|examples] IMO.

If you really think that the data is not package data, you should use data_files , in which case pip will install it correctly, and I would say that setup.py install install it in the wrong place. But, in my opinion, in this case it is package_data, since it refers to a package and is not used by other software.

Answer 2

You can download the package data pkgutil.get_data () , it will find where exactly the package data is installed.

Here is a good blog post about including data files in packages: Including Data Files in Python Packages