My goal is to distribute a Python package with several other widely used Python packages as dependencies. My package depends on well-written Pypi indexing packages such as pandas, scipy, and numpy, and indicates in the setup.py file that certain versions or higher ones are needed. "numpy> = 1.5".
I found that for Unix users who are not experts in Python packaging (even if they know how to write Python), it is incredibly difficult and almost impossible to install a package like mine, even when using what should be Use package managers easily. I am wondering if there is an alternative to this torturous process that someone can offer, or if my experience reflects the very difficult current state of Python packaging and distribution.
Suppose users download your package to their system. Most will try to install it “naively” using something like:
$ python setup.py install
Since if you provided Google with instructions for installing Python packages, this usually happens. This will fail for the vast majority of users, as most of them do not have root access on their Unix / Linux servers. With a lot of searches, they will find the "--prefix" parameter and try:
$ python setup.py install --prefix=/some/local/dir
Since users are not aware of the intricacies of Python packaging, they will choose an arbitrary directory as an argument to --prefix , for example. "~/software/mypackage/" . It will not be a pure curatorial directory where all other Python packages will reside, because again most users are not aware of these details. If they install another “myotherpackage” package, they can pass it "~/software/myotherpackage" and you can imagine how in the future this will lead to frustrating PYTHONPATH hacking and other complications.
Continuing the installation process, calling "setup.py install" with "--prefix" also fail if users try to use the package even if it was installed correctly, because one of the dependencies may be missing (for example, pandas, scipy or numpy) and the package manager is not used. They will try to install these packages separately. Even if they are successful, packages will inevitably not be in PYTHONPATH because of the non-standard directories given by "--prefix" , and user users will try to modify their PYTHONPATH to get dependencies to be visible.
At this point, an experienced Python friend might say that they should use the package manager, for example, "easy_install" , the main manager, to install the software and run the dependencies. After installing "easy_install" , which can be difficult, they will try:
$ easy_install setup.py
This also will not succeed, because users, as a rule, usually do not have permission to install software on a global level on production Unix servers. With a lot of reading, they will learn about the "--user" option and try:
$ easy_install setup.py --user
They will get the error:
usage: easy_install [options] requirement_or_url ... or: easy_install --help error: option --user not recognized
They will be extremely puzzled why their easy_install does not have the --user option, where there are clear pages on the Internet describing this option. They may try to upgrade their easy_install to the latest version and find that they still do not work.
If they continue and contact the Python packaging expert, they will find that there are two versions of easy_install , both called " easy_install" , to maximize confusion, but one part is "distribute" and the other part is "setuptools". It happens that only the "easy_install" from the "distribute" supports "--user" , and the vast majority of / sys server administrators install the "setuptools" easy_install , and therefore the local installation will be aware that these differences are between the "distribute" and "setuptools" meaningless and hard to understand for people who are not experts in Python package management.
At this point, I would lose 90% of even the most determined, experienced and patient users who are trying to install my software package - and rightly so! They wanted to install a piece of software that turned out to be written in Python, and not become experts in the modern Python distribution, and this is too confusing and complicated. They will give up and get upset when they get lost.
A small minority of users who go ahead and ask more Python experts will be told that they should use pip/virtualenv instead of easy_install . Installing pip and virtualenv and figuring out how these tools work and how they differ from the usual "python setup.py" or "easy_install" calls are themselves time-consuming and complicated, and again too much to ask users who simply wanted to install a simple piece of Python software and use it. Even those who pursue this path will be confused as to whether all the dependencies that they installed using easy_install or setup.py install --prefix be used with pip/virtualenv or if everything needs to be reinstalled from scratch.
This problem is exacerbated if one or more of the packages in question depend on installing a different version of Python than the default one. The difficulty of ensuring that your Python package box uses the version of Python you want, and that the necessary dependencies are installed in the appropriate Python 2.x directory, and not Python 2.y, will endlessly disappoint users that they will certainly refuse this stage.
Is there an easier way to install Python software that does not require users to familiarize themselves with all these technical details of Python packages, paths, and locations? For example, I'm not a big Java user, but sometimes I use some Java tools and I don’t remember having to worry about the dependencies of X and Y on the Java software that I installed, and I don’t know how the Java package is (and I'm glad that I don’t), I just wanted to use a tool that was written in Java.) I recall that if you download the Jar, you just get it, and it tends to work,
Is there an equivalent for Python? A way to distribute software in a way that is independent of how users should pursue all of these dependencies and versions? Is there a way, perhaps, to compile all the relevant packages into something self-contained, which can simply be downloaded and used as binary?
I would like to emphasize that this frustration occurs even with the narrow goal of distributing the package to experienced Unix users, which makes the problem easier without worrying about cross-platform issues, etc. I assume that users are Unix savvy, and may even know Python, but just don’t know (and don’t want to be recognized) about the features of Python packaging and the many internal complications / rivalries of different package managers. An alarming feature of this problem is that this happens even when all your Python package dependencies are well-known, well-written, and well-maintained Pypi-accessible packages such as pandas, Scipy, and Numpy. This is not like I relied on some obscure dependencies that were not formed correctly: rather, I used the most common packages that many could rely on.
Any help or advice on this would be greatly appreciated. I think Python is a great language with large libraries, but it's almost impossible for me to distribute the software that I write in it (after it has dependencies) in such a way that people can easily install locally and just run it. I would like to clarify that the software that I write is not a Python library for programmatic use, but software that has executable scripts that users run as separate programs. Thank.