Geek Answers Handbook

How to extract dependencies from PyPi package

My goal is simple, I want to remotely get the PyPi package dependency without downloading it completely.

It seems I understand (reading the pip code) that pip, when resolving dependencies, seems to read the egg as soon as the package has been loaded ...

Is there another way?

+4
source share
5 answers

I just need to find a way to do this, and this is what I came up with (stolen from pip).

def dist_metadata(setup_py): '''Get the dist object for setup.py file''' with open(setup_py) as f: d = f.read() try: # we have to do this with current globals else # imports will fail. secure? not really. A # problem? not really if your setup.py sources are # trusted exec d in globals(), globals() except SystemExit: pass return distutils.core._setup_distribution

fooobar.com/questions/273838 / ... answers why the exec spell is subtle and hard to reach.

+4
source

Unfortunately, pip does not have this feature. The metadata available for PyPI packages does not contain dependency information.

You can usually find a detailed dependency on the README file from the project website.

pip search may provide some package information. He can tell you what it is based on.

 $ pip search flask Flask - A microframework based on Werkzeug, Jinja2 and good intentions
+2
source

As Jinghli notes, there is currently no reliable way to get dependency on an arbitrary PyPi package remotely without downloading it completely. And in fact, dependencies sometimes depend on your environment, so in general you need an approach like Brian to execute setup.py code.

The way Python ecosystems handled dependencies began to evolve in 1990 before the problem was well understood. PEP 508 - The Python Software Package Dependency Specification sets us on course to improve the situation and the “desirable” approach to the project in PEP 426 - Metadata for Python 2.0 software packages can improve it in the future, combined with reimplementing PyPI as Warehouse .

The current situation is well described in the Pendon Dependency Resolution .

PyPI provides a json interface for loading metadata for each package. The info.requires_dist object contains a list of names of required packages with additional restrictions on versions, etc. This is often missing, but this is one place to run.

eg. Django (json) indicates:

{ "info": { ... "requires_dist": [ "bcrypt; extra == 'bcrypt'", "argon2-cffi (>=16.1.0); extra == 'argon2'", "pytz" ], ... }

0
source

Use pipdeptree to view the dependencies of packages installed by PyPI.

Installation:

 pip install pipdeptree

Then run:

 pipdeptree

You will see something like this:

 Warning!!! Possible conflicting dependencies found: * Mako==0.9.1 -> MarkupSafe [required: >=0.9.2, installed: 0.18] Jinja2==2.7.2 -> MarkupSafe [installed: 0.18] ------------------------------------------------------------------------ Lookupy==0.1 wsgiref==0.1.2 argparse==1.2.1 psycopg2==2.5.2 Flask-Script==0.6.6 - Flask [installed: 0.10.1] - Werkzeug [required: >=0.7, installed: 0.9.4] - Jinja2 [required: >=2.4, installed: 2.7.2] - MarkupSafe [installed: 0.18] - itsdangerous [required: >=0.21, installed: 0.23] alembic==0.6.2 - SQLAlchemy [required: >=0.7.3, installed: 0.9.1] - Mako [installed: 0.9.1] - MarkupSafe [required: >=0.9.2, installed: 0.18] ipython==2.0.0 slugify==0.0.1 redis==2.9.1
0
source

The ideal solution is to simply try installing the package and see what it depends on using the new virtualenv and pip freeze .

For example, let's find out thefuck dependency list :

Create a new virtual environment and activate it:

 virtualenv env --python=python3.6 source env/bin/activate

pip freeze shows that the package is not installed:

 (env) pip3 freeze

Install the package and you will already see the installed dependencies:

 (env) pip3 install thefuck ... Installing collected packages: wcwidth, pyte, psutil, colorama, decorator, six, thefuck Successfully installed colorama-0.3.9 decorator-4.1.2 psutil-5.4.2 pyte-0.7.0 six-1.11.0 thefuck-3.25 wcwidth-0.1.7

pip freeze to view all installed:

 (env) pip3 freeze colorama==0.3.9 decorator==4.1.2 psutil==5.4.2 pyte==0.7.0 six==1.11.0 thefuck==3.25 wcwidth==0.1.7
0
source

More articles:


All Articles