Best practice for handling path / executables in project scripts in Python (e.g. something like Django manage.py or fabric)

Question

I work on different projects (I am a scientist) in a fairly standardized directory structure. eg:.

project /analyses/ /lib /doc /results /bin 

I put all my various utility scripts in / bin / because cleanliness is next to piety. However, I have hard code codes (like .. / .. / x / y / z) and then I have to run things inside. / Bin / or they break.

I used Django, and it has /manage.py, which runs various django things and automatically processes the path. I also used fabric to run various custom functions.

Question: How do I do something like this? and what is the best way? I can easily write something in the /manage.py file to insert the root directory in sys.path, etc., but then I would like to make "./manage.py foo" that would run / bin / foo .py. Or can you make the fabric call executable files from a specific directory?

Basically - I need something light and low maintenance. I want to be able to remove the executable script / file / whatever in. / Bin / and not deal with path problems or import problems.

What is the best way to do this?

Answer 1

Continue execution with TLD

In general, try keeping runtime at the top level. This will greatly simplify your import.

If you need to do a lot of import addressing with relative import, perhaps the best way.

Path change

Other posters mentioned PYTHONPATH . This is a great way to do this forever in your shell.

If you do not want / cannot control the PYTHONPATH project trace directly, you can use sys.path to get yourself out of the adrenaline rush.

Using sys.path.append

sys.path is just a list inside. You can add to it to add material to your path.

Let's say that I'm in /bin and there is a markdown library in lib/ . You can add relative paths with sys.path to import what you want.

 import sys sys.path.append('../lib') import markdown print markdown.markdown(""" Hello world! ------------ """) 

A word to the wise: Not too crazy with sys.path add-ons. Keep your circuit simple to avoid too much confusion.

Too impatient imports can sometimes lead to cases where the python module must import itself, after which execution will stop!

Using packages and __init__.py

Another important trick is to create python packages by adding __init__.py files. __init__.py loads before any other modules in the directory, so this is a great way to add imports across the entire directory. This makes it an ideal place to add the sys.path hacker.

You do not even need to add anything to the file. Just make touch __init__.py on the console to make the directory a package.

See this SO post for more details .

Answer 2

In the shell script that you run (do not run) in the current shell, you set the following environment variables:

 PATH=$PATH:$PROJECTDIR/bin PYTHONPATH=$PROJECTDIR/lib 

Then you put your Python modules and package tree into your projects. / lib. Python automatically adds the PYTHONPATH environment variable to sys.path .

Then you can run any of the top-level script from the shell without specifying a path, and any imports from your library modules will be searched in the lib directory.

I recommend very simple top-level scripts, for example:

 #!/usr/bin/python import sys import mytool mytool.main(sys.argv) 

Then you will never have to change this, you just edit the module code and also benefit from caching byte code.

Answer 3

You can easily achieve your goals by creating a mini-package that contains all your projects. Use paste scripts to create a simple project skeleton. To make it executable, simply install it through setup.py develop . Now your bin scripts just have to import the entry point into this package and execute it.