Automatically call common initialization code without creating __init__.py file

Question

I have two directories in my project:

project/ src/ scripts/ 

"src" contains my polished code, and "scripts" contains one-time Python scripts.

I would like all scripts to add "../src" to their sys.path so that they can access the modules under the "src" tree. One way to do this is to write a script file / __ init__.py with the contents:

 scripts/__init__.py: import sys sys.path.append("../src") 

This works, but has an undesirable side effect of putting all my scripts in a package called “scripts”. Is there any other way to get all my scripts to call the above initialization code automatically?

I could just change the PYTHONPATH environment variable in my .bashrc, but I want my scripts to work out of the box without requiring the user to play with PYTHONPATH. In addition, I do not like the fact that you need to make changes to the account only to host this one project.

Answer 1

Even if you have other distribution plans, it might be worth building the base setup.py in your src folder. Thus, you can run setup.py develop so that distutils places a link to your code on your default path (which means that any changes you make will be displayed in place without the need for "reinstallation", and all modules "just work", no matter where your scripts are). This will be a one-time step, but it is one more step than zero, so it depends on whether there are more problems than updating .bashrc . If you use pip , the equivalent will be pip install -e /path/to/src .

A more reliable solution - especially if you intend to mirror / manage these scenarios on multiple development machines - is to do your development in a controlled virtual environment. It turns out that virtualenv even has built-in support for creating custom bootstrap settings . It looks like you just need the after_install() hook to either configure sitecustomize , run pip install -e , or add a simple .pth file to the site packages. A custom bootstrap can live in your original control along with other scripts and must be run once for each installation of the developer. You will also have the usual benefits of using virtualenv (explicit dependency versioning, isolation from a system-wide configuration, and standardization between disparate machines, to name a few).

If you really do not want to have any configuration steps and are ready to run these scripts only from the "project" directory, you can use __init__.py as such:

 project/ src/ some_module.py scripts/ __init__.py # special "magic" some_script.py 

And here is what your files look like:

 # file: project/src/some_module.py print("importing %r" % __name__) def some_function(): print("called some_function() inside %s" % __name__) -------------------------------------------------------- # file: project/scripts/some_script.py import some_module if __name__ == '__main__': some_module.some_function() -------------------------------------------------------- # file: project/scripts/__init__.py import sys from os.path import dirname, abspath, join print("doing magic!") sys.path.insert(0, join(dirname(dirname(abspath(__file__))), 'src')) 

Then you have to run the scripts as follows:

 [~/project] $ python -m scripts.some_script doing magic! importing 'some_module' called some_function() inside some_module 

Caution! Scripts can only be called from within project/ :

 [~/otherdir] $ python -m scripts.some_script ImportError: no module named scripts 

To enable this, you will return to editing .bashrc or use one of the options above. The last option should be the last resort; as @Simon said, you are really struggling with the language at this moment.

Answer 2

If you want your scripts to be executed (I guess from the command line), they should be somewhere in the way.

Something sounds weird about what you're trying to do, though. Can you show us an example of what you are trying to achieve?

Answer 3

you can add the file "pathHack.py" to the dir project to add something like this to it:

 import os import sys pkgDir = os.path.dirname(__file__) sys.path.insert(os.path.join(pkgDir, 'scripts') 

then in the python file in the project directory start with:

 import pathHack 

and now you can import material from the script catalog without "scripts". prefix. If there is only one file in this directory and you do not need to hide such things, you can include this snippet.