How to import an add-on module without exec __init__.py in a package

Question

When importing a helper module from a package, the __init__.py file in the package folder will execute exec first, how can I disable this. Sometimes I only need one function in a package, importing the whole package is a little heavy.

For example, the pandas.util.clipboard module is independent of any other functions in pandas.

from pandas.util.clipboard import clipboard_get imports this function, but also imports all pandas common modules. Is there any method that just imports the buffer module, since it is a module in my own application folder.

Answer 1

No, no design. If you want to avoid big overhead when importing submodules, you simply use the empty __init__.py to define the packages. Thus, the overhead costs of importing the package are almost zero.

If pandas does not do this, you have no way to import pandas.util.clipboard without importing pandas and util . What you can do, however, is a huge hack and the equivalent is not to import the clipboard module as a regular module, and not as a submodule. You just need to find the place where pandas installed (e.g. /usr/lib/pythonX.Y/dist-packages/ ) and insert the path of the parent package in sys.path ( /usr/lib/pythonX.Y/dist-packages/pandas/util in your case). Then you can import the clipboard package by doing:

 import clipboard 

Note that:

 import clipboard from pandas.util import clipboard as clipboard2 print(clipboard == clipboard2) 

False will be printed. In fact, this can break a lot of code, since you fundamentally violate some of the invariants that the import mechanism suggests.

In particular, if a submodule refers to other submodules using relative imports, the import will fail, and there are other situations where it will behave incorrectly. Another example when this fails is if you are dealing with pickled objects. If you have any objects pickled using a module imported as pandas.util.clipboard , you can not break them using the imported clipboard module, as described above.

So not ! I suggest either:

• Live with him if the time taken to import the package is not a real problem.
• Or: try finding a replacement. If you only need pandas.util.clipboard and not the rest of pandas , you should probably not use pandas in the first place, and you should use a smaller package that only implements clipboard functionality.

---

If you look at the pandas.util.clipboard source code , you will find that it is actually just a pyperclip version of module 1.3. You can simply add this module to your site-packages and use it instead of the one provided by pandas . In fact, the pandas command only added the following to the end of the source code:

 ## pandas aliases clipboard_get = paste clipboard_set = copy 

---

Expand a bit about why python imports work this way.

As you know, there are objects in python modules. And it also happens that packages are modules, although not every module is a package. When you import a package like in:

 import pandas.util.clipboard 

Python should:

• Create an instance of module pandas
• Create an instance of module util and add it as an attribute in pandas
• Create an instance of module clipboard and add it as an attribute in util .

To create an instance of module , python must execute the code in the module.

Import form:

 from pandas.util import clipboard 

Only syntactic sugar for:

 import pandas.util.clipboard clipboard = pandas.util.clipboard del pandas.util 

Note that in the case of from clipboard could be either module / package, or just something specific inside util . To verify this, the interpreter must also import util , and for this it must also import pandas .

Answer 2

I found a method that uses sys.meta_path to connect the import process:

 import imp, sys class DummyLoader(object): def load_module(self, name): names = name.split("__") path = None for name in names: f, path, info = imp.find_module(name, path) path = [path] return imp.load_module(name, f, path[0], info) def find_module(self, name, path=None): if "__" in name and not name.startswith("__"): return DummyLoader() else: return None if not sys.meta_path: sys.meta_path.append(DummyLoader()) else: sys.meta_path[0] = DummyLoader() 

Use "__" instead of ".". to download only the file:

 import pandas__util__clipboard as clip 

or use the function to download the file:

 import imp def load_module(name): names = name.split(".") path = None for name in names: f, path, info = imp.find_module(name, path) path = [path] return imp.load_module(name, f, path[0], info) clip = load_module("pandas.util.clipboard") 

Answer 3

I tried these methods but could not get them to work. Apparently, by design, it shouldn't work.

If you need to do this, create a new branch in the repo from where you are trying to import, or initialize the repo:

 git checkout -b without_init 

.. then delete __init__.py !

Wherever you try to import, you can verify that Python is in the correct branch, for example:

 import subprocess print ("Current branch is:", subprocess.check_output(["git rev-parse --abbrev-ref HEAD"], shell=True).strip().decode()) >> without_init 

Answer 4

With Python3, you can import a file if you know its path without executing __init__ in the directory in which the file is located. From [importlib example] :( How to import only a submodule without exec __init__.py into a package )

 import importlib.util import sys # For illustrative purposes. import tokenize file_path = tokenize.__file__ module_name = tokenize.__name__ spec = importlib.util.spec_from_file_location(module_name, file_path) module = importlib.util.module_from_spec(spec) sys.modules[module_name] = module spec.loader.exec_module(module)