Geek Answers Handbook

Python - restrictions on the number of imported goods?

I have 1000 custom (compiled in '.so') modules that I would like to use in python at the same time. Each such module has a size ( 100 [KB] ) on average.

My question is: is there a limit on the number of imports in python ? What are the consequences of many imports ? (for example, it will require a lot of memory)?

Can I import 10,000 custom modules? 50,000 modules?

+8
python
source share
3 answers

There is no Python limit for the amount of import in a module. If there is a limit in any particular implementation, this is probably due to resource limitations outside the Python interpreter.

+9
source share

The amount of memory consumed by one imported module will be no less than the size of the module on disk. The overhead is determined by the OS itself (to load the dynamic module) and Python overhead when importing the module.

So, if your module has an average size of 100 KB, then importing out of 10,000 of them will take at least 1 GB of address space. Importing 50,000 of them will work on more than 5 GB. It is better to use an operating system with 64-bit address space.

+7
source share

CPython has no restrictions on the number of imports. However, each .so file will be opened with dlopen() , which is located outside the Python control, as well as a character table that must continue to grow to collect information about your extension modules. Regardless of whether they have a practical limit, it is also beyond the scope of Python. CPython itself takes some memory for every module you import, so that as long as you have enough memory, everything should be fine.

+3
source share

More articles:


All Articles