Plugin architecture - Plugin Manager vs check from plugin import *

Question

I am currently writing an application that allows the user to extend it using a plug-in architecture. They can write additional python classes based on the BaseClass object that I provide, and they load with various application signals. The exact number and names of the classes loaded as plugins are unknown before the application starts, but are loaded only once at startup.

During my research on the best way to solve this problem, I came up with two general solutions.

Option 1 - collapse your own using imp, pkgutil, etc.
See for example this answer or this one .

Option 2 - Using the Plugin Manager Library
Randomly choosing a pair

• straight.plugin
• yapsy
• this approach

My question is, provided that the application needs to be restarted to download new plugins, are there any advantages of the above methods over something inspired by this SO answer and this one , for example:

import inspect import sys import my_plugins def predicate(c): # filter to classes return inspect.isclass(c) def load_plugins(): for name, obj in inspect.getmembers(sys.modules['my_plugins'], predicate): obj.register_signals() 

Are there any disadvantages to this approach compared to the above? (except for all plugins should be in one file) Thank you!

EDIT
Comments require additional information ... the only additional feature I can add is that plugins use the blinker library to provide the signals they sign. Each plugin can subscribe to different signals of different types and, therefore, must have its own specific method of "registration".

Answer 1

The metaclass approach is useful for this problem in Python <3.6 (see @quasoft answer for Python 3.6+). It is very simple and acts automatically on any imported module. In addition, complex logic can be applied to register a plugin with minimal effort. This requires:

The metaclass approach works as follows:

1) A custom metaclass PluginMount that supports a list of all plugins

2) A Plugin class is defined that sets PluginMount as its metaclass

3) When an object obtained from Plugin is imported - for example, MyPlugin , it runs the __init__ method in the metaclass. This registers the plugin and performs any specific application logic and event subscription.

Alternatively, if you put the PluginMount.__init__ in the PluginMount.__new__ , it is called when a new instance of the Plugin derived class is created.

 class PluginMount(type): """ A plugin mount point derived from: http://martyalchin.com/2008/jan/10/simple-plugin-framework/ Acts as a metaclass which creates anything inheriting from Plugin """ def __init__(cls, name, bases, attrs): """Called when a Plugin derived class is imported""" if not hasattr(cls, 'plugins'): # Called when the metaclass is first instantiated cls.plugins = [] else: # Called when a plugin class is imported cls.register_plugin(cls) def register_plugin(cls, plugin): """Add the plugin to the plugin list and perform any registration logic""" # create a plugin instance and store it # optionally you could just store the plugin class and lazily instantiate instance = plugin() # save the plugin reference cls.plugins.append(instance) # apply plugin logic - in this case connect the plugin to blinker signals # this must be defined in the derived class instance.register_signals() 

Then the plugin base class is as follows:

 class Plugin(object): """A plugin which must provide a register_signals() method""" __metaclass__ = PluginMount 

Finally, the actual plugin class will look like this:

 class MyPlugin(Plugin): def register_signals(self): print "Class created and registering signals" def other_plugin_stuff(self): print "I can do other plugin stuff" 

Access to plugins can be obtained from any python module that imported Plugin :

 for plugin in Plugin.plugins: plugin.other_plugin_stuff() 

See full working example.

Answer 2

Since Python 3.6 is a new method of the __init_subclass__ class, which is called in the base class, whenever a new subclass is created.

This method can further simplify the solution proposed by the will-chorus above by removing the metaclass.

__init_subclass__ was introduced with PEP 487: Simplified class creation setup . PEP comes with a minimal example for the plugin architecture:

Now you can configure the creation of a subclass without using a metaclass. The new __init_subclass__ method will call the base class whenever a new subclass is created:

 class PluginBase: subclasses = [] def __init_subclass__(cls, **kwargs): super().__init_subclass__(**kwargs) cls.subclasses.append(cls) class Plugin1(PluginBase): pass class Plugin2(PluginBase): pass 

The PEP example above stores class references in the Plugin.plugins field.

If you want to store instances of plugin classes, you can use this structure:

 class Plugin: """Base class for all plugins. Singleton instances of subclasses are created automatically and stored in Plugin.plugins class field.""" plugins = [] def __init_subclass__(cls, **kwargs): super().__init_subclass__(**kwargs) cls.plugins.append(cls()) class MyPlugin1(Plugin): def __init__(self): print("MyPlugin1 instance created") def do_work(self): print("Do something") class MyPlugin2(Plugin): def __init__(self): print("MyPlugin2 instance created") def do_work(self): print("Do something else") for plugin in Plugin.plugins: plugin.do_work() 

which outputs:

 MyPlugin1 instance created MyPlugin2 instance created Do something Do something else 

Answer 3

The free will approach was most useful to me! So that I need more control, I wrapped the Plugin Base class in functions like:

 def get_plugin_base(name='Plugin', cls=object, metaclass=PluginMount): def iter_func(self): for mod in self._models: yield mod bases = not isinstance(cls, tuple) and (cls,) or cls class_dict = dict( _models=None, session=None ) class_dict['__iter__'] = iter_func return metaclass(name, bases, class_dict) 

and then:

 from plugin import get_plugin_base Plugin = get_plugin_base() 

This allows you to add additional base classes or switch to another metaclass.