Dynamic adapter selection based on installed library (s)

Question

I am developing a library with adapters that support a wide range of libraries. I want the library to dynamically select which adapter always has the library that it uses, installed on the computer when importing certain classes.

The goal is to be able to modify the library on which the program depends, without making changes to the code. This feature consists in handling RabbitMQ connections, since we had a lot of problems with pika , we want to be able to switch to another library, for example pyAMPQ or rabbitpy without changing the base code,

I was thinking of implementing something like this in the __init__.py servicelibrary.simple file.

 try: #import pika # Is pika installed? from servicelibrary.simple.synchronous import Publisher from servicelibrary.simple.synchronous import Consumer except ImportError: #import ampq # Is ampq installed? from servicelibrary.simple.alternative import Publisher from servicelibrary.simple.alternative import Consumer 

Then when the user imports the library

 from servicelibrary.simple import Publisher 

The underlying layer looks something like this:

alternative.py

 import amqp class Publisher(object): ...... class Consumer(object): ...... 

synchronous.py

 import pika class Publisher(object): ...... class Consumer(object): ...... 

This will automatically select the second when the first is not installed.

Is there a better way to implement something like this? If anyone could link the library / adapter with a similar implementation, which would also be useful.

[Change]

What would be the cleanest way to implement something like this? In the future, I would also like to change the default preference. Ultimately, I can simply agree to use the installed library, as I can control this, but that would be a nice feature.

Alexanders suggestion is interesting, but I would like to know if there is a cleaner way.

[Edit2]

The original example has been simplified. Each module can contain several types of imports, for example. Consumer and publisher.

Answer 1

importlib.import_module can do what you need:

 INSTALLED = ['syncronous', 'alternative'] for mod_name in INSTALLED: try: module = importlib.import_module('servicelibrary.simple.' + mod_name) Publisher = getattr(module, 'Publisher') if Publisher: break # found, what we needed except ImportError: continue 

I think this is not the most preliminary method, but the idea should be clear. And you can take a look at the imp module.

Answer 2

Flexible solution using importlib . This is the complete, working solution that I tested.

Heading first:

 import importlib parent = 'servicelib.simple' modules = {'.synchronous':['.alternative', '.alternative_2']} success = False #an indicator, default is False, #changed to True when the import succeeds. 

We import the required module, install our indicator and indicate our modules. modules - a dictionary, with the key set as the default module and a value in the form of a list of alternatives.

Next, the import- ant part:

 #Obtain the module for default, alternatives in modules.items(): try: #we will try to import the default module first mod = importlib.import_module(parent+default) success = True except ImportError: #the default module fails, try the alternatives for alt in alternatives: try: #try the first alternative, if it still fails, try the next one. mod = importlib.import_module(parent+alt) success = True #Stop searching for alternatives! break except ImportError: continue print 'Success: ', success 

And to have classes just do:

 Publisher = mod.Publisher Consumer = mod.Consumer 

With this solution, you can have several alternatives at once. For example, as an alternative, you can use both rabbitpy and pyAMPQ.

Note. Works with both Python 2 and Python 3.

If you have more questions, feel free to comment and ask!

Answer 3

You have the right idea. Your case works because each subobject has the same classes, for example. both APIs have a class called Publisher , and you can just make sure the correct version is imported.

If this is incorrect (if possible, the implementation of A and B is not similar), you write your own facade, which is simply your own simple API, which then calls the real API with the correct methods / parameters for this library.

Obviously, switching over between selections may require some overhead (I don’t know your business, but let's say you have two libraries to go through an open file and the library handles opening the file. Switch to the second library in the middle of the file and wait that it will start by stopping the first library). But this is just a conservation issue:

 accessmethods = {} try: from modA.modB import classX as apiA_classX from modA.modB import classY as apiA_classY accessmethods['apiA'] = [apiA_classX, apiA_classY] classX = apiA_classX classY = apiA_classY except: pass try: from modC.modD import classX as apiB_classX from modC.modD import classY as apiB_classY accessmethods['apiB'] = [apiB_classX, apiB_classY] classX = apiB_classX classY = apiB_classY except: pass def switchMethod(method): global classX global classY try: classX, classY = accessmethods[method] except KeyError as e: raise ValueError, 'Method %s not currently available'%method 

etc..

Answer 4

I know two methods, one of them is wildly used, and the other is my guess. You can choose one for your situation.

The first one that is widely used, for example from tornado.concurrent import Future .

 try: from concurrent import futures except ImportError: futures = None #define _DummyFuture balabala... if futures is None: Future = _DummyFuture else: Future = futures.Future 

Then you can use from tornado.concurrent import Future in other files.

The second, which is my hunch, and I am writing a simple demo, but I do not use it in a production environment because I do not need it.

 import sys try: import servicelibrary.simple.synchronous except ImportError: import servicelibrary.simple.alternative sys.modules['servicelibrary.simple.synchronous'] = servicelibrary.simple.alternative 

You can run the script before another script import servicelibrary.simple.synchronous . Then you can use the script as before:

 from servicelibrary.simple.synchronous import Publisher from servicelibrary.simple.synchronous import Consumer 

The only thing that interests me is the consequences my guesses.

Answer 5

Based on the answers, I got the following implementation for Python 2.7.

Examples are simplified for stackoverflow.

 from importlib import import_module PARENT = 'myservicelib.rabbitmq' MODULES = ['test_adapter', 'test_two_adapter'] SUCCESS = False for _module in MODULES: try: __module = import_module('{0}.{1}'.format(PARENT, _module)) Consumer = getattr(__module, 'Consumer') Publisher = getattr(__module, 'Publisher') SUCCESS = True break except ImportError: pass if not SUCCESS: raise NotImplementedError('no supported rabbitmq library installed.') 

Although, since I also had some of my projects running Python 2.6, I had to either modify the code or enable importlib . The problem with the production platform is that it is not always easy to include new dependencies.

This is the tradeoff I came across based on __import__ instead of importlib .

It might be worth checking to see if sys.modules contains a namespace, so you are not getting a KeyError , but that is unlikely.

 import sys PARENT = 'myservicelib.rabbitmq' MODULES = ['test_adapter', 'test_two_adapter'] SUCCESS = False for _module in MODULES: try: __module_namespace = '{0}.{1}'.format(PARENT, _module) __import__(__module_namespace) __module = sys.modules[__module_namespace] Consumer = getattr(__module, 'Consumer') Publisher = getattr(__module, 'Publisher') SUCCESS = True break except ImportError: pass if not SUCCESS: raise NotImplementedError('no supported rabbitmq library installed.')