Mocking ImportError in Python

Question

I've been trying this for almost two hours with no luck.

I have a module that looks like this:

try: from zope.component import queryUtility # and things like this except ImportError: # do some fallback operations <-- how to test this? 

Later in code:

 try: queryUtility(foo) except NameError: # do some fallback actions <-- this one is easy with mocking # zope.component.queryUtility to raise a NameError 

Any ideas?

EDIT:

Alex's suggestion doesn't seem to work:

 >>> import __builtin__ >>> realimport = __builtin__.__import__ >>> def fakeimport(name, *args, **kw): ... if name == 'zope.component': ... raise ImportError ... realimport(name, *args, **kw) ... >>> __builtin__.__import__ = fakeimport 

When running tests:

 aatiis@aiur ~/work/ao.shorturl $ ./bin/test --coverage . Running zope.testing.testrunner.layer.UnitTests tests: Set up zope.testing.testrunner.layer.UnitTests in 0.000 seconds. Error in test /home/aatiis/work/ao.shorturl/src/ao/shorturl/shorturl.txt Traceback (most recent call last): File "/usr/lib64/python2.5/unittest.py", line 260, in run testMethod() File "/usr/lib64/python2.5/doctest.py", line 2123, in runTest test, out=new.write, clear_globs=False) File "/usr/lib64/python2.5/doctest.py", line 1361, in run return self.__run(test, compileflags, out) File "/usr/lib64/python2.5/doctest.py", line 1282, in __run exc_info) File "/usr/lib64/python2.5/doctest.py", line 1148, in report_unexpected_exception 'Exception raised:\n' + _indent(_exception_traceback(exc_info))) File "/usr/lib64/python2.5/doctest.py", line 1163, in _failure_header out.append(_indent(source)) File "/usr/lib64/python2.5/doctest.py", line 224, in _indent return re.sub('(?m)^(?!$)', indent*' ', s) File "/usr/lib64/python2.5/re.py", line 150, in sub return _compile(pattern, 0).sub(repl, string, count) File "/usr/lib64/python2.5/re.py", line 239, in _compile p = sre_compile.compile(pattern, flags) File "/usr/lib64/python2.5/sre_compile.py", line 507, in compile p = sre_parse.parse(p, flags) AttributeError: 'NoneType' object has no attribute 'parse' Error in test BaseShortUrlHandler (ao.shorturl) Traceback (most recent call last): File "/usr/lib64/python2.5/unittest.py", line 260, in run testMethod() File "/usr/lib64/python2.5/doctest.py", line 2123, in runTest test, out=new.write, clear_globs=False) File "/usr/lib64/python2.5/doctest.py", line 1351, in run self.debugger = _OutputRedirectingPdb(save_stdout) File "/usr/lib64/python2.5/doctest.py", line 324, in __init__ pdb.Pdb.__init__(self, stdout=out) File "/usr/lib64/python2.5/pdb.py", line 57, in __init__ cmd.Cmd.__init__(self, completekey, stdin, stdout) File "/usr/lib64/python2.5/cmd.py", line 90, in __init__ import sys File "<doctest shorturl.txt[10]>", line 4, in fakeimport NameError: global name 'realimport' is not defined 

However, it works when I run the same code from the python interactive console.

MORE EDIT:

I use zope.testing and the test file shorturl.txt , which has all the tests specific to this part of my module. First, I import the module with zope.component to demonstrate and test normal use. The absence of zope.* Packages is considered edge, so I will test it later. So, I have to reload() my module, after zope.* Was unavailable, somehow.

So far, I have even tried using tempfile.mktempdir() and empty zope/__init__.py and zope/component/__init__.py files in tempdir, then inserting tempdir in sys.path[0] and removing the old zope.* Packages from sys.modules .

Does not work.

EVEN MORE EDITING:

In the meantime, I tried this:

 >>> class NoZope(object): ... def find_module(self, fullname, path): ... if fullname.startswith('zope'): ... raise ImportError ... >>> import sys >>> sys.path.insert(0, NoZope()) 

And it works well for the test case namespace (= for all imports in shorturl.txt ), but it fails in my main ao.shorturl module. Even when I reload() . Any idea why?

 >>> import zope # ok, this raises an ImportError >>> reload(ao.shorturl) <module ...> 

Importing zope.interfaces raises ImportError , so it does not fall into the part where I import zope.component , but it remains in the ao.shorturl namespace . Why?

 >>> ao.shorturl.zope.component # why?! <module ...> 

Answer 1

Just select monkeypatch in builtins your own version of __import__ - it can pick up anything you want when it finds out that it is called on specific modules for which you want to mock errors. See documents for details. Roughly speaking:

 try: import builtins except ImportError: import __builtin__ as builtins realimport = builtins.__import__ def myimport(name, globals, locals, fromlist, level): if ...: raise ImportError return realimport(name, globals, locals, fromlist, level) builtins.__import__ = myimport 

Instead ... you can hardcode name == 'zope.component' or organize things more flexibly with your own callback, which can do import on demand in different cases depending on your specific testing needs, without requiring you to code several __import__ -other functions; -).

Please note that if you use import zope.component or from zope.component import something , from zope import component , then name will be 'zope' , and 'component' will be the only element in fromlist .

Edit : the docs for the __import__ function say that the name to import is builtin (as in Python 3), but in fact you need __builtins__ - I edited the code above to work anyway.

Answer 2

This is what I justified in my unittests.

It uses PEP-302 "New Imported Hooks" . (Warning: PEP-302 and the more concise release notes I linked are not entirely accurate.)

I use meta_path because it is as early as possible in the import sequence.

If the module has already been imported (as in my case, because the previous unittests faked it), then you must remove it from sys.modules before doing reload in the dependent module.

 Ensure we fallback to using ~/.pif if XDG doesn't exist. >>> import sys >>> class _(): ... def __init__(self, modules): ... self.modules = modules ... ... def find_module(self, fullname, path=None): ... if fullname in self.modules: ... raise ImportError('Debug import failure for %s' % fullname) >>> fail_loader = _(['xdg.BaseDirectory']) >>> sys.meta_path.append(fail_loader) >>> del sys.modules['xdg.BaseDirectory'] >>> reload(pif.index) #doctest: +ELLIPSIS <module 'pif.index' from '...'> >>> pif.index.CONFIG_DIR == os.path.expanduser('~/.pif') True >>> sys.meta_path.remove(fail_loader) 

Where the code inside pif.index looks like:

 try: import xdg.BaseDirectory CONFIG_DIR = os.path.join(xdg.BaseDirectory.xdg_data_home, 'pif') except ImportError: CONFIG_DIR = os.path.expanduser('~/.pif') 

---

To answer the question of why the new rebooted module has the properties of the old and the new load, here are two sample files.

The first is the y module with the import failure case.

 # y.py try: import sys _loaded_with = 'sys' except ImportError: import os _loaded_with = 'os' 

The second is x , which demonstrates how deleting descriptors for a module can affect its properties upon reboot.

 # x.py import sys import y assert y._loaded_with == 'sys' assert y.sys class _(): def __init__(self, modules): self.modules = modules def find_module(self, fullname, path=None): if fullname in self.modules: raise ImportError('Debug import failure for %s' % fullname) # Importing sys will not raise an ImportError. fail_loader = _(['sys']) sys.meta_path.append(fail_loader) # Demonstrate that reloading doesn't work if the module is already in the # cache. reload(y) assert y._loaded_with == 'sys' assert y.sys # Now we remove sys from the modules cache, and try again. del sys.modules['sys'] reload(y) assert y._loaded_with == 'os' assert y.sys assert y.os # Now we remove the handles to the old y so it can get garbage-collected. del sys.modules['y'] del y import y assert y._loaded_with == 'os' try: assert y.sys except AttributeError: pass assert y.os 

Answer 3

If you don't mind changing your program, you can also put an import call into a function and schedule it in your tests.