Catching indescribable exceptions and re-raising

Question

This is a continuation of my question. Hang a Python script using SQLAlchemy and multiprocessing . As discussed in this question, etching exceptions are problematic in Python. This is usually not a problem, but in one case when errors occur in the python multiprocessing module. Since multiprocessing moves objects around by etching, if an error occurs in the multiprocessing process, the whole process may freeze, as shown in this question.

One possible approach is to fix all problematic exceptions, as discussed in this question. This is not easy, as it is not easy to know in advance what exceptions can be called. An alternative approach proposed by lbolla in answering the question is to catch an exception, build an equivalent harmless exception, and then reconstruct. However, I do not know exactly how to do this. Consider the following code.

class BadExc(Exception): def __init__(self, message, a): '''Non-optional param in the constructor.''' Exception.__init__(self, message) self.a = a import sys try: try: #print foo raise BadExc("bad exception error message", "a") except Exception, e: raise Exception(e.__class__.__name__ + ": " +str(e)), None, sys.exc_info()[2] except Exception, f: pass import cPickle a = cPickle.dumps(f) l = cPickle.loads(a) print "raising error" raise sys.exc_info()[0], sys.exc_info()[1], sys.exc_info()[2] 

This code smoothes and unpacks the exception and then throws it throwing an error

 raising error Traceback (most recent call last): File "<stdin>", line 11, in <module> Exception: BadExc: bad exception error message 

Credits for Glenn Maynard answer the question "Internal Exception" (with tracing) in Python? . This has important stuff, namely trace, error message and exception type, so this may be the best thing to do. But ideally, I would like something similar to the original exception, namely

 Traceback (most recent call last): File "<stdin>", line 11, in <module> __main__.BadExc: bad exception error message 

or, more generally, with the name of the exception in front, not Exception . Is it possible?

Alternatively, instead of the BadExc class, you can use the print foo BadExc instead, which gives a NameError . However, this exception does not require special handling.

Answer 1

You can override sys.excepthook to achieve what you want. This at least works for this example, but it's pretty hacks, so please check and not promises :-)

 import sys def excepthook_wrapper(type, value, traceback): if len(value.args) == 2: name, msg = value.args value.args = (msg,) sys.__excepthook__(name, value, traceback) else: sys.__excepthook__(type, value, traceback) sys.excepthook = excepthook_wrapper 

( Edit: I am not very happy with this, because now the “normal” exceptions with two arguments will be handled differently. A possible solution is to “tag” your special exceptions by passing “PICKLED” as the first argument, and then check this, instead of checking args lengths.)

And then throw an Exception with two arguments, the name ( __module__.__class__ ) and the Exception message ( str(e) ):

 try: try: #print foo raise BadExc("bad exception error message", "a") except Exception, e: cls = e.__class__ if hasattr(cls, '__module__'): name = '{0}.{1}'.format(cls.__module__, cls.__name__) else: name = cls.__name__ raise Exception(name, str(e)), None, sys.exc_info()[2] except Exception, f: pass 

Then this:

 import cPickle a = cPickle.dumps(f) l = cPickle.loads(a) print "raising error" raise sys.exc_info()[0], sys.exc_info()[1], sys.exc_info()[2] 

Print

 raising error Traceback (most recent call last): File "test.py", line 18, in <module> raise BadExc("bad exception error message", "a") __main__.BadExc: bad exception error message