Simplified Exception Identification in Python

One of the features that makes Python a
great programming language is exceptions for error handling.
Exceptions are convenient in many ways for handling errors and
special conditions in a program. But, if several kinds of
exceptions occur in short sections of code and in several parts of
a program, it quickly becomes tedious and error-causing to recode
the "except:" chunks of the code. Error recovery, especially, is a
place where you want to have well-tested, clear and simple chunks
of code. This article suggests an approach that helps users achieve
this goal.A second factor that makes Python so flexible is it does not
reinvent the wheel when its API interfaces with the operating
system. Any C or C++ programmer familiar with standard *NIX system
calls and libraries can leverage his/her previous knowledge when
moving into Python application development. On the other hand, the
Python socket module generates exceptions that differ from one
platform to another. As an example, the Connection refused error
exception is assigned the number 111 under Linux, but it is 10061
in another popular operating system. Again, it quickly becomes
boring to code multiple except: clauses for use on many different
platforms.Always searching for a better and easier way to do things,
let's look now at how we can identify and categorize exceptions in
Python in order to simplify error recovery. In addition, let's try
to do this in a way that can be applied to multiple operating
systems.Anatomy of an ExceptionThere is more than one generic way to discover the identity
of an exception. First, you need to code a catch-all except:
clause, like this:

     try:
         ...some statements here...
     except:
         ...exception handling...

In the exception handling code, we want to have as few lines
of code as possible. Also, it is highly desirable to funnel both
normal exceptions (disconnect, connection refused) alongside the
weirder ones (Attribute error!), running both through a single
execution path. You will, of course, need more statements to do the
precise action required by the condition. But, if you can do the
first four or five steps in a generic way, it will making testing
things later as easier task.In our example, Python offers two ways to access the
exception information. For both, the Python script first must have
import sys before the try: ..
except: portion of the code. With the first method, the
function sys.exc_type gives the name of the exception, and
sys.exc_value gives more details about the exception. For example,
in a NameError exception the sys.exc_value command might return
"There is no variable named 'x'", when x was
referenced without first having been assigned a value. This method,
however, is not thread-safe. As a result, it is not that useful,
because most network applications are multithreaded.The second way to access exception information is with
sys.exc_info(). This function is thread-safe and also is more
flexible, although it might look intimidating at first. If you run
the following code:

     import sys
     try:
         x = x + 1
     except:
         print sys.exc_info()

You will see this message:

     (<class exceptions.NameError at 007C5B2C>, <exceptions.NameError
     instance at 007F5E3C>, <traceback object at 007F5E10>)

How's that for cryptic! But, with a few lines of code we can
unravel this into rather useful chunks of information. Suppose that
you run the following code instead:

     import sys
     import traceback
     def formatExceptionInfo(maxTBlevel=5):
         cla, exc, trbk = sys.exc_info()
         excName = cla.__name__
         try:
             excArgs = exc.__dict__["args"]
         except KeyError:
             excArgs = "<no args>"
         excTb = traceback.format_tb(trbk, maxTBlevel)
         return (excName, excArgs, excTb)
     try:
         x = x + 1
     except:
         print formatExceptionInfo()

This will display:

     ('NameError', ("There is no variable named 'x'",), ['  File "<stdin>",
     line 14, in ?\n'])

The function formatExceptionInfo() takes the three-element
tuple returned by sys.exc_info() and transforms each element into a
more convenient form, a string. cla.__name__ gives the name of the
exception class, while exc.__dict__["args"] gives other details
about the exception. In the case of socket exceptions, these
details will be in a two-element tuple, like ("error",
(32, 'Broken pipe'). Lastly, traceback.format_tb()
formats the traceback information into a string. The optional
argument (maxTBlevel> in the sample code) allows users to
control the depth of the traceback that will be formatted. The
traceback information is not essential to identify or categorize
exceptions, but if you want to log all the spurious unknown
exceptions your program encounters, it is useful to write that
traceback string in the log.With the first two elements--the name of the exception class
and the exception details--we can now try to identify the exception
and reduce it to a well-known one from a set of previously
recognized exception patterns.A Dictionary of Known Exception PatternsThe name of the exception class is not enough to generalize
properly the handling of exceptions. As an example, the socket
module generates many different exceptions for various conditions,
and they all are named "error". What makes each condition distinct
lies in the exceptions details. The tuple ('error', (111,
'Connection refused') ) is different from ('error', (32, 'Broken
pipe') ).So what we will do instead is create a dictionary whose keys
define exact exceptions. Each of those keys will have a simple
integer value that reduces multiple cases to a smaller number. The
following dictionary is not complete, but it gives a good idea of
how to proceed and build a larger one.

     EXC_DISCONNECT  = 1
     EXC_SERVER_NOT_AVAILABLE = 2
     ExcDiagDict = {
        # another famous OS:
        repr( ('error', (10061, 'Connection refused') ) ) :
     EXC_SERVER_NOT_AVAILABLE,
        # Linux 2.4:
        repr( ('error', (111, 'Connection refused') ) )   :
     EXC_SERVER_NOT_AVAILABLE,
        repr( ('error', (32, 'Broken pipe') ) )           : EXC_DISCONNECT,
        repr( ('error', (107, 'Transport endpoint is not connected') ) ) :
     EXC_DISCONNECT,
      }

In this exception "diagnosis" dictionary, a repr() surrounds
each of the tuples that makes up a key. In this example, strings
are the only practical way to retrieve the expected results from
the dictionary. Using a tuple directly does not work--no match
would ever be found, unless the exact same instances of objects and
tuples were used in the look-up. String keys are used because they
are less restrictive and work according to their lexical
values.In the dictionary as it is initialized above, we have two
keys associated with the constant EXC_SERVER_NOT_AVAILABLE. Each of
these two keys represents an ('error', (number, 'Connection
refused')) exception, but the value of the number varies according
to the operating system being used. The last two keys are two
completely different exceptions that can happen under Linux, but
they basically indicate the same event--a disconnect on a socket.
The dictionary can be expanded to include similar exception
patterns on more platforms, as well as more exception patterns
related to one or more basic events.If you take the previous import statements, the previous
definition of the function formatExceptionInfo() and the dictionary
above, you can append the following code:

     def getDiagnosisFromExcInfo(excInfoTuple):
         try:
           excPattern = (excInfoTuple[0], excInfoTuple[1])
           return ExcDiagDict[ repr(excPattern) ]
         except KeyError:
           return None
         except:
           return None    # maybe was not tuple?
     from socket import *
     try:
         s = socket(AF_INET, SOCK_STREAM)
         s.connect( ("127.0.0.1", 7788) )
         s.close()
     except:
         exInfo = formatExceptionInfo()
         diag = getDiagnosisFromExcInfo(exInfo)
         print "diag:", diag, "exc:", exInfo

When you run that code, the following should be
returned:

     diag: 2 exc: ('error', (111, 'Connection refused'), ['File "<stdin>",
     line 3, in ?\n', ...]

The "2" is the value of the constant
EXC_SERVER_NOT_AVAILABLE, which is indeed the constant associated
with the Connection refused exception pattern in the sample
dictionary. From experience, it is safe to say that error recovery
code that uses formatExceptionInfo() and getDiagnosisFromExcInfo()
is much simpler and, therefore, more reliable.ConclusionWith the function formatExceptionInfo(), we have found a way
to learn important information about any exception from the Python
sys.exc_info() function. Also, the exception patterns dictionary
makes it possible to simplify the identification of various
exceptions, even if they differ from one OS to another or if they
actually refer to the same event.Jean-Francois Touchette has
been developing software for 20 years. He has written several
gateways and servers with various protocols since 1985. He has been
using C and UNIX since that time. When Python is suitable, he
prefers it.

email: jftouchette@yahoo.com