Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #44289

Finding the source of an exception in a python multiprocessing program

Show key headers only | View raw

I run a bit of python code that monitors my connection to the greater Internet.  It checks connectivity to the requested target IP addresses, logging both successes and failures, once every 15 seconds.  I see failures quite regularly, predictably on Sunday nights after midnight when various networks are undergoing maintenance.  I'm trying to use python's multiprocessing library to run multiple copies in parallel to check connectivity to different parts of the country (they in no way interact with each other).

On rare occasions (maybe once every couple of months) I get the following exception and traceback:

Traceback (most recent call last):
  File "./CM_Harness.py", line 12, in <module>
    Foo = pool.map(monitor, targets)    # and hands off two targets
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/multiprocessing/pool.py", line 227, in map
    return self.map_async(func, iterable, chunksize).get()
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/multiprocessing/pool.py", line 528, in get
    raise self._value
IndexError: list index out of range

The code where the traceback occurs is:

#!/usr/bin/env python

""" Harness to call multiple parallel copies
    of the basic monitor program
"""    

from multiprocessing import Pool
from Connection_Monitor import monitor

targets = ["8.8.8.8", "www.ncsa.edu"]
pool = Pool(processes=2)            # start 2 worker processes
Foo = pool.map(monitor, targets)    # and hands off two targets


Line 12, in my code is simply the line that launches the underlying monitor code.  I'm assuming that the real error is occurring in the monitor program that is being launched, but I'm at a loss as to what to do to get a better handle on what's going wrong. Since, as I said, I see failures quite regularly, typically on Sunday nights after midnight when various networks are undergoing maintenance, I don't _think_ the exception is being triggered by that sort of failure.

When I look at the pool module, the error is occurring in get(self, timeout=None) on the line after the final else:

    def get(self, timeout=None):
        self.wait(timeout)
        if not self._ready:
            raise TimeoutError
        if self._success:
            return self._value
        else:
            raise self._value


Python v 2.7.3, from Python.org, running on Mac OS-X 10.8.3

Thanks for any suggestions,
Bill

Back to comp.lang.python | Previous | Next — Next in thread | Find similar | Unroll thread

Thread

Finding the source of an exception in a python multiprocessing program William Ray Wing <wrw@mac.com> - 2013-04-24 15:25 -0400
  Re: Finding the source of an exception in a python multiprocessing program Neil Cerutti <neilc@norwich.edu> - 2013-04-24 20:31 +0000
    Re: Finding the source of an exception in a python multiprocessing program William Ray Wing <wrw@mac.com> - 2013-04-24 17:09 -0400
      Re: Finding the source of an exception in a python multiprocessing program Neil Cerutti <neilc@norwich.edu> - 2013-04-25 12:37 +0000
    Re: Finding the source of an exception in a python multiprocessing program Dave Angel <davea@davea.name> - 2013-04-24 19:26 -0400
    Re: Finding the source of an exception in a python multiprocessing program Oscar Benjamin <oscar.j.benjamin@gmail.com> - 2013-04-25 01:00 +0100
    Re: Finding the source of an exception in a python multiprocessing program Dave Angel <davea@davea.name> - 2013-04-24 20:11 -0400

csiph-web