Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #92891

Re: Catching exceptions with multi-processing

Show all headers | View raw

On 06/19/2015 04:25 PM, Andres Riancho wrote:
> Fabien,
>
>     My recommendation is that you should pass some extra arguments to the task:
>      * A unique task id
>      * A result multiprocessing.Queue
>
>      When an exception is raised you put (unique_id, exception) to the
> queue. When it succeeds you put (unique_id, None). In the main process
> you consume the queue and do your error handling.
>
>      Note that some exceptions can't be serialized, there is where
> tblib [0] comes handy.
>
> [0]https://pypi.python.org/pypi/tblib
>
> Regards,

Thanks, I wasn't aware of the multiprocessing.Queue workflow. It seems 
like its going to require some changes in the actual code of the tasks 
though. Did I get it right that I should stop raising exceptions then?

Something like:

def task_1(path, q):
     # Do stuffs
     if dont_work:
         q.put(RuntimeError("didnt work"))
	return
     # finished
     q.put(None)
     return


Fabien

Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Catching exceptions with multi-processing Fabien <fabien.maussion@gmail.com> - 2015-06-19 16:01 +0200
  Re: Catching exceptions with multi-processing Andres Riancho <andres.riancho@gmail.com> - 2015-06-19 11:25 -0300
    Re: Catching exceptions with multi-processing Fabien <fabien.maussion@gmail.com> - 2015-06-19 18:16 +0200
      Re: Catching exceptions with multi-processing Cameron Simpson <cs@zip.com.au> - 2015-06-20 13:14 +1000
        Re: Catching exceptions with multi-processing Fabien <fabien.maussion@gmail.com> - 2015-06-20 11:03 +0200
  Re: Catching exceptions with multi-processing Oscar Benjamin <oscar.j.benjamin@gmail.com> - 2015-06-19 16:01 +0100
  Re: Catching exceptions with multi-processing Steven D'Aprano <steve@pearwood.info> - 2015-06-20 01:41 +1000
    Re: Catching exceptions with multi-processing Fabien <fabien.maussion@gmail.com> - 2015-06-19 18:07 +0200
    Re: Catching exceptions with multi-processing Chris Angelico <rosuav@gmail.com> - 2015-06-20 06:58 +1000
      Re: Catching exceptions with multi-processing Fabien <fabien.maussion@gmail.com> - 2015-06-20 11:01 +0200
  Re: Catching exceptions with multi-processing Paul Rubin <no.email@nospam.invalid> - 2015-06-21 13:27 -0700

csiph-web