Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > comp.lang.python > #88958
| References | (1 earlier) <mailman.277.1428944943.12925.python-list@python.org> <mgguou$nn0$1@speranza.aioe.org> <mailman.280.1428984319.12925.python-list@python.org> <mgihao$cjc$1@speranza.aioe.org> <552d115e$0$13010$c3e8da3$5496439d@news.astraweb.com> |
|---|---|
| Date | 2015-04-14 23:45 +1000 |
| Subject | Re: Pickle based workflow - looking for advice |
| From | Chris Angelico <rosuav@gmail.com> |
| Newsgroups | comp.lang.python |
| Message-ID | <mailman.289.1429019150.12925.python-list@python.org> (permalink) |
On Tue, Apr 14, 2015 at 11:08 PM, Steven D'Aprano <steve+comp.lang.python@pearwood.info> wrote: > On Tue, 14 Apr 2015 05:58 pm, Fabien wrote: > >> On 14.04.2015 06:05, Chris Angelico wrote: >>> Not sure what you mean, here. Any given file will be written by >>> exactly one process? No possible problem. Multiprocessing within one >>> application doesn't change that. >> >> yes that's what I meant. Thanks! > > It's not that simple though. If you require files to be written in precisely > a certain order, then parallel processing requires synchronisation. > > Suppose you write A, then B, then C, then D, each in it's own process (or > thread). So the B process has to wait for A to finish, the C process has to > wait for B to finish, and so on. Otherwise you could find yourself with C > reading the data from B before B is finished writing it. Sure, which is a matter of writer/reader conflicts on a single file - nothing to do with "writing multiple files simultaneously" which was the question raised. ChrisA
Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread
Pickle based workflow - looking for advice Fabien <fabien.maussion@gmail.com> - 2015-04-13 16:58 +0200
Re: Pickle based workflow - looking for advice Devin Jeanpierre <jeanpierreda@gmail.com> - 2015-04-13 11:45 -0400
Re: Pickle based workflow - looking for advice Fabien <fabien.maussion@gmail.com> - 2015-04-13 19:39 +0200
Re: Pickle based workflow - looking for advice Robin Becker <robin@reportlab.com> - 2015-04-13 16:53 +0100
Re: Pickle based workflow - looking for advice Dave Angel <davea@davea.name> - 2015-04-13 12:25 -0400
Re: Pickle based workflow - looking for advice Fabien <fabien.maussion@gmail.com> - 2015-04-13 19:30 +0200
Re: Pickle based workflow - looking for advice Peter Otten <__peter__@web.de> - 2015-04-13 19:08 +0200
Re: Pickle based workflow - looking for advice Fabien <fabien.maussion@gmail.com> - 2015-04-13 19:35 +0200
Re: Pickle based workflow - looking for advice Chris Angelico <rosuav@gmail.com> - 2015-04-14 14:05 +1000
Re: Pickle based workflow - looking for advice Fabien <fabien.maussion@gmail.com> - 2015-04-14 09:58 +0200
Re: Pickle based workflow - looking for advice Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-04-14 23:08 +1000
Re: Pickle based workflow - looking for advice Chris Angelico <rosuav@gmail.com> - 2015-04-14 23:45 +1000
Re: Pickle based workflow - looking for advice Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-04-15 00:14 +1000
Re: Pickle based workflow - looking for advice Chris Angelico <rosuav@gmail.com> - 2015-04-15 00:30 +1000
csiph-web