Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #39961 > unrolled thread

Writing to same file from two threads

Back to article view | Back to comp.lang.python

Contents

  Writing to same file from two threads jt@toerring.de (Jens Thoms Toerring) - 2013-02-26 15:17 +0000
    Re: Writing to same file from two threads Paul Rubin <no.email@nospam.invalid> - 2013-02-26 09:32 -0800
      Re: Writing to same file from two threads jt@toerring.de (Jens Thoms Toerring) - 2013-02-26 20:08 +0000
        Re: Writing to same file from two threads Antoine Pitrou <solipsis@pitrou.net> - 2013-02-27 13:26 +0000
          Re: Writing to same file from two threads jt@toerring.de (Jens Thoms Toerring) - 2013-02-27 16:05 +0000
          Re: Writing to same file from two threads Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2013-02-28 01:05 +0000
            Re: Writing to same file from two threads Antoine Pitrou <solipsis@pitrou.net> - 2013-03-01 14:54 +0000

#39961 — Writing to same file from two threads

Hi,

   I noticed in someone elses program that it writes single
lines to the same file from (what I call for loss of a better
name) the "main thread" of the program and from a thread sub-
sequentally started. This got me worried if it might result
in garbled output (i.e. having some output from A inside a
line written by B or vice versae) because the "main thread" or
the other thread could be interrupted during a call of write().
Is this a valid concern (and thus locking the file object is
required before writing to it) or am I guaranteed that this
can't happen? In the latter case I would be grateful for an
explanation what mechanism is responsible for this never to
happen.
                  Thanks and best regards, Jens

PS: I already have determined experimentally that a context
    switch definitely can happen between two calls of write()
    (and I expected nothing else), what I'm worried about are
    context switches somewhere within the very innards of what
    write() does.
-- 
  \   Jens Thoms Toerring  ___      jt@toerring.de
   \__________________________      http://toerring.de

[toc] | [next] | [standalone]

#39977

jt@toerring.de (Jens Thoms Toerring) writes:
> in garbled output (i.e. having some output from A inside a
> line written by B or vice versae) because the "main thread" or

Yes they do get garbled like that.  Preferred Python style is put a
single thread in charge of all the i/o to that file, and communicate
with it by message passing through Queue objects.  That is safer than
directly using locks.

[toc] | [prev] | [next] | [standalone]

#40001

Paul Rubin <no.email@nospam.invalid> wrote:
> jt@toerring.de (Jens Thoms Toerring) writes:
> > in garbled output (i.e. having some output from A inside a
> > line written by B or vice versae) because the "main thread" or

> Yes they do get garbled like that.  Preferred Python style is put a
> single thread in charge of all the i/o to that file, and communicate
> with it by message passing through Queue objects.  That is safer than
> directly using locks.

Thank you for confirmig my suspicion;-) But you have induced
another question: why is using a Queue safer than locking (not
that I doubt that it might be more elegant etc.). Is it "safer"
because it's less likely that one gets it wrong (e.g. by for-
grtting to acquire the lock) or is there something inherently
unsafe about locks?

                  Thank you and best regards, Jens
-- 
  \   Jens Thoms Toerring  ___      jt@toerring.de
   \__________________________      http://toerring.de

[toc] | [prev] | [next] | [standalone]

#40061

Jens Thoms Toerring <jt <at> toerring.de> writes:
> 
> Paul Rubin <no.email <at> nospam.invalid> wrote:
> > jt <at> toerring.de (Jens Thoms Toerring) writes:
> > > in garbled output (i.e. having some output from A inside a
> > > line written by B or vice versae) because the "main thread" or
> 
> > Yes they do get garbled like that.  Preferred Python style is put a
> > single thread in charge of all the i/o to that file, and communicate
> > with it by message passing through Queue objects.  That is safer than
> > directly using locks.
> 
> Thank you for confirmig my suspicion But you have induced
> another question: why is using a Queue safer than locking (not
> that I doubt that it might be more elegant etc.). Is it "safer"
> because it's less likely that one gets it wrong (e.g. by for-
> grtting to acquire the lock) or is there something inherently
> unsafe about locks?

For the record, binary files are thread-safe in Python 3, but text files
are not.
Locks are safe if you use them well. As you point out, if you forget
to acquire your lock, or if you devise a situation where there is a
deadlock between competing locks, you can have difficult to diagnose
issues. Queues have their internal locking all done for you.

Regards

Antoine.

[toc] | [prev] | [next] | [standalone]

#40068

Antoine Pitrou <solipsis@pitrou.net> wrote:
> Jens Thoms Toerring <jt <at> toerring.de> writes:
> > 
> > Paul Rubin <no.email <at> nospam.invalid> wrote:
> > > jt <at> toerring.de (Jens Thoms Toerring) writes:
> > > > in garbled output (i.e. having some output from A inside a
> > > > line written by B or vice versae) because the "main thread" or
> > 
> > > Yes they do get garbled like that.  Preferred Python style is put a
> > > single thread in charge of all the i/o to that file, and communicate
> > > with it by message passing through Queue objects.  That is safer than
> > > directly using locks.
> > 
> > Thank you for confirmig my suspicion But you have induced
> > another question: why is using a Queue safer than locking (not
> > that I doubt that it might be more elegant etc.). Is it "safer"
> > because it's less likely that one gets it wrong (e.g. by for-
> > grtting to acquire the lock) or is there something inherently
> > unsafe about locks?

> For the record, binary files are thread-safe in Python 3, but text files
> are not.
> Locks are safe if you use them well. As you point out, if you forget
> to acquire your lock, or if you devise a situation where there is a
> deadlock between competing locks, you can have difficult to diagnose
> issues. Queues have their internal locking all done for you.

Thank you for your kind answers!
                                  Best regards, Jens
-- 
  \   Jens Thoms Toerring  ___      jt@toerring.de
   \__________________________      http://toerring.de

[toc] | [prev] | [next] | [standalone]

#40089

On Wed, 27 Feb 2013 13:26:18 +0000, Antoine Pitrou wrote:

> For the record, binary files are thread-safe in Python 3, but text files
> are not.

Where is this documented please?



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#40259

Steven D'Aprano <steve+comp.lang.python <at> pearwood.info> writes:
> 
> On Wed, 27 Feb 2013 13:26:18 +0000, Antoine Pitrou wrote:
> 
> > For the record, binary files are thread-safe in Python 3, but text files
> > are not.
> 
> Where is this documented please?

In the documentation, of course ;)
http://docs.python.org/3.3/library/io.html#multi-threading

Regards

Antoine.

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web