Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #26395

Re: CRC-checksum failed in gzip

Show all headers | View raw

> One last thing I would like to do before I add this fix is to actually
> be able to reproduce this behaviour, and I thought I could just do the
> following:
>
> import gzip
> import threading
>
>
> class OpenAndRead(threading.Thread):
>      def run(self):
>          fz = gzip.open('out2.txt.gz')
>          fz.read()
>          fz.close()
>
>
> if __name__ == '__main__':
>      for i in range(100):
>          OpenAndRead().start()
>
>
> But no matter how many threads I start, I can't reproduce the CRC
> error, any idea how I can try to help it happening?
Your example did not share the file object between threads. Here an 
example that does that:

class OpenAndRead(threading.Thread):
     def run(self):
	global fz
	fz.read(100)

if __name__ == '__main__':
    fz = gzip.open('out2.txt.gz')
    for i in range(10):
         OpenAndRead().start()

Try this with a huge file. And here is the one that should never throw 
CRC error, because the file object is protected by a lock:

class OpenAndRead(threading.Thread):
     def run(self):
         global fz
         global fl
         with fl:
             fz.read(100)

if __name__ == '__main__':
    fz = gzip.open('out2.txt.gz')
    fl = threading.Lock()
    for i in range(2):
         OpenAndRead().start()

>
> The code in run should be shared by all the threads since there are no
> locks, right?
The code is shared but the file object is not. In your example, a new 
file object is created, every time a thread is started.

Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Re: CRC-checksum failed in gzip andrea crotti <andrea.crotti.0@gmail.com> - 2012-08-01 14:01 +0100
  Re: CRC-checksum failed in gzip Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2012-08-01 16:17 +0000
    Re: CRC-checksum failed in gzip andrea crotti <andrea.crotti.0@gmail.com> - 2012-08-01 17:38 +0100
    Re: CRC-checksum failed in gzip Laszlo Nagy <gandalf@shopzeus.com> - 2012-08-01 19:05 +0200
    Re: CRC-checksum failed in gzip andrea crotti <andrea.crotti.0@gmail.com> - 2012-08-01 18:17 +0100
    Re: CRC-checksum failed in gzip Laszlo Nagy <gandalf@shopzeus.com> - 2012-08-01 19:57 +0200
      Re: CRC-checksum failed in gzip Ulrich Eckhardt <ulrich.eckhardt@dominolaser.com> - 2012-08-02 10:49 +0200
        Re: CRC-checksum failed in gzip Laszlo Nagy <gandalf@shopzeus.com> - 2012-08-02 12:14 +0200
    Re: CRC-checksum failed in gzip andrea crotti <andrea.crotti.0@gmail.com> - 2012-08-02 10:26 +0100
    Re: CRC-checksum failed in gzip Laszlo Nagy <gandalf@shopzeus.com> - 2012-08-02 12:21 +0200
    Re: CRC-checksum failed in gzip andrea crotti <andrea.crotti.0@gmail.com> - 2012-08-02 11:57 +0100
    Re: CRC-checksum failed in gzip andrea crotti <andrea.crotti.0@gmail.com> - 2012-08-02 11:59 +0100

csiph-web