Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #38759 > unrolled thread

Re: Generate 16+MAX_WBITS decompressable data

Back to article view | Back to comp.lang.python

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: Generate 16+MAX_WBITS decompressable data Terry Reedy <tjreedy@udel.edu> - 2013-02-12 10:27 -0500
    Re: Generate 16+MAX_WBITS decompressable data Marc Christiansen <usenet@solar-empire.de> - 2013-02-12 21:39 +0100
      Re: Generate 16+MAX_WBITS decompressable data Fayaz Yusuf Khan <fayaz@dexetra.com> - 2013-02-13 10:48 +0530
      Re: Generate 16+MAX_WBITS decompressable data Terry Reedy <tjreedy@udel.edu> - 2013-02-13 01:30 -0500

#38759 — Re: Generate 16+MAX_WBITS decompressable data

On 2/12/2013 7:47 AM, Fayaz Yusuf Khan wrote:
> I'm trying write unit-tests for some of my old code and have run into this
> piece of code.
>
> dcomp = zlib.decompressobj(16+zlib.MAX_WBITS)

Since zlib.MAX_WBITS is the largest value that should be passed (15), 
adding 16 makes no sense. Since it is also the default, there is also no 
point in providing it explicitly. "Its absolute value should be between 
8 and 15 for the most recent versions of the zlib library".

> chunk = ''.join(f.chunks())
> received_data =  dcomp.decompress(chunk)

Since decompressobj is intended for data that will not all fit in memory 
at once, and since chunk does, just use zlib.decompress(chunk)

> How do I generate the chunk here? From what I've been trying I'm getting
> this exception:
>>>> import zlib
>>>> zlib.compress('hello')
> 'x\x9c\xcbH\xcd\xc9\xc9\x07\x00\x06,\x02\x15'
>>>> zlib.decompress(_, 16+zlib.MAX_WBITS)
> Traceback (most recent call last):
>    File "<stdin>", line 1, in <module>
> zlib.error: Error -3 while decompressing data: incorrect header check

You asked for an internal buffer of 2**31 = 2 gigabytes.

> zlib.decompress without the second argument works, but I can't really go
> ahead into my project file and remove it.

-- 
Terry Jan Reedy

[toc] | [next] | [standalone]

#38782

Terry Reedy <tjreedy@udel.edu> wrote:
> On 2/12/2013 7:47 AM, Fayaz Yusuf Khan wrote:
>> dcomp = zlib.decompressobj(16+zlib.MAX_WBITS)
> 
> Since zlib.MAX_WBITS is the largest value that should be passed (15), 
> adding 16 makes no sense. Since it is also the default, there is also no 
> point in providing it explicitly. "Its absolute value should be between 
> 8 and 15 for the most recent versions of the zlib library".

The above code uses a feature of the zlib library which isn't really
widely known.
From http://www.zlib.net/manual.html#Advanced (inflateInit2):
  windowBits can also be greater than 15 for optional gzip decoding. Add
  32 to windowBits to enable zlib and gzip decoding with automatic
  header detection, or add 16 to decode only the gzip format (the zlib
  format will return a Z_DATA_ERROR).


>> How do I generate the chunk here? From what I've been trying I'm getting
>> this exception:
>>>>> import zlib
>>>>> zlib.compress('hello')
>> 'x\x9c\xcbH\xcd\xc9\xc9\x07\x00\x06,\x02\x15'
>>>>> zlib.decompress(_, 16+zlib.MAX_WBITS)
>> Traceback (most recent call last):
>>    File "<stdin>", line 1, in <module>
>> zlib.error: Error -3 while decompressing data: incorrect header check

Try using a compressobj with 24 <= wbits < 32. It should work, but I
didn't try.

Marc

[toc] | [prev] | [next] | [standalone]

#38807

Marc Christiansen wrote:

> Try using a compressobj with 24 <= wbits < 32. It should work, but I
> didn't try.
> 
Er, the problem is that compressobj doesn't accept a WBIT argument.
-- 
Fayaz Yusuf Khan
Cloud architect, Dexetra SS, India
fayaz.yusuf.khan_AT_gmail_DOT_com, fayaz_AT_dexetra_DOT_com
+91-9746-830-823

[toc] | [prev] | [next] | [standalone]

#38811

On 2/13/2013 12:18 AM, Fayaz Yusuf Khan wrote:
> Marc Christiansen wrote:
>
>> Try using a compressobj with 24 <= wbits < 32. It should work, but I
>> didn't try.
>>
> Er, the problem is that compressobj doesn't accept a WBIT argument.

"Changed in version 3.3: Added the method, wbits, memlevel, strategy and 
zdict parameters."

-- 
Terry Jan Reedy

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web