Path: csiph.com!v102.xanadu-bbs.net!xanadu-bbs.net!feeder.erje.net!eu.feeder.erje.net!xlned.com!feeder1.xlned.com!newsfeed.xs4all.nl!newsfeed2a.news.xs4all.nl!xs4all!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail Return-Path: X-Original-To: python-list@python.org Delivered-To: python-list@mail.python.org X-Spam-Status: OK 0.000 X-Spam-Evidence: '*H*': 1.00; '*S*': 0.00; 'string.': 0.05; 'binary': 0.07; 'python3': 0.07; 'subject:file': 0.07; 'subject:skip:b 10': 0.07; 'string': 0.09; 'received:209.85.219': 0.09; 'cc:addr :python-list': 0.11; 'python': 0.11; 'bug': 0.12; "'rb')": 0.16; '(without': 0.16; '-tkc': 0.16; 'expects': 0.16; 'posting,': 0.16; 'stringio': 0.16; 'to:addr:python.list': 0.16; 'to:addr:tim.thechases.com': 0.16; 'to:name:tim chase': 0.16; 'wrote:': 0.18; 'feb': 0.22; '>>>': 0.22; 'import': 0.22; 'cc:addr:python.org': 0.22; '>>>': 0.24; 'bytes': 0.24; 'cc:2**0': 0.24; '>': 0.26; 'this:': 0.26; 'header:In-Reply- To:1': 0.27; 'function': 0.29; 'correct': 0.29; 'tim': 0.29; 'message-id:@mail.gmail.com': 0.30; 'chase': 0.31; 'with,': 0.31; 'file': 0.32; 'skip:d 20': 0.34; 'received:209.85': 0.35; 'received:google.com': 0.35; 'received:209': 0.37; 'skip:o 20': 0.38; 'skip:& 10': 0.38; 'handle': 0.38; 'fact': 0.38; 'pm,': 0.38; 'skip:& 20': 0.39; 'bad': 0.39; 'either': 0.39; 'skip:o 30': 0.61; 'more': 0.64; 'world': 0.66; 'subject:handle': 0.84; '2013,': 0.91 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-type; bh=JjKNCDAfArRzoh8OrZfz0Ac0qAVUlnJJpgM8fupx3Jo=; b=V1z80VTMr0qxpr0VMTsYh48XQfC3SmLEMh4iqnXiljs0fKC2fSs+qoxJw+r/u7d3Kz I6QaO7MJ1wAVoQaM/+77w0IPYipNlUrRO/mYtEtIo9e8mLXvOsRpXKO1wEv+vW38/l4a bmRwuMkdubSHvrdLF1PfdYIim8pBgZQ9J59sYHDtoHgeQIbyh0Q3aXHZzfOppreMA5a+ /QYmOelWkKprK7o+lqqHui7z8zlhnf635fhRSoUcePRiXuTn8crEJNVe2F8q/yZZ1iY+ /KRHgmTFJWker7/KKa/WYvLbf+CnjulykM660aFqngFJfn6SP21cILWs/GBEec8FTRxC vl/Q== X-Gm-Message-State: ALoCoQnzOeTOZYmXc3YHJR0/W4jJm60Vl3isVnsP9dXUH1DQYz/407VzPBVHOKONcRA0XSn3VSFL X-Received: by 10.60.81.227 with SMTP id d3mr26616458oey.41.1400467102346; Sun, 18 May 2014 19:38:22 -0700 (PDT) MIME-Version: 1.0 In-Reply-To: <20140518213256.5a0f2d71@bigbox.christie.dr> References: <20140518213256.5a0f2d71@bigbox.christie.dr> From: Vincent Davis Date: Sun, 18 May 2014 20:38:02 -0600 Subject: Re: bz2.decompress as file handle To: Tim Chase Content-Type: multipart/alternative; boundary=047d7b417fbb27c30904f9b7a80c Cc: "python-list@python.org" X-BeenThere: python-list@python.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: General discussion list for the Python programming language List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Newsgroups: comp.lang.python Message-ID: Lines: 125 NNTP-Posting-Host: 2001:888:2000:d::a6 X-Trace: 1400467110 news.xs4all.nl 2924 [2001:888:2000:d::a6]:37691 X-Complaints-To: abuse@xs4all.nl Xref: csiph.com comp.lang.python:71742 --047d7b417fbb27c30904f9b7a80c Content-Type: text/plain; charset=UTF-8 Well after posting, I think I figured it out. The key is to use StringIO to get a file handle on the string. The fact that it is binary just complicates it a little. with open('Tests/Affy/affy_v3_ex.CEL.bz2', 'rb') as handle: cel_data = StringIO(decompress(handle.read()).decode('ascii')) Vincent Davis 720-301-3003 On Sun, May 18, 2014 at 8:32 PM, Tim Chase wrote: > On 2014-05-18 19:53, Vincent Davis wrote: > > I have a file compressed with bz2 and a function that expects a > > file handle. When I decompress the bz2 file I get a string (binary) > > not a file handle. > > > from bz2 import decompress, > > > > with open('Tests/Affy/affy_v3_ex.CEL.bz2', 'rb') as handle: > > cel_data = decompress(handle.read()) > > When I try (without the Bio.Affy which isn't part of the stdlib), I > get correct bytes from this: > > tim@bigbox:~$ echo hello world > test.txt > tim@bigbox:~$ bzip2 -9 test.txt > tim@bigbox:~$ python3 > Python 3.2.3 (default, Feb 20 2013, 14:44:27) > [GCC 4.7.2] on linux2 > Type "help", "copyright", "credits" or "license" for more information. > >>> from bz2 import decompress > >>> with open('test.txt.bz2', 'rb') as f: > ... data = decompress(f.read()) > ... > >>> data > b'hello world\n' > > > > c = CelFile.read(cel_data) > > So either you have bad data in the file to begin with, or your > CelFile.read() function has a bug in it. > > -tkc > > > > --047d7b417fbb27c30904f9b7a80c Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Well after posting, I think I figured it out.= =C2=A0
The key is to use StringIO to get a file handle on the string. The fact tha= t it is binary just complicates it a little.

with = open('Tests/Affy/affy_v3_ex.CEL.bz2', 'rb') as handle:
=C2=A0 =C2=A0 cel_data =3D StringIO(decompress(handle.read()).decode(= 9;ascii'))

Vincent = Davis
720-301-3003


On Sun, May 18, 2014 at 8:32 PM, Tim Cha= se <python.list@tim.thechases.com> wrote:
On 2014-05-18 19:53, Vincent Davis wrote:
> I have a file compressed with bz2 and a function that expects a
> file handle. When I decompress the bz2 file I get a string (binary) > not a file handle.

> from bz2 import decompress,
>
> with open('Tests/Affy/affy_v3_ex.CEL.bz2', 'rb') as ha= ndle:
> =C2=A0 =C2=A0 cel_data =3D decompress(handle.read())

When I try (without the Bio.Affy which isn't part of the stdlib),= I
get correct bytes from this:

tim@bigbox:~$ echo hello world > test.txt
tim@bigbox:~$ bzip2 -9 test.txt
tim@bigbox:~$ python3
Python 3.2.3 (default, Feb 20 2013, 14:44:27)
[GCC 4.7.2] on linux2
Type "help", "copyright", "credits" or "= license" for more information.
>>> from bz2 import decompress
>>> with open('test.txt.bz2', 'rb') as f:
... =C2=A0 =C2=A0 data =3D decompress(f.read())
...
>>> data
b'hello world\n'


> c =3D CelFile.read(cel_data)

So either you have bad data in the file to begin with, or your
CelFile.read() function has a bug in it.

-tkc




--047d7b417fbb27c30904f9b7a80c--