Re: Posting gzip'd image file - server says Malformed Upload?

Return-Path	<rosuav@gmail.com>
X-Original-To	python-list@python.org
Delivered-To	python-list@mail.python.org
X-Spam-Status	OK 0.001
X-Spam-Evidence	'H': 1.00; 'S': 0.00; 'reason,': 0.07; 'subject:file': 0.07; 'wednesday,': 0.07; "'rb')": 0.09; 'here?': 0.09; 'issue?': 0.09; 'stringio': 0.09; 'subject:Posting': 0.09; 'cc:addr:python-list': 0.10; 'file,': 0.15; 'thu,': 0.15; '10:45': 0.16; 'big,': 0.16; 'example?': 0.16; 'from:addr:rosuav': 0.16; 'from:name:chris angelico': 0.16; 'helps!': 0.16; 'inclined': 0.16; 'iteration': 0.16; 'perfect.': 0.16; 'skip:g 50': 0.16; 'subject:image': 0.16; 'subject:server': 0.16; 'swallows': 0.16; 'terse': 0.16; 'wrote:': 0.16; 'passes': 0.18; 'student': 0.20; 'fix': 0.21; 'work,': 0.21; 'cc:2**0': 0.21; 'cc:addr:python.org': 0.21; "skip:' 40": 0.22; 'am,': 0.23; '2015': 0.23; 'header:In- Reply-To:1': 0.24; 'paul': 0.24; 'chris': 0.26; 'plain': 0.27; 'message-id:@mail.gmail.com': 0.28; "i'm": 0.29; 'appear': 0.29; 'away.': 0.29; 'once.': 0.29; "i'd": 0.31; 'posting': 0.32; 'probably': 0.32; 'subject:?': 0.34; 'previous': 0.34; 'file': 0.34; 'received:google.com': 0.34; 'that,': 0.34; 'could': 0.35; 'text.': 0.35; 'really': 0.35; "isn't": 0.35; 'but': 0.36; 'too': 0.36; 'should': 0.37; 'subject:: ': 0.37; 'difference': 0.38; 'rather': 0.38; 'version': 0.38; 'data': 0.40; 'sure': 0.40; 'some': 0.40; 'your': 0.60; 'back': 0.61; 'hope': 0.61; 'avoid': 0.61; 'entire': 0.61; 'was:': 0.66; 'situation': 0.67; 'now:': 0.72; 'blob': 0.84; 'chrisa': 0.84; 'utc-4,': 0.84; 'to:none': 0.90
DKIM-Signature	v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:cc :content-type; bh=4CtyMUz9r1XZnxJ5+sTboNA6WXZS2rVol4uSPqLcNBA=; b=goyh7h5FEf6+kIJdJNo7Gavbyz6RE5N+dXcjEzIEt/y0Rmq6Cx5/xBcDj0Dbv6fFOU ZMR0GeAfb5xkKw1Y4OCUsLDtC4Xreh6YDfHtmJexRzlB9BuVeQzorM9VJTL9lQx/IBsZ lss7808/X4Dd57MtaA5sFvEfUR0ybbXth3MZAFk8kY1qveI6j2Mb4ics7EM21xX/2BH/ M3XOnZGDpWKJlkjaQiChvTh4Ulgx9UYgexsWnRYR+SrobCgv10AKoAIX4hfQ6V0SoRIV t9Demaw/J5I7Pb5LLCPjG+6KmKYljgvGx8/tEndsSkxa70aHElSECZ4idXXAIgneY+RI 23ow==
MIME-Version	1.0
X-Received	by 10.50.141.164 with SMTP id rp4mr39547505igb.2.1434591963132; Wed, 17 Jun 2015 18:46:03 -0700 (PDT)
In-Reply-To	<0dd32144-8969-4e15-9e78-4c9698fb9d8c@googlegroups.com>
References	<59756062-6632-46b4-a92c-aa3a260718a7@googlegroups.com> <mailman.575.1434587009.13271.python-list@python.org> <0dd32144-8969-4e15-9e78-4c9698fb9d8c@googlegroups.com>
Date	Thu, 18 Jun 2015 11:46:03 +1000
Subject	Re: Posting gzip'd image file - server says Malformed Upload?
From	Chris Angelico <rosuav@gmail.com>
Cc	"python-list@python.org" <python-list@python.org>
Content-Type	text/plain; charset=UTF-8
X-BeenThere	python-list@python.org
X-Mailman-Version	2.1.20+
Precedence	list
List-Id	General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe	<https://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive	<http://mail.python.org/pipermail/python-list/>
List-Post	<mailto:python-list@python.org>
List-Help	<mailto:python-list-request@python.org?subject=help>
List-Subscribe	<https://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups	comp.lang.python
Message-ID	<mailman.577.1434591971.13271.python-list@python.org> (permalink)
Lines	42
NNTP-Posting-Host	2001:888:2000:d::a6
X-Trace	1434591971 news.xs4all.nl 2831 [2001:888:2000:d::a6]:52109
X-Complaints-To	abuse@xs4all.nl
Path	csiph.com!usenet.pasdenom.info!news.stben.net!border1.nntp.ams1.giganews.com!nntp.giganews.com!newsfeed.xs4all.nl!newsfeed7.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
Xref	csiph.com comp.lang.python:92800

Show key headers only | View raw

On Thu, Jun 18, 2015 at 10:45 AM, Paul Hubert <phbrt25@gmail.com> wrote:
> On Wednesday, June 17, 2015 at 8:24:17 PM UTC-4, Chris Angelico wrote:
>
>> Are you sure you want iteration and writelines() here? I would be
>> inclined to avoid those for any situation that isn't plain text. If
>> the file isn't too big, I'd just read it all in a single blob and then
>> write it all out at once.
>>
>> ChrisA
>
> Do you think that would fix my issue? Could you give me an example?

Sorry for the abrupt and terse previous email; I had a student arrive
just as I was posting that, and hit Ctrl-Enter when I should really
have just left the email as a draft. Here's what I'm thinking:

# Was:
f_in = open(dafile, 'rb')
f_out = gzip.open('/Users/Paul/Desktop/scripts/pic.jpg.gz', 'wb')
f_out.writelines(f_in)
f_out.close()
f_in.close()

# Now:
gz = '/Users/Paul/Desktop/scripts/pic.jpg.gz'
with open(dafile, 'rb') as f_in, gzip.open(gz, 'wb') as f_out:
    f_out.write(f_in.read())

You might actually be able to write to a StringIO rather than to a
file, given that you appear to be just reading the data back again
straight away. But in case you want to keep the file around for some
other reason, this still works the exact same way you had it.

The main difference is that this version swallows the entire file in a
single gulp, then passes it all to the gzip writer. If your file is
tiny (under 1MB), this is perfect. If it's huge (over 1GB), you may
have problems. In between, it'll probably work, but might be
inefficient.

Hope that helps!

ChrisA

Thread

Posting gzip'd image file - server says Malformed Upload? Paul Hubert <phbrt25@gmail.com> - 2015-06-17 14:55 -0700
  Re: Posting gzip'd image file - server says Malformed Upload? Chris Angelico <rosuav@gmail.com> - 2015-06-18 10:23 +1000
    Re: Posting gzip'd image file - server says Malformed Upload? Paul Hubert <phbrt25@gmail.com> - 2015-06-17 17:45 -0700
      Re: Posting gzip'd image file - server says Malformed Upload? Michael Torrie <torriem@gmail.com> - 2015-06-17 19:10 -0600
      Re: Posting gzip'd image file - server says Malformed Upload? Chris Angelico <rosuav@gmail.com> - 2015-06-18 11:46 +1000
        Re: Posting gzip'd image file - server says Malformed Upload? Paul Hubert <phbrt25@gmail.com> - 2015-06-17 20:48 -0700
          Re: Posting gzip'd image file - server says Malformed Upload? Chris Angelico <rosuav@gmail.com> - 2015-06-18 13:56 +1000
          Re: Posting gzip'd image file - server says Malformed Upload? Michael Torrie <torriem@gmail.com> - 2015-06-17 21:59 -0600
          Re: Posting gzip'd image file - server says Malformed Upload? Laura Creighton <lac@openend.se> - 2015-06-18 23:44 +0200
  Re: Posting gzip'd image file - server says Malformed Upload? random832@fastmail.us - 2015-06-18 08:38 -0400
  Re: Posting gzip'd image file - server says Malformed Upload? Chris Angelico <rosuav@gmail.com> - 2015-06-18 22:42 +1000

csiph-web