Return-Path: <rosuav@gmail.com>
X-Original-To: python-list@python.org
Delivered-To: python-list@mail.python.org
X-Spam-Status: OK 0.001
X-Spam-Evidence: '*H*': 1.00; '*S*': 0.00; 'reason,': 0.07; 'subject:file': 0.07; 'wednesday,': 0.07; "'rb')": 0.09; 'here?': 0.09; 'issue?': 0.09; 'stringio': 0.09; 'subject:Posting': 0.09; 'cc:addr:python-list': 0.10; 'file,': 0.15; 'thu,': 0.15; '10:45': 0.16; 'big,': 0.16; 'example?': 0.16; 'from:addr:rosuav': 0.16; 'from:name:chris angelico': 0.16; 'helps!': 0.16; 'inclined': 0.16; 'iteration': 0.16; 'perfect.': 0.16; 'skip:g 50': 0.16; 'subject:image': 0.16; 'subject:server': 0.16; 'swallows': 0.16; 'terse': 0.16; 'wrote:': 0.16; 'passes': 0.18; 'student': 0.20; 'fix': 0.21; 'work,': 0.21; 'cc:2**0': 0.21; 'cc:addr:python.org': 0.21; "skip:' 40": 0.22; 'am,': 0.23; '2015': 0.23; 'header:In- Reply-To:1': 0.24; 'paul': 0.24; 'chris': 0.26; 'plain': 0.27; 'message-id:@mail.gmail.com': 0.28; "i'm": 0.29; 'appear': 0.29; 'away.': 0.29; 'once.': 0.29; "i'd": 0.31; 'posting': 0.32; 'probably': 0.32; 'subject:?': 0.34; 'previous': 0.34; 'file': 0.34; 'received:google.com': 0.34; 'that,': 0.34; 'could': 0.35; 'text.': 0.35; 'really': 0.35; "isn't": 0.35; 'but': 0.36; 'too': 0.36; 'should': 0.37; 'subject:: ': 0.37; 'difference': 0.38; 'rather': 0.38; 'version': 0.38; 'data': 0.40; 'sure': 0.40; 'some': 0.40; 'your': 0.60; 'back': 0.61; 'hope': 0.61; 'avoid': 0.61; 'entire': 0.61; 'was:': 0.66; 'situation': 0.67; 'now:': 0.72; 'blob': 0.84; 'chrisa': 0.84; 'utc-4,': 0.84; 'to:none': 0.90
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:cc :content-type; bh=4CtyMUz9r1XZnxJ5+sTboNA6WXZS2rVol4uSPqLcNBA=; b=goyh7h5FEf6+kIJdJNo7Gavbyz6RE5N+dXcjEzIEt/y0Rmq6Cx5/xBcDj0Dbv6fFOU ZMR0GeAfb5xkKw1Y4OCUsLDtC4Xreh6YDfHtmJexRzlB9BuVeQzorM9VJTL9lQx/IBsZ lss7808/X4Dd57MtaA5sFvEfUR0ybbXth3MZAFk8kY1qveI6j2Mb4ics7EM21xX/2BH/ M3XOnZGDpWKJlkjaQiChvTh4Ulgx9UYgexsWnRYR+SrobCgv10AKoAIX4hfQ6V0SoRIV t9Demaw/J5I7Pb5LLCPjG+6KmKYljgvGx8/tEndsSkxa70aHElSECZ4idXXAIgneY+RI 23ow==
MIME-Version: 1.0
X-Received: by 10.50.141.164 with SMTP id rp4mr39547505igb.2.1434591963132; Wed, 17 Jun 2015 18:46:03 -0700 (PDT)
In-Reply-To: <0dd32144-8969-4e15-9e78-4c9698fb9d8c@googlegroups.com>
References: <59756062-6632-46b4-a92c-aa3a260718a7@googlegroups.com> <mailman.575.1434587009.13271.python-list@python.org> <0dd32144-8969-4e15-9e78-4c9698fb9d8c@googlegroups.com>
Date: Thu, 18 Jun 2015 11:46:03 +1000
Subject: Re: Posting gzip'd image file - server says Malformed Upload?
From: Chris Angelico <rosuav@gmail.com>
Cc: "python-list@python.org" <python-list@python.org>
Content-Type: text/plain; charset=UTF-8
X-BeenThere: python-list@python.org
X-Mailman-Version: 2.1.20+
Precedence: list
List-Id: General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe: <https://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive: <http://mail.python.org/pipermail/python-list/>
List-Post: <mailto:python-list@python.org>
List-Help: <mailto:python-list-request@python.org?subject=help>
List-Subscribe: <https://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups: comp.lang.python
Message-ID: <mailman.577.1434591971.13271.python-list@python.org>
Lines: 42
NNTP-Posting-Host: 2001:888:2000:d::a6
X-Trace: 1434591971 news.xs4all.nl 2831 [2001:888:2000:d::a6]:52109
X-Complaints-To: abuse@xs4all.nl
Path: csiph.com!usenet.pasdenom.info!news.stben.net!border1.nntp.ams1.giganews.com!nntp.giganews.com!newsfeed.xs4all.nl!newsfeed7.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
Xref: csiph.com comp.lang.python:92800

On Thu, Jun 18, 2015 at 10:45 AM, Paul Hubert <phbrt25@gmail.com> wrote:
> On Wednesday, June 17, 2015 at 8:24:17 PM UTC-4, Chris Angelico wrote:
>
>> Are you sure you want iteration and writelines() here? I would be
>> inclined to avoid those for any situation that isn't plain text. If
>> the file isn't too big, I'd just read it all in a single blob and then
>> write it all out at once.
>>
>> ChrisA
>
> Do you think that would fix my issue? Could you give me an example?

Sorry for the abrupt and terse previous email; I had a student arrive
just as I was posting that, and hit Ctrl-Enter when I should really
have just left the email as a draft. Here's what I'm thinking:

# Was:
f_in = open(dafile, 'rb')
f_out = gzip.open('/Users/Paul/Desktop/scripts/pic.jpg.gz', 'wb')
f_out.writelines(f_in)
f_out.close()
f_in.close()

# Now:
gz = '/Users/Paul/Desktop/scripts/pic.jpg.gz'
with open(dafile, 'rb') as f_in, gzip.open(gz, 'wb') as f_out:
    f_out.write(f_in.read())

You might actually be able to write to a StringIO rather than to a
file, given that you appear to be just reading the data back again
straight away. But in case you want to keep the file around for some
other reason, this still works the exact same way you had it.

The main difference is that this version swallows the entire file in a
single gulp, then passes it all to the gzip writer. If your file is
tiny (under 1MB), this is perfect. If it's huge (over 1GB), you may
have problems. In between, it'll probably work, but might be
inefficient.

Hope that helps!

ChrisA