Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #92784 > unrolled thread

Posting gzip'd image file - server says Malformed Upload?

Back to article view | Back to comp.lang.python

Contents

  Posting gzip'd image file - server says Malformed Upload? Paul Hubert <phbrt25@gmail.com> - 2015-06-17 14:55 -0700
    Re: Posting gzip'd image file - server says Malformed Upload? Chris Angelico <rosuav@gmail.com> - 2015-06-18 10:23 +1000
      Re: Posting gzip'd image file - server says Malformed Upload? Paul Hubert <phbrt25@gmail.com> - 2015-06-17 17:45 -0700
        Re: Posting gzip'd image file - server says Malformed Upload? Michael Torrie <torriem@gmail.com> - 2015-06-17 19:10 -0600
        Re: Posting gzip'd image file - server says Malformed Upload? Chris Angelico <rosuav@gmail.com> - 2015-06-18 11:46 +1000
          Re: Posting gzip'd image file - server says Malformed Upload? Paul Hubert <phbrt25@gmail.com> - 2015-06-17 20:48 -0700
            Re: Posting gzip'd image file - server says Malformed Upload? Chris Angelico <rosuav@gmail.com> - 2015-06-18 13:56 +1000
            Re: Posting gzip'd image file - server says Malformed Upload? Michael Torrie <torriem@gmail.com> - 2015-06-17 21:59 -0600
            Re: Posting gzip'd image file - server says Malformed Upload? Laura Creighton <lac@openend.se> - 2015-06-18 23:44 +0200
    Re: Posting gzip'd image file - server says Malformed Upload? random832@fastmail.us - 2015-06-18 08:38 -0400
    Re: Posting gzip'd image file - server says Malformed Upload? Chris Angelico <rosuav@gmail.com> - 2015-06-18 22:42 +1000

#92784 — Posting gzip'd image file - server says Malformed Upload?

Any idea why a server might be returning the message, in json format, "Malformed Upload"? The image is gzipped as the server requires... Someone on another forum said that he thinks Python requests automatically gzips?

I have the code working in vb.net... no idea why it wont work in Python.

This is my code.

def uploadPhoto(url, sid, dafile):

headers = {'User-Agent':useragent, 
                     'Session-Id':sessionid, 
                     'Content-Type':'image/jpeg', 
                     'Accept-Encoding':'gzip, deflate'}

f_in = open(dafile, 'rb')
f_out = gzip.open('/Users/Paul/Desktop/scripts/pic.jpg.gz', 'wb')
f_out.writelines(f_in)
f_out.close()
f_in.close()

files = {('Image', open('/Users/Paul/Desktop/scripts/pic.jpg.gz', 'rb'))}
r = requests.post(url, headers=headers, proxies=proxies, files=files)    

print r.status_code
print r.request.headers
return str(r.text)



The server replies with:

{"message":"Malformed Upload","code":1012}

[toc] | [next] | [standalone]

#92797

On Thu, Jun 18, 2015 at 7:55 AM, Paul Hubert <phbrt25@gmail.com> wrote:
> f_in = open(dafile, 'rb')
> f_out = gzip.open('/Users/Paul/Desktop/scripts/pic.jpg.gz', 'wb')
> f_out.writelines(f_in)
> f_out.close()
> f_in.close()

Are you sure you want iteration and writelines() here? I would be
inclined to avoid those for any situation that isn't plain text. If
the file isn't too big, I'd just read it all in a single blob and then
write it all out at once.

ChrisA

[toc] | [prev] | [next] | [standalone]

#92798

On Wednesday, June 17, 2015 at 8:24:17 PM UTC-4, Chris Angelico wrote:

> Are you sure you want iteration and writelines() here? I would be
> inclined to avoid those for any situation that isn't plain text. If
> the file isn't too big, I'd just read it all in a single blob and then
> write it all out at once.
> 
> ChrisA

Do you think that would fix my issue? Could you give me an example? 

[toc] | [prev] | [next] | [standalone]

#92799

On 06/17/2015 06:45 PM, Paul Hubert wrote:
> On Wednesday, June 17, 2015 at 8:24:17 PM UTC-4, Chris Angelico wrote:
> 
>> Are you sure you want iteration and writelines() here? I would be
>> inclined to avoid those for any situation that isn't plain text. If
>> the file isn't too big, I'd just read it all in a single blob and then
>> write it all out at once.
>>
>> ChrisA
> 
> Do you think that would fix my issue? 

I recommend you try it and see.  That's the only way to know.  We don't
know any details of your program or situation really, so we can only guess.

> Could you give me an example?

Instead of writelines(), just write() the data.  There are examples in
the official python docs I'm sure.

[toc] | [prev] | [next] | [standalone]

#92800

On Thu, Jun 18, 2015 at 10:45 AM, Paul Hubert <phbrt25@gmail.com> wrote:
> On Wednesday, June 17, 2015 at 8:24:17 PM UTC-4, Chris Angelico wrote:
>
>> Are you sure you want iteration and writelines() here? I would be
>> inclined to avoid those for any situation that isn't plain text. If
>> the file isn't too big, I'd just read it all in a single blob and then
>> write it all out at once.
>>
>> ChrisA
>
> Do you think that would fix my issue? Could you give me an example?

Sorry for the abrupt and terse previous email; I had a student arrive
just as I was posting that, and hit Ctrl-Enter when I should really
have just left the email as a draft. Here's what I'm thinking:

# Was:
f_in = open(dafile, 'rb')
f_out = gzip.open('/Users/Paul/Desktop/scripts/pic.jpg.gz', 'wb')
f_out.writelines(f_in)
f_out.close()
f_in.close()

# Now:
gz = '/Users/Paul/Desktop/scripts/pic.jpg.gz'
with open(dafile, 'rb') as f_in, gzip.open(gz, 'wb') as f_out:
    f_out.write(f_in.read())

You might actually be able to write to a StringIO rather than to a
file, given that you appear to be just reading the data back again
straight away. But in case you want to keep the file around for some
other reason, this still works the exact same way you had it.

The main difference is that this version swallows the entire file in a
single gulp, then passes it all to the gzip writer. If your file is
tiny (under 1MB), this is perfect. If it's huge (over 1GB), you may
have problems. In between, it'll probably work, but might be
inefficient.

Hope that helps!

ChrisA

[toc] | [prev] | [next] | [standalone]

#92801

On Wednesday, June 17, 2015 at 9:46:25 PM UTC-4, Chris Angelico wrote:
> On Thu, Jun 18, 2015 at 10:45 AM, Paul Hubert <phbrt25@gmail.com> wrote:
> > On Wednesday, June 17, 2015 at 8:24:17 PM UTC-4, Chris Angelico wrote:
> >
> >> Are you sure you want iteration and writelines() here? I would be
> >> inclined to avoid those for any situation that isn't plain text. If
> >> the file isn't too big, I'd just read it all in a single blob and then
> >> write it all out at once.
> >>
> >> ChrisA
> >
> > Do you think that would fix my issue? Could you give me an example?
> 
> Sorry for the abrupt and terse previous email; I had a student arrive
> just as I was posting that, and hit Ctrl-Enter when I should really
> have just left the email as a draft. Here's what I'm thinking:
> 
> # Was:
> f_in = open(dafile, 'rb')
> f_out = gzip.open('/Users/Paul/Desktop/scripts/pic.jpg.gz', 'wb')
> f_out.writelines(f_in)
> f_out.close()
> f_in.close()
> 
> # Now:
> gz = '/Users/Paul/Desktop/scripts/pic.jpg.gz'
> with open(dafile, 'rb') as f_in, gzip.open(gz, 'wb') as f_out:
>     f_out.write(f_in.read())
> 
> You might actually be able to write to a StringIO rather than to a
> file, given that you appear to be just reading the data back again
> straight away. But in case you want to keep the file around for some
> other reason, this still works the exact same way you had it.
> 
> The main difference is that this version swallows the entire file in a
> single gulp, then passes it all to the gzip writer. If your file is
> tiny (under 1MB), this is perfect. If it's huge (over 1GB), you may
> have problems. In between, it'll probably work, but might be
> inefficient.
> 
> Hope that helps!
> 
> ChrisA

Same result - server says malformed upload. :/

[toc] | [prev] | [next] | [standalone]

#92802

On Thu, Jun 18, 2015 at 1:48 PM, Paul Hubert <phbrt25@gmail.com> wrote:
>> # Now:
>> gz = '/Users/Paul/Desktop/scripts/pic.jpg.gz'
>> with open(dafile, 'rb') as f_in, gzip.open(gz, 'wb') as f_out:
>>     f_out.write(f_in.read())
>>
>
> Same result - server says malformed upload. :/

Oh well, was worth a shot. Since the file's sitting around still, you
should be able to attempt to decompress it with the stand-alone gzip
command; does that give the right result? Also, you should be able to
manually send that .gz file up to the server, which would be worth
confirming.

Are you certain that the content should be gzipped as a file? It's not
meant to be done as a transfer-level encoding?

ChrisA

[toc] | [prev] | [next] | [standalone]

#92803

On 06/17/2015 09:48 PM, Paul Hubert wrote:
> Same result - server says malformed upload. :/

You may want to run a sniffer like wireshark and see what the difference
is between the packets coming from your C# program and coming from Python.

[toc] | [prev] | [next] | [standalone]

#92855

I got to this party late.

One way to get the malformed upload message is is you gzip something
that already is gzipped, and send that up the pipe.

worth checking.

Laura

[toc] | [prev] | [next] | [standalone]

#92827

On Wed, Jun 17, 2015, at 20:23, Chris Angelico wrote:
> On Thu, Jun 18, 2015 at 7:55 AM, Paul Hubert <phbrt25@gmail.com> wrote:
> > f_in = open(dafile, 'rb')
> > f_out = gzip.open('/Users/Paul/Desktop/scripts/pic.jpg.gz', 'wb')
> > f_out.writelines(f_in)
> > f_out.close()
> > f_in.close()
> 
> Are you sure you want iteration and writelines() here? I would be
> inclined to avoid those for any situation that isn't plain text. If
> the file isn't too big, I'd just read it all in a single blob and then
> write it all out at once.

Is there a reason not to use shutil.copyfileobj?

[toc] | [prev] | [next] | [standalone]

#92828

On Thu, Jun 18, 2015 at 10:38 PM,  <random832@fastmail.us> wrote:
> On Wed, Jun 17, 2015, at 20:23, Chris Angelico wrote:
>> On Thu, Jun 18, 2015 at 7:55 AM, Paul Hubert <phbrt25@gmail.com> wrote:
>> > f_in = open(dafile, 'rb')
>> > f_out = gzip.open('/Users/Paul/Desktop/scripts/pic.jpg.gz', 'wb')
>> > f_out.writelines(f_in)
>> > f_out.close()
>> > f_in.close()
>>
>> Are you sure you want iteration and writelines() here? I would be
>> inclined to avoid those for any situation that isn't plain text. If
>> the file isn't too big, I'd just read it all in a single blob and then
>> write it all out at once.
>
> Is there a reason not to use shutil.copyfileobj?

If the file is too big (or might be too big) to want to fit into
memory, then sure. But a typical JPEG image isn't going to be
gigabytes of content; chances are it's going to be a meg or so at
most, and that doesn't justify the overhead of chunking. Just read it,
write it, job done.

ChrisA

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web