Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #7722

pycurl and MAX_RECV_SPEED_LARGE

Show key headers only | View raw

Hello Folks,


I have a problem with pycurl. I need to do a download with a lower rate, 
+ or - 1 Kb / 128bytes.

I use the  MAX_RECV_SPEED_LARGE setting with 128 as value.

My problem is:  The download take a long time to be finished.

File:  test.jpg 92 KB, with 128 rate,  take 2.38 Minutes.

The strange behavior is that:   The connection keep ESTABLISHED only 1/5 
from all time ( 2.38 minutes )


Analyzing with wireshark, I see that: After pass 1/5 of all time, do not 
change packages anymore until reach 2.38 minutes, and client send a RST ACK.

Have a high delay between the transfer and the finished connection.


I need to solution that I can download with a lower rate, include using 
Multi-Threading with many simultaneous downloads.



Follow my test code:


import pycurl
import random
import os



def download1(t_url):
         headers = [
                    'Accept: 
text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
                    'Accept-Language: en-us,pt-br;q=0.8,pt;q=0.5,en;q=0.3',
                    'Accept-Encoding: gzip,deflate',
                    'Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7',
                    'Cache-control: no-cache',
                    'Pragma: no-cache',
                    'Connection: Keep-Alive',
                    'Keep-Alive: 300']

         c = pycurl.Curl()
         f = "/tmp/teste.tmp"
         c.setopt(c.URL, t_url)
        c.setopt(c.FAILONERROR, 1)
#       c.setopt(c.VERBOSE, 1)
         c.setopt(c.MAX_RECV_SPEED_LARGE, 128)
#        c.setopt(c.TIMEOUT, timeout)
         c.setopt(c.NOSIGNAL, 1)
         c.setopt(pycurl.HTTPHEADER, headers)
         c.setopt(c.CONNECTTIMEOUT,10)
         c.setopt(c.WRITEDATA, file(f,"w"))
         c.setopt(c.USERAGENT,"XADASSDASDASD")
         try:
                 c.perform()
         except pycurl.error:
                 print "Connection Problem"
                 c.close()
                 return 2
         else:
                 print "Download Complete"
                 print f
                 c.close()
                 return 0

download1("http://192.168.111.128/teste.jpg")


Thank You.

Emanuel

Back to comp.lang.python | Previous | Next | Find similar | Unroll thread

Thread

pycurl and MAX_RECV_SPEED_LARGE Emanuel dos Reis Rodrigues <emanueldosreis@gmail.com> - 2011-06-15 22:28 -0400

csiph-web