Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #71014 > unrolled thread

data: protocol

Back to article view | Back to comp.lang.python

Contents

  data: protocol Robin Becker <robin@reportlab.com> - 2014-05-07 11:42 +0100
    Re: data: protocol Steven D'Aprano <steve@pearwood.info> - 2014-05-08 03:46 +0000
      Re: data: protocol Robin Becker <robin@reportlab.com> - 2014-05-08 11:34 +0100
      Re: data: protocol Ian Kelly <ian.g.kelly@gmail.com> - 2014-05-08 09:07 -0600
      Re: data: protocol Robin Becker <robin@reportlab.com> - 2014-05-08 17:52 +0100

#71014 — data: protocol

I have an outstanding request for ReportLab to allow images to be opened using 
the data: scheme. That used to be supported in python 2.7 using urllib, but in 
python 3.3 urllib2 --> urllib and at least the default urlopener doesn't support 
data:

Is there a way to use the residual legacy of the old urllib code that's now in 
urllib.URLopener to open unusual schemes? I know it can be used directly eg

urllib.request.URLopener().open('data:.........')

but that seems to leave the splitting & testing logic up to me when it logically 
belongs in some central place ie urllib.request.urlopen.
-- 
Robin Becker

[toc] | [next] | [standalone]

#71075

On Wed, 07 May 2014 11:42:24 +0100, Robin Becker wrote:

> I have an outstanding request for ReportLab to allow images to be opened
> using the data: scheme. That used to be supported in python 2.7 using
> urllib, but in python 3.3 urllib2 --> urllib and at least the default
> urlopener doesn't support data:


It looks like you intended to show an example, but left it out.

> Is there a way to use the residual legacy of the old urllib code that's
> now in urllib.URLopener to open unusual schemes? I know it can be used
> directly eg
> 
> urllib.request.URLopener().open('data:.........')
> 
> but that seems to leave the splitting & testing logic up to me when it
> logically belongs in some central place ie urllib.request.urlopen.

You may need to explain in a little more detail. When you say "splitting 
and testing", what are you splitting and testing? It may also help if you 
show some Python 2.7 code that works, and what happens in 3.3.


-- 
Steven

[toc] | [prev] | [next] | [standalone]

#71090

On 08/05/2014 04:46, Steven D'Aprano wrote:
> On Wed, 07 May 2014 11:42:24 +0100, Robin Becker wrote:
>
>> I have an outstanding request for ReportLab to allow images to be opened
>> using the data: scheme. That used to be supported in python 2.7 using
>> urllib, but in python 3.3 urllib2 --> urllib and at least the default
>> urlopener doesn't support data:
>
>
> It looks like you intended to show an example, but left it out.
>
>> Is there a way to use the residual legacy of the old urllib code that's
>> now in urllib.URLopener to open unusual schemes? I know it can be used
>> directly eg
>>
>> urllib.request.URLopener().open('data:.........')
>>
>> but that seems to leave the splitting & testing logic up to me when it
>> logically belongs in some central place ie urllib.request.urlopen.
>
> You may need to explain in a little more detail. When you say "splitting
> and testing", what are you splitting and testing? It may also help if you
> show some Python 2.7 code that works, and what happens in 3.3.
>
>
OK not sure about 3.4, but in 3.3 the urllib module cannot open a request like this

C:\code-trunk\hg-repos\reportlab\tests>\python33\python.exe
Python 3.3.3 (v3.3.3:c3896275c0f6, Nov 18 2013, 21:18:40) [MSC v.1600 32 bit 
(Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
 >>> import urllib.request
 >>> 
urllib.request.urlopen('data:image/gif;base64,R0lGODdhAQABAIAAAP///////ywAAAAAAQABAAACAkQBADs=').read()
Traceback (most recent call last):
   File "<stdin>", line 1, in <module>
   File "C:\python33\lib\urllib\request.py", line 156, in urlopen
     return opener.open(url, data, timeout)
   File "C:\python33\lib\urllib\request.py", line 469, in open
     response = self._open(req, data)
   File "C:\python33\lib\urllib\request.py", line 492, in _open
     'unknown_open', req)
   File "C:\python33\lib\urllib\request.py", line 447, in _call_chain
     result = func(*args)
   File "C:\python33\lib\urllib\request.py", line 1310, in unknown_open
     raise URLError('unknown url type: %s' % type)
urllib.error.URLError: <urlopen error unknown url type: data>
 >>>

in python27 one can do

C:\tmp>python
Python 2.7.6 (default, Nov 10 2013, 19:24:18) [MSC v.1500 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
 >>> import urllib
 >>> 
data=urllib.urlopen('data:image/gif;base64,R0lGODdhAQABAIAAAP///////ywAAAAAAQABAAACAkQBADs=').read()
 >>> len(data)
35
 >>>

and as indicated  by Ian Kelly in 3.4
C:\tmp>\python34\python.exe
Python 3.4.0 (v3.4.0:04f714765c13, Mar 16 2014, 19:24:06) [MSC v.1600 32 bit 
(Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
 >>> import urllib.request
 >>> 
data=urllib.request.urlopen('data:image/gif;base64,R0lGODdhAQABAIAAAP///////ywAAAAAAQABAAACAkQBADs=').read()
 >>> len(data)
35



in 3.3 we have the old code URLopener class. However, when I use that I see this

C:\code-trunk\hg-repos\reportlab\tests>\python33\python.exe
Python 3.3.3 (v3.3.3:c3896275c0f6, Nov 18 2013, 21:18:40) [MSC v.1600 32 bit 
(Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
 >>> from urllib.request import URLopener
 >>> data = 
URLopener().open('data:image/gif;base64,R0lGODdhAQABAIAAAP///////ywAAAAAAQABAAACAkQBADs=').read()
 >>> len(data)
115
 >>> data
'Date: Thu, 08 May 2014 10:21:45 GMT\nContent-type: image/gif\nContent-Length: 
35\n\nGIF87a\x01\x00\x01\x00\x80\x00\x00├
┐├┐├┐├┐├┐├┐,\x00\x00\x00\x00\x01\x00\x01\x00\x00\x02\x02D\x01\x00;'
 >>>

so I seem to be getting the real data and some headers now. I think this is 
different from what is expected, but that code is labelled as old/deprecated and 
possibly going away.

Since urllib doesn't always work as expected in 3.3 I've had to write a small 
stub for the special data: case. Doing all the splitting off of the headers 
seems harder than just doing the special case.

However, there are a lot of these 'schemes' so should I be doing this sort of 
thing? Apparently it's taken 4 versions of python to get urllib in 3.4 to do 
this so it's not clear to me whether all schemes are supposed to hang off 
urllib.request.urlopen or if instead of special casing the 3.3 data: I should 
have special cased a handler for it and injected that into my opener (or 
possibly the default opener). Doing the handler means I do have to handle the 
headers stuff whereas my stub is just returning the data bits.
-- 
Robin Becker

[toc] | [prev] | [next] | [standalone]

#71110

On Thu, May 8, 2014 at 4:34 AM, Robin Becker <robin@reportlab.com> wrote:
> Since urllib doesn't always work as expected in 3.3 I've had to write a
> small stub for the special data: case. Doing all the splitting off of the
> headers seems harder than just doing the special case.
>
> However, there are a lot of these 'schemes' so should I be doing this sort
> of thing? Apparently it's taken 4 versions of python to get urllib in 3.4 to
> do this so it's not clear to me whether all schemes are supposed to hang off
> urllib.request.urlopen or if instead of special casing the 3.3 data: I
> should have special cased a handler for it and injected that into my opener
> (or possibly the default opener). Doing the handler means I do have to
> handle the headers stuff whereas my stub is just returning the data bits.

The DataHandler class in 3.4 is all of 14 lines of code. My first
instinct would be to backport that and add it to some OpenerDirector
instance.

[toc] | [prev] | [next] | [standalone]

#71114

On 08/05/2014 16:07, Ian Kelly wrote:
........
> The DataHandler class in 3.4 is all of 14 lines of code. My first

which makes me wonder why the old URLopener is still there and why all this 
wasn't done before 3.0 appeared.

> instinct would be to backport that and add it to some OpenerDirector
> instance.
>
Using a director would be possible, but it's not much cleaner than special 
casing the function and I don't have to worry about which opener to use; I use 
the one provided by the user
-- 
Robin Becker

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web