Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #10535 > unrolled thread

What is xrange?

Back to article view | Back to comp.lang.python

Contents

  What is xrange? Billy Mays <noway@nohow.com> - 2011-07-29 15:36 -0400
    Re: What is xrange? harrismh777 <harmar@member.fsf.org> - 2011-07-29 14:47 -0500
    Re: What is xrange? Thomas Jollans <t@jollybox.de> - 2011-07-29 22:31 +0200
    Re: What is xrange? Jerry Hill <malaclypse2@gmail.com> - 2011-07-29 16:36 -0400
    Re: What is xrange? Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2011-07-30 11:22 +1000
      Re: What is xrange? Brian Blais <bblais@bryant.edu> - 2011-07-30 06:23 -0400
        Re: What is xrange? Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2011-07-30 21:36 +1000
          Re: What is xrange? Brian Blais <bblais@bryant.edu> - 2011-07-30 09:46 -0400
          Re: What is xrange? Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2011-07-30 17:06 -0700
          Re: What is xrange? Chris Angelico <rosuav@gmail.com> - 2011-07-31 01:10 +0100
          Re: What is xrange? Ethan Furman <ethan@stoneleaf.us> - 2011-07-30 23:42 -0700
        Re: What is xrange? Gregory Ewing <greg.ewing@canterbury.ac.nz> - 2011-07-31 12:18 +1200

#10535 — What is xrange?

Is xrange not a generator?  I know it doesn't return a tuple or list, so 
what exactly is it?  Y doesn't ever complete, but x does.

x = (i for i in range(10))
y = xrange(10)

print "===X==="
while True:
     for i in x:
         print i
         break
     else:
         break

print "===Y==="
while True:
     for i in y:
         print i
         break
     else:
         break

[toc] | [next] | [standalone]

#10537

Billy Mays wrote:
> Is xrange not a generator?  I know it doesn't return a tuple or list, so
> what exactly is it?  Y doesn't ever complete, but x does.

      range(n) creates a list containing all the integers 0..n-1. This 
is a problem if you do range(1000000), because you'll end up with a >4Mb 
list. xrange deals with this by returning an object that pretends to be 
a list, but just works out the number needed from the index asked for, 
and returns that.

      range() can actually be faster in some cases - eg. if iterating 
over the same sequence multiple times. xrange has to reconstruct the 
integer object every time, but range will have real integer objects. (It 
will always perform worse in terms of memory however)

      xrange isn't usable in all cases where a real list is needed. For 
instance, it doesn't support slices, or any list methods.



-- 
m harris

FSF  ...free as in freedom/
http://webpages.charter.net/harrismh777/gnulinux/gnulinux.htm

[toc] | [prev] | [next] | [standalone]

#10543

On 29/07/11 21:36, Billy Mays wrote:
> Is xrange not a generator?  I know it doesn't return a tuple or list,
> so what exactly is it?  Y doesn't ever complete, but x does.
>
> x = (i for i in range(10))
> y = xrange(10)
>
> print "===X==="
> while True:
>     for i in x:
>         print i
>         break
>     else:
>         break
>
> print "===Y==="
> while True:
>     for i in y:
>         print i
>         break
>     else:
>         break

Every for loop calls gets a new iterator from the object you're
iterating over. (__iter__) -- Apparently, xrange is implemented in such
a way (as are lists) that you can iterate over the object many times,
while each generator object (and how could it be otherwise can only be
iterated over once. What is xrange(foo)? It is an object that supports
list-like indices, the iterator protocol, and probably a few other
things that you will find in the stdlib docs. Generators also support
the iterator protocol, but that's about as far as the similarity goes
(in general)

 - Thomas

[toc] | [prev] | [next] | [standalone]

#10544

On Fri, Jul 29, 2011 at 3:36 PM, Billy Mays <noway@nohow.com> wrote:
> Is xrange not a generator?  I know it doesn't return a tuple or list, so
> what exactly is it?  Y doesn't ever complete, but x does.
>
> x = (i for i in range(10))
> y = xrange(10)

xrange() does not return a generator.  It returns an iterable xrange
object.  If you want the iterator derived from the iterable xrange
object, you can get it like this:  iterator = y.__iter__()

See http://docs.python.org/library/functions.html#xrange for the
definition of the xrange object.

http://www.learningpython.com/2009/02/23/iterators-iterables-and-generators-oh-my/
seems to cover the differences between iterables, iterators, and
generators pretty well.

Some more reading:
http://docs.python.org/howto/functional.html
http://www.python.org/dev/peps/pep-0255/
http://www.python.org/dev/peps/pep-0289/

-- 
Jerry

[toc] | [prev] | [next] | [standalone]

#10563

Billy Mays wrote:

> Is xrange not a generator?  I know it doesn't return a tuple or list, so
> what exactly is it?  

xrange pre-dates generators by approximately forever. It returns a
special "xrange object", which generates values suitable for use in
for-loops using the old __getitem__ protocol.

You can consider xrange to be implemented something vaguely like this:

class My_xrange:
    def __init__(self, start, end=None, step=1):
        if end is None:
            end = start
            start = 0
        self.start = start
        self.end = end
        self.step = step
    def __getitem__(self, index):
        if self.step != 1:
            raise NotImplementedError('too lazy to support step values')
        start, end = self.start, self.end
        if start <= index < end:
            return start + index
        raise IndexError('out of range')



> Y doesn't ever complete, but x does. 
> 
> x = (i for i in range(10))

That is better written as iter(range(10)).


> y = xrange(10)

Y doesn't complete because xrange objects are restartable. The for-loop ends
up using the original iteration protocol:

i = y[0]
i = y[1]
i = y[2]
...

until IndexError is raised. You break after calling y[0], but the next loop
starts at y[0] again, and so you never progress beyond the first item in
the xrange object.


-- 
Steven

[toc] | [prev] | [next] | [standalone]

#10576

On Jul 29, 2011, at 9:22 PM, Steven D'Aprano wrote:

> Billy Mays wrote:
> 
>> Is xrange not a generator?  I know it doesn't return a tuple or list, so
>> what exactly is it?  
> 
> xrange pre-dates generators by approximately forever. It returns a
> special "xrange object", which generates values suitable for use in
> for-loops using the old __getitem__ protocol.
> 

interesting...I never really thought about this.  Is there a reason that it wasn't reimplemented when generators came out?  Is there a use-case for the current implementation that wouldn't work as a generator?


				bb
-- 
Brian Blais
bblais@bryant.edu
http://web.bryant.edu/~bblais
http://bblais.blogspot.com/

[toc] | [prev] | [next] | [standalone]

#10577

Brian Blais wrote:

> On Jul 29, 2011, at 9:22 PM, Steven D'Aprano wrote:
> 
>> Billy Mays wrote:
>> 
>>> Is xrange not a generator?  I know it doesn't return a tuple or list, so
>>> what exactly is it?
>> 
>> xrange pre-dates generators by approximately forever. It returns a
>> special "xrange object", which generates values suitable for use in
>> for-loops using the old __getitem__ protocol.
>> 
> 
> interesting...I never really thought about this.  Is there a reason that
> it wasn't reimplemented when generators came out?  Is there a use-case for
> the current implementation that wouldn't work as a generator?

It certainly couldn't be re-implemented before Python 3, because that would
change the behaviour and therefore break people's code that expected to be
able to treat xrange objects as sequences.

And now that Python 3.2 is out, and 3.3 is being worked on, it can't be
changed now either, for the same reason. There was a very narrow window of
opportunity for the functionality to be changed, and it was missed.

Probably because nobody thought about it, but if it had been proposed to
change xrange into an iterator, I'm pretty confident that the suggestion
would have been rejected. xrange objects might be lazily generated, but
they're also sequence types: you can get their length, and you can index
them. (However you can't slice them.) Iterators are not sequence types:
they aren't indexable and you can't get their length.

So why bother *taking away* functionality from xrange just to make it a less
powerful and more restricted iterator? It works fine just the way it is,
there's no need to change it.

It isn't like the Python developers are sitting around bored, looking for
things to do. They are overworked with far too much to do and not enough
manpower or time to do it. There are a huge number of much more important
bug fixes and features that haven't been dealt with for them to bother with
something like this.



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#10579

On Jul 30, 2011, at 7:36 AM, Steven D'Aprano wrote:

> xrange objects might be lazily generated, but
> they're also sequence types: you can get their length, and you can index
> them. (However you can't slice them.) Iterators are not sequence types:
> they aren't indexable and you can't get their length.

ah!  now that makes sense.  I guess I never check their length, or index them, and only use them like generators.  :)

thanks for the clarification!

> It isn't like the Python developers are sitting around bored, looking for
> things to do. They are overworked with far too much to do and not enough
> manpower or time to do it. There are a huge number of much more important
> bug fixes and features that haven't been dealt with for them to bother with
> something like this.

Now, that seems a little harsh, and nothing that I was intending.  I figured (before learning about getting its length and indexing) that if the xrange object was basically a pre-generator hack, that it would make sense from a code-clarity point of view to reimplement them as generators -- it would gave made the developer's life easier.  Now that I realize that xrange has different functionality than a generator (which was the point of my question...just curious *what* different functionality) it clearly doesn't make sense to implement them as generators.  I certainly wasn't trying to imply that the developers are lazy, or bored!


			bb
> 
> 
> 
> -- 
> Steven
> 
> -- 
> http://mail.python.org/mailman/listinfo/python-list

-- 
Brian Blais
bblais@bryant.edu
http://web.bryant.edu/~bblais
http://bblais.blogspot.com/

[toc] | [prev] | [next] | [standalone]

#10605

On Sat, 30 Jul 2011 21:36:41 +1000, Steven D'Aprano
<steve+comp.lang.python@pearwood.info> declaimed the following in
gmane.comp.python.general:


> Probably because nobody thought about it, but if it had been proposed to
> change xrange into an iterator, I'm pretty confident that the suggestion
> would have been rejected. xrange objects might be lazily generated, but
> they're also sequence types: you can get their length, and you can index
> them. (However you can't slice them.) Iterators are not sequence types:
> they aren't indexable and you can't get their length.
>
	Ah, but what did they change range() into -- I seem to recall
reading the Python 3.x turned the regular range() into something else...
(from Python <3.x returning a full list)
-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
        wlfraed@ix.netcom.com    HTTP://wlfraed.home.netcom.com/

[toc] | [prev] | [next] | [standalone]

#10607

On Sun, Jul 31, 2011 at 1:06 AM, Dennis Lee Bieber
<wlfraed@ix.netcom.com> wrote:
>        Ah, but what did they change range() into -- I seem to recall
> reading the Python 3.x turned the regular range() into something else...
> (from Python <3.x returning a full list)
>

xrange got renamed to range, as I understand it.

ChrisA

[toc] | [prev] | [next] | [standalone]

#10617

Dennis Lee Bieber wrote:
> On Sat, 30 Jul 2011 21:36:41 +1000, Steven D'Aprano
> <steve+comp.lang.python@pearwood.info> declaimed the following in
> gmane.comp.python.general:
> 
> 
>> Probably because nobody thought about it, but if it had been proposed to
>> change xrange into an iterator, I'm pretty confident that the suggestion
>> would have been rejected. xrange objects might be lazily generated, but
>> they're also sequence types: you can get their length, and you can index
>> them. (However you can't slice them.) Iterators are not sequence types:
>> they aren't indexable and you can't get their length.
>>
> 	Ah, but what did they change range() into -- I seem to recall
> reading the Python 3.x turned the regular range() into something else...
> (from Python <3.x returning a full list)

 From the What's New docs, in the Views And Iterators Instead Of Lists 
section:

range() now behaves like xrange() used to behave, except it works with 
values of arbitrary size. The latter no longer exists.

~Ethan~

[toc] | [prev] | [next] | [standalone]

#10608

Brian Blais wrote:
> On Jul 29, 2011, at 9:22 PM, Steven D'Aprano wrote:
> 
>>Billy Mays wrote:
>>
>>>Is xrange not a generator?  I know it doesn't return a tuple or list, so
>>>what exactly is it?

It's an iterable object.

There are advantages in having xrange (or range in python3) return
an iterable object, rather than creating an iterator directly.
One of them is that you can iterate over the same object more
than once.

This can be useful if you're iterating over a multi-dimensional
space of some kind. Some efficiency can be gained by pre-creating
an xrange object for each of the inner loops and re-using them.

-- 
Greg

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web