Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #85394 > unrolled thread

__next__ and StopIteration

Back to article view | Back to comp.lang.python

Contents

  __next__ and StopIteration Charles Hixson <charleshixsn@earthlink.net> - 2015-02-09 11:14 -0800
    Re: __next__ and StopIteration Rob Gaddi <rgaddi@technologyhighland.invalid> - 2015-02-09 19:27 +0000
    Re: __next__ and StopIteration Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-02-10 10:30 +1100
      Re: __next__ and StopIteration Ian Kelly <ian.g.kelly@gmail.com> - 2015-02-09 16:56 -0700
        Re: __next__ and StopIteration Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-02-10 11:42 +1100
          Re: __next__ and StopIteration Chris Angelico <rosuav@gmail.com> - 2015-02-10 11:54 +1100
            Re: __next__ and StopIteration Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-02-10 12:11 +1100
              Re: __next__ and StopIteration Chris Angelico <rosuav@gmail.com> - 2015-02-10 12:58 +1100
          Re: __next__ and StopIteration Chris Kaynor <ckaynor@zindagigames.com> - 2015-02-09 16:59 -0800
            Re: __next__ and StopIteration Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-02-10 16:54 +1100
          Re: __next__ and StopIteration Ian Kelly <ian.g.kelly@gmail.com> - 2015-02-10 01:30 -0700
          Re: __next__ and StopIteration Chris Kaynor <ckaynor@zindagigames.com> - 2015-02-10 09:27 -0800
      Re: __next__ and StopIteration Charles Hixson <charleshixsn@earthlink.net> - 2015-02-09 20:33 -0800
      Re: __next__ and StopIteration Chris Angelico <rosuav@gmail.com> - 2015-02-10 15:46 +1100
      Re: __next__ and StopIteration Charles Hixson <charleshixsn@earthlink.net> - 2015-02-09 22:16 -0800
      Re: __next__ and StopIteration Chris Angelico <rosuav@gmail.com> - 2015-02-10 17:38 +1100
      Re: __next__ and StopIteration Ethan Furman <ethan@stoneleaf.us> - 2015-02-10 08:44 -0800
      Re: __next__ and StopIteration Ian Kelly <ian.g.kelly@gmail.com> - 2015-02-10 09:53 -0700
      Re: __next__ and StopIteration Ethan Furman <ethan@stoneleaf.us> - 2015-02-10 09:33 -0800

#85394 — __next__ and StopIteration

I'm trying to write a correct iteration over a doubly indexed container, 
and what I've got so far is:    def __next__ (self):
         for row    in    range(self._rows):
             for col in range(self._cols):
                 if self._grid[row][col]:
                     yield    self._grid[row][col]
                 #end    if
             #end    for col
         #end    for row
         raise    StopIteration

What bothers me is that it doesn't look like it would continue to raise 
StopIteration if it were called again, which is what 
https://docs.python.org/3/library/stdtypes.html#iterator.__next__ says 
is correct.  How should this be fixed?

[toc] | [next] | [standalone]

#85397

On Mon, 09 Feb 2015 11:14:55 -0800, Charles Hixson wrote:

> I'm trying to write a correct iteration over a doubly indexed container,
> and what I've got so far is:    def __next__ (self):
>          for row    in    range(self._rows):
>              for col in range(self._cols):
>                  if self._grid[row][col]:
>                      yield    self._grid[row][col]
>                  #end    if
>              #end    for col
>          #end    for row raise    StopIteration
> 
> What bothers me is that it doesn't look like it would continue to raise
> StopIteration if it were called again, which is what
> https://docs.python.org/3/library/stdtypes.html#iterator.__next__ says
> is correct.  How should this be fixed?

You're mixing metaphors.  You don't iterate over containers, you iterate 
over iterators that are returned to you by containers.  So the container 
class itself has a __iter__ method, which returns a custom iterator 
object that has a __next__ method.  Which is a lot of work.

Or, you write your __iter__ like so:

def __iter__ (self):
    for row in range(self._rows):
        for col in range(self._cols):
            if self._grid[row][col]:
                yield self._grid[row][col]

In which case, Python's magic handling of yield handles the creation of 
the iterator object and the raising of StopIteration at the end of the 
__iter__ function all by itself.

Or you write your __iter__ like so:

def __iter__(self):
    return (self._grid[row][col]
        for col in range(self._cols)
        for row in range(self._rows)
        if self._grid[row][col]
    )

In which case you've returned a generator expression, which is a 
shorthand way of making an iterator.  All extremely equivalent and a 
matter of personal taste (I'd probably opt for the yield one myself).

-- 
Rob Gaddi, Highland Technology -- www.highlandtechnology.com
Email address domain is currently out of order.  See above to fix.

[toc] | [prev] | [next] | [standalone]

#85409

Charles Hixson wrote:

> I'm trying to write a correct iteration over a doubly indexed container,
> and what I've got so far is:    def __next__ (self):
>          for row    in    range(self._rows):
>              for col in range(self._cols):
>                  if self._grid[row][col]:
>                      yield    self._grid[row][col]
>                  #end    if
>              #end    for col
>          #end    for row
>          raise    StopIteration

That's wrong, you don't use yield in __next__ as that will turn it into a
generator function. Here is a simplified example demonstrating the problem:


py> class X(object):
...     def __next__(self):
...             yield 1
...             yield 2
...
py> x = X()
py> next(x)
<generator object __next__ at 0xb7b0e504>
py> next(x)
<generator object __next__ at 0xb7b0e52c>
py> next(x)
<generator object __next__ at 0xb7b0e504>
py> next(x)
<generator object __next__ at 0xb7b0e52c>


The way you write iterators is like this:


Method 1 (the hard way):

- Give your class an __iter__ method which simply returns self:

    def __iter__(self):
        return self

- Give your class a __next__ method (`next` in Python 2) which *returns* a
value. You will need to track which value to return yourself. It must raise
StopIteration when there are no more values to return. Don't use yield.

    def __next__(self):
        value = self.value
        if value is None:
            raise StopIteration
        self.value = self.calculate_the_next_value()
        return value

Your class is itself an iterator.

Method 2 (the easy way):

- Don't write a __next__ method at all.

- Give your class an __iter__ method which returns an iterator. E.g.:

    def __iter__(self):
        return iter(self.values)

In this case, your class is not itself an iterator, but it is iterable:
calling iter(myinstance) will return an iterator, which is enough.

If you don't have a convenient collection of values to return, you can
conveniently use a generator, yielding values you want and just falling off
the end (or returning) when you are done. E.g.:

    def __iter__(self):
        while self.value is not None:
            yield self.value
            self.calculate_the_next_value()

In your case, it looks to me that what you need is something like:


    def __iter__(self):
        for row in range(self._rows):
            for col in range(self._cols):
                if self._grid[row][col]:
                    yield self._grid[row][col]

which is probably better written as:

    # untested -- I may have the row/col order backwards
    def __iter__(self):
        for column in self._grid:
            for item in column:
                if value:
                    yield value


As a general rule, Python is not Fortran. If you find yourself wanting to
write code that iterates over an index, then indexes into a list or array,
99.9% of the time you are better off just iterating over the list or array
directly:

# not this!
for i in range(len(mylist)):
    value = mylist[i]
    print(value)

# instead use this
for value in mylist:
    print(value)


If you need both the index and the list item, use enumerate:

for i, value in enumerate(mylist):
    print(i, value)



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#85411

On Mon, Feb 9, 2015 at 4:30 PM, Steven D'Aprano
<steve+comp.lang.python@pearwood.info> wrote:
> The way you write iterators is like this:
>
>
> Method 1 (the hard way):
>
> - Give your class an __iter__ method which simply returns self:
>
>     def __iter__(self):
>         return self
>
> - Give your class a __next__ method (`next` in Python 2) which *returns* a
> value. You will need to track which value to return yourself. It must raise
> StopIteration when there are no more values to return. Don't use yield.
>
>     def __next__(self):
>         value = self.value
>         if value is None:
>             raise StopIteration
>         self.value = self.calculate_the_next_value()
>         return value
>
> Your class is itself an iterator.

This is an anti-pattern, so don't even suggest it. Iterables should
never be their own iterators. Otherwise, your iterable can only be
iterated over once!

The proper version of the "hard way" is:

1) The __iter__ method of the iterable constructs a new iterator
instance and returns it.

2) The __iter__ method of the *iterator* simply returns itself.

3) The __next__ method of the iterator tracks the current value and
returns the next value. Note that the iterator should never store the
iterable's data internally, unless the iterable is immutable and the
calculation is trivial (e.g. a range object). Instead, it should
determine the next value by referring to its source iterable.

[toc] | [prev] | [next] | [standalone]

#85415

Ian Kelly wrote:

> On Mon, Feb 9, 2015 at 4:30 PM, Steven D'Aprano
> <steve+comp.lang.python@pearwood.info> wrote:
[...]
>> Your class is itself an iterator.
> 
> This is an anti-pattern, so don't even suggest it. Iterables should
> never be their own iterators. Otherwise, your iterable can only be
> iterated over once!

Hmmm, good point.

However, I will point out a couple of factors:

Ultimately, that's the correct behaviour for *iterator* classes. With the
rich set of tools available to build iterators from built-in parts, it is
rare that you need to write your own class with a __next__ method, but if
you do, that's the way you want it to behave.

Whether that *iterator* class should be the same class as the *iterable*
class is another story. In the built-ins, they mostly (always?) come in
pairs:

tuple <-> tuple_iterator
list <-> list_iterator
dict <-> dict_keyiterator
set <-> set_iterator
range <-> range_iterator

so that's an excellent sign that doing so is best practice, but it should
not be seen as *required*. After all, perhaps you have good reason for
wanting your iterable class to only be iterated over once.

Also, *technically* iterators may be re-iterable. The docs say that
iterators which fail to raise StopIteration forever once they are exhausted
are "broken", but the docs do not forbid broken iterators. Consenting
adults and all that. You might want an iterator with a reset() method. Even
an outright broken iterator!

    def __next__(self):
        if random.random() < 0.1: raise StopIteration
        return random.random()

Why you would want one, I don't know, but if you have a hankering for such a
beast, Python lets you do it.


> The proper version of the "hard way" is:
> 
> 1) The __iter__ method of the iterable constructs a new iterator
> instance and returns it.
> 
> 2) The __iter__ method of the *iterator* simply returns itself.
> 
> 3) The __next__ method of the iterator tracks the current value and
> returns the next value. Note that the iterator should never store the
> iterable's data internally, unless the iterable is immutable and the
> calculation is trivial (e.g. a range object). Instead, it should
> determine the next value by referring to its source iterable.


-- 
Steven

[toc] | [prev] | [next] | [standalone]

#85416

On Tue, Feb 10, 2015 at 11:42 AM, Steven D'Aprano
<steve+comp.lang.python@pearwood.info> wrote:
> Also, *technically* iterators may be re-iterable. The docs say that
> iterators which fail to raise StopIteration forever once they are exhausted
> are "broken", but the docs do not forbid broken iterators. Consenting
> adults and all that. You might want an iterator with a reset() method. Even
> an outright broken iterator!
>
>     def __next__(self):
>         if random.random() < 0.1: raise StopIteration
>         return random.random()
>
> Why you would want one, I don't know, but if you have a hankering for such a
> beast, Python lets you do it.

Yes, it is allowed. But when you write code that's documented as being
"broken", you should expect annoying, subtle errors, maybe a long way
down the track. You know all those niggling problems you get when you
find some NaNs in a series of numbers, and suddenly things don't sort
stably and such? You'll hanker for those well-defined days. :)

ChrisA

[toc] | [prev] | [next] | [standalone]

#85418

Chris Angelico wrote:

> On Tue, Feb 10, 2015 at 11:42 AM, Steven D'Aprano
> <steve+comp.lang.python@pearwood.info> wrote:
>> Also, *technically* iterators may be re-iterable. The docs say that
>> iterators which fail to raise StopIteration forever once they are
>> exhausted are "broken", but the docs do not forbid broken iterators.
>> Consenting adults and all that. You might want an iterator with a reset()
>> method. Even an outright broken iterator!
>>
>>     def __next__(self):
>>         if random.random() < 0.1: raise StopIteration
>>         return random.random()
>>
>> Why you would want one, I don't know, but if you have a hankering for
>> such a beast, Python lets you do it.
> 
> Yes, it is allowed. But when you write code that's documented as being
> "broken", you should expect annoying, subtle errors, maybe a long way
> down the track.

Well, that depends, don't it?

If you're writing library code, you need to be much more careful and
thoughtful about both your API and implementation.

But if you're writing something quick and dirty for a one-off script, even a
script that is going to be used in perpetuity, you can afford a lot more
sloppiness. Better something sloppy that works today than something perfect
next year. Say my script just outputs a bunch of random numbers, one per
line, and I pipe the output to a file in the shell:

mkrnd.py > the_numbers.txt

Am I really going to care if the iterator used to generate those random
numbers is technically "broken"? Probably not. And if I do, some time in
the distant future, oh well, I'll "fix" it then.



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#85420

On Tue, Feb 10, 2015 at 12:11 PM, Steven D'Aprano
<steve+comp.lang.python@pearwood.info> wrote:
>> Yes, it is allowed. But when you write code that's documented as being
>> "broken", you should expect annoying, subtle errors, maybe a long way
>> down the track.
>
> Well, that depends, don't it?
>
> Am I really going to care if the iterator used to generate those random
> numbers is technically "broken"? Probably not. And if I do, some time in
> the distant future, oh well, I'll "fix" it then.

If you inhale asbestos, you should expect annoying, maybe subtle,
errors, maybe a long way down the track. Good news is, the lab boys
say the symptoms of asbestos poisoning show a median latency of 44.6
years, so if you're thirty or older, you're laughing. Worst case
scenario, you miss out on a few rounds of canasta. Do you care that
your lungs are technically "broken"? Probably not. It's something to
be aware of when you consider working in certain environments, but "be
aware of" doesn't mean "absolutely always avoid".

Okay, maybe it does with asbestos... but not with broken iterators. :)

ChrisA

[toc] | [prev] | [next] | [standalone]

#85417

On Mon, Feb 9, 2015 at 4:42 PM, Steven D'Aprano
<steve+comp.lang.python@pearwood.info> wrote:
> so that's an excellent sign that doing so is best practice, but it should
> not be seen as *required*. After all, perhaps you have good reason for
> wanting your iterable class to only be iterated over once.

In fact, there is one in the stdlib, the "file" object, which has a
__iter__ which returns self. The code below shows this,

There are a number of good reasons: perhaps there is no good way to
reset the iterator or there is outside state that has to be managed.
I'm also sure there are other reasons I cannot think of right now.

>
> Also, *technically* iterators may be re-iterable. The docs say that
> iterators which fail to raise StopIteration forever once they are exhausted
> are "broken", but the docs do not forbid broken iterators. Consenting
> adults and all that. You might want an iterator with a reset() method. Even
> an outright broken iterator!
>
>     def __next__(self):
>         if random.random() < 0.1: raise StopIteration
>         return random.random()
>
> Why you would want one, I don't know, but if you have a hankering for such a
> beast, Python lets you do it.

The "file" object is also an example of this. It is technically a
broken iterator according to the docs:

Python 3.4.2 (v3.4.2:ab2c023a9432, Oct  6 2014, 22:15:05) [MSC v.1600
32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> f = open("d:/test.txt")
>>> iter(f) is f
True
>>> for l in f:
...     print(l)
...
line 1

line 2

line 3

>>> for l in f:
...     print(l)
...
>>> f.seek(0)
0
>>> for l in f:
...     print(l)
...
line 1

line 2

line 3

>>> next(f)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
StopIteration
>>> f.seek(0)
0
>>> next(f) # This should throw StopIteration as it has previously thrown StopIteration.
'line 1\n'
>>>



Chris

[toc] | [prev] | [next] | [standalone]

#85428

Chris Kaynor wrote:

> On Mon, Feb 9, 2015 at 4:42 PM, Steven D'Aprano
> <steve+comp.lang.python@pearwood.info> wrote:
>> so that's an excellent sign that doing so is best practice, but it should
>> not be seen as *required*. After all, perhaps you have good reason for
>> wanting your iterable class to only be iterated over once.
> 
> In fact, there is one in the stdlib, the "file" object, which has a
> __iter__ which returns self. The code below shows this,
[...]
> The "file" object is also an example of this. It is technically a
> broken iterator according to the docs:


Awesome example!

-- 
Steve

[toc] | [prev] | [next] | [standalone]

#85439

On Mon, Feb 9, 2015 at 5:59 PM, Chris Kaynor <ckaynor@zindagigames.com> wrote:
> On Mon, Feb 9, 2015 at 4:42 PM, Steven D'Aprano
> <steve+comp.lang.python@pearwood.info> wrote:
>> so that's an excellent sign that doing so is best practice, but it should
>> not be seen as *required*. After all, perhaps you have good reason for
>> wanting your iterable class to only be iterated over once.
>
> In fact, there is one in the stdlib, the "file" object, which has a
> __iter__ which returns self. The code below shows this,

Fair point. I suppose that's because the file paradigm itself has a
concept of current position that can't easily be abstracted away.

> The "file" object is also an example of this. It is technically a
> broken iterator according to the docs:
>
> Python 3.4.2 (v3.4.2:ab2c023a9432, Oct  6 2014, 22:15:05) [MSC v.1600
> 32 bit (Intel)] on win32
> Type "help", "copyright", "credits" or "license" for more information.
>>>> f = open("d:/test.txt")
>>>> iter(f) is f
> True
>>>> for l in f:
> ...     print(l)
> ...
> line 1
>
> line 2
>
> line 3
>
>>>> for l in f:
> ...     print(l)
> ...
>>>> f.seek(0)
> 0
>>>> for l in f:
> ...     print(l)
> ...
> line 1
>
> line 2
>
> line 3
>
>>>> next(f)
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
> StopIteration
>>>> f.seek(0)
> 0
>>>> next(f) # This should throw StopIteration as it has previously thrown StopIteration.
> 'line 1\n'

IIRC the docs also warn that mixing file iteration with other file
methods results in undefined or broken behavior. Maybe that's only for
Python 2 files, however.

[toc] | [prev] | [next] | [standalone]

#85458

On Tue, Feb 10, 2015 at 12:30 AM, Ian Kelly <ian.g.kelly@gmail.com> wrote:
>
> On Mon, Feb 9, 2015 at 5:59 PM, Chris Kaynor <ckaynor@zindagigames.com> wrote:
> > On Mon, Feb 9, 2015 at 4:42 PM, Steven D'Aprano
> > <steve+comp.lang.python@pearwood.info> wrote:
> >> so that's an excellent sign that doing so is best practice, but it should
> >> not be seen as *required*. After all, perhaps you have good reason for
> >> wanting your iterable class to only be iterated over once.
> >
> > In fact, there is one in the stdlib, the "file" object, which has a
> > __iter__ which returns self. The code below shows this,
>
> Fair point. I suppose that's because the file paradigm itself has a
> concept of current position that can't easily be abstracted away.

That is basically my thought as well. It would be possible to seek
around the file while iterating (seek to the current iteration
position, read, then seek back to the previous location), however that
would slow down iteration for little practical gain. It would also
require locking and possibly introduce odd corner cases.

>
> > The "file" object is also an example of this. It is technically a
> > broken iterator according to the docs:
> >
> > Python 3.4.2 (v3.4.2:ab2c023a9432, Oct  6 2014, 22:15:05) [MSC v.1600
> > 32 bit (Intel)] on win32
> > Type "help", "copyright", "credits" or "license" for more information.
> >>>> f = open("d:/test.txt")
> >>>> iter(f) is f
> > True
> >>>> for l in f:
> > ...     print(l)
> > ...
> > line 1
> >
> > line 2
> >
> > line 3
> >
> >>>> for l in f:
> > ...     print(l)
> > ...
> >>>> f.seek(0)
> > 0
> >>>> for l in f:
> > ...     print(l)
> > ...
> > line 1
> >
> > line 2
> >
> > line 3
> >
> >>>> next(f)
> > Traceback (most recent call last):
> >   File "<stdin>", line 1, in <module>
> > StopIteration
> >>>> f.seek(0)
> > 0
> >>>> next(f) # This should throw StopIteration as it has previously thrown StopIteration.
> > 'line 1\n'
>
> IIRC the docs also warn that mixing file iteration with other file
> methods results in undefined or broken behavior. Maybe that's only for
> Python 2 files, however.


>From the 2.7.9 docs (https://docs.python.org/2/library/stdtypes.html#file.next):
> As a consequence of using a read-ahead buffer, combining next() with other file methods (like readline()) does not work right. However, using seek() to reposition the file to an absolute position will flush the read-ahead buffer.

So it explicitly states that seek, the method I used, should work
fine. I could not find any similar text in the 3.4 docs.

Chris

[toc] | [prev] | [next] | [standalone]

#85424

On 02/09/2015 03:56 PM, Ian Kelly wrote:
> On Mon, Feb 9, 2015 at 4:30 PM, Steven D'Aprano
> <steve+comp.lang.python@pearwood.info> wrote:
>> The way you write iterators is like this:
>>
>>
>> Method 1 (the hard way):
>>
>> - Give your class an __iter__ method which simply returns self:
>>
>>      def __iter__(self):
>>          return self
>>
>> - Give your class a __next__ method (`next` in Python 2) which *returns* a
>> value. You will need to track which value to return yourself. It must raise
>> StopIteration when there are no more values to return. Don't use yield.
>>
>>      def __next__(self):
>>          value = self.value
>>          if value is None:
>>              raise StopIteration
>>          self.value = self.calculate_the_next_value()
>>          return value
>>
>> Your class is itself an iterator.
> This is an anti-pattern, so don't even suggest it. Iterables should
> never be their own iterators. Otherwise, your iterable can only be
> iterated over once!
>
> The proper version of the "hard way" is:
>
> 1) The __iter__ method of the iterable constructs a new iterator
> instance and returns it.
>
> 2) The __iter__ method of the *iterator* simply returns itself.
>
> 3) The __next__ method of the iterator tracks the current value and
> returns the next value. Note that the iterator should never store the
> iterable's data internally, unless the iterable is immutable and the
> calculation is trivial (e.g. a range object). Instead, it should
> determine the next value by referring to its source iterable.
So if I'm understanding this correctly, I should implement as an 
internal class within Grid something like:
     class GridIter(Iterator):
         """ A class to iterate over the cells of a Grid instance
         Vars:
           row  ::  The row currently being iterated over.
           col  ::  Yhe column currently being iterated over.
           grid ::  The grid instance being iterated over.
         """

         def __init__ (self, grid):
             """
             Params:
               grid  ::  The instance of the grid to iterate over.
             """
             self.row    =    -1
             self.col    =    -1
             self.grid    =    grid
         #end    __init__

         def __iter__ (self):
             return    self

         def __next__ (self):
             if self.row == -1 and self.col == -1:
                 self.row    =    0
                 self.col    =    0
             elif self.col >= grid.cols():
                 self.row    =    self.row + 1
                 self.col    =    0
             if self.row > grid.rows():
                 raise    StopIteration
             if not    [row, col] in self.grid:
                 return    self.__next__()
             return    grid(row, col)

[toc] | [prev] | [next] | [standalone]

#85425

On Tue, Feb 10, 2015 at 3:33 PM, Charles Hixson
<charleshixsn@earthlink.net> wrote:
>> The proper version of the "hard way" is:
>>
>> 1) The __iter__ method of the iterable constructs a new iterator
>> instance and returns it.
>>
>> 2) The __iter__ method of the *iterator* simply returns itself.
>>
>> 3) The __next__ method of the iterator tracks the current value and
>> returns the next value. Note that the iterator should never store the
>> iterable's data internally, unless the iterable is immutable and the
>> calculation is trivial (e.g. a range object). Instead, it should
>> determine the next value by referring to its source iterable.
>
> So if I'm understanding this correctly, I should implement as an internal
> class within Grid something like:
>     class GridIter(Iterator):

Apart from the fact that you shouldn't have to explicitly subclass
Iterator, yes. But this is the hard way to do things. The easy way is
to simply define an __iter__ method on your Grid which returns an
iterator - and one excellent form of iterator, for custom classes like
this, is a generator object. Your original code can slot happily in,
with one tiny change:

class Grid:
    blah blah

    def __iter__(self):
        for row in range(self._rows):
            for col in range(self._cols):
                if self._grid[row][col]:
                    yield self._grid[row][col]

The only change is to remove the explicit StopIteration at the end;
once your generator function terminates, the generator object will
raise StopIteration forever afterward, without any help from you.
(Also, post-PEP479, the explicit raise will actually cause
RuntimeError, so it's not just superfluous, but actually a problem.)

But I'm guessing that your grid and rows are actually iterable
themselves. If they are, you can cut the code down to this:

    def __iter__(self):
        for row in self._grid:
            for cell in row:
                if cell: yield cell

or a generator expression:

    def __iter__(self):
        return (cell for row in self._grid for cell in row if cell)

or itertools.chain and filter, if you so desired. As long as you
return an iterator, you're fine. This is far and away the easiest way
to make a class iterable.

ChrisA

[toc] | [prev] | [next] | [standalone]

#85429

On 02/09/2015 08:46 PM, Chris Angelico wrote:
> On Tue, Feb 10, 2015 at 3:33 PM, Charles Hixson
> <charleshixsn@earthlink.net> wrote:
>>> The proper version of the "hard way" is:
>>>
>>> 1) The __iter__ method of the iterable constructs a new iterator
>>> instance and returns it.
>>>
>>> 2) The __iter__ method of the *iterator* simply returns itself.
>>>
>>> 3) The __next__ method of the iterator tracks the current value and
>>> returns the next value. Note that the iterator should never store the
>>> iterable's data internally, unless the iterable is immutable and the
>>> calculation is trivial (e.g. a range object). Instead, it should
>>> determine the next value by referring to its source iterable.
>> So if I'm understanding this correctly, I should implement as an internal
>> class within Grid something like:
>>      class GridIter(Iterator):
> Apart from the fact that you shouldn't have to explicitly subclass
> Iterator, yes. But this is the hard way to do things. The easy way is
> to simply define an __iter__ method on your Grid which returns an
> iterator - and one excellent form of iterator, for custom classes like
> this, is a generator object. Your original code can slot happily in,
> with one tiny change:
>
> class Grid:
>      blah blah
>
>      def __iter__(self):
>          for row in range(self._rows):
>              for col in range(self._cols):
>                  if self._grid[row][col]:
>                      yield self._grid[row][col]
>
> The only change is to remove the explicit StopIteration at the end;
> once your generator function terminates, the generator object will
> raise StopIteration forever afterward, without any help from you.
> (Also, post-PEP479, the explicit raise will actually cause
> RuntimeError, so it's not just superfluous, but actually a problem.)
>
> But I'm guessing that your grid and rows are actually iterable
> themselves. If they are, you can cut the code down to this:
>
>      def __iter__(self):
>          for row in self._grid:
>              for cell in row:
>                  if cell: yield cell
>
> or a generator expression:
>
>      def __iter__(self):
>          return (cell for row in self._grid for cell in row if cell)
>
> or itertools.chain and filter, if you so desired. As long as you
> return an iterator, you're fine. This is far and away the easiest way
> to make a class iterable.
>
> ChrisA
Yes, rows and cols are lists, but I'm going to need to iterate through 
them more than once.  I'd rather do without a included class, but if a 
properly formed iterator can only be cycled through once, and if I 
understand properly that means I can't use the "class instance is it's 
own iterator" form.

[toc] | [prev] | [next] | [standalone]

#85430

On Tue, Feb 10, 2015 at 5:16 PM, Charles Hixson
<charleshixsn@earthlink.net> wrote:
> Yes, rows and cols are lists, but I'm going to need to iterate through them
> more than once.  I'd rather do without a included class, but if a properly
> formed iterator can only be cycled through once, and if I understand
> properly that means I can't use the "class instance is it's own iterator"
> form.

When you write __iter__ as a generator function, what you actually
have is a factory for iterator objects. (Generators are iterators.)
When something calls __iter__ on your Grid, it gets back a brand new
iterator which is independent of every other iterator from that or any
other Grid, so you can iterate over the same Grid more than once.
You're not writing __next__ here, so you're not using the class
instance as its own iterator, which means that that consideration
doesn't apply.

ChrisA

[toc] | [prev] | [next] | [standalone]

#85455

[Multipart message — attachments visible in raw view] — view raw

On 02/09/2015 08:46 PM, Chris Angelico wrote:
> 
> class Grid:
>     blah blah
> 
>     def __iter__(self):
>         for row in range(self._rows):
>             for col in range(self._cols):
>                 if self._grid[row][col]:
>                     yield self._grid[row][col]

I strongly suggest you remove the

  if self._grid[row][col]:

line.

Best case scenario: the entire grid is blank, and iterating through it does nothing.

Worst case scenario:  only some elements evaluate as False, so your loop doesn't execute the full number of times; i.e.
with a grid of 4x5 with 7 blank cells you get 13 iterations -- probably not what was expected.

--
~Ethan~

[toc] | [prev] | [next] | [standalone]

#85456

On Tue, Feb 10, 2015 at 9:44 AM, Ethan Furman <ethan@stoneleaf.us> wrote:
> On 02/09/2015 08:46 PM, Chris Angelico wrote:
>>
>> class Grid:
>>     blah blah
>>
>>     def __iter__(self):
>>         for row in range(self._rows):
>>             for col in range(self._cols):
>>                 if self._grid[row][col]:
>>                     yield self._grid[row][col]
>
> I strongly suggest you remove the
>
>   if self._grid[row][col]:
>
> line.
>
> Best case scenario: the entire grid is blank, and iterating through it does nothing.
>
> Worst case scenario:  only some elements evaluate as False, so your loop doesn't execute the full number of times; i.e.
> with a grid of 4x5 with 7 blank cells you get 13 iterations -- probably not what was expected.

Depends on what the expected behavior is -- is every grid position
something that should be included in the iteration, or are we looking
at elements of a container where some possible locations may be empty?
You don't expect a dict iteration to yield empty buckets, for example.

I have some code that looks similar to this, which is an iterator for
a chess board that yields the contained pieces. It doesn't really make
sense in that case to yield empty squares.

[toc] | [prev] | [next] | [standalone]

#85459

[Multipart message — attachments visible in raw view] — view raw

On 02/10/2015 08:53 AM, Ian Kelly wrote:
> On Tue, Feb 10, 2015 at 9:44 AM, Ethan Furman <ethan@stoneleaf.us> wrote:
>> On 02/09/2015 08:46 PM, Chris Angelico wrote:
>>>
>>> class Grid:
>>>     blah blah
>>>
>>>     def __iter__(self):
>>>         for row in range(self._rows):
>>>             for col in range(self._cols):
>>>                 if self._grid[row][col]:
>>>                     yield self._grid[row][col]
>>
>> I strongly suggest you remove the
>>
>>   if self._grid[row][col]:
>>
>> line.
>>
>> Best case scenario: the entire grid is blank, and iterating through it does nothing.
>>
>> Worst case scenario:  only some elements evaluate as False, so your loop doesn't execute the full number of times; i.e.
>> with a grid of 4x5 with 7 blank cells you get 13 iterations -- probably not what was expected.
> 
> Depends on what the expected behavior is -- is every grid position
> something that should be included in the iteration, or are we looking
> at elements of a container where some possible locations may be empty?
> You don't expect a dict iteration to yield empty buckets, for example.
> 
> I have some code that looks similar to this, which is an iterator for
> a chess board that yields the contained pieces. It doesn't really make
> sense in that case to yield empty squares.

Cool, thanks for the correction.

--
~Ethan~

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web