Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #100123 > unrolled thread

Help on for loop understanding

Back to article view | Back to comp.lang.python

Contents

  Help on for loop understanding Robert <rxjwg98@gmail.com> - 2015-12-07 17:05 -0800
    Re: Help on for loop understanding Robin Koch <robin.koch@t-online.de> - 2015-12-08 02:14 +0100
      Re: Help on for loop understanding Robert <rxjwg98@gmail.com> - 2015-12-07 17:31 -0800
        Re: Help on for loop understanding Robert <rxjwg98@gmail.com> - 2015-12-07 17:39 -0800
          Re: Help on for loop understanding Robert <rxjwg98@gmail.com> - 2015-12-07 17:50 -0800
            Re: Help on for loop understanding Erik <python@lucidity.plus.com> - 2015-12-08 02:36 +0000
            Re: Help on for loop understanding Chris Angelico <rosuav@gmail.com> - 2015-12-08 14:23 +1100
              Re: Help on for loop understanding Robert <rxjwg98@gmail.com> - 2015-12-08 06:59 -0800
          Re: Help on for loop understanding Erik <python@lucidity.plus.com> - 2015-12-08 01:53 +0000

#100123 — Help on for loop understanding

Hi,
When I learn for loop with below link:

http://www.shutupandship.com/2012/01/understanding-python-iterables-and.html

it has such explanation:

\\\\\\\\\
for loop under the hood

First let's look at the for loop under the hood. When Python executes the
 for loop, it first invokes the __iter__() method of the container to get the
 iterator of the container. It then repeatedly calls the next() method
 (__next__() method in Python 3.x) of the iterator until the iterator raises a
 StopIteration exception. Once the exception is raised, the for loop ends.
\\\\\\\\\

When I follow a list example from the above link, and one example of myself:

//////////
xx=[1,2,3,4,5]

xx.next
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-76-dd0716c641b1> in <module>()
----> 1 xx.next

AttributeError: 'list' object has no attribute 'next' 

xx.__iter__
Out[77]: <method-wrapper '__iter__' of list object at 0x000000000A1ACE08>

for c in xx: print c
1
2
3
4
5
//////////////

I am puzzled that the list examples have no next method, but it can run as
a for loop. Could you explain it to me the difference?

Thanks,

[toc] | [next] | [standalone]

#100124

Am 08.12.2015 um 02:05 schrieb Robert:
> Hi,
> When I learn for loop with below link:
>
> http://www.shutupandship.com/2012/01/understanding-python-iterables-and.html
>
> it has such explanation:
>
> \\\\\\\\\
> for loop under the hood
>
> First let's look at the for loop under the hood. When Python executes the
>   for loop, it first invokes the __iter__() method of the container to get the
>   iterator of the container. It then repeatedly calls the next() method
>   (__next__() method in Python 3.x) of the iterator until the iterator raises a
>   StopIteration exception. Once the exception is raised, the for loop ends.
> \\\\\\\\\
>
> When I follow a list example from the above link, and one example of myself:
>
> //////////
> xx=[1,2,3,4,5]
>
> xx.next
> ---------------------------------------------------------------------------
> AttributeError                            Traceback (most recent call last)
> <ipython-input-76-dd0716c641b1> in <module>()
> ----> 1 xx.next
>
> AttributeError: 'list' object has no attribute 'next'
>
> xx.__iter__
> Out[77]: <method-wrapper '__iter__' of list object at 0x000000000A1ACE08>
>
> for c in xx: print c
> 1
> 2
> 3
> 4
> 5
> //////////////
>
> I am puzzled that the list examples have no next method, but it can run as
> a for loop. Could you explain it to me the difference?

Lists don't have a next method. Their iterators have:

xx.__iter__().__next__()

or

xxIterator = xx.__iter__()
xxIterator.__next__()
xxIterator.__next__()
xxIterator.__next__()
xxIterator.__next__()
xxIterator.__next__()

That's also what your quoted paragraph states:

| It then repeatedly calls the next() method
| (__next__() method in Python 3.x) of the iterator

-- 
Robin Koch

[toc] | [prev] | [next] | [standalone]

#100125

On Monday, December 7, 2015 at 8:14:46 PM UTC-5, Robin Koch wrote:
> Am 08.12.2015 um 02:05 schrieb Robert:
> > Hi,
> > When I learn for loop with below link:
> >
> > http://www.shutupandship.com/2012/01/understanding-python-iterables-and.html
> >
> > it has such explanation:
> >
> > \\\\\\\\\
> > for loop under the hood
> >
> > First let's look at the for loop under the hood. When Python executes the
> >   for loop, it first invokes the __iter__() method of the container to get the
> >   iterator of the container. It then repeatedly calls the next() method
> >   (__next__() method in Python 3.x) of the iterator until the iterator raises a
> >   StopIteration exception. Once the exception is raised, the for loop ends.
> > \\\\\\\\\
> >
> > When I follow a list example from the above link, and one example of myself:
> >
> > //////////
> > xx=[1,2,3,4,5]
> >
> > xx.next
> > ---------------------------------------------------------------------------
> > AttributeError                            Traceback (most recent call last)
> > <ipython-input-76-dd0716c641b1> in <module>()
> > ----> 1 xx.next
> >
> > AttributeError: 'list' object has no attribute 'next'
> >
> > xx.__iter__
> > Out[77]: <method-wrapper '__iter__' of list object at 0x000000000A1ACE08>
> >
> > for c in xx: print c
> > 1
> > 2
> > 3
> > 4
> > 5
> > //////////////
> >
> > I am puzzled that the list examples have no next method, but it can run as
> > a for loop. Could you explain it to me the difference?
> 
> Lists don't have a next method. Their iterators have:
> 
> xx.__iter__().__next__()
> 
> or
> 
> xxIterator = xx.__iter__()
> xxIterator.__next__()
> xxIterator.__next__()
> xxIterator.__next__()
> xxIterator.__next__()
> xxIterator.__next__()
> 
> That's also what your quoted paragraph states:
> 
> | It then repeatedly calls the next() method
> | (__next__() method in Python 3.x) of the iterator
> 
> -- 
> Robin Koch

I use Python 2.7. I have tried these commands:

xx=[1,2,3,4,5]

xx.__iter__
Out[2]: <method-wrapper '__iter__' of list object at 0x000000000A1B85C8>

xx.__iter__().__next__()
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-3-980bcf1bbf42> in <module>()
----> 1 xx.__iter__().__next__()

AttributeError: 'listiterator' object has no attribute '__next__' 

xx.__iter__.__next__
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-4-df8d8c6955e2> in <module>()
----> 1 xx.__iter__.__next__

AttributeError: 'method-wrapper' object has no attribute '__next__' 

xx.__iter__()
Out[5]: <listiterator at 0xa211780>

xx.__iter__().__next__
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-6-92f21313d62e> in <module>()
----> 1 xx.__iter__().__next__

AttributeError: 'listiterator' object has no attribute '__next__' 

xxIterator = xx.__iter__()

xxIterator
Out[8]: <listiterator at 0xa2115c0>

xxIterator.__next__()
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-9-5b74e35c2c6e> in <module>()
----> 1 xxIterator.__next__()

AttributeError: 'listiterator' object has no attribute '__next__' 

for c in xx: print c
1
2
3
4
5
------------

I don't find a way to show __next__ yet. 
Can we explicitly get the iterator for a list?
Thanks,

[toc] | [prev] | [next] | [standalone]

#100126

On Monday, December 7, 2015 at 8:32:30 PM UTC-5, Robert wrote:
> On Monday, December 7, 2015 at 8:14:46 PM UTC-5, Robin Koch wrote:
> > Am 08.12.2015 um 02:05 schrieb Robert:
> > > Hi,
> > > When I learn for loop with below link:
> > >
> > > http://www.shutupandship.com/2012/01/understanding-python-iterables-and.html
> > >
> > > it has such explanation:
> > >
> > > \\\\\\\\\
> > > for loop under the hood
> > >
> > > First let's look at the for loop under the hood. When Python executes the
> > >   for loop, it first invokes the __iter__() method of the container to get the
> > >   iterator of the container. It then repeatedly calls the next() method
> > >   (__next__() method in Python 3.x) of the iterator until the iterator raises a
> > >   StopIteration exception. Once the exception is raised, the for loop ends.
> > > \\\\\\\\\
> > >
> > > When I follow a list example from the above link, and one example of myself:
> > >
> > > //////////
> > > xx=[1,2,3,4,5]
> > >
> > > xx.next
> > > ---------------------------------------------------------------------------
> > > AttributeError                            Traceback (most recent call last)
> > > <ipython-input-76-dd0716c641b1> in <module>()
> > > ----> 1 xx.next
> > >
> > > AttributeError: 'list' object has no attribute 'next'
> > >
> > > xx.__iter__
> > > Out[77]: <method-wrapper '__iter__' of list object at 0x000000000A1ACE08>
> > >
> > > for c in xx: print c
> > > 1
> > > 2
> > > 3
> > > 4
> > > 5
> > > //////////////
> > >
> > > I am puzzled that the list examples have no next method, but it can run as
> > > a for loop. Could you explain it to me the difference?
> > 
> > Lists don't have a next method. Their iterators have:
> > 
> > xx.__iter__().__next__()
> > 
> > or
> > 
> > xxIterator = xx.__iter__()
> > xxIterator.__next__()
> > xxIterator.__next__()
> > xxIterator.__next__()
> > xxIterator.__next__()
> > xxIterator.__next__()
> > 
> > That's also what your quoted paragraph states:
> > 
> > | It then repeatedly calls the next() method
> > | (__next__() method in Python 3.x) of the iterator
> > 
> > -- 
> > Robin Koch
> 
> I use Python 2.7. I have tried these commands:
> 
> xx=[1,2,3,4,5]
> 
> xx.__iter__
> Out[2]: <method-wrapper '__iter__' of list object at 0x000000000A1B85C8>
> 
> xx.__iter__().__next__()
> ---------------------------------------------------------------------------
> AttributeError                            Traceback (most recent call last)
> <ipython-input-3-980bcf1bbf42> in <module>()
> ----> 1 xx.__iter__().__next__()
> 
> AttributeError: 'listiterator' object has no attribute '__next__' 
> 
> xx.__iter__.__next__
> ---------------------------------------------------------------------------
> AttributeError                            Traceback (most recent call last)
> <ipython-input-4-df8d8c6955e2> in <module>()
> ----> 1 xx.__iter__.__next__
> 
> AttributeError: 'method-wrapper' object has no attribute '__next__' 
> 
> xx.__iter__()
> Out[5]: <listiterator at 0xa211780>
> 
> xx.__iter__().__next__
> ---------------------------------------------------------------------------
> AttributeError                            Traceback (most recent call last)
> <ipython-input-6-92f21313d62e> in <module>()
> ----> 1 xx.__iter__().__next__
> 
> AttributeError: 'listiterator' object has no attribute '__next__' 
> 
> xxIterator = xx.__iter__()
> 
> xxIterator
> Out[8]: <listiterator at 0xa2115c0>
> 
> xxIterator.__next__()
> ---------------------------------------------------------------------------
> AttributeError                            Traceback (most recent call last)
> <ipython-input-9-5b74e35c2c6e> in <module>()
> ----> 1 xxIterator.__next__()
> 
> AttributeError: 'listiterator' object has no attribute '__next__' 
> 
> for c in xx: print c
> 1
> 2
> 3
> 4
> 5
> ------------
> 
> I don't find a way to show __next__ yet. 
> Can we explicitly get the iterator for a list?
> Thanks,

Excuse me. I find it as the following:

xx.__iter__().next
Out[16]: <method-wrapper 'next' of listiterator object at 0x0000000008B38AC8>

xx.__iter__().next()
Out[17]: 1

[toc] | [prev] | [next] | [standalone]

#100127

On Monday, December 7, 2015 at 8:39:48 PM UTC-5, Robert wrote:
> On Monday, December 7, 2015 at 8:32:30 PM UTC-5, Robert wrote:
> > On Monday, December 7, 2015 at 8:14:46 PM UTC-5, Robin Koch wrote:
> > > Am 08.12.2015 um 02:05 schrieb Robert:
> > > > Hi,
> > > > When I learn for loop with below link:
> > > >
> > > > http://www.shutupandship.com/2012/01/understanding-python-iterables-and.html
> > > >
> > > > it has such explanation:
> > > >
> > > > \\\\\\\\\
> > > > for loop under the hood
> > > >
> > > > First let's look at the for loop under the hood. When Python executes the
> > > >   for loop, it first invokes the __iter__() method of the container to get the
> > > >   iterator of the container. It then repeatedly calls the next() method
> > > >   (__next__() method in Python 3.x) of the iterator until the iterator raises a
> > > >   StopIteration exception. Once the exception is raised, the for loop ends.
> > > > \\\\\\\\\
> > > >
> > > > When I follow a list example from the above link, and one example of myself:
> > > >
> > > > //////////
> > > > xx=[1,2,3,4,5]
> > > >
> > > > xx.next
> > > > ---------------------------------------------------------------------------
> > > > AttributeError                            Traceback (most recent call last)
> > > > <ipython-input-76-dd0716c641b1> in <module>()
> > > > ----> 1 xx.next
> > > >
> > > > AttributeError: 'list' object has no attribute 'next'
> > > >
> > > > xx.__iter__
> > > > Out[77]: <method-wrapper '__iter__' of list object at 0x000000000A1ACE08>
> > > >
> > > > for c in xx: print c
> > > > 1
> > > > 2
> > > > 3
> > > > 4
> > > > 5
> > > > //////////////
> > > >
> > > > I am puzzled that the list examples have no next method, but it can run as
> > > > a for loop. Could you explain it to me the difference?
> > > 
> > > Lists don't have a next method. Their iterators have:
> > > 
> > > xx.__iter__().__next__()
> > > 
> > > or
> > > 
> > > xxIterator = xx.__iter__()
> > > xxIterator.__next__()
> > > xxIterator.__next__()
> > > xxIterator.__next__()
> > > xxIterator.__next__()
> > > xxIterator.__next__()
> > > 
> > > That's also what your quoted paragraph states:
> > > 
> > > | It then repeatedly calls the next() method
> > > | (__next__() method in Python 3.x) of the iterator
> > > 
> > > -- 
> > > Robin Koch
> > 
> > I use Python 2.7. I have tried these commands:
> > 
> > xx=[1,2,3,4,5]
> > 
> > xx.__iter__
> > Out[2]: <method-wrapper '__iter__' of list object at 0x000000000A1B85C8>
> > 
> > xx.__iter__().__next__()
> > ---------------------------------------------------------------------------
> > AttributeError                            Traceback (most recent call last)
> > <ipython-input-3-980bcf1bbf42> in <module>()
> > ----> 1 xx.__iter__().__next__()
> > 
> > AttributeError: 'listiterator' object has no attribute '__next__' 
> > 
> > xx.__iter__.__next__
> > ---------------------------------------------------------------------------
> > AttributeError                            Traceback (most recent call last)
> > <ipython-input-4-df8d8c6955e2> in <module>()
> > ----> 1 xx.__iter__.__next__
> > 
> > AttributeError: 'method-wrapper' object has no attribute '__next__' 
> > 
> > xx.__iter__()
> > Out[5]: <listiterator at 0xa211780>
> > 
> > xx.__iter__().__next__
> > ---------------------------------------------------------------------------
> > AttributeError                            Traceback (most recent call last)
> > <ipython-input-6-92f21313d62e> in <module>()
> > ----> 1 xx.__iter__().__next__
> > 
> > AttributeError: 'listiterator' object has no attribute '__next__' 
> > 
> > xxIterator = xx.__iter__()
> > 
> > xxIterator
> > Out[8]: <listiterator at 0xa2115c0>
> > 
> > xxIterator.__next__()
> > ---------------------------------------------------------------------------
> > AttributeError                            Traceback (most recent call last)
> > <ipython-input-9-5b74e35c2c6e> in <module>()
> > ----> 1 xxIterator.__next__()
> > 
> > AttributeError: 'listiterator' object has no attribute '__next__' 
> > 
> > for c in xx: print c
> > 1
> > 2
> > 3
> > 4
> > 5
> > ------------
> > 
> > I don't find a way to show __next__ yet. 
> > Can we explicitly get the iterator for a list?
> > Thanks,
> 
> Excuse me. I find it as the following:
> 
> xx.__iter__().next
> Out[16]: <method-wrapper 'next' of listiterator object at 0x0000000008B38AC8>
> 
> xx.__iter__().next()
> Out[17]: 1

One example, see below please, is in the above mentioned link. I don't see
the purpose of the example.


\\\\\\\\\\\
class MyList(list):
    def __iter__(self):
        return MyListIter(self)
    
class MyListIter(object):
    """ A sample implementation of a list iterator. NOTE: This is just a 
    demonstration of concept!!! YOU SHOULD NEVER IMPLEMENT SOMETHING LIKE THIS!
    Even if you have to (for any reason), there are many better ways to 
    implement this."""
    def __init__(self, lst):
        self.lst = lst
        self.i = -1
    def __iter__(self):
        return self
    def next(self):
        if self.i<len(self.lst)-1:
            self.i += 1         
            return self.lst[self.i]
        else:
            raise StopIteration

if __name__ == '__main__':
    a = MyList([1, 2, 3, 4])
    ia = iter(a)
    print 'type(a): %r, type(ia): %r' %(type(a), type(ia))
    for i in a: 
        print i,
\\\\\\\\\\\\\\\

I have the following two same results. What do they tell me? Thanks,

for c in a: print c
1
2
3
4

for c in ia: print c
1
2
3
4

[toc] | [prev] | [next] | [standalone]

#100129

On 08/12/15 01:50, Robert wrote:
> One example, see below please, is in the above mentioned link. I don't see
> the purpose of the example.

OK, so there are two parts to this.

The first, is "how do I iterate over something". The answer to that is 
using "for" or using "iter()" followed by zero or more calls to 
"next()". In this case, by using the correct syntax/function calls your 
script can work under various versions of Python without change.

The second is "how do I make my own classes iterable". This is what the 
example you pasted is trying to show. In this case, you are implementing 
things which are "internal" to the way the version of Python you are 
using does things (which is why the code Robin posted wasn't quite right 
for you).

> I have the following two same results. What do they tell me?

They tell you that the implementation does the same thing as the default 
implementation for a list. That perhaps doesn't help much - especially 
with the comment in the example telling you not to do it!

Instead, try the following (Python 2):

class MyList(list):
     def __iter__(self):
         return MyListIter(self)

class MyListIter(object):
     def __init__(self, lst):
         self.lst = lst
         self.i = -1

     def __iter__(self):
         return self

     def next(self):
         if self.i >= -len(self.lst):
             item = self.lst[self.i]
             self.i -= 1
             return item

         raise StopIteration

if __name__ == "__main__":
     a = MyList([1, 2, 3, 4])
     ia = iter(a)
     print 'type(a): %r, type(ia): %r' %(type(a), type(ia))
     for i in a:
         print i,
     print

     while True:
         print next(ia)

What we have here is the same class that subclasses 'list'. It's just a 
list. However, it has a custom iterator. In this implementation the 
iterator works BACKWARDS through the list - the final element is 
returned first, the penultimate element second, and so on. After the 
first element has been returned, the iterator raises StopIteration. This 
tells you not to call next() again, or if in a for loop, the loop is exited.

So, you can write your class's iterator to do anything that makes sense 
when someone says "for i in myclassinstance:".

If your class is a subclass of a class ("is-a") that already has a 
defined iterator (such as a list or a dict) and the behaviour of that is 
correct for you, then you need to do nothing (you inherit that class's 
__iter__() method).

If your class should iterate over an embedded object ("has-a") that 
already has a defined iterator, then your __iter__() method can just 
delegate to that object's iterator using something like:

def __iter__(self):
     return iter(self.embedded_thing)

Does that make more sense?
E.

[toc] | [prev] | [next] | [standalone]

#100130

On Tue, Dec 8, 2015 at 1:36 PM, Erik <python@lucidity.plus.com> wrote:
> So, you can write your class's iterator to do anything that makes sense when
> someone says "for i in myclassinstance:".
>
> If your class is a subclass of a class ("is-a") that already has a defined
> iterator (such as a list or a dict) and the behaviour of that is correct for
> you, then you need to do nothing (you inherit that class's __iter__()
> method).
>
> If your class should iterate over an embedded object ("has-a") that already
> has a defined iterator, then your __iter__() method can just delegate to
> that object's iterator using something like:
>
> def __iter__(self):
>     return iter(self.embedded_thing)

Another great way to write an __iter__ method is as a generator.

def __iter__(self):
    yield "thing"
    yield from self.things
    yield "other thing"

Like returning an embedded object's iterator, this saves you having to
write a __next__ method. The less work you do, the less bugs you get.

ChrisA

[toc] | [prev] | [next] | [standalone]

#100162

On Monday, December 7, 2015 at 10:24:09 PM UTC-5, Chris Angelico wrote:
> On Tue, Dec 8, 2015 at 1:36 PM, Erik <pylucidity.plus.com> wrote:
> > So, you can write your class's iterator to do anything that makes sense when
> > someone says "for i in myclassinstance:".
> >
> > If your class is a subclass of a class ("is-a") that already has a defined
> > iterator (such as a list or a dict) and the behaviour of that is correct for
> > you, then you need to do nothing (you inherit that class's __iter__()
> > method).
> >
> > If your class should iterate over an embedded object ("has-a") that already
> > has a defined iterator, then your __iter__() method can just delegate to
> > that object's iterator using something like:
> >
> > def __iter__(self):
> >     return iter(self.embedded_thing)
> 
> Another great way to write an __iter__ method is as a generator.
> 
> def __iter__(self):
>     yield "thing"
>     yield from self.things
>     yield "other thing"
> 
> Like returning an embedded object's iterator, this saves you having to
> write a __next__ method. The less work you do, the less bugs you get.
> 
> ChrisA

Thanks. One more question is here. I see the following code snippet. 
It can run as expected. That is, zeros array is iterable, but I don't see
the iterator of it. And I do see some codes using np.nditer, but the 
following without it.


seq = [ im for im in zeros((20,240,320), int)]

1. What difference for iterate with and without np.nditer?
2. How can I know one object whether it is iterable (only by testing?)

[toc] | [prev] | [next] | [standalone]

#100128

Hi Robert,

On 08/12/15 01:39, Robert wrote:
>> I don't find a way to show __next__ yet.
>> Can we explicitly get the iterator for a list?
>> Thanks,
>
> Excuse me. I find it as the following:
>
> xx.__iter__().next
> Out[16]: <method-wrapper 'next' of listiterator object at 0x0000000008B38AC8>
>
> xx.__iter__().next()
> Out[17]: 1

Robin has told you how things work under the hood for the particular 
version of Python that he is running (Python 3). As you've seen, it 
works a bit different under the hood in your version (Python 2).

This is why you should not be calling the __ ("dunder") methods 
directly. Until you understand more and want to write your own classes 
that support being iterated using the 'for' keyword, you should probably 
ignore them.

Instead, the way to do this which works on all versions is:

x = [1, 2, 3, 4]

xit = iter(x)
next(xit)
next(xit)
next(xit)
next(xit)
next(xit)

This is what the 'for' keyword is doing for you - it first gets an 
iterator for the list (using iter()) and then processes each element 
that the iterator returns (from next()) until it raises the exception. 
It then suppresses the exception (so you don't have to catch it 
yourself) and exits the for loop.

Of course, it might actually do this using the __ methods and other 
things as a shortcut internally, but that's just an implementation detail.

E.

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web