Path: csiph.com!eternal-september.org!feeder.eternal-september.org!border1.nntp.ams1.giganews.com!nntp.giganews.com!newsfeed.xs4all.nl!newsfeed7.news.xs4all.nl!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
MIME-Version: 1.0
In-Reply-To: <b1c0218c-05dd-402b-b2f0-5e0836146f76@googlegroups.com>
References: <88256581-75d4-4f77-81f0-9e3e25baecbc@googlegroups.com> <mailman.2.1439112286.3627.python-list@python.org> <b1c0218c-05dd-402b-b2f0-5e0836146f76@googlegroups.com>
Date: Sun, 9 Aug 2015 20:13:13 +1000
Subject: Re: Iterators membership testing
From: Chris Angelico <rosuav@gmail.com>
Cc: "python-list@python.org" <python-list@python.org>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
Precedence: list
Newsgroups: comp.lang.python
Message-ID: <mailman.3.1439115197.3627.python-list@python.org>
Lines: 56
NNTP-Posting-Host: 2001:888:2000:d::a6
Xref: csiph.com comp.lang.python:95184

On Sun, Aug 9, 2015 at 7:55 PM, Pierre Quentel <pierre.quentel@gmail.com> w=
rote:
> Thanks for the explanation. I understand that an iterator can't test memb=
ership any other way, but I'm still worried about how the documentation exp=
lains it. Reading it, I naively expected that an iterator which produces th=
e integer 0 such as the one included in my post would say that "0 in iterat=
or" is True, because "some value z with 0 =3D=3D z is produced while iterat=
ing over y".
>
> Shouldn't it be more clear if the documentation said something like : "Fo=
r user-defined classes which do not define __contains__() but do define __i=
ter__(), x in y consumes the iterator y until some value z with x =3D=3D z =
is produced" ?
>

Consuming the iterator is implicit in the description, because the
only way to iterate over an iterator is to call next() on it, which
consumes values. But the description covers all *iterables*, and the
caveat applies only to *iterators*. Compare:

class A:
    def __init__(self, x):
        self.x =3D x

    def __iter__(self):
        counter =3D -1
        while counter < self.x:
            counter +=3D 1
            yield counter

This is the same code as you had, except that it's an iterable that
returns a _different_ object, and thus is not itself an iterator (in
this case, A().__iter__() returns a generator object). Note the
similarity of code to your example, and how easy it is to convert your
code to be a generator and make it reiterable. Now let's do that
membership test:

>>> iterable =3D A(10)
>>> for i in iterable:
...     assert i in iterable
...

No error. We can repeatedly iterate over this generator factory, and
each iteration starts a new instance of the generator, which thus
contains all the same values. Adding a print() will show that every
number is indeed iterated over.

The trap you're seeing here is that iterating over an iterator always
consumes it, but mentally, you're expecting this to be iterating over
a new instance of the same sequence. That's perfectly understandable,
but due to the extreme flexibility of the iterator and iterable
protocols, there's no way for the interpreter to say anything
different. Make your object reiterable, and then it'll behave the way
your brain is expecting... but the docs aren't incorrect here.

ChrisA