Path: csiph.com!v102.xanadu-bbs.net!xanadu-bbs.net!news.mixmin.net!feeds.phibee-telecom.net!newsfeed.xs4all.nl!newsfeed1a.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail Return-Path: X-Original-To: python-list@python.org Delivered-To: python-list@mail.python.org X-Spam-Status: OK 0.000 X-Spam-Evidence: '*H*': 1.00; '*S*': 0.00; 'example:': 0.03; 'argument': 0.05; 'i?m': 0.05; 'say,': 0.05; 'subject:Python': 0.06; 'definitions': 0.07; 'i?ve': 0.07; 'problem:': 0.07; 'string': 0.09; '[1,': 0.09; 'args)': 0.09; 'assuming': 0.09; 'can?t': 0.09; 'falls': 0.09; 'file)': 0.09; 'list).': 0.09; 'next,': 0.09; 'posting.': 0.09; 'seemed': 0.09; 'subject:using': 0.09; 'topic:': 0.09; 'types:': 0.09; 'python': 0.11; 'def': 0.12; 'wrote': 0.14; "wouldn't": 0.14; '"dirty': 0.16; '"to': 0.16; "('b',": 0.16; "('c',": 0.16; '?to': 0.16; "[('a',": 0.16; '[])': 0.16; 'bug:': 0.16; 'definition.': 0.16; 'emulation': 0.16; 'happily': 0.16; 'investigate': 0.16; 'iterable': 0.16; 'iterable,': 0.16; 'iterables': 0.16; 'iterator': 0.16; 'iterators': 0.16; 'ivan': 0.16; 'mean,': 0.16; 'operation.': 0.16; 're-read': 0.16; 'res': 0.16; 'typeerror:': 0.16; 'we?ve': 0.16; 'why,': 0.16; 'applies': 0.16; 'trying': 0.19; 'everyone,': 0.19; 'it?s': 0.19; 'properly': 0.19; "python's": 0.19; 'examples': 0.20; '(the': 0.22; '>>>': 0.22; 'example': 0.22; 'coding': 0.22; 'issue.': 0.22; 'python?': 0.22; '>>>': 0.24; '2.x': 0.24; 'integer': 0.24; 'questions:': 0.24; 'fine': 0.24; 'looks': 0.24; 'question': 0.24; "i've": 0.25; 'this:': 0.26; 'pass': 0.26; 'least': 0.26; 'code:': 0.26; 'tried': 0.27; 'function': 0.29; 'related': 0.29; "doesn't": 0.30; 'statement': 0.30; "i'm": 0.30; 'went': 0.31; 'code': 0.31; "skip:' 10": 0.31; '"",': 0.31; '3.x': 0.31; 'clever': 0.31; 'once,': 0.31; 'python"': 0.31; 'raised': 0.31; 'though.': 0.31; 'trace': 0.31; 'tuples': 0.31; 'file': 0.32; 'lists': 0.32; 'checked': 0.32; 'run': 0.32; 'another': 0.32; 'url:python': 0.33; '(most': 0.33; 'skip:& 30': 0.33; 'moment': 0.34; 'maybe': 0.34; "can't": 0.35; 'but': 0.35; 'received:google.com': 0.35; 'there': 0.35; 'url:rec- html40': 0.35; 'false': 0.36; 'much.': 0.36; 'yield': 0.36; 'charset:us-ascii': 0.36; 'thanks': 0.36; "i'll": 0.36; 'possible': 0.36; 'url:org': 0.36; 'should': 0.36; 'skip:4 10': 0.37; 'too': 0.37; 'url:microsoft': 0.37; 'list': 0.37; 'clear': 0.37; 'message-id:@gmail.com': 0.38; 'skip:& 10': 0.38; 'ends': 0.38; 'url:library': 0.38; 'here:': 0.62; 'times': 0.62; 'kind': 0.63; 'soon': 0.63; 'today': 0.64; 'skip:n 10': 0.64; 'map': 0.64; 'zip': 0.64; 'finally': 0.65; 'here': 0.66; 'received:109': 0.72; 'links,': 0.74; 'attention': 0.75; '--->': 0.84; 'adapted': 0.84; 'avoids': 0.84; 'links:': 0.84; 'lived': 0.84; 'played': 0.84; 'song': 0.84; 'forever.': 0.91; 'on?': 0.91; 'promised': 0.91 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:to:subject:date:message-id:mime-version:content-type :thread-index:content-language; bh=DxDSVVz06Li58Koo80GJxv68IXap8Vx49jV+MDygPcM=; b=PP52I7nmMKo1gxgK08FbvZ6fPoQAYozW/r0Eax3WCaBOtXdCXrHYPsMr2D8eYPFHLx 3eW25OrHVSI/PmYDVtVTcx3BIwBkTWKJwQOL/aE+ClUbXZK1HuWkd3mRfXnX7SleDvEQ hKDmBiMbkZrfVD5cHr4X7zXlaUMoB/HWSYyvskawl16QFHdtFPAWQToNNUCm4kIgcghM oKhPqQIn64tti5fHAL3x4jgfKPgXktb3EvTFff1h7/hJpOJY/yuR1v0U4KGPEMia1W4K eDCEF2vtuYoCT/Fp0me1XyFqgQxT+VBmqO6XZ9k3ZLNllvlTEqyn0xlb0vOLD8fT7PR3 kgdw== X-Received: by 10.194.7.97 with SMTP id i1mr23745062wja.31.1417884322023; Sat, 06 Dec 2014 08:45:22 -0800 (PST) From: "Ivan Evstegneev" To: Subject: Python Iterables struggling using map() built-in Date: Sat, 6 Dec 2014 18:44:17 +0200 MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_004F_01D01184.A4F131D0" X-Mailer: Microsoft Outlook 14.0 Thread-Index: AdARcSqMNuT0s6RoSC+WYhuDZWkEjw== Content-Language: en-us X-Mailman-Approved-At: Sun, 07 Dec 2014 11:48:59 +0100 X-BeenThere: python-list@python.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: General discussion list for the Python programming language List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Newsgroups: comp.lang.python Message-ID: Lines: 1503 NNTP-Posting-Host: 2001:888:2000:d::a6 X-Trace: 1417949341 news.xs4all.nl 2835 [2001:888:2000:d::a6]:44596 X-Complaints-To: abuse@xs4all.nl Xref: csiph.com comp.lang.python:82052 This is a multipart message in MIME format. ------=_NextPart_000_004F_01D01184.A4F131D0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Hello everyone, I'm currently in the process of self-study journey, so I have some questions arisen from time to time. Today I would like to talk about iterables and iterators,(ask for your help actually ^_^). Before I'll continue, just wanted to draw your attention to the fact that I did some RTFM before posting. Links: 1. map() built-in definitions: https://docs.python.org/3/library/functions.html#map - for Python 3.X https://docs.python.org/2.6/library/functions.html#map - for Python 2.6.X 2. Glossary definitions of "iterable " and "iterator": https://docs.python.org/3/glossary.html?highlight=glossary 3. Iterator Types: https://docs.python.org/2/library/stdtypes.html#typeiter 4. iter() definition: https://docs.python.org/2/library/functions.html#iter 5. Some StackOverflow links, related to the topic: http://stackoverflow.com/questions/13054057/confused-with-python-lists-are-t hey-or-are-they-not-iterators http://stackoverflow.com/questions/9884132/understanding-pythons-iterator-it erable-and-iteration-protocols-what-exact http://stackoverflow.com/questions/19523563/python-typeerror-int-object-is-n ot-iterable http://stackoverflow.com/questions/538346/iterating-over-a-string 6. And of course, re-read couple of times a relevant parts of the book ('Learning Python" by Mark Lutz). But the questions still persist, maybe because those examples look too esoteric though. Another warning: Despite all my attempts to make my questions as short as possible it still looks huge. My apologies. The problem: Here is the book's example: Consider the following clever alternative coding for this chapter's zip emulation examples, adapted from one in Python's manuals at the time I wrote these words: def myzip(*args): iters = map(iter, args) while iters: res = [next(i) for i in iters] yield tuple(res) Because this code uses iter and next, it works on any type of iterable. Note that there is no reason to catch the StopIteration raised by the next(it) inside the comprehension here when any one of the arguments' iterators is exhausted-allowing it to pass ends this generator function and has the same effect that a return statement would. The while iters: suffices to loop if at least one argument is passed, and avoids an infinite loop otherwise (the list comprehension would always return an empty list). This code works fine in Python 2.X as is: >>> list(myzip('abc', 'lmnop')) [('a', 'l'), ('b', 'm'), ('c', 'n')] But it falls into an infinite loop and fails in Python 3.X, because the 3.X map returns a one-shot iterable object instead of a list as in 2.X. In 3.X, as soon as we've run the list comprehension inside the loop once, iters will be exhausted but still True (and res will be []) forever. To make this work in 3.X, we need to use the list built-in function to create an object that can support multiple iterations: >>>def myzip(*args): iters = list(map(iter, args)) # Allow multiple scans ...rest as is... Run this on your own to trace its operation. The lesson here: wrapping map calls in list calls in 3.X is not just for display! *********END OF THE BOOK EXAMPLE************ According to the book , in order to get things work properly in Python 3.X, I should write this code: >>> def myzip(*args): iters = list(map(iter, args)) while iters: res = [next(i) for i in iters] yield tuple(res) And all seemed to be clear here, but, when I tried to run this thing: >>> k= myzip(1, 2, 3, 4) >>> next(k) I got this result: Traceback (most recent call last): File "", line 1, in next(k) File "", line 2, in myzip iters = list(map(iter, args)) TypeError: 'int' object is not iterable Finding the problem: I started to investigate further in order to catch the bug: What I've tried? 1. >>> L = [1, 2, 3, 4] >>> iter(L) is L False ---> According to the "theory" it's OK, because list doesn't have self iterators. >>> k = iter(L) >>>print(k) >>> next(k) 1 >>> print(list(map(iter, L))) Traceback (most recent call last): File "", line 1, in print(list(map(iter, L))) TypeError: 'int' object is not iterable 2. I went to strings: >>> S = 'spam' >>> iter(S) is S False ---> Same about strings >>> string = iter(S) >>> string >>> next(string) 's' and so on. And then just tried this one: >>> print(list(map(iter, S))) [, , , ] -----> At this moment "Wrecking Ball" song played in my head %))))))) >>> k = list(map(iter,S)) >>> next(k[0]) 's' >>> next(k[0]) Traceback (most recent call last): File "", line 1, in next(k[0]) StopIteration So I may admit that I found the problem, but can't explain it to myself. Finally I've checked the map() definition. According to python.org: "Return an iterator that applies function to every item of iterable, yielding the results." The questions: And as I've promised the question section: 1. What actually map() trying to do in Python 3.X? I mean, why is this works fine: >>>L = [1, 2, 3, 4] >>> k = iter(L) >>> next(k) 1 and so on. But not this: >>> list(map(iter, L)) Traceback (most recent call last): File "", line 1, in print(list(map(iter, L))) TypeError: 'int' object is not iterable 2. Why strings are allowed "to become" an iterators(self-iterators)? Maybe because of files(reading from file) ? I mean why, is this possible: >>> print(list(map(iter, S))) [, , , ] What I'm trying to say, is that if I wouldn't tried to run the book's example with integer arguments(or tuples or lists as arguments) it wouldn't alarm this issue. And I've lived happily assuming that I understand iterables. )))) This is kind of a "dirty trick", because author do not talk about this anywhere. 3. The last question Author says: " But it falls into an infinite loop and fails in Python 3.X, because the 3.X map returns a one-shot iterable object instead of a list as in 2.X. In 3.X, as soon as we've run the list comprehension inside the loop once, iters will be exhausted but still True. To make this work in 3.X, we need to use the list built-in function to create an object that can support multiple iterations". (Like:"Wat?!" ^_^) Why the infinite loop would be there and why should list() to make it finite? o_0 Hope you have not get annoyed too much. Thanks in advance, Ivan ------=_NextPart_000_004F_01D01184.A4F131D0 Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Hello = everyone,

 

I’m currently in the process of self-study = journey, so I have some questions arisen from time to = time.

Today I would like to talk = about iterables and iterators,(ask for your help actually = ^_^).

 

Before I’ll continue, just wanted to draw your = attention  to the fact that I did some RTFM before = posting.

Links:

1.       = map() built-in = definitions:

https://doc= s.python.org/3/library/functions.html#map –    = for Python 3.X

https://d= ocs.python.org/2.6/library/functions.html#map – for Python = 2.6.X

 

2.       = Glossary definitions of = “iterable “ and “iterator”:

htt= ps://docs.python.org/3/glossary.html?highlight=3Dglossary<= /p>

 

3.       = Iterator = Types:

https:/= /docs.python.org/2/library/stdtypes.html#typeiter

 

4.       = iter() = definition:

https://do= cs.python.org/2/library/functions.html#iter

 

 

5.       = Some StackOverflow links, = related to the topic:

http://stackoverflow.com/questi= ons/13054057/confused-with-python-lists-are-they-or-are-they-not-iterator= s

http://stackoverflo= w.com/questions/9884132/understanding-pythons-iterator-iterable-and-itera= tion-protocols-what-exact

http://stackoverflow.com/questions/19523563/pytho= n-typeerror-int-object-is-not-iterable

http://stackoverflow.com/questions/538346/iterating-over-a-string

 

6.       = And of course, re-read =  couple of times  a relevant parts of the book = (‘Learning Python” by Mark Lutz).

 

But the = questions  still persist, maybe because those examples look too = esoteric though…

 

Another warning: Despite all my attempts = to make my questions as short as possible it still looks huge. My = apologies.

 

 

=

 

=

The problem:

 

=

Here is the book’s = example:

 

=

Consider the following = clever alternative coding for this chapter’s zip emulation = examples,

adapted from one in = Python’s manuals at the time I wrote these = words:

 

=

def = myzip(*args):

iters =3D map(iter, = args)

while = iters:

    &nbs= p;       res =3D [next(i) for i in = iters]

yield = tuple(res)

 

=

Because this code uses iter = and next, it works on any type of iterable. Note that = there

is no reason to catch the = StopIteration raised by the next(it) inside the = comprehension

here when any one of the = arguments’ iterators is exhausted—allowing it to pass = ends

this generator function and = has the same effect that a return statement would. = The

while iters: suffices to = loop if at least one argument is passed, and avoids an = infinite

loop otherwise (the list = comprehension would always return an empty = list).

 

=

This code works fine in = Python 2.X as is:

 

=

>>> = list(myzip('abc', 'lmnop'))

[('a', 'l'), ('b', 'm'), = ('c', 'n')]

 

=

But it falls into an = infinite loop and fails in Python 3.X, because the 3.X map returns = a

one-shot iterable object = instead of a list as in 2.X. In 3.X, as soon as we’ve run the = list

comprehension inside the = loop once, iters will be exhausted but still True (and res = will

be []) forever. To make = this work in 3.X, we need to use the list built-in function = to

create an object that can = support multiple iterations:

 

=

>>>def = myzip(*args):

iters =3D list(map(iter, = args)) # Allow multiple scans

...rest as = is...

 =

Run this on your own to = trace its operation. The lesson here: wrapping map calls = in

list calls in 3.X is not = just for display!

 =

*********END OF THE BOOK = EXAMPLE************

 

=

 

=

 

=

According to the book , in = order to get things work properly in Python 3.X, I should write this = code:

 

=

>>> def = myzip(*args):

    &nbs= p;       iters =3D list(map(iter, = args))

    &nbs= p;       while = iters:

    &nbs= p;            = ;       res =3D [next(i) for i in = iters]

    &nbs= p;            = ;       yield = tuple(res)

 

=

And all seemed to be clear = here, but, when I tried to run this thing:

 

=

>>> k=3D myzip(1, = 2, 3, 4)

>>> = next(k)

 

=

I got this result: =

 

=

Traceback (most recent call = last):

  File = "<pyshell#73>", line 1, in = <module>

    = next(k)

  File = "<pyshell#65>", line 2, in = myzip

    iters = =3D list(map(iter, args))

TypeError: 'int' object is = not iterable

 =

Finding the = problem:

 

I started to investigate = further in order to catch the bug:

 

=

What I’ve = tried?

 

=

1.     >>> L =3D [1, 2, = 3, 4]

 

=

>>> iter(L) is = L

False ---> =  According to the “theory” it’s OK, because list = doesn’t have self iterators.

 

=

>>> k =3D = iter(L)

 

=

>>>print(k)

<list_iterator object at = 0x03233F90>

 

=

>>> = next(k)

1

 

=

>>> = print(list(map(iter, L)))

Traceback (most recent call = last):

  File = "<pyshell#88>", line 1, in = <module>

    = print(list(map(iter, L)))

TypeError: 'int' object is = not iterable

 

=

2.     I = went to strings:

 

=

>>> S =3D = 'spam'

>>> iter(S) is = S

False  ---> =        Same about strings =

 

=

>>> string =3D = iter(S)

 

=

>>> = string

<str_iterator object at = 0x02E24F30>

 

=

>>> = next(string)

's'

 

=

and so = on…

 

=

And then just tried this = one:

 

=

>>> = print(list(map(iter, S)))

[<str_iterator object at = 0x02E24FF0>,

<str_iterator object at = 0x02E24CF0>,

<str_iterator object at = 0x02E24E10>,

<str_iterator object at = 0x02E24DF0>]     -----> At this moment =   “Wrecking Ball” song played in my head = %)))))))

 

=

 

=

>>> k =3D = list(map(iter,S))

>>> = next(k[0])

's'

>>> = next(k[0])

Traceback (most recent call = last):

  File = "<pyshell#99>", line 1, in = <module>

    = next(k[0])

StopIteration

 

=

So I may admit that I found = the problem, but can’t explain it to = myself…

 

=

 

=

 

=

Finally I’ve checked =  the map() definition.

According to python.org: = “Return an iterator that = applies function to every item of iterable, = yielding the results.”

 

=

The = questions:

 

And as I’ve promised = the question section:

 

=

1.     What = actually map() trying to do in Python 3.X?

I mean, why is this works = fine:

 

=

>>>L =3D [1, 2, 3, = 4]

>>> k =3D = iter(L)

>>> = next(k)

1

and so = on…

 

=

But not this: =

    &nbs= p;       >>> list(map(iter, = L))

 

=

Traceback (most recent call = last):

File = "<pyshell#88>", line 1, in = <module>

    &nbs= p;        print(list(map(iter, = L)))

TypeError: 'int' object is = not iterable

 

=

2.     Why = strings are allowed “to become” an = iterators(self-iterators)? Maybe because of files(reading from file) ? =

 

=

I mean why, is this = possible:

 

=

>>> = print(list(map(iter, S)))

[<str_iterator object at = 0x02E24FF0>,

<str_iterator object at = 0x02E24CF0>,

<str_iterator object at = 0x02E24E10>,

<str_iterator object at = 0x02E24DF0>]    

 

What I’m trying to say, is that if I = wouldn’t tried to run the book’s example with integer = arguments(or tuples or lists as arguments)  it wouldn’t alarm = this issue.

And = I’ve lived happily assuming that I understand iterables… = ))))

This is = kind of a “dirty trick”, because author do not talk about = this anywhere.

 

 

 

3.       = The last question

Author says:

 

” But it falls into an = infinite loop and fails in Python 3.X, because the 3.X map returns a =

one-shot iterable object = instead of a list as in 2.X. In 3.X, as soon as we’ve run the list =

comprehension inside the = loop once, iters will be exhausted but still True. To make this =

work in 3.X, we need to use = the list built-in function to create an object that can support =

multiple iterations”. = (Like:”Wat?!” = ^_^)

 

 

Why the infinite loop would be there and why = should list() to make it finite?  o_0

 

 

Hope  you have not get annoyed too = much.

 

Thanks in advance,

 

Ivan

 

 

 

 

=

 

=

 

=

 

=

 

=

 

=

 

=

 

=

 

=

 

=

 

=

 

=

 

=

 

=

 

 

 

=

 =

 

 

 

 

------=_NextPart_000_004F_01D01184.A4F131D0--