Path: csiph.com!optima2.xanadu-bbs.net!xanadu-bbs.net!feeder.erje.net!1.eu.feeder.erje.net!newsfeed.xs4all.nl!newsfeed8.news.xs4all.nl!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
MIME-Version: 1.0
In-Reply-To: <55A8FAF6.5090104@rece.vub.ac.be>
References: <CAD+URuj-4e62-9hvpiLsrBP1M6kXRyZhZ6XyL01gWWnDoShvNg@mail.gmail.com> <mailman.499.1436874241.3674.python-list@python.org> <87d1zunctp.fsf@elektro.pacujo.net> <mailman.503.1436878550.3674.python-list@python.org> <87k2u2eu67.fsf@elektro.pacujo.net> <CAPTjJmqtTRHpNZHByn1pKNSfbsG6X--s2d_G0FopKxDugqpjdA@mail.gmail.com> <55A51662.4090007@rece.vub.ac.be> <mo4a7g$mkg$1@ger.gmane.org> <mailman.531.1436951405.3674.python-list@python.org> <d0o60cF8ubiU1@mid.individual.net> <55A75DE0.1070101@rece.vub.ac.be> <CAPTjJmoWqhZ_P890qug3Zf=VjJOBmO5kOSaX08PDF3FdNyLFNA@mail.gmail.com> <55A7B80B.6090905@rece.vub.ac.be> <CAPTjJmp3mupnenUYYNmvJpNr27zx_CdHQ9m6wkEFi-Lz+Evw9A@mail.gmail.com> <55A7C094.7060604@rece.vub.ac.be> <CAPTjJmozknJAqPvE1W1JZ66u72hNc_R5j5=wo+2JJK30Z5oc6w@mail.gmail.com> <55A8DD75.5000403@rece.vub.ac.be> <CAPTjJmrK=A7=4MYLbFdRBGjkr1qm442b_Ci-SQVmf6_2z97KtA@mail.gmail.com> <55A8EA7D.2040005@rece.vub.ac.be> <CAPTjJmrf9drR3Z7QUHSt+qCjY32EOm9aECR-gjiBRm3x-A+ang@mail.gmail.com> <55A8FAF6.5090104@rece.vub.ac.be>
Date: Fri, 17 Jul 2015 23:12:20 +1000
Subject: Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)
From: Chris Angelico <rosuav@gmail.com>
Cc: "python-list@python.org" <python-list@python.org>
Content-Type: text/plain; charset=UTF-8
Precedence: list
Newsgroups: comp.lang.python
Message-ID: <mailman.648.1437138743.3674.python-list@python.org>
Lines: 86
NNTP-Posting-Host: 2001:888:2000:d::a6
Xref: csiph.com comp.lang.python:94011

On Fri, Jul 17, 2015 at 10:54 PM, Antoon Pardon
<antoon.pardon@rece.vub.ac.be> wrote:
> On 07/17/2015 01:49 PM, Chris Angelico wrote:
>> On Fri, Jul 17, 2015 at 9:43 PM, Antoon Pardon
>> <antoon.pardon@rece.vub.ac.be> wrote:
>>
>>
>>> Sure, but in this case, the generator is still active. The Runtime
>>> would be able to jump to and somehow activates it's stack record
>>> for the next value. So why would we expect it to be impossible to
>>> include this trace back record in a stack trace?
>> Python could also give you stack traces for any other threads that are
>> concurrently running, on the off-chance that one of them affected it.
>> But the only influence the generator has on the loop is to yield a
>> value or signal termination; if an exception is thrown in the loop
>> itself, the local name 'stuff' should have all the information about
>> that cause.
>
> But the local name 'stuff' may only have the information for the immediate
> cause. The underlying cause may be available in the generator. Suppose
> you have a generator that should only generate positive numbers that you
> use to divide some other number by. Your loop crashes because of a DivideByZeroError
> Sure the local name shows the dividor to be zero, but you have no
> information on why your generator produced a zero, but there may be a
> clue in the trace back record of the generator.

Indeed, but there's nothing special about generators here. The same
sequence could have been a concrete list, or it could have been some
other kind of iterator (any object with __iter__ and __next__), which
won't have a stack frame. Special cases aren't special enough to warp
exception handling around.

>>  Python isn't a mind-reader, no matter how much it may look
>> like one, and it can't know that this function's return value should
>> be shown as part of a completely different function's stack trace.
>
> It is not a matter of mindreading. And it is not a completely different
> functions stack trace. It is the trace back record of a generator that
> is used by the process/thread that crashed. And AFAIK an active generator
> belongs to one specific thread. You can't have it yield a value to a different
> thread and you can't send it a value from an other thread. So I really
> see no reason to exclude the trace back records of active generators
> from a stack trace of a crashed thread.

No, generators are fine across threads:

rosuav@sikorsky:~$ python3 threadgen.py
Starting!
First yielded value
Continuing!
Second yielded value
Terminating.
Traceback (most recent call last):
  File "threadgen.py", line 20, in <module>
    print(next(gen))
StopIteration
rosuav@sikorsky:~$ cat threadgen.py
import threading
import time

def thread():
    time.sleep(0.5)
    print(next(gen))

threading.Thread(target=thread).start()

def generator():rosuav@sikorsky:~$

    print("Starting!")
    yield "First yielded value"
    print("Continuing!")
    yield "Second yielded value"
    print("Terminating.")

gen = generator()
print(next(gen))
time.sleep(1)
print(next(gen))
rosuav@sikorsky:~$

In fact, a generator doesn't have a stack unless it's currently
executing, so all you could get is whatever's actually inside it (that
is, if there's a deep tree of 'yield from's, you could dig up that
part of the stack). I'm not sure this would really help you very much.

ChrisA