Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #93942

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)

References (20 earlier) <55A7C094.7060604@rece.vub.ac.be> <CAPTjJmozknJAqPvE1W1JZ66u72hNc_R5j5=wo+2JJK30Z5oc6w@mail.gmail.com> <CAB1GNpQw_gu4r3OmnQUtw8AhTWyPY4hpC42145UTPD2qMovYEw@mail.gmail.com> <CAPTjJmrsrcHd0FUHA=D0uqrqxO04_1W=jMaOVzDFWvZ7MyOWfw@mail.gmail.com> <CAB1GNpTcxJTeYXPAheP+EJh5kL5Sx13qSjepJj7B4mJQfYPP8g@mail.gmail.com>
Date 2015-07-17 03:49 +1000
Subject Re: Possibly Pythonic Tail Call Optimization (TCO/TRE)
From Chris Angelico <rosuav@gmail.com>
Newsgroups comp.lang.python
Message-ID <mailman.597.1437068955.3674.python-list@python.org> (permalink)

Show all headers | View raw

On Fri, Jul 17, 2015 at 3:34 AM, Joonas Liik <liik.joonas@gmail.com> wrote:
> That all sounds reasonable. However that can be looked another way.
> Soppose you have some code that traverses some tree, a strange
> imbalanced tree (say from some xml)
>
> It is, semantically at least, a reasonable aproach to process such a
> structure with some recursive function.
> Lets say we have a function that counts all <li> elements in a
> document for example. and recursively traverses the element tree.
> Because most xml is relatively flat (let's assume its rare to find
> more than 100 levels of nesting say) this function would perform
> perfectly well for most cases.
> however if some guy sent you an xml document with 1000 levels of
> nesting your program would crash.

This sounds like a denial-of-service attack. If you can state that no
reasonable document will ever have more than 100 levels of nesting,
then you can equally state that cutting the parser off with a tidy
exception if it exceeds 100 levels is safe.

> Now i admit that it is possible to have infinite recursion but it is
> also possiblew to have infinite loops. and we don't kill your code
> after 1000 iterations of a while loop so why should we treat recursion
> any differently?

Partly because infinite recursion requires infinite storage too. If it
truly is tail calls, then you can turn it into a while loop and not
have the limit; otherwise, you run the risk of blowing out your RAM.

> Having a user defined maximum stack limit might be a good idea, eg if
> my stack takes up 100MB its prolly broke, but it should be my duty as
> a programmer to specify such a limit, not have it inflicted upon me
> (worse, in a manner that cannot be changed!).

Actually, it is up to you. There's a default, but you can change it.

>>> def recurse(n): return n and recurse(n-1)
...
>>> recurse(998)
0
>>> recurse(999)
(throws RuntimeError)
>>> sys.getrecursionlimit()
1000
>>> sys.setrecursionlimit(5)
>>> recurse(5)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 1, in recurse
  File "<stdin>", line 1, in recurse
  File "<stdin>", line 1, in recurse
  File "<stdin>", line 1, in recurse
RuntimeError: maximum recursion depth exceeded
>>> sys.setrecursionlimit(5000)
>>> recurse(5000)
(throws RuntimeError with a gigantic traceback)
>>> sys.setrecursionlimit(50000)
>>> recurse(50000)
Segmentation fault

Though as you can see, CPython does have other issues. If you crank
the recursion limit up far enough, you *will* blow out your C stack.
Other Pythons may behave differently, and different builds of CPython
may crash at different points. But within reason, you can expand this
limit, and you can certainly shrink it (eg to reduce the effect of a
tree-parser DOS attack).

ChrisA

Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Chris Angelico <rosuav@gmail.com> - 2015-07-14 00:05 +1000
  Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Steven D'Aprano <steve@pearwood.info> - 2015-07-14 13:20 +1000
    Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Antoon Pardon <antoon.pardon@rece.vub.ac.be> - 2015-07-14 09:26 +0200
    Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Ian Kelly <ian.g.kelly@gmail.com> - 2015-07-14 02:34 -0600
      Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Grant Edwards <invalid@invalid.invalid> - 2015-07-14 14:03 +0000
    Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Antoon Pardon <antoon.pardon@rece.vub.ac.be> - 2015-07-14 12:43 +0200
  Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Paul Rubin <no.email@nospam.invalid> - 2015-07-13 22:15 -0700
    Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Chris Angelico <rosuav@gmail.com> - 2015-07-14 15:25 +1000
      Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Paul Rubin <no.email@nospam.invalid> - 2015-07-13 22:41 -0700
        Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Chris Angelico <rosuav@gmail.com> - 2015-07-14 21:43 +1000
          Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Marko Rauhamaa <marko@pacujo.net> - 2015-07-14 15:28 +0300
            Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Chris Angelico <rosuav@gmail.com> - 2015-07-14 22:55 +1000
              Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Marko Rauhamaa <marko@pacujo.net> - 2015-07-14 16:38 +0300
                Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Chris Angelico <rosuav@gmail.com> - 2015-07-14 23:43 +1000
                Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Marko Rauhamaa <marko@pacujo.net> - 2015-07-14 20:29 +0300
                Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Steven D'Aprano <steve@pearwood.info> - 2015-07-15 03:48 +1000
                Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Antoon Pardon <antoon.pardon@rece.vub.ac.be> - 2015-07-15 10:28 +0200
                Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Antoon Pardon <antoon.pardon@rece.vub.ac.be> - 2015-07-14 16:02 +0200
                Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Chris Angelico <rosuav@gmail.com> - 2015-07-15 00:07 +1000
                Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Marko Rauhamaa <marko@pacujo.net> - 2015-07-14 20:31 +0300
                Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Terry Reedy <tjreedy@udel.edu> - 2015-07-14 20:41 -0400
                Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Antoon Pardon <antoon.pardon@rece.vub.ac.be> - 2015-07-15 11:10 +0200
                Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Gregory Ewing <greg.ewing@canterbury.ac.nz> - 2015-07-16 10:43 +1200
                Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Antoon Pardon <antoon.pardon@rece.vub.ac.be> - 2015-07-16 09:31 +0200
                Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Gregory Ewing <greg.ewing@canterbury.ac.nz> - 2015-07-17 10:41 +1200
                Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Chris Angelico <rosuav@gmail.com> - 2015-07-16 21:45 +1000
                Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Antoon Pardon <antoon.pardon@rece.vub.ac.be> - 2015-07-16 15:56 +0200
                Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Chris Angelico <rosuav@gmail.com> - 2015-07-17 00:00 +1000
                Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Antoon Pardon <antoon.pardon@rece.vub.ac.be> - 2015-07-16 16:32 +0200
                Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Chris Angelico <rosuav@gmail.com> - 2015-07-17 02:43 +1000
                Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Joonas Liik <liik.joonas@gmail.com> - 2015-07-16 19:50 +0300
                Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Chris Angelico <rosuav@gmail.com> - 2015-07-17 03:03 +1000
                Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Ethan Furman <ethan@stoneleaf.us> - 2015-07-16 10:04 -0700
                Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Joonas Liik <liik.joonas@gmail.com> - 2015-07-16 20:34 +0300
                Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Steven D'Aprano <steve@pearwood.info> - 2015-07-17 04:58 +1000
                Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Joonas Liik <liik.joonas@gmail.com> - 2015-07-16 22:14 +0300
                Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Gregory Ewing <greg.ewing@canterbury.ac.nz> - 2015-07-17 11:41 +1200
                Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Ethan Furman <ethan@stoneleaf.us> - 2015-07-16 12:52 -0700
                Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Chris Angelico <rosuav@gmail.com> - 2015-07-17 03:49 +1000
                Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Joonas Liik <liik.joonas@gmail.com> - 2015-07-16 21:23 +0300
                Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Chris Angelico <rosuav@gmail.com> - 2015-07-17 04:29 +1000
                Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Terry Reedy <tjreedy@udel.edu> - 2015-07-16 16:19 -0400
                Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Antoon Pardon <antoon.pardon@rece.vub.ac.be> - 2015-07-17 12:48 +0200
                Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Gregory Ewing <greg.ewing@canterbury.ac.nz> - 2015-07-19 10:57 +1200
                Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Chris Angelico <rosuav@gmail.com> - 2015-07-17 21:05 +1000
                Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Antoon Pardon <antoon.pardon@rece.vub.ac.be> - 2015-07-17 13:43 +0200
                Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Chris Angelico <rosuav@gmail.com> - 2015-07-17 21:49 +1000
                Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Antoon Pardon <antoon.pardon@rece.vub.ac.be> - 2015-07-17 14:54 +0200
                Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Chris Angelico <rosuav@gmail.com> - 2015-07-17 23:12 +1000
                Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Terry Reedy <tjreedy@udel.edu> - 2015-07-17 16:27 -0400
          Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Gregory Ewing <greg.ewing@canterbury.ac.nz> - 2015-07-15 21:13 +1200
            Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Marko Rauhamaa <marko@pacujo.net> - 2015-07-15 12:55 +0300
            Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Mark Lawrence <breamoreboy@yahoo.co.uk> - 2015-07-15 12:22 +0100
              Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Gregory Ewing <greg.ewing@canterbury.ac.nz> - 2015-07-16 10:34 +1200
                Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Mark Lawrence <breamoreboy@yahoo.co.uk> - 2015-07-16 00:34 +0100
            Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) MRAB <python@mrabarnett.plus.com> - 2015-07-15 13:00 +0100
            Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Chris Angelico <rosuav@gmail.com> - 2015-07-15 22:13 +1000
              Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Gregory Ewing <greg.ewing@canterbury.ac.nz> - 2015-07-16 10:34 +1200
            Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Chris Angelico <rosuav@gmail.com> - 2015-07-15 22:22 +1000
      Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Marko Rauhamaa <marko@pacujo.net> - 2015-07-14 08:41 +0300
        Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Chris Angelico <rosuav@gmail.com> - 2015-07-14 21:33 +1000
          Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Marko Rauhamaa <marko@pacujo.net> - 2015-07-14 15:08 +0300
    Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Chris Angelico <rosuav@gmail.com> - 2015-07-14 15:31 +1000
    Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Ian Kelly <ian.g.kelly@gmail.com> - 2015-07-13 23:46 -0600
      Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Marko Rauhamaa <marko@pacujo.net> - 2015-07-14 08:57 +0300
        Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Gregory Ewing <greg.ewing@canterbury.ac.nz> - 2015-07-14 18:29 +1200
          Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Marko Rauhamaa <marko@pacujo.net> - 2015-07-14 10:05 +0300
            Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Gregory Ewing <greg.ewing@canterbury.ac.nz> - 2015-07-15 17:52 +1200
              Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Marko Rauhamaa <marko@pacujo.net> - 2015-07-15 09:44 +0300
                Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Ned Batchelder <ned@nedbatchelder.com> - 2015-07-15 03:45 -0700
                Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Marko Rauhamaa <marko@pacujo.net> - 2015-07-15 13:55 +0300
                Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Antoon Pardon <antoon.pardon@rece.vub.ac.be> - 2015-07-15 13:04 +0200
                Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Ned Batchelder <ned@nedbatchelder.com> - 2015-07-15 04:12 -0700
                Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Marko Rauhamaa <marko@pacujo.net> - 2015-07-15 14:24 +0300
                Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Rustom Mody <rustompmody@gmail.com> - 2015-07-16 21:28 -0700
          Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Mark Lawrence <breamoreboy@yahoo.co.uk> - 2015-07-14 08:47 +0100
        Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Ian Kelly <ian.g.kelly@gmail.com> - 2015-07-14 02:13 -0600
          Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Marko Rauhamaa <marko@pacujo.net> - 2015-07-14 11:33 +0300
            Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Ian Kelly <ian.g.kelly@gmail.com> - 2015-07-14 07:15 -0600
              Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Marko Rauhamaa <marko@pacujo.net> - 2015-07-14 20:27 +0300
                Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Ian Kelly <ian.g.kelly@gmail.com> - 2015-07-15 08:28 -0600
                Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Marko Rauhamaa <marko@pacujo.net> - 2015-07-15 17:55 +0300
            Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Steven D'Aprano <steve@pearwood.info> - 2015-07-15 03:34 +1000
              Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Marko Rauhamaa <marko@pacujo.net> - 2015-07-14 20:43 +0300
                Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Chris Angelico <rosuav@gmail.com> - 2015-07-15 03:52 +1000
                Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Steven D'Aprano <steve@pearwood.info> - 2015-07-15 04:06 +1000
                Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Marko Rauhamaa <marko@pacujo.net> - 2015-07-14 21:37 +0300
                Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Chris Angelico <rosuav@gmail.com> - 2015-07-15 04:55 +1000
                Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Marko Rauhamaa <marko@pacujo.net> - 2015-07-14 22:12 +0300
                Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Terry Reedy <tjreedy@udel.edu> - 2015-07-14 21:36 -0400
                Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Terry Reedy <tjreedy@udel.edu> - 2015-07-14 21:01 -0400
                Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Chris Angelico <rosuav@gmail.com> - 2015-07-15 13:05 +1000
              Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Paul Rubin <no.email@nospam.invalid> - 2015-07-14 10:54 -0700
    Re: Possibly Pythonic Tail Call Optimization (TCO/TRE) Terry Reedy <tjreedy@udel.edu> - 2015-07-14 20:23 -0400

csiph-web