Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #12473

Re: PC locks up with list operations

Show all headers | View raw

Steven D'Aprano wrote:

> Twice in a couple of weeks, I have locked up my PC by running a Python 2.5
> script that tries to create a list that is insanely too big.
> 
> In the first case, I (stupidly) did something like:
> 
> mylist = [0]*12345678901234
> 
> After leaving the machine for THREE DAYS (!!!) I eventually was able to
> get to a console and kill the Python process. Amazingly, it never raised
> MemoryError in that time.
> 
> The second time was a little less stupid, but not much:
> 
> mylist = []
> for x in itertools.combinations_with_replacement(some_big_list, 20):
>     mylist.append(func(x))
> 
> After three hours, the desktop is still locked up. I'm waiting to see what
> happens in the morning before rebooting.
> 
> Apart from "Then don't do that!", is there anything I can do to prevent
> this sort of thing in the future? Like instruct Python not to request more
> memory than my PC has?
> 
> I am using Linux desktops; both incidents were with Python 2.5. Do newer
> versions of Python respond to this sort of situation more gracefully?

If you are starting these scripts from the shell, how about ulimit?

$ ulimit -v 40000
$ python -c'print range(10**5)[-1]'
99999
$ python -c'print range(10**6)[-1]'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
MemoryError
$

Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

PC locks up with list operations Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2011-08-31 22:33 +1000
  Re: PC locks up with list operations Chris Withers <chris@simplistix.co.uk> - 2011-08-31 13:40 +0100
    Re: PC locks up with list operations Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2011-08-31 22:47 +1000
      Re: PC locks up with list operations Gregory Ewing <greg.ewing@canterbury.ac.nz> - 2011-09-01 11:31 +1200
        Re: PC locks up with list operations Tim Chase <python.list@tim.thechases.com> - 2011-08-31 19:53 -0500
      Re: PC locks up with list operations Nobody <nobody@nowhere.com> - 2011-09-12 13:19 +0100
    Re: PC locks up with list operations Robert Spanjaard <spamtrap@arumes.com> - 2011-08-31 16:06 +0200
    Re: PC locks up with list operations John Nagle <nagle@animats.com> - 2011-09-15 12:27 -0700
  Re: PC locks up with list operations Benjamin Kaplan <benjamin.kaplan@case.edu> - 2011-08-31 08:49 -0400
    Re: PC locks up with list operations Carl Banks <pavlovevidence@gmail.com> - 2011-09-13 06:31 -0700
  Re: PC locks up with list operations Peter Otten <__peter__@web.de> - 2011-08-31 14:56 +0200
  Re: PC locks up with list operations Rodrick Brown <rodrick.brown@gmail.com> - 2011-08-31 10:13 -0400
  Re: PC locks up with list operations Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2011-09-12 18:44 +1000
    Re: PC locks up with list operations Roy Smith <roy@panix.com> - 2011-09-12 07:40 -0400
      Re: PC locks up with list operations Terry Reedy <tjreedy@udel.edu> - 2011-09-12 16:53 -0400

csiph-web