Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #2730

Re: Sandboxed Python: memory limits?

Show key headers only | View raw

On Thu, Apr 7, 2011 at 6:38 AM, Martin v. Loewis <martin@v.loewis.de> wrote:
> You can adjust the implementations of PyMem_Malloc and PyObject_Malloc.
> This would catch many allocations, but not all of them. If you adjust
> PyMem_MALLOC instead of PyMem_Malloc, you catch even more allocations -
> but extensions modules which directly call malloc() still would bypass
> this accounting.

I'm not too concerned about extensions, here; in any case, I lock most
of them off. I just want to prevent stupid stuff like this:

a='a'
while True:
    a+=a

from bringing the entire node to its knees. Obviously that will
eventually bomb with MemoryError, but I'd rather it be some time
*before* the poor computer starts thrashing virtual memory.

(Hmm. I tried the above code in Python 2.6.6 on my scratch box, with
3GB of memory, and it actually died with "OverflowError: strings are
too large to concat" at 1GB. Must be the 32-bit Python on there, heh.
But repeating the exercise in the same Python with a second variable
produces the expected MemoryError.)

If it's too difficult, I'll probably just tell my boss that we need
8GB of physical memory in these things, and then disable virtual
memory. That'll ensure that MemoryError happens before the hard disk
starts grinding performance into dust :)

Chris Angelico

Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar

Thread

Sandboxed Python: memory limits? Chris Angelico <rosuav@gmail.com> - 2011-04-06 11:59 +1000
  Re: Sandboxed Python: memory limits? "Martin v. Loewis" <martin@v.loewis.de> - 2011-04-06 22:38 +0200
    Re: Sandboxed Python: memory limits? Chris Angelico <rosuav@gmail.com> - 2011-04-07 10:06 +1000
      Re: Sandboxed Python: memory limits? "Martin v. Loewis" <martin@v.loewis.de> - 2011-04-07 10:01 +0200
      Re: Sandboxed Python: memory limits? "Martin v. Loewis" <martin@v.loewis.de> - 2011-04-07 10:01 +0200
    Re: Sandboxed Python: memory limits? Chris Angelico <rosuav@gmail.com> - 2011-04-07 10:10 +1000
      Re: Sandboxed Python: memory limits? David Bolen <db3l.net@gmail.com> - 2011-04-07 14:36 -0400
        Re: Sandboxed Python: memory limits? Chris Angelico <rosuav@gmail.com> - 2011-04-08 04:59 +1000

csiph-web