Re: Python is not bad ;-)

From	Cecil Westerhof <Cecil@decebal.nl>
Newsgroups	comp.lang.python
Subject	Re: Python is not bad ;-)
Organization	Decebal Computing
References	<87mw1q9jqw.fsf@Equus.decebal.nl> <mailman.130.1430402596.3680.python-list@python.org> <87d22lvbnj.fsf@Equus.decebal.nl> <mhtqfg$gcj$1@dont-email.me>
Date	2015-04-30 22:05 +0200
Message-ID	<874mnxtmaa.fsf@Equus.decebal.nl> (permalink)

Show all headers | View raw

Op Thursday 30 Apr 2015 19:59 CEST schreef Christian Gollwitzer:

> Am 30.04.15 um 18:11 schrieb Cecil Westerhof:
>> Op Thursday 30 Apr 2015 16:03 CEST schreef Michael Torrie:
>>
>>> On 04/30/2015 01:07 AM, Cecil Westerhof wrote:
>>>> When I do that the computer is freezed a few times. That is a
>>>> little less nice. Does not happen with Clojure when it gets an
>>>> out of memory.
>>>
>>> A system freeze is probably due to thrashing by your operating
>>> system as a process (in this case Python) uses more and more
>>> memory, causing massive swapping. Clojure's heap, being a
>>> JVM-based language, is based on JVM settings, so it may be maxing
>>> out at just a couple of GB. Whereas Python will happily max out
>>> your swap if your program demands the memory.
>>
>> I just posted a message about that. This gets the problem also:
>> l = range(int(1E9))
>>
>> The problem is that after this other processes are swapped out and
>> have a bad performance. There is nothing that can be done about it?
>>
>> So there is a positive point for working with the JVM. :-D
>>
>
> No, you understood this the wrong way. The JVM has a limit on the
> max memory size on startup, which you can give by the -Xmx option on
> the Oracle machine, so for instance
>
> java -Xmx512M -jar myjar.jar
>
> limits the memory that your program may consume to 512 megs, until
> the JVM kills it. The standard limit is usually fairly low and
> probably below your real memory, so the java program does not have
> the chance to max out your memory and make your computer swap.

Well, I did play a little with that. My experience is that this is to
high. For example by lowering it, a certain program did not swap
anymore. Sound counter intuitive, but not that strange if you know the
inner workings of the JVM.


> CPython, au contraire, uses all memory it can get from the OS. The
> OS kills it if it uses too much. On Linux, you can set this limit
> yourself using ulimit. The analogue to the java call would therefore
> be something like
>
> ulimit -m 512M
> python mypython.py

I use startup scripts for the different Java (or Scala, or Clojure)
programs to set the memory size. I should do that also for my (or
anothers) Python scripts.


> If you set the ulimit to something smaller than your physical memory
> size, you also cause the program to be killed before it can use up

Nope, the program gets a MemoryError and will not be killed. Much
better I think.


> should set both limits to the same value and see how far you can
> get.

You are right.

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof

Back to comp.lang.python | Previous | Next — Previous in thread | Find similar | Unroll thread

Thread

Python is not bad ;-) Cecil Westerhof <Cecil@decebal.nl> - 2015-04-30 09:07 +0200
  Re: Python is not bad ;-) Ben Finney <ben+python@benfinney.id.au> - 2015-04-30 19:10 +1000
    Re: Python is not bad ;-) Marko Rauhamaa <marko@pacujo.net> - 2015-04-30 13:16 +0300
      Re: Python is not bad ;-) Chris Angelico <rosuav@gmail.com> - 2015-04-30 20:52 +1000
      Re: Python is not bad ;-) Cecil Westerhof <Cecil@decebal.nl> - 2015-04-30 13:30 +0200
        Re: Python is not bad ;-) Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-05-01 17:03 +1000
          Re: Python is not bad ;-) Cecil Westerhof <Cecil@decebal.nl> - 2015-05-01 09:47 +0200
          Re: Python is not bad ;-) Christian Gollwitzer <auriocus@gmx.de> - 2015-05-01 19:56 +0200
          Re: Python is not bad ;-) Gregory Ewing <greg.ewing@canterbury.ac.nz> - 2015-05-02 19:44 +1200
          Re: Python is not bad ;-) Cecil Westerhof <Cecil@decebal.nl> - 2015-05-02 10:26 +0200
            Re: Python is not bad ;-) Marko Rauhamaa <marko@pacujo.net> - 2015-05-02 12:10 +0300
              Re: Python is not bad ;-) Chris Angelico <rosuav@gmail.com> - 2015-05-02 19:25 +1000
                Re: Python is not bad ;-) Marko Rauhamaa <marko@pacujo.net> - 2015-05-02 12:58 +0300
                Re: Python is not bad ;-) Dave Angel <davea@davea.name> - 2015-05-02 06:22 -0400
                Re: Python is not bad ;-) Chris Angelico <rosuav@gmail.com> - 2015-05-02 20:42 +1000
                Re: Python is not bad ;-) Christian Gollwitzer <auriocus@gmx.de> - 2015-05-02 13:07 +0200
                Re: Python is not bad ;-) Chris Angelico <rosuav@gmail.com> - 2015-05-02 21:21 +1000
                Re: Python is not bad ;-) Christian Gollwitzer <auriocus@gmx.de> - 2015-05-02 13:32 +0200
                Re: Python is not bad ;-) Marko Rauhamaa <marko@pacujo.net> - 2015-05-02 14:42 +0300
                Re: Python is not bad ;-) Ian Kelly <ian.g.kelly@gmail.com> - 2015-05-02 09:45 -0600
                Re: Python is not bad ;-) Chris Angelico <rosuav@gmail.com> - 2015-05-03 01:55 +1000
                Re: Python is not bad ;-) Joonas Liik <liik.joonas@gmail.com> - 2015-05-02 16:50 +0300
                Re: Python is not bad ;-) Joonas Liik <liik.joonas@gmail.com> - 2015-05-02 18:53 +0300
                Re: Python is not bad ;-) Ian Kelly <ian.g.kelly@gmail.com> - 2015-05-02 11:00 -0600
                Re: Python is not bad ;-) Ian Kelly <ian.g.kelly@gmail.com> - 2015-05-02 11:17 -0600
                Re: Python is not bad ;-) Mark Lawrence <breamoreboy@yahoo.co.uk> - 2015-05-02 18:22 +0100
              Re: Python is not bad ;-) Cecil Westerhof <Cecil@decebal.nl> - 2015-05-02 12:29 +0200
            Re: Python is not bad ;-) Cecil Westerhof <Cecil@decebal.nl> - 2015-05-02 11:33 +0200
              Re: Python is not bad ;-) Dave Angel <davea@davea.name> - 2015-05-02 06:35 -0400
                Re: Python is not bad ;-) Cecil Westerhof <Cecil@decebal.nl> - 2015-05-02 13:12 +0200
              Re: Python is not bad ;-) Ian Kelly <ian.g.kelly@gmail.com> - 2015-05-02 09:31 -0600
      Re: Python is not bad ;-) Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-05-01 15:56 +1000
    Re: Python is not bad ;-) Cecil Westerhof <Cecil@decebal.nl> - 2015-04-30 13:10 +0200
  Re: Python is not bad ;-) Michael Torrie <torriem@gmail.com> - 2015-04-30 08:03 -0600
    Re: Python is not bad ;-) Cecil Westerhof <Cecil@decebal.nl> - 2015-04-30 18:11 +0200
      Re: Python is not bad ;-) Christian Gollwitzer <auriocus@gmx.de> - 2015-04-30 19:59 +0200
        Re: Python is not bad ;-) Cecil Westerhof <Cecil@decebal.nl> - 2015-04-30 22:05 +0200

csiph-web