Path: csiph.com!news.swapon.de!eternal-september.org!feeder.eternal-september.org!news.eternal-september.org!.POSTED!not-for-mail
From: Cecil Westerhof <Cecil@decebal.nl>
Newsgroups: comp.lang.java.programmer
Subject: Re: Restricting the memory usage of a Java program
Date: Fri, 12 Aug 2016 10:34:15 +0200
Organization: Decebal Computing
Lines: 114
Message-ID: <87wpjml720.fsf@Equus.decebal.nl>
References: <8760r7m18z.fsf@Equus.decebal.nl> <noisp2$12e7$1@gioia.aioe.org>
Mime-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
Injection-Info: mx02.eternal-september.org; posting-host="12baf5125b19322b899c79c2e3878015"; logging-data="9184"; mail-complaints-to="abuse@eternal-september.org";	posting-account="U2FsdGVkX1+lOBVIlql2mk4l6kp4cH2qCFobdgQI9I8="
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.3 (gnu/linux)
Cancel-Lock: sha1:ZrDQ+iDT2xSDz5ovCw86q1vd3sc= sha1:Qt+FHhy115VzbTogW6+Ma0GmZPE=
X-Homepage: http://www.decebal.nl/
Xref: csiph.com comp.lang.java.programmer:35266

On Friday 12 Aug 2016 00:04 CEST, Arne Vajhøj wrote:

> On 8/11/2016 5:42 PM, Cecil Westerhof wrote:
>> I made a little program for cropping images. When just starting it
>> as a Java program the memory usage was just way to much. So I wrote
>> the following script: #!/usr/bin/env bash
>>
>> declare -r CLASS=CropImage
>> declare -r MIN_MEM=128M
>> declare -r MAX_MEM=256M
>>
>> # Make sure you are in the right directory. Only start the Java
>> # application when moving to the right directory was succesful. cd
>> # "${0%/*}" && java "-Xms${MIN_MEM}" "-Xmx${MAX_MEM}" "${CLASS}" &
>>
>> And now the memory consumption is much less and as far as I can see
>> without negative consequences for the program.
>>
>> The values I use (128 MB and 256 MB) are just values I kind of
>> randomly choose. Is there a way to find ‘optimal’ values?
>
> System.gc();
> Runtime rt = Runtime.getRuntime();
> long actual_value = rt.totalMemory() - rt.freeMemory();
> long good_value = SOME_MAGIC_FACTOR * actual_value;

It is funny: I was always told that calling gc does not have an effect
on memory and that it can have a negative impact on performance. But I
saw a real difference. Maybe it is a good idea to call it once an
hour?

I added the following to my constructor:
    ses.scheduleAtFixedRate(new Runnable() {
            @Override
            public void run() {
                long divideValue = 1024 * 1024;
                long freeMemory  = rt.freeMemory()  / divideValue;
                long totalMemory = rt.totalMemory() / divideValue;

                giveMessage(String.format("Maximum to use %d",
                                          rt.maxMemory() / divideValue));
                giveMessage(String.format("Before %d - %d = %d",
                                          totalMemory, freeMemory,
                                          totalMemory - freeMemory));
                System.gc();
                freeMemory  = rt.freeMemory()  / divideValue;
                totalMemory = rt.totalMemory() / divideValue;
                giveMessage(String.format("After  %d - %d = %d",
                                          totalMemory, freeMemory,
                                          totalMemory - freeMemory));
            }
        }, 15, 60, TimeUnit.MINUTES);

The function giveMessage prints the input string with a time stamp to
standard out.

The first time I ran it with:
        }, 0, 15, TimeUnit.MINUTES);

I ran it for little more as ten hours and with both I saw that the
free memory was the same after the gc. So it is really useful to get
an indication.


Maybe I should rewrite it as an utility function, so I could use it in
all my applications. And of-course I should log to a (SQLite) database
instead of standard out. (It should be called logMessage instead of
giveMessage.)


I also have an instance running without the gc. I should let it run
for at least a day to see what happens.

They start with:
- script version
09:43:54: Maximum to use 228
09:43:54: Before 123 - 64 = 59
09:43:54: After  123 - 69 = 54

- Java version
09:48:20: Maximum to use 3540
09:48:20: Before 240 - 172 = 68
09:48:20: After  240 - 186 = 54

- Java version without gc
09:49:23: Maximum to use 3540
09:49:23: Before 240 - 172 = 68
09:49:23: After  240 - 172 = 68

When it runs at least a day I will give an update.


It is clear that max memory default is quit large.
Without the script total memory is higher, but not that much. So maybe
I do not need the scripts to curtail memory usage. I save work and
when sometimes the application needs a lot more it is not curtailed.
Of-course 120 MB is a lot on an embedded device and when you would
have 20 Java applications it is going to count also, but at the moment
I am running on a desktop with enough memory and am not running scores
of Java applications.
I can always start limiting memory when it becomes tight and when I
also have some information about the usage …

Just to be sure: maxMemory cannot change during the run of a Java
application?


There is 28 MB difference between -Xmx and what maxMemory returns.
Where did it go?

-- 
Cecil Westerhof
Senior Software Engineer
LinkedIn: http://www.linkedin.com/in/cecilwesterhof