Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.java.programmer > #28634 > unrolled thread

why running multiple instances of the same program is faster than running single instance?

Back to article view | Back to comp.lang.java.programmer

Contents

  why running multiple instances of the same program is faster than running single instance? www <xsli2@yahoo.com> - 2014-06-02 08:12 -0700
    Re: why running multiple instances of the same program is faster than running single instance? Joerg Meier <joergmmeier@arcor.de> - 2014-06-02 23:59 +0200
      Re: why running multiple instances of the same program is faster than running single instance? Arivald <arivald_@AT_interia_DOT_.pl> - 2014-06-03 13:25 +0200
    Re: why running multiple instances of the same program is faster than running single instance? Josip Almasi <joe@vrspace.org> - 2014-06-03 09:40 +0200
    Re: why running multiple instances of the same program is faster than running single instance? lipska the kat <lipskathekat@example.com> - 2014-06-03 10:02 +0100
    Re: why running multiple instances of the same program is faster than running single instance? Rupert Smith <rupertlssmith@googlemail.com> - 2014-06-03 03:24 -0700
      Re: why running multiple instances of the same program is faster than running single instance? Rupert Smith <rupertlssmith@googlemail.com> - 2014-06-03 03:26 -0700
    Re: why running multiple instances of the same program is faster than running single instance? Roedy Green <see_website@mindprod.com.invalid> - 2014-06-06 19:50 -0700

#28634 — why running multiple instances of the same program is faster than running single instance?

Hi:

We have a software(some code from ours and some from the 3rd party) doing scientific calculation. It runs for several hours for finishing two years calculation. We use linux and it has four CPUs. Our code is not con-current multi-threaded yet.

We are puzzled that when we run multiple instances on the same linux computer, the run finishes earlier than running singe instance. When we run multiple instances, each instance is independently doing its own two year calculation. For example, one instance is calculating 1980-1981, another instance is calculating 1982-1983, etc. If running single instance for 1980-1981 takes 120 minutes to finish, when running together with other instances, it only needs 100 minutes to finish 1980-1981. 

Any suggestions why this is happening? Thank you very much.

[toc] | [next] | [standalone]

#28639

On Mon, 2 Jun 2014 08:12:31 -0700 (PDT), www wrote:

> We have a software(some code from ours and some from the 3rd party) doing scientific calculation. It runs for several hours for finishing two years calculation. We use linux and it has four CPUs. Our code is not con-current multi-threaded yet.

> We are puzzled that when we run multiple instances on the same linux computer, the run finishes earlier than running singe instance. When we run multiple instances, each instance is independently doing its own two year calculation. For example, one instance is calculating 1980-1981, another instance is calculating 1982-1983, etc. If running single instance for 1980-1981 takes 120 minutes to finish, when running together with other instances, it only needs 100 minutes to finish 1980-1981. 

> Any suggestions why this is happening? Thank you very much.

Without code or any more detail we can only speculate. My guess: you have a
bug or ill written piece in your code that slows down your progress and
additional instances somehow affect the bug. 

Liebe Gruesse,
		Joerg

-- 
Ich lese meine Emails nicht, replies to Email bleiben also leider
ungelesen.

[toc] | [prev] | [next] | [standalone]

#28644

W dniu 2014-06-02 23:59, Joerg Meier pisze:
> On Mon, 2 Jun 2014 08:12:31 -0700 (PDT), www wrote:
>
>> We have a software(some code from ours and some from the 3rd party) doing scientific calculation. It runs for several hours for finishing two years calculation. We use linux and it has four CPUs. Our code is not con-current multi-threaded yet.
>
>> We are puzzled that when we run multiple instances on the same linux computer, the run finishes earlier than running singe instance. When we run multiple instances, each instance is independently doing its own two year calculation. For example, one instance is calculating 1980-1981, another instance is calculating 1982-1983, etc. If running single instance for 1980-1981 takes 120 minutes to finish, when running together with other instances, it only needs 100 minutes to finish 1980-1981.
>
>> Any suggestions why this is happening? Thank you very much.
>
> Without code or any more detail we can only speculate. My guess: you have a
> bug or ill written piece in your code that slows down your progress and
> additional instances somehow affect the bug.
>


There could be a bug related to the intermediate data, if You save them 
on disk or other shared storage.
If both processes use same filenames, it may be that one process 
generated some data, then other one saw them already generated, then 
skipped to next task. But in this case both processes may used incorrect 
data for further processing, data belonging to the other process.

-- 
Arivald

[toc] | [prev] | [next] | [standalone]

#28640

On 06/02/2014 05:12 PM, www wrote:
> Hi:
>
> We have a software(some code from ours and some from the 3rd party) doing scientific calculation.
> It runs for several hours for finishing two years calculation. We use linux and it has four CPUs.
> Our code is not con-current multi-threaded yet.
>
> We are puzzled that when we run multiple instances on the same linux computer, the run finishes
> earlier than running singe instance. When we run multiple instances, each instance is independently
> doing its own two year calculation. For example, one instance is calculating 1980-1981, another
> instance is calculating 1982-1983, etc. If running single instance for 1980-1981 takes 120 minutes
> to finish, when running together with other instances, it only needs 100 minutes to finish 1980-1981.
>
> Any suggestions why this is happening? Thank you very much.

Maybe you're lucky, and read-ahead kicked in.
But I have no clue how to monitor this on linux.
I/O may be faster with multiple instances, either due to readahead, or 
due to dentry/inode caching.
Maybe slaptop and vmstat provide some useful info.

Regards...

[toc] | [prev] | [next] | [standalone]

#28641

On 02/06/14 16:12, www wrote:
> Hi:
>
> We have a software(some code from ours and some from the 3rd party) doing scientific calculation. It runs for several hours for finishing two years calculation. We use linux and it has four CPUs. Our code is not con-current multi-threaded yet.
>
> We are puzzled that when we run multiple instances on the same linux computer, the run finishes earlier than running singe instance. When we run multiple instances, each instance is independently doing its own two year calculation. For example, one instance is calculating 1980-1981, another instance is calculating 1982-1983, etc. If running single instance for 1980-1981 takes 120 minutes to finish, when running together with other instances, it only needs 100 minutes to finish 1980-1981.
>
> Any suggestions why this is happening? Thank you very much.
>

Might it have something to do with the IO overhead of paging?

If you have a single process with a given amount of memory that runs for 
a long time then eventually some paging will occur.

If you have multiple processes with a (similar?) given amount of memory 
then there will be less need for paging, what happens if you increase 
the heap space of the single process (at least double) and reduce the 
heap space of the multiple processes (by half)

-- 
lipska the kat - treacherous feline.
Proudly nominated for IPOTY 2014 - LIGAF
GNU/Linux user #560883 - linuxcounter.net

[toc] | [prev] | [next] | [standalone]

#28642

On Monday, June 2, 2014 4:12:31 PM UTC+1, www wrote:
> Hi:
> 
> 
> 
> We have a software(some code from ours and some from the 3rd party) doing scientific calculation. It runs for several hours for finishing two years calculation. We use linux and it has four CPUs. Our code is not con-current multi-threaded yet.
> 
> 
> 
> We are puzzled that when we run multiple instances on the same linux computer, the run finishes earlier than running singe instance. When we run multiple instances, each instance is independently doing its own two year calculation. For example, one instance is calculating 1980-1981, another instance is calculating 1982-1983, etc. If running single instance for 1980-1981 takes 120 minutes to finish, when running together with other instances, it only needs 100 minutes to finish 1980-1981. 
> 
> 
> 
> Any suggestions why this is happening? Thank you very much.

Its definitely possible, and not necessarily anything to do with Java.

If your processes read from files, they could be IO bound, and not using much CPU, resulting in the CPU running at a lower clock speed. Running a second one may be using more CPU, pushing up the clock speed.

A hyper-threaded CPU will be more efficient with 2 single-threaded processes running, as pipeline stalls on one process may get filled in with the other process doing useful work.

Rupert

[toc] | [prev] | [next] | [standalone]

#28643

On Tuesday, June 3, 2014 11:24:38 AM UTC+1, Rupert Smith wrote:
> If your processes read from files, they could be IO bound, and not using much CPU, resulting in the CPU running at a lower clock speed. Running a second one may be using more CPU, pushing up the clock speed.
> 
> 
> 
> A hyper-threaded CPU will be more efficient with 2 single-threaded processes running, as pipeline stalls on one process may get filled in with the other process doing useful work.

Try turning hyper-threading off in the BIOS, and setting the CPU to always run at maximum clock speed (performance mode), and see how they compare then.

Rupert

[toc] | [prev] | [next] | [standalone]

#28679

On Mon, 2 Jun 2014 08:12:31 -0700 (PDT), www <xsli2@yahoo.com> wrote,
quoted or indirectly quoted someone who said :

>Any suggestions why this is happening? Thank you very much.

I am not following what your anomaly is.

Some things to consider.

You would only expect it to run twice as fast if both instances have
all the ram they want.  If you run threads, code can be shared rather
than two separate JVMs which will have two copies of the app in ram.

You would expect it to run two run twice as fast if i/o is not the
limiting factor.  You need to experiment with buffer size to find the
optimal.

Use the control panel monitor to see how busy ram and each core is.

It can happen that the CPU has turned all but one core off. Make sure
that has not happened.

You may need to up your physical ram to exploit the extra cores.
-- 
Roedy Green Canadian Mind Products http://mindprod.com
In former times, people who craved absolute power became gangsters.
Today, they become computer programmers.

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.java.programmer

csiph-web