Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]


Groups > comp.soft-sys.math.mathematica > #8304

Dynamically launching more kernels *during* a parallel calculation -- is it safe?

From Szabolcs Horvát <szhorvat@gmail.com>
Newsgroups comp.soft-sys.math.mathematica
Subject Dynamically launching more kernels *during* a parallel calculation -- is it safe?
Date 2012-02-02 10:06 +0000
Organization Steven M. Christensen and Associates, Inc and MathTensor, Inc.
Message-ID <jgdn69$qq4$1@smc.vnet.net> (permalink)

Show all headers | View raw


It appears that it is possible to launch more kernels *during* a
parallel calculation, and they will be utilized for the rest of the
calculation.

Here's a simple way to test and illustrate (in a fresh kernel):

LaunchKernels[2]

Parallelize[
 Table[Pause@RandomInteger[20]; $KernelID, {i, 8}],
 Method -> "FinestGrained"
]

Now use Evaluation -> Interrupt Evaluation... to interrupt the
evaluation and go into a subsession.  In the subsession evaluate

$KernelCount (* check that we only have 2 kernels *)

LaunchKernels[4] (* launch 4 more *)

Return (* return from the dialog *)

Notice that when the Table[] finally finishes, we get results between
1 and 6.  This means that all 6 kernels have been used for the rest of
the evaluation, even though some kernels were only launched in the
middle of the calculation.

Question:

Is it safe to do this?  Can this break anything?

Consider some code like this:

manageKernels[] :=(* uses LaunchKernels[] to launch more is compute
resources become available *)

SetSharedFunction[manageKernels] (* ensure that manageKernels[] is
always evaluated on the master kernel *)

Parallelize[
 Table[manageKernels[]; compute[i], {i, 100}],
 Method -> "FinestGrained"
]

Is this going to be safe?  What if manageKernels[] uses CloseKernels[] as well?

Back to comp.soft-sys.math.mathematica | Previous | Next | Find similar | Unroll thread


Thread

Dynamically launching more kernels *during* a parallel calculation -- is it safe? Szabolcs Horvát <szhorvat@gmail.com> - 2012-02-02 10:06 +0000

csiph-web