Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > comp.soft-sys.math.mathematica > #8304
| From | Szabolcs Horvát <szhorvat@gmail.com> |
|---|---|
| Newsgroups | comp.soft-sys.math.mathematica |
| Subject | Dynamically launching more kernels *during* a parallel calculation -- is it safe? |
| Date | 2012-02-02 10:06 +0000 |
| Organization | Steven M. Christensen and Associates, Inc and MathTensor, Inc. |
| Message-ID | <jgdn69$qq4$1@smc.vnet.net> (permalink) |
It appears that it is possible to launch more kernels *during* a
parallel calculation, and they will be utilized for the rest of the
calculation.
Here's a simple way to test and illustrate (in a fresh kernel):
LaunchKernels[2]
Parallelize[
Table[Pause@RandomInteger[20]; $KernelID, {i, 8}],
Method -> "FinestGrained"
]
Now use Evaluation -> Interrupt Evaluation... to interrupt the
evaluation and go into a subsession. In the subsession evaluate
$KernelCount (* check that we only have 2 kernels *)
LaunchKernels[4] (* launch 4 more *)
Return (* return from the dialog *)
Notice that when the Table[] finally finishes, we get results between
1 and 6. This means that all 6 kernels have been used for the rest of
the evaluation, even though some kernels were only launched in the
middle of the calculation.
Question:
Is it safe to do this? Can this break anything?
Consider some code like this:
manageKernels[] :=(* uses LaunchKernels[] to launch more is compute
resources become available *)
SetSharedFunction[manageKernels] (* ensure that manageKernels[] is
always evaluated on the master kernel *)
Parallelize[
Table[manageKernels[]; compute[i], {i, 100}],
Method -> "FinestGrained"
]
Is this going to be safe? What if manageKernels[] uses CloseKernels[] as well?
Back to comp.soft-sys.math.mathematica | Previous | Next | Find similar | Unroll thread
Dynamically launching more kernels *during* a parallel calculation -- is it safe? Szabolcs Horvát <szhorvat@gmail.com> - 2012-02-02 10:06 +0000
csiph-web