Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.soft-sys.math.mathematica > #16803 > unrolled thread

Tracking progress in ParallelDo

Back to article view | Back to comp.soft-sys.math.mathematica

Contents

  Tracking progress in ParallelDo Brian Beckage <Brian.Beckage@uvm.edu> - 2014-04-14 09:28 +0000
    Re: Tracking progress in ParallelDo Szabolcs Horvát <szhorvat@gmail.com> - 2014-04-17 09:10 +0000

#16803 — Tracking progress in ParallelDo

I'm applying a function repeatedly using ParallelDo.  I would like to keep track of its progress.  I've tried using both my own counter variable as well as using the increments within the ParallelDo function (below). Neither provides a way of sequentially tracking how many of the iterations have been spawned (not completed--but just kicked off). 

How could I track that iteration 1, 2, 3, 4, 5 etc. have been started?  Is there a better way of monitoring progress of the ParalleDo?

Thanks for your help!

Brian


USING MY OWN COUNTER: myCounter:

parallelBootstrapFireMultiplier[nBoot_, fireMultiplierRange_, parmList1_,
  nYears_, initLandscapeProbs_] := Module[{outIter, myCounter},

  ParallelEvaluate[Clear[myCounter]];
  myCounter = 0;
  SetSharedVariable[myCounter];
  Reap[
   ParallelDo[
    myCounter++;
    Print["Starting parallelBootstrapMultiplier step ", myCounter, " out of ",
      nBoot];
    outIter =
     iterFireMultiplier[fireMultiplierRange, parmList1, nYears,
      initLandscapeProbs];
    ParallelSow[outIter],
    {nBoot}]
   ]
  ]

RESULTS in:

(kernel 2) Starting parallelBootstrapMultiplier step 2 out of 6
(kernel 1) Starting parallelBootstrapMultiplier step 2 out of 6
(kernel 1) Starting parallelBootstrapMultiplier step 4 out of 6
(kernel 2) Starting parallelBootstrapMultiplier step 4 out of 6
(kernel 2) Starting parallelBootstrapMultiplier step 5 out of 6
(kernel 1) Starting parallelBootstrapMultiplier step 6 out of 6

USING THE INCREMENT i FROM ParallelDo:

parallelBootstrapFireMultiplier[nBoot_,fireMultiplierRange_,parmList1_,nYears_,initLandscapeProbs_]:= Module[{outIter},
Reap[
ParallelDo[
Print["Starting parallelBootstrapMultiplier step ",i, " out of ",nBoot];
outIter=iterFireMultiplier[fireMultiplierRange,parmList1,nYears,initLandscapeProbs]; (*This returns a list*)
ParallelSow[outIter],
{i,nBoot}]
]
]

RESULTS in:
Starting parallelBootstrapMultiplier step 1 out of 6
Starting parallelBootstrapMultiplier step 4 out of 6
Starting parallelBootstrapMultiplier step 5 out of 6
Starting parallelBootstrapMultiplier step 2 out of 6
Starting parallelBootstrapMultiplier step 3 out of 6
Starting parallelBootstrapMultiplier step 6 out of 6

[toc] | [next] | [standalone]

#16819

There was a very similar question asked on StackExchange.  You might be 
interested in the answers there.

http://mathematica.stackexchange.com/questions/1548/monitor-doesnt-work-with-paralleltable

Here's a slightly modified version of Leonid's solution:

SetSharedVariable[progress]

progress = 0;
Monitor[
  ParallelTable[progress++; Pause[0.5]; i, {i, 100}],
  progress
]

The caveat here is that SetSharedVariable causes 'progress' to be always 
evaluated on the main kernel.  Accessing the main kernel is expensive 
(it takes a long time).  So this only makes sense if:

  - it is reasonable to break the parallel evaluation into the smallest 
possible pieces (equivalent of Method -> "FinestGrained")

  - each single evaluation in your ParallelDo takes considerably longer 
than the overhead due to the callback to the main kernel

Otherwise doing this will kill performance.  Personally I do not 
recommend doing this unless you are experienced with parallelization in 
Mathematica and understand what factors affect performance.

Do check my link for more complex but also more efficient variations on 
the same idea (Eli Lansey's answer).

Szabolcs


On 2014-4-14, 5:28 , Brian Beckage wrote:
> I'm applying a function repeatedly using ParallelDo.  I would like to keep track of its progress.  I've tried using both my own counter variable as well as using the increments within the ParallelDo function (below). Neither provides a way of sequentially tracking how many of the iterations have been spawned (not completed--but just kicked off).
>
> How could I track that iteration 1, 2, 3, 4, 5 etc. have been started?  Is there a better way of monitoring progress of the ParalleDo?
>
> Thanks for your help!
>
> Brian
>
>
> USING MY OWN COUNTER: myCounter:
>
> parallelBootstrapFireMultiplier[nBoot_, fireMultiplierRange_, parmList1_,
>    nYears_, initLandscapeProbs_] := Module[{outIter, myCounter},
>
>    ParallelEvaluate[Clear[myCounter]];
>    myCounter = 0;
>    SetSharedVariable[myCounter];
>    Reap[
>     ParallelDo[
>      myCounter++;
>      Print["Starting parallelBootstrapMultiplier step ", myCounter, " out of ",
>        nBoot];
>      outIter =
>       iterFireMultiplier[fireMultiplierRange, parmList1, nYears,
>        initLandscapeProbs];
>      ParallelSow[outIter],
>      {nBoot}]
>     ]
>    ]
>
> RESULTS in:
>
> (kernel 2) Starting parallelBootstrapMultiplier step 2 out of 6
> (kernel 1) Starting parallelBootstrapMultiplier step 2 out of 6
> (kernel 1) Starting parallelBootstrapMultiplier step 4 out of 6
> (kernel 2) Starting parallelBootstrapMultiplier step 4 out of 6
> (kernel 2) Starting parallelBootstrapMultiplier step 5 out of 6
> (kernel 1) Starting parallelBootstrapMultiplier step 6 out of 6
>
> USING THE INCREMENT i FROM ParallelDo:
>
> parallelBootstrapFireMultiplier[nBoot_,fireMultiplierRange_,parmList1_,nYears_,initLandscapeProbs_]:= Module[{outIter},
> Reap[
> ParallelDo[
> Print["Starting parallelBootstrapMultiplier step ",i, " out of ",nBoot];
> outIter=iterFireMultiplier[fireMultiplierRange,parmList1,nYears,initLandscapeProbs]; (*This returns a list*)
> ParallelSow[outIter],
> {i,nBoot}]
> ]
> ]
>
> RESULTS in:
> Starting parallelBootstrapMultiplier step 1 out of 6
> Starting parallelBootstrapMultiplier step 4 out of 6
> Starting parallelBootstrapMultiplier step 5 out of 6
> Starting parallelBootstrapMultiplier step 2 out of 6
> Starting parallelBootstrapMultiplier step 3 out of 6
> Starting parallelBootstrapMultiplier step 6 out of 6
>

[toc] | [prev] | [standalone]

Back to top | Article view | comp.soft-sys.math.mathematica

csiph-web