Path: csiph.com!fu-berlin.de!uni-berlin.de!not-for-mail
From: Noah <noah-list@enabled.com>
Newsgroups: comp.lang.python
Subject: multiprocessing not quite working
Date: Tue, 24 May 2016 15:17:02 -0700
Lines: 62
Message-ID: <mailman.69.1464128684.20402.python-list@python.org>
References: <f803eff5-786e-703f-4625-69f5e64c6a6e@enabled.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:45.0) Gecko/20100101 Thunderbird/45.1.0
Precedence: list
Xref: csiph.com comp.lang.python:109091


Hi,

I am using this example: 
http://spartanideas.msu.edu/2014/06/20/an-introduction-to-parallel-programming-using-pythons-multiprocessing-module/

I am sending and receiving communication from the worker processes.

Two issues. the join is only getting to the process and waiting.
When I comment out the .join() process the output.get() appends the 
previous process and therefore the returned output keeps getting longer 
and longer after each process returns its output.

hostnames is an array of hostnames.

here is my code from main():

     # Define an output queue
     output = mp.Queue()

     # Setup a list of processes that we want to run
     processes = [mp.Process(target=worker, args=(hostnames[x], output)) 
for x in range(len(hostnames))]

     # Run processes
     for p in processes:
         print "start: {}".format(p)
         p.start()

     time.sleep(6)
     print "processes: {}".format(processes)

     # Exit the completed processes
     '''for p in processes:
         print "join: {}".format(p)
         p.join()'''

     print "got here"
     # Get process results from the output queue
     # results = [output.get() for p in processes]

     io = StringIO()
     count = 0
     for p in processes:
         found_output = output.get()
         print "returned {}".format(p)
         io.write (found_output)
         zipArchive.writestr(hostnames[count] + "." + 
content['sitename'] + '.config.txt', io.getvalue())
         count = count + 1
     io.close()


def worker(hostname, output):
.
.
.
         output.put(template_output)


Cheers