Path: csiph.com!x330-a1.tempe.blueboxinc.net!usenet.pasdenom.info!news.dougwise.org!news-transit.tcx.org.uk!de-l.enfer-du-nord.net!feeder1.enfer-du-nord.net!feeds.phibee-telecom.net!newsfeed.kpn.net!pfeed08.wxs.nl!newsfeed.xs4all.nl!newsfeed5.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail
Content-Type: text/plain; charset=us-ascii
Mime-Version: 1.0 (Apple Message framework v1082)
Subject: Re: multiprocessing
From: Philip Semanchuk <philip@semanchuk.com>
In-Reply-To: <COL113-W293751B1719D5A92C6FCBDDCA70@phx.gbl>
Date: Thu, 7 Apr 2011 21:05:23 -0400
Content-Transfer-Encoding: quoted-printable
References: <75a6974b-d954-4528-8bbc-9ab7b6b8b80b@f11g2000vbx.googlegroups.com>, <BANLkTinjS8+dHt4eGDjjLfJogiJeaX7+qg@mail.gmail.com> <COL113-W293751B1719D5A92C6FCBDDCA70@phx.gbl>
To: Python list <python-list@python.org>
Precedence: list
Newsgroups: comp.lang.python
Message-ID: <mailman.127.1302224729.9059.python-list@python.org>
Lines: 123
NNTP-Posting-Host: 82.94.164.166
Xref: x330-a1.tempe.blueboxinc.net comp.lang.python:2820


On Apr 7, 2011, at 8:57 PM, Kerensa McElroy wrote:

>=20
> Hi,
>=20
> thanks for your response.
>=20
> I checked out multiprocessing.value, however from what I can make out, =
it works with object of only a very limited type. Is there a way to do =
this for more complex objects? (In reality, my object is a large =
multi-dimensional numpy array).

Elsa,=20
Are you following the current thread in this list which is talking about =
sharing numpy arrays via multiprocessing?

http://mail.python.org/pipermail/python-list/2011-April/1269173.html





> Date: Wed, 6 Apr 2011 22:20:06 -0700
> Subject: Re: multiprocessing
> From: drsalists@gmail.com
> To: kerensaelise@hotmail.com
> CC: python-list@python.org
>=20
>=20
> On Wed, Apr 6, 2011 at 9:06 PM, elsa <kerensaelise@hotmail.com> wrote:
>=20
> Hi guys,
>=20
>=20
>=20
> I want to try out some pooling of processors, but I'm not sure if it
>=20
> is possible to do what I want to do. Basically, I want to have a
>=20
> global object, that is updated during the execution of a function, and
>=20
> I want to be able to run this function several times on parallel
>=20
> processors. The order in which the function runs doesn't matter, and
>=20
> the value of the object doesn't matter to the function, but I do want
>=20
> the processors to take turns 'nicely' when updating the object, so
>=20
> there are no collisions. Here is an extremely simplified and trivial
>=20
> example of what I have in mind:
>=20
>=20
>=20
> from multiprocessing import Pool
>=20
> import random
>=20
>=20
>=20
> p=3DPool(4)
>=20
> myDict=3D{}
>=20
>=20
>=20
> def update(value):
>=20
>    global myDict
>=20
>    index=3Drandom.random()
>=20
>    myDict[index]+=3Dvalue
>=20
>=20
>=20
> total=3D1000
>=20
>=20
>=20
> p.map(update,range(total))
>=20
>=20
>=20
>=20
>=20
> After, I would also like to be able to use several processors to
>=20
> access the global object (but not modify it). Again, order doesn't
>=20
> matter:
>=20
>=20
>=20
> p1=3DPool(4)
>=20
>=20
>=20
> def getValues(index):
>=20
>    global myDict
>=20
>    print myDict[index]
>=20
>=20
>=20
> p1.map(getValues,keys.myDict)
>=20
>=20
>=20
> Is there a way to do this=20
> This should give you a synchronized wrapper around an object in shared =
memory:
>=20
> =
http://docs.python.org/library/multiprocessing.html#multiprocessing.Value
>=20
>=20
> 		 	   		  --=20
> http://mail.python.org/mailman/listinfo/python-list