Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #28895

Re: submit jobs on multi-core

Show all headers | View raw

On Mon, Sep 10, 2012 at 11:53 PM, Laszlo Nagy <gandalf@shopzeus.com> wrote:
> On 2012-09-11 06:16, Dhananjay wrote:
>>
>> Dear all,
>>
>> I have a python script in which I have a list of files to input one by one
>> and for each file I get a number as an output.
>> I used for loop to submit the file to script.
>> My script uses one file at a time and returns the output.
>>
>> My computers has 8 cores.
>> Is there any way that I could submit 8 jobs at a time and get all the
>> output faster ?
>> In other words, how can I modify my script so that I could submit 8 jobs
>> together on 8 different processors ?
>>
>> I am bit new to this stuff, please suggest me some directions.
>
> You should first look at the multiprocessing module. It is part of the
> standard library.
>
> http://docs.python.org/library/multiprocessing.html

Or if Python 3.2 is an option, the concurrent.futures module would be
very well suited for this task.

Back to comp.lang.python | Previous | Next | Find similar | Unroll thread

Thread

Re: submit jobs on multi-core Ian Kelly <ian.g.kelly@gmail.com> - 2012-09-11 12:41 -0600

csiph-web