Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.programming > #2272 > unrolled thread

Threadpool engien and scalability [1]

Back to article view | Back to comp.programming

Contents

  Threadpool engien and scalability [1] "aminer" <aminer@videotron.ca> - 2012-10-01 12:59 -0500
    Re: Threadpool engien and scalability [1] "aminer" <aminer@videotron.ca> - 2012-10-01 13:34 -0500
    Re: Threadpool engien and scalability [1] "aminer" <aminer@videotron.ca> - 2012-10-01 13:48 -0500
    Re: Threadpool engien and scalability [1] "aminer" <aminer@videotron.ca> - 2012-10-01 14:02 -0500

#2272 — Threadpool engien and scalability [1]

Hello,

I have corrected my mistakes , please read again:

If you take a look at my threadpool engine:

http://pages.videotron.com/aminer/

you will notice that it does not scale in the execute() method
cause LockedIncLong(balance1) is expensive.

here is the execute method:

==

function TThreadPool.execute(func:TMyProc;const Context: Pointer): Boolean;

var
local_balance,local_count:long;

params: PParams;
p:pointer;

begin
new(params);
setlength(params^,1);
params^[0].proc:=func;
params^[0].data:=context;

local_balance:=LockedIncLong(balance1) mod FThreadCount;
while not Queues[local_balance].push(tobject(params)) do sleep(0);
events[local_balance].setevent;
end;

===

Now to be able to scale the execute() method we have to add a new
execute1() method that will be called from mutiple threads, i have to
use LockedIncLong() but it will be called only once:

==

function TThreadPool.execute1(func:TMyProc;const Context:
Pointer;index:long): Boolean;

params: PParams;
p:pointer;

begin
new(params);
setlength(params^,1);
params^[0].proc:=func;
params^[0].data:=context;

if index = 0
then
begin
index:=LockedIncLong(balance1) mod FThreadCount;
end;

while not Queues[index].push(tobject(params)) do sleep(0);
events[index].setevent;
end;

=


so we have to call the execute1() method from mutiple threads like this:

and we have to initialize index to zero before:, like this:

ThreadPool.execute1(myproc,data,index)

I am using mutiple lockfree FIFO queues inside my threadpool engine,
so that there is less contention and using work-stealing also, so after
adding execute1() methods , this threadpool engine will become scalable.
this threadpool engine can also be used as a scalable queue
(not a strict FIFO queue, but it's also useful).

This threadpool engine is useful and that is what's important.



I will try to update my threadpol engine soon.



Thank you,
Amine Moulay Ramdane.

[toc] | [next] | [standalone]

#2273

Hello,

As you will notice, inside the new exucuted1() i have used the same method
as with my implementation of the Distributed Reader-Writer Mutex 1.04
and i am using a variable number of mrews to lower the contention
more in my implementation of Distributed Reader-Writer Mutex 1.04
based on the Dmitry Vyukov C++ Distributed Reader-Writer Mutex,
and as i said i am using mutiple lockfree FIFO queues inside my threadpool 
engine,
so that there is less contention and using work-stealing also, so after 
adding the execute1()
method , this threadpool engine will become scalable and this threadpool 
engine can also
be used as a scalable queue(not a strict FIFO queue, but almost, and it will 
be also useful).

This threadpool engine is useful and that is what's important.

I will try to update my threadpol engine soon.

You can download Distributed Reader-Writer Mutex and my other paralle 
libraries
from:

http://pages.videotron.com/aminer/




Amine Moulay Ramdane.



"aminer" <aminer@videotron.ca> wrote in message 
news:k4ci68$8ho$1@dont-email.me...
> Hello,
>
> I have corrected my mistakes , please read again:
>
> If you take a look at my threadpool engine:
>
> http://pages.videotron.com/aminer/
>
> you will notice that it does not scale in the execute() method
> cause LockedIncLong(balance1) is expensive.
>
> here is the execute method:
>
> ==
>
> function TThreadPool.execute(func:TMyProc;const Context: Pointer): 
> Boolean;
>
> var
> local_balance,local_count:long;
>
> params: PParams;
> p:pointer;
>
> begin
> new(params);
> setlength(params^,1);
> params^[0].proc:=func;
> params^[0].data:=context;
>
> local_balance:=LockedIncLong(balance1) mod FThreadCount;
> while not Queues[local_balance].push(tobject(params)) do sleep(0);
> events[local_balance].setevent;
> end;
>
> ===
>
> Now to be able to scale the execute() method we have to add a new
> execute1() method that will be called from mutiple threads, i have to
> use LockedIncLong() but it will be called only once:
>
> ==
>
> function TThreadPool.execute1(func:TMyProc;const Context:
> Pointer;index:long): Boolean;
>
> params: PParams;
> p:pointer;
>
> begin
> new(params);
> setlength(params^,1);
> params^[0].proc:=func;
> params^[0].data:=context;
>
> if index = 0
> then
> begin
> index:=LockedIncLong(balance1) mod FThreadCount;
> end;
>
> while not Queues[index].push(tobject(params)) do sleep(0);
> events[index].setevent;
> end;
>
> =
>
>
> so we have to call the execute1() method from mutiple threads like this:
>
> and we have to initialize index to zero before:, like this:
>
> ThreadPool.execute1(myproc,data,index)
>
> I am using mutiple lockfree FIFO queues inside my threadpool engine,
> so that there is less contention and using work-stealing also, so after
> adding execute1() methods , this threadpool engine will become scalable.
> this threadpool engine can also be used as a scalable queue
> (not a strict FIFO queue, but it's also useful).
>
> This threadpool engine is useful and that is what's important.
>
>
>
> I will try to update my threadpol engine soon.
>
>
>
> Thank you,
> Amine Moulay Ramdane.
>
>
>
> 

[toc] | [prev] | [next] | [standalone]

#2274

I have wrote my parallel libraries with love and i have
given them with love , my libraries look like the following
song of Bob Marley, and i am a socialist in the deep of
my soul and my heart:


http://www.youtube.com/watch?v=CHekNnySAfM



Amine Moulay Ramdane.


"aminer" <aminer@videotron.ca> wrote in message 
news:k4ci68$8ho$1@dont-email.me...
> Hello,
>
> I have corrected my mistakes , please read again:
>
> If you take a look at my threadpool engine:
>
> http://pages.videotron.com/aminer/
>
> you will notice that it does not scale in the execute() method
> cause LockedIncLong(balance1) is expensive.
>
> here is the execute method:
>
> ==
>
> function TThreadPool.execute(func:TMyProc;const Context: Pointer): 
> Boolean;
>
> var
> local_balance,local_count:long;
>
> params: PParams;
> p:pointer;
>
> begin
> new(params);
> setlength(params^,1);
> params^[0].proc:=func;
> params^[0].data:=context;
>
> local_balance:=LockedIncLong(balance1) mod FThreadCount;
> while not Queues[local_balance].push(tobject(params)) do sleep(0);
> events[local_balance].setevent;
> end;
>
> ===
>
> Now to be able to scale the execute() method we have to add a new
> execute1() method that will be called from mutiple threads, i have to
> use LockedIncLong() but it will be called only once:
>
> ==
>
> function TThreadPool.execute1(func:TMyProc;const Context:
> Pointer;index:long): Boolean;
>
> params: PParams;
> p:pointer;
>
> begin
> new(params);
> setlength(params^,1);
> params^[0].proc:=func;
> params^[0].data:=context;
>
> if index = 0
> then
> begin
> index:=LockedIncLong(balance1) mod FThreadCount;
> end;
>
> while not Queues[index].push(tobject(params)) do sleep(0);
> events[index].setevent;
> end;
>
> =
>
>
> so we have to call the execute1() method from mutiple threads like this:
>
> and we have to initialize index to zero before:, like this:
>
> ThreadPool.execute1(myproc,data,index)
>
> I am using mutiple lockfree FIFO queues inside my threadpool engine,
> so that there is less contention and using work-stealing also, so after
> adding execute1() methods , this threadpool engine will become scalable.
> this threadpool engine can also be used as a scalable queue
> (not a strict FIFO queue, but it's also useful).
>
> This threadpool engine is useful and that is what's important.
>
>
>
> I will try to update my threadpol engine soon.
>
>
>
> Thank you,
> Amine Moulay Ramdane.
>
>
>
> 

[toc] | [prev] | [next] | [standalone]

#2275

It's a socialist soldier from in the heart of America !


http://www.youtube.com/watch?v=S5FCdx7Dn0o




Amine Moulay Ramdane.


"aminer" <aminer@videotron.ca> wrote in message 
news:k4ci68$8ho$1@dont-email.me...
> Hello,
>
> I have corrected my mistakes , please read again:
>
> If you take a look at my threadpool engine:
>
> http://pages.videotron.com/aminer/
>
> you will notice that it does not scale in the execute() method
> cause LockedIncLong(balance1) is expensive.
>
> here is the execute method:
>
> ==
>
> function TThreadPool.execute(func:TMyProc;const Context: Pointer): 
> Boolean;
>
> var
> local_balance,local_count:long;
>
> params: PParams;
> p:pointer;
>
> begin
> new(params);
> setlength(params^,1);
> params^[0].proc:=func;
> params^[0].data:=context;
>
> local_balance:=LockedIncLong(balance1) mod FThreadCount;
> while not Queues[local_balance].push(tobject(params)) do sleep(0);
> events[local_balance].setevent;
> end;
>
> ===
>
> Now to be able to scale the execute() method we have to add a new
> execute1() method that will be called from mutiple threads, i have to
> use LockedIncLong() but it will be called only once:
>
> ==
>
> function TThreadPool.execute1(func:TMyProc;const Context:
> Pointer;index:long): Boolean;
>
> params: PParams;
> p:pointer;
>
> begin
> new(params);
> setlength(params^,1);
> params^[0].proc:=func;
> params^[0].data:=context;
>
> if index = 0
> then
> begin
> index:=LockedIncLong(balance1) mod FThreadCount;
> end;
>
> while not Queues[index].push(tobject(params)) do sleep(0);
> events[index].setevent;
> end;
>
> =
>
>
> so we have to call the execute1() method from mutiple threads like this:
>
> and we have to initialize index to zero before:, like this:
>
> ThreadPool.execute1(myproc,data,index)
>
> I am using mutiple lockfree FIFO queues inside my threadpool engine,
> so that there is less contention and using work-stealing also, so after
> adding execute1() methods , this threadpool engine will become scalable.
> this threadpool engine can also be used as a scalable queue
> (not a strict FIFO queue, but it's also useful).
>
> This threadpool engine is useful and that is what's important.
>
>
>
> I will try to update my threadpol engine soon.
>
>
>
> Thank you,
> Amine Moulay Ramdane.
>
>
>
> 

[toc] | [prev] | [standalone]

Back to top | Article view | comp.programming

csiph-web