Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.programming.threads > #341 > unrolled thread

Fastest way to extract data from producer-consumer shared queue

Back to article view | Back to comp.programming.threads

Contents

  Fastest way to extract data from producer-consumer shared queue ittium <ittium@gmail.com> - 2011-06-21 03:13 -0700
    Re: Fastest way to extract data from producer-consumer shared queue "Balog Pal" <pasa@lib.hu> - 2011-06-21 16:53 +0200
      Re: Fastest way to extract data from producer-consumer shared queue ittium <ittium@gmail.com> - 2011-06-23 06:19 -0700
        Re: Fastest way to extract data from producer-consumer shared queue "Balog Pal" <pasa@lib.hu> - 2011-06-23 17:08 +0200

#341 — Fastest way to extract data from producer-consumer shared queue

Hi,
I am using condition varibles to share data (STL queue) between two
threads (producer and consumer). Following is the standard pseudo code

Consumer
--------------

While(1)
{

lock the mutex
if(queue not empty)
	extract the elements
else
	wait on condition variable
	Extract element from queue
unlock the mutex

process the extracted element

}


Producer
--------
lock the data
	insert data on queue
unlock

Processing on extracted element is very cpu intensive.

I am worried about extract element procedure. If I copy the elements
from queue to some other data structure (another queue or vector) then
this copy might become bottle neck.

if I extract one element each time then mutex contention between
producer and consumer can become bottleneck.

Please suggest what is the best way.

thanks
Ittium

[toc] | [next] | [standalone]

#345

"ittium" <ittium@gmail.com>

> I am worried about extract element procedure. If I copy the elements
> from queue to some other data structure (another queue or vector) then
> this copy might become bottle neck.

Why copy? Use list<unique_ptr<T>> for your queue. Or something equivalent.

> if I extract one element each time then mutex contention between
> producer and consumer can become bottleneck.

You said processing is the low thing. that means mutex operations will 
hardly be a problem.

But the other tech I used was to just swap() the queue with an empty local 
one. Works fine with simple list<T> and similar queues.

[toc] | [prev] | [next] | [standalone]

#358

On Jun 21, 7:53 pm, "Balog Pal" <p...@lib.hu> wrote:
> "ittium" <itt...@gmail.com>
>
> > I am worried about extract element procedure. If I copy the elements
> > from queue to some other data structure (another queue or vector) then
> > this copy might become bottle neck.
>
> Why copy? Use list<unique_ptr<T>> for your queue. Or something equivalent.
>
> > if I extract one element each time then mutex contention between
> > producer and consumer can become bottleneck.
>
> You said processing is the low thing. that means mutex operations will
> hardly be a problem.
>
> But the other tech I used was to just swap() the queue with an empty local
> one. Works fine with simple list<T> and similar queues.

thanks, looks great, so you are saying to put the data in link list
and while extracting just pass the head of link list. In producer,
start from zero elements link list again.

I am not sure if swapping is a efficient operation, efficiency of such
an operation might be O(n). If you think otherwise please send me some
relevant link.

Ittium

[toc] | [prev] | [next] | [standalone]

#359

"ittium" <ittium@gmail.com>
>I am not sure if swapping is a efficient operation, efficiency of such
>an operation might be O(n). If you think otherwise please send me some
>relevant link.

Swapping std:: or any other healthy containers is cheap and constant time. 
It only swaps a few pointers. 

[toc] | [prev] | [standalone]

Back to top | Article view | comp.programming.threads

csiph-web