Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.programming.threads > #1442

Re: Symmetric threads and fair processing of fds

Show all headers | View raw

On Friday, March 22, 2013 11:16:54 AM UTC+5:30, itt ium wrote:
> Hi Group, 
> 
> I am developing a multithreaded software based on idea of symmetric thread suggested by  David Schwartz.
> 
>  Here all thread will run the same procedure. Thread can have following work to do
> 
> -	Handle timer message
> 
> -	Receive message from socket fd
> 
> -	Send message from send Q
> 
> -	Poll for work
> 
> We will have a shared work Q from where any thread can pick the work and start executing. If a free thread finds out, that there is no work available, it will poll for work.
> 
> 
> 
> While reading the data from fd, a thread will keep reading till it gets EAGAIN. 
> 
> The software runs fine when all the fds are getting similar amount of data.  When there is more data on some of the fds. Some threads are busy servicing same fd and  certain fds are not processed for long time. This leads to temporary buffer (e.g buffer used to keep message on getting EAGAIN) filling up making the system unstable. To resolve this issue I am now thinking about processing only configurable number of message whenever a (symmetric) thread picks up a work. Unfortunately, traffic on fds can not be predicted so it might takes lot of experiment to make it right for certain network conditions. Is there any other way to ensure (dynamically), symmetric threads process all the fds for fair amount of time. I will appreciate, if you can send me any discussion link, details of your previous experiences handling similar scenarios or algorithms that you might have developed for handing this scenario.
> 
> thanks
> 
> Ittium

Other way to look at the problem is, I have a pool of threads and pool of fds. Some of the fds have more data to read/write than other. I would like to assign threads in such a way, that fds with more data are processed by more number of threads (although, I will have to solve concurrency issues). This way more cpu cycles will be assigned to fd with more data to transfer.
One way to solve this issue is your information about the fds (which peer will send how much data) but this is static information and will work as long as fds behave as assumed. I want to develop some dynamic scheme if possible.

Back to comp.programming.threads | Previous | Next — Previous in thread | Find similar

Thread

Symmetric threads and fair processing of fds itt ium <ittium@gmail.com> - 2013-03-21 22:46 -0700
  Re: Symmetric threads and fair processing of fds itt ium <ittium@gmail.com> - 2013-03-22 20:56 -0700

csiph-web