Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.unix.programmer > #947 > unrolled thread

Non blocking socket operations in a multithreaded solution

Back to article view | Back to comp.unix.programmer

Contents

  Non blocking socket operations in a multithreaded solution itt ium <ittium@gmail.com> - 2011-07-15 03:10 -0700
    Re: Non blocking socket operations in a multithreaded solution David Schwartz <davids@webmaster.com> - 2011-07-15 08:19 -0700
    Re: Non blocking socket operations in a multithreaded solution Rainer Weikusat <rweikusat@mssgmbh.com> - 2011-07-15 22:08 +0100

#947 — Non blocking socket operations in a multithreaded solution

I have a multithreaded program where various thread uses socket for
sending and receiving data from network. I would not like to block my
threads so I am thinking of following solution.

Solution written below are for a single socket connection used for
send and recv. In future, I might like to extend it for handling
multiple connections.

--------------
Solution 1
---------------
Keep a thread that does following

Mark the socket as non blocking, keep a send buffer
while(1)
{
	if (data in send buffer && socket writable)
		send the data. This call may send less data than available so adjust
the pointer in send buffer accordingly

	if(socket readable)
		read the data from socket process it, if you need to send a message
as part of processing put the data at the end of send Q

	call select/poll/epoll with socket fd to be polled for reading  and
writing both

}

Advatage
---------------
	Above solution can be easily extended to multiple socket fds
(connection). One send buffer per socket will be needed.
Problems
--------------
One possible problem that I see here is following

After reading if I process the data in the same thread, socket will
get a chance to send only after received data has been processed. If
processing is involved socket queue might get full during the
processing time. I am thinking of putting processing in a separate
thread so that this thread only receive data and signal the processor
thread through some mechanism (like condition variable)
---------------
Solution 2
---------------
Keep the fd as blocking
Keep a separate thread for reading and writing
When some thread has to send data it writes the data to send Q and
signal the send thread
Dedicated recv thread recv the data and puts it on recv q and signal
some other processor(worker) thread.

This solution may not be appropriate for handling large number of fds
since, system may have issues in having large number thread.

I will appreciate your thought on this.

thanks
Ittium

[toc] | [next] | [standalone]

#949

On Jul 15, 3:10 am, itt ium <itt...@gmail.com> wrote:
> I have a multithreaded program where various thread uses socket for
> sending and receiving data from network. I would not like to block my
> threads so I am thinking of following solution.
[snip]

The ideal solution depends on a lot of factors. The biggest one is how
much effort you can justify putting into getting sockets "right". If
you're only handling a small number of connections or doing much more
processing than I/O, getting this perfect is wasted effort.

The "good enough" solution for most projects is to have one thread in
a big poll/select/epoll loop. Generally, you want that thread to do as
little other processing as possible so that it doesn't become a
bottleneck.

DS

[toc] | [prev] | [next] | [standalone]

#950

itt ium <ittium@gmail.com> writes:
> I have a multithreaded program where various thread uses socket for
> sending and receiving data from network. I would not like to block my
> threads so I am thinking of following solution.
>
> Solution written below are for a single socket connection used for
> send and recv. In future, I might like to extend it for handling
> multiple connections.

Another option (if the required facilities for that are available) is
to use an edge-triggered I/O readiness notification mechanism and a
bunch of thread blocking in the notification routine. Whenever
something becomes ready, one of the threads gets a notification and
then goes off doing I/O until it that is either completed or would
block at which point it goes back into the notification routine until
something else becomes ready.

If you need to handle an arbitrarily large number of 'descriptors
which are ready' with a limited number of threads, each 'working'
thread would need to maintain some list of 'descriptors which are
ready' and do round-robin I/O and (while it is still busy) do
occasional non-blocking calls into the notification routine in order
to pick up new 'ready descriptors' nobody is looking after so far.

[toc] | [prev] | [standalone]

Back to top | Article view | comp.unix.programmer

csiph-web