Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.unix.programmer > #1091

Re: Non-deterministic behavior of multithreaded software

Show all headers | View raw

On 08-08-2011 AM 06:39, BGB wrote:
> On 8/7/2011 12:02 PM, Rainer Weikusat wrote:
>> BGB<cr88192@hotmail.com> writes:
>>> On 8/6/2011 6:59 AM, Rainer Weikusat wrote:
>>
>> [...]
>>
>>>> Context switches are a specific kind of operation taking
>>>> place when one thread 'passes work' to some other thread: Assuming the
>>>> server isn't already overloaded and actually buffering requests on the
>>>> queue because they arrive faster than the responses can be generated,
>>>> it means that a currently running thread needs to be put to sleep
>>>> despite it has all the information ('the request') necessary to
>>>> continue processing available while a currently sleeping thread that
>>>> doesn't have this information available needs to be awoken in order to
>>>> receive it so that it can do the processing. Technically, that's just a
>>>> nonsense operation because it is pure overhead. The advantage is that
>>>> the design becomes simpler when threads are used as 'specialized
>>>> execution units' as if they were independent parts of a real machine
>>>> design to perform a specific function and incapable of doing anything
>>>> else. But this limitation of physical things doesn't really exist for
>>>> threads: They are all created equal in the sense that every thread can
>>>> run all of the code any other thread can also run which implies that
>>>> the transport overhead implicit in the 'geographically distant
>>>> funtional units with different purposes' model can be avoided.
>>>
>>> I have before measured the time to go to sleep, switch threads, and
>>> get control back, to typically be about 50-100ns or so on modern
>>> systems (granted, this was in the case of multi-threaded programs on
>>> Windows).
>>>
>>> (the above is in contrast to the 1-2us needed to lock a mutex object,
>>> for whatever reason it takes this long...).
>>
>> According to a very simplistic experiment I just made, namely,
>>
>> ----------
>> #include<pthread.h>
>>
>> unsigned a;
>>
>> int main(void)
>> {
>> pthread_mutex_t lock;
>> unsigned b;
>>
>> b = 0xfffffff;
>> while (--b) {
>> pthread_mutex_lock(&lock);
>> nop(++a);
>> pthread_mutex_unlock(&lock);
>> }
>>
>> return a;
>> }
>> ----------
>>
>> with 'nop' being
>>
>> void nop(unsigned x) { }
>>
>> defined in another file in order to 'fool' the gcc optimizer, locking
>> and unlocking an uncontended mutex takes about 11.6ns on an Intel Core
>> i3 3.2Ghz-using computer. This means assuming your numbers are
>> accurate, the operation is an about an order of magnitude faster than
>> 'go to sleep, switch threads ...'. However, I don't quite understand
>> what you're up to, especially since passing a data item from one
>> thread to another 'fast' (ie, without copying it through the kernel)
>> will usually involve operations on a mutex and a condition variable on
>> both ends.
>>
>
> I should have written this more clearly (the context was assumed because
> the prior statement mentioned Windows and the following statement had
> parenthesis, and so was a side-note related to the former).
>
> I was talking at the moment about the "mutex object" on Windows (handled
> with "CreateMutex()" and "WaitForSingleObject()" API calls), which is,
> generally, *absurdly* slow... (why?... who knows...). it is well
> documented as being slow though (I guess the usual answer is because it
> does multiple-process synchronization or similar, but it is a mystery
> why this itself takes as long as it does).
>
> OTOH, Windows offers "critical section" support which is a good deal
> faster, but still not terribly fast (it is single process, but I have
> read that it tries to be "fair" by granting locks in a FIFO order, ...).
>
>
> the fastest option was generally to roll ones' own mutex support (in my
> case, essentially a spin-lock with sleeping), which gets performance
> fairly similar to your pthread-based example (on a 2.8GHz AthlonII).
>
>
>>> if the operation itself is something which takes, say,< 10us or so,
>>> then likely the granularity was too small, and a single thread (or
>>> multiple independent/parallel threads) would probably have made more
>>> sense.
>>
>> Also, I still don't understand what 'granularity' is supposed to mean
>> here.
>>
>
> the the size as the amount of time needed to complete a request.
>
> smaller granularity would mean a larger number of smaller requests.
> larger granularity would mean a smaller number of much larger requests.
>
> the relative cost of scheduling overhead/... would likely depend
> somewhat on the number, type, and size of the requests (scheduling
> overhead is much lower if the requests are relatively few but take a
> reasonably large amount of time to process).
>
>
>> [...]
>>
>>>>> also, I was assuming that the operations would be queued up and
>>>>> handled asynchronously. if synchronous handling is needed, well then,
>>>>> there is no good reason to be using multiple threads in this case, as
>>>>> multiple threads will not improve performance in this case (one thread
>>>>> will simply end up sitting idly waiting for the other to complete).
>>>>
>>>> Multiple threads can handle multiple requests 'synchronously' in
>>>> parallell if such parallell request handling is possible.
>>>
>>> potentially.
>>
>> Provided that multiple threads can run in parallell (because more than
>> a single 'execution unit' exists), they will be able to do parallell
>> request processing.
>
> yep.
>
>
> this is one case where using threads makes sense, but typically threads
> would be used in a more-or-less asynchronous manner (pass a bunch of
> requests and wait for them to start finishing), since if one does a
> synchronous operation (pass request, wait for it to complete, pass next
> request, ...), more time is wasted with one thread waiting for another
> thread to go do its thing, essentially eliminating much of the possible
> speedup (since time spent sleeping is not time spent doing something).
>
>
> or such...

Thanks BGB  and Rainer, for detailed discussion on this. I went through 
the whole threads, I think some more information will help in 
understanding my system better

We are finding out readyness of the fd through edge-triggered mechanism 
(epoll). All threads are equally likely to read the message using 
non-blocking call. We are currently targeting message based 
prototocol(sctp/udp) so no locks are needed. In future we might add 
support for byte stream protocol (like TCP), we might need mutex then. 
There is zero context switching in this model.

My software is handing the call session. The problem occur, if event for 
a call are delivered out of order. I have modified the software such 
that message belonging to a call are processed by same thread, in this 
way ordering of events for the call state machine is maintained. But 
this has introduced a pinner thread that assign the messages to 
different worker threads based on a a hash function that take input as 
call ID.

1. Thread ID may not be uniformly distributed, any logic of work 
division might load few thread.

2. A context switch is introduced because of pinner. Earlier a single 
thread was doing all the work (from reading from socket to delivering it 
to other modules.

Back to comp.unix.programmer | Previous | Next — Previous in thread | Find similar | Unroll thread

Thread

Non-deterministic behavior of multithreaded software itt ium <ittium@gmail.com> - 2011-08-05 02:38 -0700
  Re: Non-deterministic behavior of multithreaded software Rainer Weikusat <rweikusat@mssgmbh.com> - 2011-08-05 15:41 +0100
  Re: Non-deterministic behavior of multithreaded software BGB <cr88192@hotmail.com> - 2011-08-05 14:46 -0700
    Re: Non-deterministic behavior of multithreaded software Rainer Weikusat <rweikusat@mssgmbh.com> - 2011-08-05 23:04 +0100
      Re: Non-deterministic behavior of multithreaded software BGB <cr88192@hotmail.com> - 2011-08-05 18:54 -0700
        Re: Non-deterministic behavior of multithreaded software Rainer Weikusat <rweikusat@mssgmbh.com> - 2011-08-06 14:59 +0100
          Re: Non-deterministic behavior of multithreaded software BGB <cr88192@hotmail.com> - 2011-08-06 20:29 -0700
            Re: Non-deterministic behavior of multithreaded software Rainer Weikusat <rweikusat@mssgmbh.com> - 2011-08-07 20:02 +0100
              Re: Non-deterministic behavior of multithreaded software BGB <cr88192@hotmail.com> - 2011-08-07 18:09 -0700
                Re: Non-deterministic behavior of multithreaded software ittium <ittium@gmail.com> - 2011-08-09 09:48 +0530

csiph-web