Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.programming.threads > #534 > unrolled thread

Thread safe logging (or protecting global variables)

Back to article view | Back to comp.programming.threads

Contents

  Thread safe logging (or protecting global variables) Jef Driesen <jefdriesen@hotmail.com.invalid> - 2011-09-05 20:19 +0200
    Re: Thread safe logging (or protecting global variables) Edek <edek.pienkowski@gmail.com> - 2011-09-05 20:30 +0200
      Re: Thread safe logging (or protecting global variables) Jef Driesen <jefdriesen@hotmail.com.invalid> - 2011-09-06 18:48 +0200
        Re: Thread safe logging (or protecting global variables) Edek <edek.pienkowski@gmail.com> - 2011-09-07 10:34 +0200
          Re: Thread safe logging (or protecting global variables) Jef Driesen <jefdriesen@hotmail.com.invalid> - 2011-09-08 22:00 +0200
            Re: Thread safe logging (or protecting global variables) Edek <edek.pienkowski@gmail.com> - 2011-09-09 18:13 +0200
              Re: Thread safe logging (or protecting global variables) Jef Driesen <jefdriesen@hotmail.com.invalid> - 2011-09-13 17:58 +0200
                Re: Thread safe logging (or protecting global variables) Joshua Maurice <joshuamaurice@gmail.com> - 2011-09-13 13:18 -0700
                  Re: Thread safe logging (or protecting global variables) Jef Driesen <jefdriesen@hotmail.com.invalid> - 2011-09-14 07:16 +0200
                    Re: Thread safe logging (or protecting global variables) Joshua Maurice <joshuamaurice@gmail.com> - 2011-09-14 13:18 -0700
                  Re: Thread safe logging (or protecting global variables) Edek <edek.pienkowski@gmail.com> - 2011-09-14 09:33 +0200
                  Re: Thread safe logging (or protecting global variables) Alexander Terekhov <terekhov@web.de> - 2011-09-14 13:17 +0200
                    Re: Thread safe logging (or protecting global variables) Joshua Maurice <joshuamaurice@gmail.com> - 2011-09-14 13:03 -0700
                      Re: Thread safe logging (or protecting global variables) Joshua Maurice <joshuamaurice@gmail.com> - 2011-09-14 13:19 -0700
                      Re: Thread safe logging (or protecting global variables) "James" <no@spam.invalid> - 2011-09-14 19:19 -0700
                        Re: Thread safe logging (or protecting global variables) Joshua Maurice <joshuamaurice@gmail.com> - 2011-09-14 21:03 -0700
                          Re: Thread safe logging (or protecting global variables) Alexander Terekhov <terekhov@web.de> - 2011-09-15 13:12 +0200
                            Re: Thread safe logging (or protecting global variables) Joshua Maurice <joshuamaurice@gmail.com> - 2011-09-15 12:24 -0700
                              Re: Thread safe logging (or protecting global variables) Alexander Terekhov <terekhov@web.de> - 2011-09-15 22:22 +0200
                                Re: Thread safe logging (or protecting global variables) "James" <no@spam.invalid> - 2011-09-15 13:33 -0700
                                  Re: Thread safe logging (or protecting global variables) Joshua Maurice <joshuamaurice@gmail.com> - 2011-09-15 15:36 -0700
                                Re: Thread safe logging (or protecting global variables) Joshua Maurice <joshuamaurice@gmail.com> - 2011-09-15 15:36 -0700
                                  Re: Thread safe logging (or protecting global variables) Alexander Terekhov <terekhov@web.de> - 2011-09-16 16:28 +0200
                                    Re: Thread safe logging (or protecting global variables) Joshua Maurice <joshuamaurice@gmail.com> - 2011-09-16 15:06 -0700
                                      Re: Thread safe logging (or protecting global variables) Alexander Terekhov <terekhov@web.de> - 2011-09-17 12:50 +0200
                      Re: Thread safe logging (or protecting global variables) Alexander Terekhov <terekhov@web.de> - 2011-09-15 12:50 +0200
                Re: Thread safe logging (or protecting global variables) Edek <edek.pienkowski@gmail.com> - 2011-09-14 09:17 +0200
    Re: Thread safe logging (or protecting global variables) Joshua Maurice <joshuamaurice@gmail.com> - 2011-09-08 16:28 -0700
      Re: Thread safe logging (or protecting global variables) Joshua Maurice <joshuamaurice@gmail.com> - 2011-09-08 16:33 -0700
        Re: Thread safe logging (or protecting global variables) Joshua Maurice <joshuamaurice@gmail.com> - 2011-09-08 20:55 -0700
      Re: Thread safe logging (or protecting global variables) Joshua Maurice <joshuamaurice@gmail.com> - 2011-09-12 00:45 -0700

Page 1 of 2  [1] 2  Next page →

#534 — Thread safe logging (or protecting global variables)

Hi,

I have a C library that is supposed to be thread-safe under the
condition that you access each library object only from a single thread.
This is done by ensuring that objects do not share any state (e.g. no
global variables) and where necessary reentrant variants of certain C
library functions are used (e.g. localtime_r).

Now, I want to add a logging feature for debugging purposes. The idea
would be to implement a logging infrastructure where the amount of
output can be controlled with a loglevel parameter (defaulting to no
output) and redirected to a file (defaulting to stderr), like in the
pseudo code below:

static int g_loglevel = 0;
static FILE *g_logfile = stderr;
/* Some other globals used for the logging here */

message_set_loglevel (int loglevel)
{
    g_loglevel = loglevel;
}

message_set_logfile (const char *logfile)
{
    /* Open/close the new/old logfiles here. */
    g_logfile = ...;
}

message (const char *fmt, ...)
{
    va_list ap;

    /* This is just an example, the real code does more
       and relies on a number of extra global variables. */
    if (g_loglevel) {
       va_start (ap, fmt);
       vfprintf (g_logfile, fmt, ap);
       va_end (ap);
    }
}

Obviously, this is not thread-safe due to the use of global variables.
Thus my next idea was to add a global mutex (e.g. pthread_mutex_t on
unix or CRITICAL_SECTION on windows) protecting these global variables.
But how do I initialize the mutex, considering that my library doesn't
have (or need) any initialization functions? For the unix pthread there
is PTHREAD_MUTEX_INITIALIZER, but as far as I know there is no such
thing for a windows critical section? How can this problem be solved?

Would it be an acceptable solution to mark the message_set_*()
functions as not thread-safe and require applications to call them
before/after any threads are created/destroyed? (For my use case such a
restriction would be acceptable, although I'm not sure it would be
fully thread-safe.) Then those functions could initialize the mutex. Or will 
that cause any trouble?

Can I even go one step futher and use simple a simple flag (like the
g_loglevel variable above) that can only be modified from the
message_set_*() functions, and which is only checked in the message()
function but not modified.

Jef

[toc] | [next] | [standalone]

#535

On 09/05/2011 08:19 PM, Jef Driesen wrote:
> Hi,
>
> I have a C library that is supposed to be thread-safe under the
> condition that you access each library object only from a single thread.
> This is done by ensuring that objects do not share any state (e.g. no
> global variables) and where necessary reentrant variants of certain C
> library functions are used (e.g. localtime_r).
>
> Now, I want to add a logging feature for debugging purposes. The idea
> would be to implement a logging infrastructure where the amount of
> output can be controlled with a loglevel parameter (defaulting to no
> output) and redirected to a file (defaulting to stderr), like in the
> pseudo code below:
>
[...]
>
> Obviously, this is not thread-safe due to the use of global variables.
> Thus my next idea was to add a global mutex (e.g. pthread_mutex_t on
> unix or CRITICAL_SECTION on windows) protecting these global variables.
> But how do I initialize the mutex, considering that my library doesn't
> have (or need) any initialization functions? For the unix pthread there
> is PTHREAD_MUTEX_INITIALIZER, but as far as I know there is no such
> thing for a windows critical section? How can this problem be solved?

By definition ;): if you're trying to use threads on win32, you are
screwed. There are too many wise people there.

>
> Would it be an acceptable solution to mark the message_set_*()
> functions as not thread-safe and require applications to call them
> before/after any threads are created/destroyed? (For my use case such a
> restriction would be acceptable, although I'm not sure it would be
> fully thread-safe.) Then those functions could initialize the mutex. Or
> will that cause any trouble?

That will not cause any trouble, as long you don't have anything that
tries to do logging before that - think other initialisations, if there
is any C++. If this is pure C, things are easier, assuming everybody
reads the API spec saying "call the init() first". The last can be
achieved by another flag used only for the purpose of detecting any
use of logging mechanisms before init(). Naturally, the flag won't
really be thread safe, but after all it is not for being really used;
it depends if you want this mechanism or not, possibly #ifdef-ed.

>
> Can I even go one step futher and use simple a simple flag (like the
> g_loglevel variable above) that can only be modified from the
> message_set_*() functions, and which is only checked in the message()
> function but not modified.

Will work on x86, but there is no guarantee it won't break on any other
CPU or on some new hardware a couple years later.

An atomic would be enough - either the new standard, or find an
assembly that makes one for the target CPU, a couple of libraries have
been doing that for a long time. For example, logging libraries.

Edek

[toc] | [prev] | [next] | [standalone]

#542

On 09/05/2011 08:30 PM, Edek wrote:
> On 09/05/2011 08:19 PM, Jef Driesen wrote:
>> Hi,
>>
>> I have a C library that is supposed to be thread-safe under the
>> condition that you access each library object only from a single thread.
>> This is done by ensuring that objects do not share any state (e.g. no
>> global variables) and where necessary reentrant variants of certain C
>> library functions are used (e.g. localtime_r).
>>
>> Now, I want to add a logging feature for debugging purposes. The idea
>> would be to implement a logging infrastructure where the amount of
>> output can be controlled with a loglevel parameter (defaulting to no
>> output) and redirected to a file (defaulting to stderr), like in the
>> pseudo code below:
>>
> [...]
>>
>> Obviously, this is not thread-safe due to the use of global variables.
>> Thus my next idea was to add a global mutex (e.g. pthread_mutex_t on
>> unix or CRITICAL_SECTION on windows) protecting these global variables.
>> But how do I initialize the mutex, considering that my library doesn't
>> have (or need) any initialization functions? For the unix pthread there
>> is PTHREAD_MUTEX_INITIALIZER, but as far as I know there is no such
>> thing for a windows critical section? How can this problem be solved?
>
> By definition ;): if you're trying to use threads on win32, you are
> screwed. There are too many wise people there.
>
>>
>> Would it be an acceptable solution to mark the message_set_*()
>> functions as not thread-safe and require applications to call them
>> before/after any threads are created/destroyed? (For my use case such a
>> restriction would be acceptable, although I'm not sure it would be
>> fully thread-safe.) Then those functions could initialize the mutex. Or
>> will that cause any trouble?
>
> That will not cause any trouble, as long you don't have anything that
> tries to do logging before that - think other initialisations, if there
> is any C++. If this is pure C, things are easier, assuming everybody
> reads the API spec saying "call the init() first". The last can be
> achieved by another flag used only for the purpose of detecting any
> use of logging mechanisms before init(). Naturally, the flag won't
> really be thread safe, but after all it is not for being really used;
> it depends if you want this mechanism or not, possibly #ifdef-ed.

It's a pure C library, so I don't have to worry about any C++ issues. I'm also 
fine with the idea that something might break when misused. Documentation exists 
for a reason :-)

However my message_set_*() functions are entirely optional. By default, the 
logging should be disabled and you only need to call them if you want to enable 
the logging. So they are not true initialization functions for the library. What 
I meant to say in my original text is that *IF* you call these functions you 
have to call them before/after any threads are created/destroyed. I'm not sure 
you interpreted my post this way.

I think that makes a very important difference, because I believe I would still 
need some global flag to track whether the functions have been called or not 
(and thus initialized the mutex or not), which is again unsafe?

>> Can I even go one step futher and use simple a simple flag (like the
>> g_loglevel variable above) that can only be modified from the
>> message_set_*() functions, and which is only checked in the message()
>> function but not modified.
>
> Will work on x86, but there is no guarantee it won't break on any other
> CPU or on some new hardware a couple years later.
>
> An atomic would be enough - either the new standard, or find an
> assembly that makes one for the target CPU, a couple of libraries have
> been doing that for a long time. For example, logging libraries.

I understand (in theory at least) how atomic operations can be used to protect a 
single variable like the g_loglevel flag in my example code. But how does this 
work (if that's even possible) if the logging state consists of multiple variables?

[toc] | [prev] | [next] | [standalone]

#545

On 09/06/2011 06:48 PM, Jef Driesen wrote:
> On 09/05/2011 08:30 PM, Edek wrote:
>> On 09/05/2011 08:19 PM, Jef Driesen wrote:
[...]


>>
>> That will not cause any trouble, as long you don't have anything that
>> tries to do logging before that - think other initialisations, if there
>> is any C++. If this is pure C, things are easier, assuming everybody
>> reads the API spec saying "call the init() first". The last can be
>> achieved by another flag used only for the purpose of detecting any
>> use of logging mechanisms before init(). Naturally, the flag won't
>> really be thread safe, but after all it is not for being really used;
>> it depends if you want this mechanism or not, possibly #ifdef-ed.
>
[...]

> However my message_set_*() functions are entirely optional. By default,
> the logging should be disabled and you only need to call them if you
> want to enable the logging. So they are not true initialization
> functions for the library.

They are not: add an init().

What I meant to say in my original text is
> that *IF* you call these functions you have to call them before/after
> any threads are created/destroyed. I'm not sure you interpreted my post
> this way.

I proposed adding a new init() function.

>
> I think that makes a very important difference, because I believe I
> would still need some global flag to track whether the functions have
> been called or not (and thus initialized the mutex or not), which is
> again unsafe?

That's what I wrote too: but this flag only needs to be in debug
builds. It is meant to fire an assertion immediately.

>
>>> Can I even go one step futher and use simple a simple flag (like the
>>> g_loglevel variable above) that can only be modified from the
>>> message_set_*() functions, and which is only checked in the message()
>>> function but not modified.
>>
>> Will work on x86, but there is no guarantee it won't break on any other
>> CPU or on some new hardware a couple years later.
>>
>> An atomic would be enough - either the new standard, or find an
>> assembly that makes one for the target CPU, a couple of libraries have
>> been doing that for a long time. For example, logging libraries.
>
> I understand (in theory at least) how atomic operations can be used to
> protect a single variable like the g_loglevel flag in my example code.
> But how does this work (if that's even possible) if the logging state
> consists of multiple variables?

I do not see why a logging library would need to access atomically
a structure containing both log levels and the sink.

If you do not care about optimisations (for the 'logging off' case),
then you can create a larger structure and make it atomic.

Atomics are supported by CPUs up to some size; if larger they need to be
emulated, usually by locking a mutex (or a spinlock) and using
the data in the structure

Edek

[toc] | [prev] | [next] | [standalone]

#549

On 09/07/2011 10:34 AM, Edek wrote:
> On 09/06/2011 06:48 PM, Jef Driesen wrote:
>> On 09/05/2011 08:30 PM, Edek wrote:
>>> On 09/05/2011 08:19 PM, Jef Driesen wrote:
> [...]
>
>
>>>
>>> That will not cause any trouble, as long you don't have anything that
>>> tries to do logging before that - think other initialisations, if there
>>> is any C++. If this is pure C, things are easier, assuming everybody
>>> reads the API spec saying "call the init() first". The last can be
>>> achieved by another flag used only for the purpose of detecting any
>>> use of logging mechanisms before init(). Naturally, the flag won't
>>> really be thread safe, but after all it is not for being really used;
>>> it depends if you want this mechanism or not, possibly #ifdef-ed.
>>
> [...]
>
>> However my message_set_*() functions are entirely optional. By default,
>> the logging should be disabled and you only need to call them if you
>> want to enable the logging. So they are not true initialization
>> functions for the library.
>
> They are not: add an init().
>
>> What I meant to say in my original text is
>> that *IF* you call these functions you have to call them before/after
>> any threads are created/destroyed. I'm not sure you interpreted my post
>> this way.
>
> I proposed adding a new init() function.

Sorry I misunderstood you then.

Anyway, adding init/exit functions have there own problems too. First of all, my 
library doesn't need those functions at all. Adding them just for the logging, 
which is just for debugging/diagnostics, feels a little heavy weight. And the 
second issue is that there might problems when calling this function multiple 
times. For example when an application calls the init function, and another 
library used by that application also calls the same init function. How would 
you deal with that?

>> I think that makes a very important difference, because I believe I
>> would still need some global flag to track whether the functions have
>> been called or not (and thus initialized the mutex or not), which is
>> again unsafe?
>
> That's what I wrote too: but this flag only needs to be in debug
> builds. It is meant to fire an assertion immediately.
>
>>
>>>> Can I even go one step futher and use simple a simple flag (like the
>>>> g_loglevel variable above) that can only be modified from the
>>>> message_set_*() functions, and which is only checked in the message()
>>>> function but not modified.
>>>
>>> Will work on x86, but there is no guarantee it won't break on any other
>>> CPU or on some new hardware a couple years later.
>>>
>>> An atomic would be enough - either the new standard, or find an
>>> assembly that makes one for the target CPU, a couple of libraries have
>>> been doing that for a long time. For example, logging libraries.
>>
>> I understand (in theory at least) how atomic operations can be used to
>> protect a single variable like the g_loglevel flag in my example code.
>> But how does this work (if that's even possible) if the logging state
>> consists of multiple variables?
>
> I do not see why a logging library would need to access atomically
> a structure containing both log levels and the sink.
>
> If you do not care about optimisations (for the 'logging off' case),
> then you can create a larger structure and make it atomic.
>
> Atomics are supported by CPUs up to some size; if larger they need to be
> emulated, usually by locking a mutex (or a spinlock) and using
> the data in the structure

Can you explain in a little more detail what you mean here? I'm a little lost.

[toc] | [prev] | [next] | [standalone]

#554

On 09/08/2011 10:00 PM, Jef Driesen wrote:

>>>>> Can I even go one step futher and use simple a simple flag (like the
>>>>> g_loglevel variable above) that can only be modified from the
>>>>> message_set_*() functions, and which is only checked in the message()
>>>>> function but not modified.
>>>>
>>>> Will work on x86, but there is no guarantee it won't break on any other
>>>> CPU or on some new hardware a couple years later.
>>>>
>>>> An atomic would be enough - either the new standard, or find an
>>>> assembly that makes one for the target CPU, a couple of libraries have
>>>> been doing that for a long time. For example, logging libraries.
>>>
>>> I understand (in theory at least) how atomic operations can be used to
>>> protect a single variable like the g_loglevel flag in my example code.
>>> But how does this work (if that's even possible) if the logging state
>>> consists of multiple variables?
>>
>> I do not see why a logging library would need to access atomically
>> a structure containing both log levels and the sink.
>>
>> If you do not care about optimisations (for the 'logging off' case),
>> then you can create a larger structure and make it atomic.
>>
>> Atomics are supported by CPUs up to some size; if larger they need to be
>> emulated, usually by locking a mutex (or a spinlock) and using
>> the data in the structure
>
> Can you explain in a little more detail what you mean here? I'm a little
> lost.

Which part?

The first one and the second one is 'why do you want to do that?'. How
are you going to write your logging framework? With a single level?
With a hierarchy of loggers? Just global structures, or each logger
having a flag?

The third one explains how atomicity is done for many variables -
as long as they can be packed in one structure.

Edek

[toc] | [prev] | [next] | [standalone]

#557

On 09/09/2011 06:13 PM, Edek wrote:
> On 09/08/2011 10:00 PM, Jef Driesen wrote:
>
>>>>>> Can I even go one step futher and use simple a simple flag (like the
>>>>>> g_loglevel variable above) that can only be modified from the
>>>>>> message_set_*() functions, and which is only checked in the message()
>>>>>> function but not modified.
>>>>>
>>>>> Will work on x86, but there is no guarantee it won't break on any other
>>>>> CPU or on some new hardware a couple years later.
>>>>>
>>>>> An atomic would be enough - either the new standard, or find an
>>>>> assembly that makes one for the target CPU, a couple of libraries have
>>>>> been doing that for a long time. For example, logging libraries.
>>>>
>>>> I understand (in theory at least) how atomic operations can be used to
>>>> protect a single variable like the g_loglevel flag in my example code.
>>>> But how does this work (if that's even possible) if the logging state
>>>> consists of multiple variables?
>>>
>>> I do not see why a logging library would need to access atomically
>>> a structure containing both log levels and the sink.
>>>
>>> If you do not care about optimisations (for the 'logging off' case),
>>> then you can create a larger structure and make it atomic.
>>>
>>> Atomics are supported by CPUs up to some size; if larger they need to be
>>> emulated, usually by locking a mutex (or a spinlock) and using
>>> the data in the structure
>>
>> Can you explain in a little more detail what you mean here? I'm a little
>> lost.
>
> Which part?

All

> The first one and the second one is 'why do you want to do that?'. How
> are you going to write your logging framework? With a single level?
> With a hierarchy of loggers? Just global structures, or each logger
> having a flag?

My logging framework is intended to be very simple. It's basically nothing more 
than a lightweight replacement for the printf() function:

int message (int loglevel, const char *fmt, ...);

The only extra it does, compared to a plain printf() call, is prefixing each 
line with a timestamp and writing the output to both stderr and a file, and of 
course the loglevel parameter. If necessary, you can view the current code 
(without the loglevel support) in the sourceforge git repository:

http://libdivecomputer.git.sourceforge.net/git/gitweb.cgi?p=libdivecomputer/libdivecomputer;a=blob;f=src/utils.c

Thus a single global logger is all need. There is no need for multiple loggers, 
or even a hierarchy.

> The third one explains how atomicity is done for many variables -
> as long as they can be packed in one structure.

I have absolutely no idea how this works.

[toc] | [prev] | [next] | [standalone]

#558

On Sep 13, 8:58 am, Jef Driesen <jefdrie...@hotmail.com.invalid>
wrote:
> [snip]

Hmm, is his newsreader filtering google groups? Or perhaps just me?
Ugg, I really need to get a real newsreader. Can you see this? Is
there something wrong with my solutions else-thread?

[toc] | [prev] | [next] | [standalone]

#559

On 09/13/2011 10:18 PM, Joshua Maurice wrote:
> On Sep 13, 8:58 am, Jef Driesen<jefdrie...@hotmail.com.invalid>
> wrote:
>> [snip]
>
> Hmm, is his newsreader filtering google groups? Or perhaps just me?
> Ugg, I really need to get a real newsreader. Can you see this? Is
> there something wrong with my solutions else-thread?

Yes, I have read your posts. I didn't comment yet because I'm still going 
through msdn to learn about all the functions you used, and also figuring out 
how your code could be applied to my logging code.

[toc] | [prev] | [next] | [standalone]

#564

On Sep 13, 10:16 pm, Jef Driesen <jefdrie...@hotmail.com.invalid>
wrote:
> On 09/13/2011 10:18 PM, Joshua Maurice wrote:
>
> > On Sep 13, 8:58 am, Jef Driesen<jefdrie...@hotmail.com.invalid>
> > wrote:
> >> [snip]
>
> > Hmm, is his newsreader filtering google groups? Or perhaps just me?
> > Ugg, I really need to get a real newsreader. Can you see this? Is
> > there something wrong with my solutions else-thread?
>
> Yes, I have read your posts. I didn't comment yet because I'm still going
> through msdn to learn about all the functions you used, and also figuring out
> how your code could be applied to my logging code.

You initially asked, more or less, "How do I write code on win32 where
I would use the static mutex initializer macro on POSIX?". On POSIX
systems, you can initialize a mutex with a static initializer (which
is guaranteed to be done before any runtime execution). On win32, you
want a mutex that you can use without intialization to guard some
other singleton. The problem is that you're trying to avoid a race
condition on the other singleton. You could make the mutex itself a
lazy initialized singleton, but then you have potential race condition
on the construction of the mutex itself. I'm sure you're well aware of
these issues judging from your first post.

The answer is to make the win32 mutex a lazy initialized singleton,
but to use some fun win32 specific stuff to make the lazy
initialization thread-safe.

If I were paranoid about speed, and if this was in a dll entirely
under my control, then I'd be writing in C++ where I could use static
constructors to initialize the mutex in the dll static init
construction time, before any dlopen call (or whatever windows calls
it) returns. I control the dll, so I know no one will be creating
threads during dll static init, so I can simply use a naive eager-
initialized initialize-on-demand lazy-initialized singleton approach
for my mutex, ala:
  CritialSection & getMutex()
  {
    bool initFlag;
    static CriticalSection mutex;
    if (!initFlag)
    {
      InitializeCriticalSection(&mutex);
      initFlag = true;
    }
    return mutex;
  }

Actually, if this were code under my control, I'd probably use Boost,
but nevermind.

If I'm stuck with C, and if I didn't want to figure out how to
register some init function to be called during dll static init, I'd
use something more or less like the code in my earlier post to
initialize and return a pointer to a singleton CriticalSection.

[toc] | [prev] | [next] | [standalone]

#561

On 09/13/2011 10:18 PM, Joshua Maurice wrote:
> On Sep 13, 8:58 am, Jef Driesen<jefdrie...@hotmail.com.invalid>
> wrote:
>> [snip]
>
> Hmm, is his newsreader filtering google groups? Or perhaps just me?
> Ugg, I really need to get a real newsreader. Can you see this? Is
> there something wrong with my solutions else-thread?

I've seen this solution for Windows before and as far as I remember it
it were along the same lines. For correctness, I do not know the API
details; for usability, it is better than an explicit init() function.

And it is great that newer systems than XP implement this requirement
themselves.

Edek

[toc] | [prev] | [next] | [standalone]

#562

Joshua Maurice wrote:
[...]
> Ugg, I really need to get a real newsreader. Can you see this? Is
> there something wrong with my solutions else-thread?

There is something wrong with your named mutex solution... compare it
to:

http://lists.boost.org/Archives/boost/2001/08/15363.php

regards,
alexander.

[toc] | [prev] | [next] | [standalone]

#563

On Sep 14, 4:17 am, Alexander Terekhov <terek...@web.de> wrote:
> Joshua Maurice wrote:
>
> [...]
>
> > Ugg, I really need to get a real newsreader. Can you see this? Is
> > there something wrong with my solutions else-thread?
>
> There is something wrong with your named mutex solution... compare it
> to:
>
> http://lists.boost.org/Archives/boost/2001/08/15363.php

The link raises several issues.  Let me reproduce the boost mailing
list code here.

> struct OnceInfo {
> DWORD counter;
> bool done;
> };
>
> static OnceInfo info = {-1,false};
>
> void doOnce(OnceInfo& info,void (*foo)())
> {
> if (InterlockedIncrement(&info.counter)==0) {
> foo();
> info.done=true;
> } else while (!info.done) Sleep(0);
> }

First is that "InterlockedIncrement" probably does not provide memory
order guarantees, probably just like C++11 std::memory_order_relaxed.
At least, it's not documented to provide anything stronger than atomic
operations on that single variable. Thus, I wouldn't use it when I
needed acquire and release semantics. This is in contract to the msdn
documentation on the interlocked /acquire/ and /release/ variants of
the interlocked functions, and that documentation is pretty clear that
they provides the basic acquire and release memory order semantics. I
would be curious if the msdn documentation is actually that broken. I
must admit to never having looked at the assembly on an IA 64
platform. I doubt it though.

Also important is that it's not clear as documented whether the memory
order acquire and release semantics apply if the exchange is not done.
I believe I coded the above code in under the assumption that the
memory order constraints are only respected when the exchange is
actually done. Again, I haven't too much research on this, which is
why I strongly suggested in my first post to double check this work.

Second, the code from the boost link is still broken, but broken in a
way that my code is not. The "second" thread coming in which reach the
busy sleep waiting loop. That "second" thread does not call any
synchronization functions, and thus there is no guarantee that it will
even see the results of foo(), let alone a guarantee that it will see
all or none of them. In C++11 terms, there is a race condition, and
thus undefined behavior. While not documented IIRC, the effective
(lack of) guarantees of win32 is the same. Compare and contrast with
my solution where every thread past the "first" will call an acquire
read which pairs with a release write which guarantees the needed
memory order constraints, or acquire the mutex which again provides
the needed memory order constraints.

Finally, with regards to thread local storage. Obviously, my initial
solution could have been sped up. I said so up front. I apologize for
not listing out some of the other ways of doing it. I was in a rush. I
could have just obtained the mutex on each call, but I opted for a
useable acquire and release semantics approach. Obviously thread local
storage can potentially speed it noticeably by removing the acquire
read on the fastpath. However, acquire reads are pretty cheap (if not
free on some systems IIRC), and TLS is a relatively rare precious
resource (only ~1000 keys available on most systems IIRC), so I'm
slightly cautious where I use TLS. Also, some versions of windows have
had a bug in TLS where if you try to free a key of value 64 or greater
(or thereabouts), it actually doesn't get freed when you call the free
tls key function, and it remains unavailable for future allocations. I
think it was an obscure windows platform that did this - I forget
which offhand. I filed a bug to Microsoft about that for my company's
product about it, and put in a hack my company's product to work
around it.

And of course, on POSIX systems, you don't need to use pthread_once
and could simply use the POSIX static mutex initializer macro.

[toc] | [prev] | [next] | [standalone]

#565

On Sep 14, 1:03 pm, Joshua Maurice <joshuamaur...@gmail.com> wrote:
> This is in contract to the msdn
> documentation on the interlocked /acquire/ and /release/ variants of
> the interlocked functions,

That should be *in contrast to* (obviously :)).

[toc] | [prev] | [next] | [standalone]

#566

"Joshua Maurice" <joshuamaurice@gmail.com> wrote in message 
news:17b0441f-0a8c-4c53-97e6-90e79cb7888d@z26g2000pre.googlegroups.com...

[...]

> First is that "InterlockedIncrement" probably does not provide memory
> order guarantees, probably just like C++11 std::memory_order_relaxed.

http://msdn.microsoft.com/en-us/library/ms683614(v=vs.85).aspx

How do you explain the following quote:

"This function generates a full memory barrier (or fence) to ensure that 
memory operations are completed in order."

?

[...] 

[toc] | [prev] | [next] | [standalone]

#567

On Sep 14, 7:19 pm, "James" <n...@spam.invalid> wrote:
> "Joshua Maurice" <joshuamaur...@gmail.com> wrote in message
>
> news:17b0441f-0a8c-4c53-97e6-90e79cb7888d@z26g2000pre.googlegroups.com...
>
> [...]
>
> > First is that "InterlockedIncrement" probably does not provide memory
> > order guarantees, probably just like C++11 std::memory_order_relaxed.
>
> http://msdn.microsoft.com/en-us/library/ms683614(v=vs.85).aspx
>
> How do you explain the following quote:
>
> "This function generates a full memory barrier (or fence) to ensure that
> memory operations are completed in order."
>
> ?

I explain it through my complete inability to read.

So, do the interlocked functions act as a full barrier? As that's ill
defined without context, do they act at least as strong as a C++11
std::memory_order_acq_rel memory operation?

Is that Boost mailing list "post" authoritative? It raised some good
points, but the lack of proper punctuation makes me question it. To
quite, "also, Interlocked stuff does not formally provide memory synch
guaranties". The above msdn citation is about as formal as you get for
win32 AFAIK. I merely assumed it'd work as documented. Anyone actually
know if it doesn't?

[toc] | [prev] | [next] | [standalone]

#569

Joshua Maurice wrote:
> 
> On Sep 14, 7:19 pm, "James" <n...@spam.invalid> wrote:
> > "Joshua Maurice" <joshuamaur...@gmail.com> wrote in message
> >
> > news:17b0441f-0a8c-4c53-97e6-90e79cb7888d@z26g2000pre.googlegroups.com...
> >
> > [...]
> >
> > > First is that "InterlockedIncrement" probably does not provide memory
> > > order guarantees, probably just like C++11 std::memory_order_relaxed.
> >
> > http://msdn.microsoft.com/en-us/library/ms683614(v=vs.85).aspx
> >
> > How do you explain the following quote:
> >
> > "This function generates a full memory barrier (or fence) to ensure that
> > memory operations are completed in order."
> >
> > ?
> 
> I explain it through my complete inability to read.
> 
> So, do the interlocked functions act as a full barrier? As that's ill
> defined without context, do they act at least as strong as a C++11
> std::memory_order_acq_rel memory operation?
> 
> Is that Boost mailing list "post" authoritative? It raised some good
> points, but the lack of proper punctuation makes me question it. To

What do you mean by "authoritative"? ;-)

Back in 2001 I was typing so fast that proper punctuation was just a
hindrance.

> quite, "also, Interlocked stuff does not formally provide memory synch
> guaranties". The above msdn citation is about as formal as you get for
> win32 AFAIK. I merely assumed it'd work as documented. Anyone actually
> know if it doesn't?

IIRC back in 2001 MS docs said absolutely nothing about memory fencing
for interlocked stuff.

Saying that "this function generates a full memory barrier (or fence) to
ensure that memory operations are completed in order" isn't really
helpful either because in the context of relaxed memory models a fully
fenced RMW needs more than one fence.

regards,
alexander.

[toc] | [prev] | [next] | [standalone]

#570

On Sep 15, 4:12 am, Alexander Terekhov <terek...@web.de> wrote:
> Joshua Maurice wrote:
>
> > On Sep 14, 7:19 pm, "James" <n...@spam.invalid> wrote:
> > > "Joshua Maurice" <joshuamaur...@gmail.com> wrote in message
>
> > >news:17b0441f-0a8c-4c53-97e6-90e79cb7888d@z26g2000pre.googlegroups.com...
>
> > > [...]
>
> > > > First is that "InterlockedIncrement" probably does not provide memory
> > > > order guarantees, probably just like C++11 std::memory_order_relaxed.
>
> > >http://msdn.microsoft.com/en-us/library/ms683614(v=vs.85).aspx
>
> > > How do you explain the following quote:
>
> > > "This function generates a full memory barrier (or fence) to ensure that
> > > memory operations are completed in order."
>
> > > ?
>
> > I explain it through my complete inability to read.
>
> > So, do the interlocked functions act as a full barrier? As that's ill
> > defined without context, do they act at least as strong as a C++11
> > std::memory_order_acq_rel memory operation?
>
> > Is that Boost mailing list "post" authoritative? It raised some good
> > points, but the lack of proper punctuation makes me question it. To
>
> What do you mean by "authoritative"? ;-)
>
> Back in 2001 I was typing so fast that proper punctuation was just a
> hindrance.
>
> > quite, "also, Interlocked stuff does not formally provide memory synch
> > guaranties". The above msdn citation is about as formal as you get for
> > win32 AFAIK. I merely assumed it'd work as documented. Anyone actually
> > know if it doesn't?
>
> IIRC back in 2001 MS docs said absolutely nothing about memory fencing
> for interlocked stuff.
>
> Saying that "this function generates a full memory barrier (or fence) to
> ensure that memory operations are completed in order" isn't really
> helpful either because in the context of relaxed memory models a fully
> fenced RMW needs more than one fence.

So, as I understand where we are now, that you know that the older
docs did not mention memory barriers of any kind. Also, the current
docs are sufficiently vague that we're not quite sure if it acts as an
atomic exchange with acquire and release semantics on all platforms. I
agree to both of those points.

My code relied simply on the "acquire" function having acquire
semantics, and the "release" function having release semantics,
according to the well known meanings of acquire and release, such as C+
+11. Do you think under the current docs that this is an unreasonable
reading of what guarantees win32 gives me? Do you think that if it
works now, Microsoft will later implement it to not have acquire and
release semantics? Have you looked at any of the assembly to see if
they have acquire and release semantics now?

Also, I guess it particularly annoyed me that the code you linked to
in that Boost mailing list bears little resemblance to my own.

Boost mailing list code:
> struct OnceInfo {
> DWORD counter;
> bool done;
> };
>
> static OnceInfo info = {-1,false};
>
> void doOnce(OnceInfo& info,void (*foo)())
> {
> if (InterlockedIncrement(&info.counter)==0) {
> foo();
> info.done=true;
> } else while (!info.done) Sleep(0);
> }

The Boost mailing list code is broken no matter how the interlocked
functions are implemented (within reason). That while loop at the end
needs to have an acquire read. If it had an acquire read at the end,
then maybe it would work. Without that, it is broken. My code used the
newer docs and arguably safer interpretations that the acquire
function gives acquire semantics, and the release function gives
release semantics.

[toc] | [prev] | [next] | [standalone]

#571

Joshua Maurice wrote:
[...]
> Also, I guess it particularly annoyed me that the code you linked to

You posted code 

  CreateMutex(0, 1, . . .

versus my code

  CreateMutex( NULL, FALSE, . . .

See:

http://lists.boost.org/Archives/boost/2001/08/15363.php 

Got it now?

regards,
alexander.

[toc] | [prev] | [next] | [standalone]

#572

"Alexander Terekhov" <terekhov@web.de> wrote in message 
news:4E725EA0.8D81AE41@web.de...
>
> Joshua Maurice wrote:
> [...]
>> Also, I guess it particularly annoyed me that the code you linked to
>
> You posted code
>
>  CreateMutex(0, 1, . . .
>
> versus my code
>
>  CreateMutex( NULL, FALSE, . . .
>
> See:
>
> http://lists.boost.org/Archives/boost/2001/08/15363.php
>
> Got it now?

AFAICT, he is recursively acquiring the mutex...


Joshua Maurice Code:

    HANDLE namedMutex = CreateMutex(0, 1,
            /* concoct some name which is unique to the process, ex:
   "com.mycompany.myproject.loggingmutex" + pid */ );

      WaitForSingleObject(namedMutex, INFINITE);


The call to WFSO should recursively lock the mutex! 

[toc] | [prev] | [next] | [standalone]

Page 1 of 2  [1] 2  Next page →

Back to top | Article view | comp.programming.threads

csiph-web