Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.programming.threads > #806 > unrolled thread

Is volatile so bad?

Back to article view | Back to comp.programming.threads

Contents

  Is volatile so bad? Mark Adler <madler@alumni.caltech.edu> - 2012-01-01 23:14 -0800
    Re: Is volatile so bad? Alexander Terekhov <terekhov@web.de> - 2012-01-02 15:49 +0100
      Re: Is volatile so bad? "Ersek, Laszlo" <lacos@caesar.elte.hu> - 2012-01-03 00:50 +0100
        Re: Is volatile so bad? Mark Adler <madler@alumni.caltech.edu> - 2012-01-05 20:52 -0800
          Re: Is volatile so bad? "Ersek, Laszlo" <lacos@caesar.elte.hu> - 2012-01-07 23:38 +0100
            Re: Is volatile so bad? Mark Adler <madler@alumni.caltech.edu> - 2012-01-08 14:14 -0800
    Re: Is volatile so bad? Richard Kettlewell <rjk@greenend.org.uk> - 2012-01-06 11:30 +0000
      Re: Is volatile so bad? Mark Adler <google@madler.net> - 2012-01-06 07:33 -0800
        Re: Is volatile so bad? Richard Kettlewell <rjk@greenend.org.uk> - 2012-01-06 22:05 +0000
          Re: Is volatile so bad? Mark Adler <madler@alumni.caltech.edu> - 2012-01-06 19:13 -0800
            Re: Is volatile so bad? Richard Kettlewell <rjk@greenend.org.uk> - 2012-01-07 08:51 +0000

#806 — Is volatile so bad?

My parallel gzip, pigz (http://zlib.net/pigz/) is written in C, uses 
pthreads, and everything seems to work fine.  Until recently the word 
"volatile" did not appear in the code.

However I got an email from a user about a global variable (local to a 
single source file) that he said needed to be volatile.  The variable 
is a pointer to the head of a linked list of jobs, and access to it is 
locked by a related pthreads mutex.  The user said:

"Without volatile the HP C compiler is doing a prefetch on this 
variable before possess/enter_mutex and uses the prefetched old value 
from a processor register still after the mutex."

("possess" is the name of a routine that calls pthread_mutex_lock.)

So I figured what the heck, it can't hurt too much so I put the 
volatile in there.  The intent is that pigz be portable over a wide 
range of operating systems, so I generally try to accommodate requests 
like this.

I realize that if pthreads is implemented properly, volatile should not 
help, since pthreads should be flushing all the write caches or 
whatever else it needs to do when you access a mutex.  However it is 
possible that pthreads is not always implemented properly, as must be 
the case with HP C.  I also realize that volatile may not be 
implemented completely by the compiler, and so might not solve such a 
problem anyway.  But apparently it helped in this case.

So what I'm wondering is how bad it could be to have volatiles in a few 
places, just in case.

I have a few more variables like that for pools of allocated memory for 
use by threads.  I'm thinking about sticking a volatile in front of 
those as well as a possible safety net.  They're not accessed all that 
often, generally when a thread finishes one job and goes back to find 
another to work on.

And thoughts on putting a few more volatiles in to possibly improve 
portability?  Is there a downside that would outweigh the benefits?

[toc] | [next] | [standalone]

#807

Mark Adler wrote:
> 
> My parallel gzip, pigz (http://zlib.net/pigz/) is written in C, uses
> pthreads, and everything seems to work fine.  Until recently the word
> "volatile" did not appear in the code.
> 
> However I got an email from a user about a global variable (local to a
> single source file) that he said needed to be volatile.  The variable
> is a pointer to the head of a linked list of jobs, and access to it is
> locked by a related pthreads mutex.  The user said:
> 
> "Without volatile the HP C compiler is doing a prefetch on this
> variable before possess/enter_mutex and uses the prefetched old value
> from a processor register still after the mutex."
> 
> ("possess" is the name of a routine that calls pthread_mutex_lock.)
> 
> So I figured what the heck, it can't hurt too much so I put the
> volatile in there.  The intent is that pigz be portable over a wide
> range of operating systems, so I generally try to accommodate requests
> like this.

If your program is data-race-free then just file a bug report against HP
C (it is really hard to believe that HP C can't deal with locking
correctly).

For use of volatile, see also:

http://www.hpl.hp.com/personal/Hans_Boehm/c++mm/user-faq.html

regards,
alexander.

[toc] | [prev] | [next] | [standalone]

#808

On Mon, 2 Jan 2012, Alexander Terekhov wrote:

> Mark Adler wrote:
>>
>> My parallel gzip, pigz (http://zlib.net/pigz/) is written in C, uses
>> pthreads, and everything seems to work fine.  Until recently the word
>> "volatile" did not appear in the code.
>>
>> However I got an email from a user about a global variable (local to a
>> single source file) that he said needed to be volatile.  The variable
>> is a pointer to the head of a linked list of jobs, and access to it is
>> locked by a related pthreads mutex.  The user said:
>>
>> "Without volatile the HP C compiler is doing a prefetch on this
>> variable before possess/enter_mutex and uses the prefetched old value
>> from a processor register still after the mutex."
>>
>> ("possess" is the name of a routine that calls pthread_mutex_lock.)
>>
>> So I figured what the heck, it can't hurt too much so I put the
>> volatile in there.  The intent is that pigz be portable over a wide
>> range of operating systems, so I generally try to accommodate requests
>> like this.
>
> If your program is data-race-free then just file a bug report against HP
> C (it is really hard to believe that HP C can't deal with locking
> correctly).

It may also depend on how the compiler is invoked.

HP-UX is certified UNIX 03 <http://www.opengroup.org/openbrand/register/>. 
Did the user make sure that the environment was set up correctly? The 
implementation should document how one can establish a conforming 
environment.

pigz binaries are made available by eg. The PAC for HP-UX 
<http://hpux.connect.org.uk/hppd/hpux/Gnu/pigz-2.1.6/> and they seem to 
build packages with the platform compiler 
<http://hpux.connect.org.uk/hppd/answers/4-4.html>. Unless the problem was 
reported for one of their binaries, I think they should be able to help 
with the env setup.

On the other hand, I notice the pigz-2.1.7 tarball doesn't contain the 
_XOPEN_SOURCE (or _POSIX_C_SOURCE) strings anywhere, so technically it 
can't expect XSI (or POSIX) conformance.

http://pubs.opengroup.org/onlinepubs/007908799/xsh/compilation.html
http://pubs.opengroup.org/onlinepubs/000095399/functions/xsh_chap02_02.html
http://pubs.opengroup.org/onlinepubs/9699919799/functions/V2_chap02.html#tag_15_02

Laszlo

[toc] | [prev] | [next] | [standalone]

#810

On 2012-01-02 15:50:20 -0800, Ersek, Laszlo said:
> On the other hand, I notice the pigz-2.1.7 tarball doesn't contain the 
> _XOPEN_SOURCE (or _POSIX_C_SOURCE) strings anywhere, so technically it 
> can't expect XSI (or POSIX) conformance.

Um, so, if I #define _XOPEN_SOURCE, then the pthread library won't have 
a bug, but if I don't #define _XOPEN_SOURCE, then the pthread library 
will have a bug?

[toc] | [prev] | [next] | [standalone]

#816

On Thu, 5 Jan 2012, Mark Adler wrote:

> On 2012-01-02 15:50:20 -0800, Ersek, Laszlo said:
>> On the other hand, I notice the pigz-2.1.7 tarball doesn't contain the 
>> _XOPEN_SOURCE (or _POSIX_C_SOURCE) strings anywhere, so technically it 
>> can't expect XSI (or POSIX) conformance.
>
> Um, so, if I #define _XOPEN_SOURCE, then the pthread library won't have a 
> bug, but if I don't #define _XOPEN_SOURCE, then the pthread library will have 
> a bug?

I have no idea. If you set _XOPEN_SOURCE before the inclusion of 
<pthread.h> (as required by the standard), <pthread.h> may generate 
compiler-specific pragmas (that may even put hints in the object file for 
the linker), function prototypes with nonportable attributes, and so on. I 
don't know.

<rant>

yarn.c in pigz-2.1.7 says:

 	/* for thread portability */
 	#define _POSIX_PTHREAD_SEMANTICS
 	#define _REENTRANT

_REENTRANT is documented like this in the glibc info:

----*----
  -- Macro: _REENTRANT
  -- Macro: _THREAD_SAFE
      If you define one of these macros, reentrant versions of several
      functions get declared.  Some of the functions are specified in
      POSIX.1c but many others are only available on a few other systems
      or are unique to GNU libc.  The problem is the delay in the
      standardization of the thread safe C library interface.

      Unlike on some other systems, no special version of the C library
      must be used for linking.  There is only one version but while
      compiling this it must have been specified to compile as thread
      safe.
----*----

(I don't know where _POSIX_PTHREAD_SEMANTICS comes from; I couldn't find 
it in either the glibc info or SUSv4.)

Those two defines are there for some specific system(s). In order to 
achieve the same goal more portably (as in, for SUSv[234] or POSIX), 
_XOPEN_SOURCE (or _POSIX_C_SOURCE) is neeed with an appropriate value 
before the inclusion of any header. The compiler invoked should also be 
c89 or c99 (dependent on the SUS version), and so on.

http://pubs.opengroup.org/onlinepubs/007908799/xcu/c89.html
http://pubs.opengroup.org/onlinepubs/000095399/utilities/c99.html
http://pubs.opengroup.org/onlinepubs/9699919799/utilities/c99.html

Otherwise, you may have to pass platform-specific defines and switches 
(like you do now, just (probably) not for the HP-UX compiler).

... I googled _POSIX_PTHREAD_SEMANTICS: it seems to be for Solaris. I 
think Solaris is a good example where you expressly don't need such 
nonportable measures. Solaris 8/9 are certified UNIX 98 
(http://www.opengroup.org/openbrand/register/), thus once the environment 
is set up correctly (see standards(5)), a program (which includes its 
Makefile) written for SUSv2 should build and run on Solaris 8/9 without a 
hitch. Ditto for Solaris 10 / UNIX 03 / SUSv3.

... A further point; I can see

 	#define _FILE_OFFSET_BITS 64

in pigz.c (and not in yarn.c). First, this might make the two derived 
object files (at least in theory) binarily incompatible with each other. 
Second, this is again something that the SUS covers: see "Programming 
Environments" under the above c89/c99 links (and the respective "getconf" 
specs).

I certainly agree that pigz shouldn't use GNU autotools (not that you 
asked for my opinion :)), but then I think it should rely on SUS.

...

/* To-do:
     - [...]
     - make source portable for Windows, VMS, etc. (see gzip source code)

If you "port" it to SUSv2, then Cygwin should give you the rest on 
Windows. The OpenVMS I last played with provided a SUSv1 support layer as 
well (at least in C); perhaps a more modern OpenVMS would give you SUSv2.

     - make build portable (currently good for Unixish)

I think you should rely on c89/c99, the .POSIX: target, getconf for large 
file support, and an _XOPEN_SOURCE macro for all files.

     - add bzip2 decompression (using threads)

I was trying to avoid mentioning lbzip2, but now I can't anymore. It 
supports multi-threaded decompression of single-stream bz2 files. I tested 
it scale up to 103 cores (almost linearly) on a SPARC Solaris 5.9.

Legacy (0.23, specifically for SUSv2, dead): http://lacos.hu

2.x (faster, better, and gnulib & autotools based, new maintainer): 
https://github.com/kjn/lbzip2

</rant>

Apologies for the unsolicited verbiage. It's perfectly possible that your 
user passed a too agressive optimization flag to the HP-UX compiler and 
triggered a bug in it.

Laszlo

[toc] | [prev] | [next] | [standalone]

#817

On 2012-01-07 14:38:36 -0800, Ersek, Laszlo said:
> http://pubs.opengroup.org/onlinepubs/9699919799/utilities/c99.html
...
>  	#define _FILE_OFFSET_BITS 64
> 
> in pigz.c (and not in yarn.c).

Thanks.  I have updated the source files.  Both now have before any includes:

/* for thread portability */
#define _XOPEN_SOURCE 700
#define _POSIX_C_SOURCE 200809L
#define _THREAD_SAFE

/* use large file functions if available */
#define _FILE_OFFSET_BITS 64

which should cover the bases.

> I was trying to avoid mentioning lbzip2, but now I can't anymore.

I have removed bzip2 from the to-do list, since there appears to 
already be something out there that does the job.

Mark

[toc] | [prev] | [next] | [standalone]

#811

Mark Adler <madler@alumni.caltech.edu> writes:
> My parallel gzip, pigz (http://zlib.net/pigz/) is written in C, uses
> pthreads, and everything seems to work fine.  Until recently the word
> "volatile" did not appear in the code.
>
> However I got an email from a user about a global variable (local to a
> single source file) that he said needed to be volatile.  The variable
> is a pointer to the head of a linked list of jobs, and access to it is
> locked by a related pthreads mutex.  The user said:
>
> "Without volatile the HP C compiler is doing a prefetch on this
> variable before possess/enter_mutex and uses the prefetched old value
> from a processor register still after the mutex."
>
> ("possess" is the name of a routine that calls pthread_mutex_lock.)

FWIW I can't persuade HP's C compiler to do prefetch across subroutine
calls as described (even without using any particular options or
#defines).

I did notice that it loads the _address_ of global objects into a
register before any subroutine calls; is it possible that your
correspondent has seen that and misunderstood it as prefetching the
value?

I've managed to build pigz on HP-UX but there are too many possess()
calls to check them all.  Did your correspondent point to a particular
one place?

    $ cc -V
    cc: HP C/aC++ B3910B A.06.15 [May 16 2007]

-- 
http://www.greenend.org.uk/rjk/

[toc] | [prev] | [next] | [standalone]

#812

On Jan 6, 3:30 am, Richard Kettlewell <r...@greenend.org.uk> wrote:
> Did your correspondent point to a particular one place?

I can infer that it was either or both of the possess() calls in
reenter() or join(), both in yarn.c.

[toc] | [prev] | [next] | [standalone]

#813

Mark Adler <google@madler.net> writes:
> Richard Kettlewell <r...@greenend.org.uk> wrote:

>> Did your correspondent point to a particular one place?
>
> I can infer that it was either or both of the possess() calls in
> reenter() or join(), both in yarn.c.

As far as I can see there's no prefetching across the lock in those
functions.  (Based on the exact compiler version I used, my somewhat
exiguous experience of IA64, etc...)

-- 
http://www.greenend.org.uk/rjk/

[toc] | [prev] | [next] | [standalone]

#814

On 2012-01-06 14:05:59 -0800, Richard Kettlewell said:
> As far as I can see there's no prefetching across the lock in those
> functions.  (Based on the exact compiler version I used, my somewhat
> exiguous experience of IA64, etc...)

Richard,

Thank you very much for looking into this.  I will get back with the 
correspondant for more information on the instance.

What version of what compiler are you using?

Mark

[toc] | [prev] | [next] | [standalone]

#815

Mark Adler <madler@alumni.caltech.edu> writes:
> On 2012-01-06 14:05:59 -0800, Richard Kettlewell said:
>> As far as I can see there's no prefetching across the lock in those
>> functions.  (Based on the exact compiler version I used, my somewhat
>> exiguous experience of IA64, etc...)
>
> Richard,
>
> Thank you very much for looking into this.  I will get back with the
> correspondant for more information on the instance.
>
> What version of what compiler are you using?

    $ cc -V
    cc: HP C/aC++ B3910B A.06.15 [May 16 2007]

-- 
http://www.greenend.org.uk/rjk/

[toc] | [prev] | [standalone]

Back to top | Article view | comp.programming.threads

csiph-web