Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.programming.threads > #206 > unrolled thread

False sharing

Back to article view | Back to comp.programming.threads

Contents

  False sharing curious <curious.oskar58@gmail.com> - 2011-05-30 04:26 -0700
    Re: False sharing Edek <edek.pienkowski@gmail.com> - 2011-05-30 22:29 +0200
      Re: False sharing curious <curious.oskar58@gmail.com> - 2011-05-31 03:21 -0700
        Re: False sharing Marcel Müller <news.5.maazl@spamgourmet.com> - 2011-05-31 15:41 +0200
          Re: False sharing curious <curious.oskar58@gmail.com> - 2011-05-31 10:18 -0700
          Re: False sharing Edek <edek.pienkowski@gmail.com> - 2011-05-31 20:10 +0200
        Re: False sharing Edek <edek.pienkowski@gmail.com> - 2011-05-31 20:37 +0200
          Re: False sharing Edek <edek.pienkowski@gmail.com> - 2011-05-31 21:18 +0200
            Re: False sharing curious <curious.oskar58@gmail.com> - 2011-05-31 14:40 -0700
              Re: False sharing "Balog Pal" <pasa@lib.hu> - 2011-06-01 03:17 +0200
              Re: False sharing Edek <edek.pienkowski@gmail.com> - 2011-06-01 22:20 +0200
          Re: False sharing curious <curious.oskar58@gmail.com> - 2011-05-31 14:29 -0700
    Re: False sharing David Schwartz <davids@webmaster.com> - 2011-05-31 14:12 -0700
    Re: False sharing "Chris M. Thomasson" <cristom@charter.net> - 2011-06-11 15:34 -0700
      Re: False sharing "Chris M. Thomasson" <cristom@charter.net> - 2011-06-11 19:56 -0700
        Re: False sharing "Chris M. Thomasson" <cristom@charter.net> - 2011-06-11 21:07 -0700

#206 — False sharing

Hi All,

I wanted to see the effects of false sharing and wrote small test
program below:


#include <pthread.h>
#include <iostream>

typedef volatile long work_t;



struct WorkArea {
  work_t __attribute__ ((aligned (128))) worker1; // making sure the
first data item is alligned to cache line
  //char buffer[128];
  work_t worker2;
  WorkArea() : worker1(0), worker2(0) {}
};


void* worker_thread(void* data);
volatile bool g_wait = true;
int main() {
  pthread_t thread1, thread2;
  WorkArea wa;
  int  iret1, iret2;
  iret1 = pthread_create( &thread1, NULL, worker_thread,
(void*)&wa.worker1 );
  iret2 = pthread_create( &thread2, NULL, worker_thread,
(void*)&wa.worker2);
  g_wait = false;
  pthread_join( thread1, NULL);
  pthread_join( thread2, NULL);
  std::cout << "worker 1 " << wa.worker1 << " worker2 " << wa.worker2
<< std::endl;

  return 0;
}

void* worker_thread(void* data)
{
  while(g_wait){}
  work_t &work = *(work_t*)data;
  for(long i =0; i < 10 * 1000L * 1000L * 1000L; ++i) {
    work += i;
  }
  return data;
}

I expected it to run much faster when the char buffer[128] would not
be commented out. However it seems that it does not affect the
runtime. For both scenarios I get almost identical results.

Typical results are all very close, here is one example
real	0m24.405s
user	0m47.910s
sys	0m0.020s

I run this on Intel(R) Core(TM)2 Duo CPU     P8700  @ 2.53GHz

Could this be because shared L2 cache it does not suffer from the
false sharing as easily?

Another interesting observation is that if I change
work += i to work = i ( so that it does not need to read the old work
value) then it runs much faster:
real	0m5.840s
user	0m11.350s
sys	0m0.020s

Which kind of makes sense, but it would be great if someone could
explain in terms of MESI how the two scenarios differ and why the huge
difference in speed.

Many Thanks!

[toc] | [next] | [standalone]

#207

On 05/30/2011 01:26 PM, curious wrote:
> Hi All,
>
> I wanted to see the effects of false sharing and wrote small test
> program below:
>

False sharing happens when data reaches RAM (or cache at least). Here,
it probably doesn't (at least not for every loop).


>
> #include<pthread.h>
> #include<iostream>
>
> typedef volatile long work_t;
>

Note: volatile, not "atomic". People often misunderstand 'volatile' as
'always reach RAM'. It might not even reach all cache levels.

>
>
> struct WorkArea {
>    work_t __attribute__ ((aligned (128))) worker1; // making sure the
> first data item is alligned to cache line

Just as a small warning: older compilers didn't align values on stack,
even if they had the right attributes; silently.
Double check if the address is really aligned.


>    //char buffer[128];
>    work_t worker2;
>    WorkArea() : worker1(0), worker2(0) {}
> };
>
>
> void* worker_thread(void* data);
> volatile bool g_wait = true;
> int main() {
>    pthread_t thread1, thread2;
>    WorkArea wa;
>    int  iret1, iret2;
>    iret1 = pthread_create(&thread1, NULL, worker_thread,
> (void*)&wa.worker1 );
>    iret2 = pthread_create(&thread2, NULL, worker_thread,
> (void*)&wa.worker2);

You can change that even to wa.worker1, wouldn't make a huge difference
(real sharing). On my CPU this adds about 3%,w hen same test varies
by ~2%. Of course, the result is undefined then.

>    g_wait = false;
>    pthread_join( thread1, NULL);
>    pthread_join( thread2, NULL);
>    std::cout<<  "worker 1 "<<  wa.worker1<<  " worker2 "<<  wa.worker2
> <<  std::endl;
>
>    return 0;
> }
>
> void* worker_thread(void* data)
> {
>    while(g_wait){}

Don't...

>    work_t&work = *(work_t*)data;
>    for(long i =0; i<  10 * 1000L * 1000L * 1000L; ++i) {
>      work += i;

What happens here: compiler cannot just compute the result instead
of coding the loop, because of volatile. CPU however knows
nothing about 'volatile'. It only knows that this is regular RAM
(so such things as write-combining are on),
which is different than IO memory - then every write and read
would have to be performed (that's what volatile is for BTW).
The part of the CPU which loads and stores data core <-> cache
  will not do anything, because it is told 'wait, I need to
load it back and change again' - why would it store anything?
It maybe stores the value sometimes, maybe it doesn't.

CPUs today are way smarter than microcontrolers. They optimise.
If one writes two values into same memory address, only the second
one will be stored, for example.

However, the CPU still has to do addition. If you just store, the thing
is much simpler. Multiplication takes about 1.5x time for addition
when multiplying zeroes,
(interesting on its own). So: not RAM access takes time.
Division: 7x addition.

>    }

At this point the unit responsible for RAM gets told 'nothing more'
and stores for sure. Maybe it did store something before, maybe not.
(Ok: it probably did, but an order of magnitude fewer times than
the loop count). Still, the other thread might not see this immediately.


>    return data;
> }
>
> I expected it to run much faster when the char buffer[128] would not
> be commented out. However it seems that it does not affect the
> runtime. For both scenarios I get almost identical results.
>
> Typical results are all very close, here is one example
> real	0m24.405s
> user	0m47.910s
> sys	0m0.020s
>
> I run this on Intel(R) Core(TM)2 Duo CPU     P8700  @ 2.53GHz
>
> Could this be because shared L2 cache it does not suffer from the
> false sharing as easily?
>
> Another interesting observation is that if I change
> work += i to work = i ( so that it does not need to read the old work
> value) then it runs much faster:
> real	0m5.840s
> user	0m11.350s
> sys	0m0.020s
>
> Which kind of makes sense, but it would be great if someone could
> explain in terms of MESI how the two scenarios differ and why the huge
> difference in speed.

cmp and jump, plus: 4 ops vs. 2 ops, and no calculation,
maybe also the fact that there is no read-after-write
changes things too:
for (...) {
    work = work;
}
also takes about the same time as addition. Probably calculation
is hidden by some bubble in handling the dependencies.
  (Q6600, similar to a pair of slower than yours core2's glued together).


if I change the loop to
for (...) {
    __sync_fetch_and_add (&work, i);
}
... then there is a huge difference if the padding is there or
not (almost 5x), because of false sharing.

__sync_synchronize() in a loop also has a similar effect, though it
is faster than two threads thrashing the bus with LOCKs. Of course,
still much slower than your version (2x).
for (...) {
    work += i;
    __sync_synchronize ();
}

They all have a different purpose.

>
> Many Thanks!

HTH
Edek

PS. What I am not saying is that false sharing may never happen when
there is no atomicity or at least a fence. But if data is not stored, 
there is no false sharing. Store is forced by a barrier or atomicity,
otherwise will happen sooner or later, with the stress on _later_.

[toc] | [prev] | [next] | [standalone]

#209

Hi Edek,

Thanks for your reply.
>
> False sharing happens when data reaches RAM (or cache at least). Here,
> it probably doesn't (at least not for every loop).
>

I was under the the understanding (most likely miss guided) that only
one CPU can have a cache line in modified state (it should be invalid
at all others), when the another cpu tries to read the data from that
line it must be written and changed to exclusive state. This would
then cause so called cache line ping pong between the two competing
CPUs. Are you saying that the two CPUs can have two different modified
copies of the cache line that they keep updating?

> Just as a small warning: older compilers didn't align values on stack,
> even if they had the right attributes; silently.
> Double check if the address is really aligned.
>
Thanks for the warning my compiler gcc 4.5.2 seems to do the right
thing.


> However, the CPU still has to do addition. If you just store, the thing
> is much simpler. Multiplication takes about 1.5x time for addition
> when multiplying zeroes,
> (interesting on its own). So: not RAM access takes time.
> Division: 7x addition.
>
Good point, the simpler loop was doing less. I changed my second
experiment so that I kept adding another number to work so that the
only difference within the loop would be reading back the work value.

// loop with read back and adding ( work += i )
.L3:
	movq	(%rax), %rcx	#* data, D.22006
	leaq	(%rdx,%rcx), %rcx	#, D.22007
	addq	$1, %rdx	#, i
	cmpq	%rsi, %rdx	# tmp71, i
	movq	%rcx, (%rax)	# D.22007,* data
	jne	.L3	#,

real	0m24.414s
user	0m47.130s
sys	0m0.680s

// loop with adding ( work = i + dummy )
.L3:
	leaq	(%rdx,%rdi), %rcx	#, D.22006
	addq	$1, %rdx	#, i
	cmpq	%rsi, %rdx	# tmp72, i
	movq	%rcx, (%rax)	# D.22006,* data
	jne	.L3	#,

real	0m8.092s
user	0m15.920s
sys	0m0.010s

So the loop with adding is still much faster than the loop with
reading and adding. It makes sense that it is faster as it does not
need to care what is the old value for the work is. I would really
like to understand what causes the difference. Is it because in the
latter case the CPU can issue "request for ownership" instead of first
reading it (shared state) and invalidating other copies (modified
state)? Or something else all together?

Also it would be interesting to know if the "buffer" would make a
bigger difference with cores that would not share any caches.

Many Thanks

[toc] | [prev] | [next] | [standalone]

#211

curious wrote:
> I was under the the understanding (most likely miss guided) that only
> one CPU can have a cache line in modified state (it should be invalid
> at all others), when the another cpu tries to read the data from that
> line it must be written and changed to exclusive state. This would
> then cause so called cache line ping pong between the two competing
> CPUs.

This in an implementation detail. CPUs might also access cache lines of 
other CPUs via their interconnects. But this is slow too, although not 
that slow than a memory transfer.

> Are you saying that the two CPUs can have two different modified
> copies of the cache line that they keep updating?

Definitely not.


Marcel

[toc] | [prev] | [next] | [standalone]

#212

On May 31, 2:41 pm, Marcel Müller <news.5.ma...@spamgourmet.com>
wrote:
> curious wrote:
> > I was under the the understanding (most likely miss guided) that only
> > one CPU can have a cache line in modified state (it should be invalid
> > at all others), when the another cpu tries to read the data from that
> > line it must be written and changed to exclusive state. This would
> > then cause so called cache line ping pong between the two competing
> > CPUs.
>
> This in an implementation detail. CPUs might also access cache lines of
> other CPUs via their interconnects. But this is slow too, although not
> that slow than a memory transfer.

I suppose I'm interested in the implementation details and how to get
most out of the modern CPUs. In this case I'm very curious why the
sharing did not seem to significantly impair the speed.

> > Are you saying that the two CPUs can have two different modified
> > copies of the cache line that they keep updating?
>
> Definitely not.

That's great, something makes sense in the multi threaded world :-)

Many thanks

[toc] | [prev] | [next] | [standalone]

#213

> curious wrote:
>> I was under the the understanding (most likely miss guided) that only
>> one CPU can have a cache line in modified state

With modern CPUs it is hard to tell, when exactly the data is in
"modified state". When the CPU knows the address, where it will store?
That plus what data is to be stored? That plus other writes, which are
write combined in the same cache line, or just pending? Cache lines
on the other hand have requests for ownership, they are well-defined.

(it should be invalid
>> at all others), when the another cpu tries to read the data from that
>> line it must be written and changed to exclusive state. This would
>> then cause so called cache line ping pong between the two competing
>> CPUs.
>
> This in an implementation detail. CPUs might also access cache lines of other CPUs via their interconnects. But this is slow too, although not that slow than a memory transfer.
>
>> Are you saying that the two CPUs can have two different modified
>> copies of the cache line that they keep updating?
>
> Definitely not.

.. but still, they may have two different copies of data at a
particular address, before L1 requests the rest of a cache line
to merge the changes. Intel Optimisation Guide says something about
the L1 waiting with RFO.

Edek

[toc] | [prev] | [next] | [standalone]

#214

On 05/31/2011 12:21 PM, curious wrote:
>
> Hi Edek,
>
> Thanks for your reply.
>>
>> False sharing happens when data reaches RAM (or cache at least). Here,
>> it probably doesn't (at least not for every loop).
>>
>
> I was under the the understanding (most likely miss guided) that only
> one CPU can have a cache line in modified state (it should be invalid
> at all others), when the another cpu tries to read the data from that
> line it must be written and changed to exclusive state. This would
> then cause so called cache line ping pong between the two competing
> CPUs. Are you saying that the two CPUs can have two different modified
> copies of the cache line that they keep updating?

No, but it is not said a cache line is modified always when there is
a store (I mean "immediately") - some stores may be pending first.

>
>
>> However, the CPU still has to do addition. If you just store, the thing
>> is much simpler. Multiplication takes about 1.5x time for addition
>> when multiplying zeroes,
>> (interesting on its own). So: not RAM access takes time.
>> Division: 7x addition.
>>
> Good point, the simpler loop was doing less. I changed my second
> experiment so that I kept adding another number to work so that the
> only difference within the loop would be reading back the work value.
>
> // loop with read back and adding ( work += i )
> .L3:
> 	movq	(%rax), %rcx	#* data, D.22006
> 	leaq	(%rdx,%rcx), %rcx	#, D.22007
> 	addq	$1, %rdx	#, i
> 	cmpq	%rsi, %rdx	# tmp71, i
> 	movq	%rcx, (%rax)	# D.22007,* data
> 	jne	.L3	#,
>
> real	0m24.414s
> user	0m47.130s
> sys	0m0.680s
>
> // loop with adding ( work = i + dummy )
> .L3:
> 	leaq	(%rdx,%rdi), %rcx	#, D.22006
> 	addq	$1, %rdx	#, i
> 	cmpq	%rsi, %rdx	# tmp72, i
> 	movq	%rcx, (%rax)	# D.22006,* data
> 	jne	.L3	#,
>
> real	0m8.092s
> user	0m15.920s
> sys	0m0.010s
>
> So the loop with adding is still much faster than the loop with
> reading and adding. It makes sense that it is faster as it does not
> need to care what is the old value for the work is. I would really
> like to understand what causes the difference.

Like I wrote before, "work = work", which is read and store back,
is just as slow. It is like your first test.
It is called "loop-carried data dependency"
(previous loop result is fed into next loop), and there is
Store-Forwarding (instead of writing to memory and reading it back
data is fed back with a shortcut and stored only later; data
dependency nevertheless in this example)


Is it because in the
> latter case the CPU can issue "request for ownership" instead of first
> reading it (shared state) and invalidating other copies (modified
> state)? Or something else all together?

See above, something else. To know better, write assembly where work is
in register (instead of memory) and measure.
That will preserve the data dependency, but make no memory handling
in the CPU necessary.
No way to write it without assembly, or gcc will optimise it out.

As a side note, CPU may not be optimised for something as stupid
as reading a memory and storing it back in a loop. That's what
registers are for.

>
> Also it would be interesting to know if the "buffer" would make a
> bigger difference with cores that would not share any caches.

The buffer makes the cores not share any caches, that's what it is for.
The point is rather if the ping-pong you are referring to takes place.
It does not in this example, and does happen with synchronisation,
can be guessed from tests.

Cache coherency has a "builtin support" for threading. It changes
some things, but the memory model is not very strict,
Write-Combining is allowed for example, which by definition
changes ordering.
Synchronisation totally changes the behaviour, forcing certain points
where data must become visible. That means making changes visible across
CPUs immediately.

>
> Many Thanks

Edek

[toc] | [prev] | [next] | [standalone]

#215

I made one more test, of (work += i). This time with a
single thread (it can be argued there is no false-sharing with
a single thread ;) ).

Two threads:
real	0m3.449s
user	0m3.274s
sys	0m0.015s

One thread:
real	0m3.465s
user	0m6.318s
sys	0m0.037s

Of course, the loop count is 10x smaller than yours.

Edek

[toc] | [prev] | [next] | [standalone]

#219

On May 31, 8:18 pm, Edek <edek.pienkow...@gmail.com> wrote:
> I made one more test, of (work += i). This time with a
> single thread (it can be argued there is no false-sharing with
> a single thread ;) ).
>
> Two threads:
> real    0m3.449s
> user    0m3.274s
> sys     0m0.015s
>
> One thread:
> real    0m3.465s
> user    0m6.318s
> sys     0m0.037s
>
> Of course, the loop count is 10x smaller than yours.
>
> Edek

I did another experiment where I have two arrays that the threads
write so that it can't combine the writes.
for(long i =0; i < 1000L * 1000L * 1000L; i++) {
    for(long k = 0; k < 10; ++k ){
      work[k] = k + i;
    }
  }
Now finally I can see the effect if the buffer is separating the
arrays:
With buffer:
real	0m4.493s
user	0m8.730s
sys	0m0.010s

No buffer:
real	0m11.801s
user	0m22.450s
sys	0m0.330s

Many thanks for your help, this has been very educational.

[toc] | [prev] | [next] | [standalone]

#220

"curious" <curious.oskar58@gmail.com>

>Many thanks for your help, this has been very educational.

Good thesis with experiments and measurements here:

http://scholarworks.sjsu.edu/cgi/viewcontent.cgi?article=1001&context=etd_projects

[toc] | [prev] | [next] | [standalone]

#224

>
> I did another experiment where I have two arrays that the threads
> write so that it can't combine the writes.

I think you may be wrong again, sorry. 10*sizeof long is 40 bytes.
If they lie in the same cache line, they will be combined. Maybe
RFO will be done sooner; or more likely, didn't you put the k array
on the cache boundary (or one of those arrays)?

OTOH, maybe you are right, 10 dependencies might be harder to keep
track of than one.

According to the Fine Manual, Core Duo has I think 6 Write Combining
buffers on L1D, of which at least 4 can be used a lot. That means,
not more than four arrays (in separate cache lines of course)
should be put in the loop: you have one.

> for(long i =0; i<  1000L * 1000L * 1000L; i++) {
>      for(long k = 0; k<  10; ++k ){
>        work[k] = k + i;
>      }
>    }
> Now finally I can see the effect if the buffer is separating the
> arrays:

I believe that part, I am not sure about the interpretation.

And, what are you exactly trying to do? Why not just do the 
synchronisation?

Edek

[toc] | [prev] | [next] | [standalone]

#218

On May 31, 7:37 pm, Edek <edek.pienkow...@gmail.com> wrote:
> On 05/31/2011 12:21 PM, curious wrote:

> Like I wrote before, "work = work", which is read and store back,
> is just as slow. It is like your first test.
> It is called "loop-carried data dependency"
> (previous loop result is fed into next loop)

Thanks! I see that it can mess up the pipeline. Is that why it is some
much slower when using work += i?

[toc] | [prev] | [next] | [standalone]

#217

On May 30, 4:26 am, curious <curious.oska...@gmail.com> wrote:

> I expected it to run much faster when the char buffer[128] would not
> be commented out. However it seems that it does not affect the
> runtime. For both scenarios I get almost identical results.

Your code is broken and may not be testing anything. You are using
POSIX threads, which requires that mutexes be used when an object may
be modified concurrently by two or more threads. You have defined your
WorkArea such that worker1 and worker2 are parts of the very same
object. So you must modify them under the protection of a mutex. Your
false sharing may just be optimized away.

The 'volatile' does not guarantee it won't be, this is a common
misunderstanding.
http://origin-software.intel.com/en-us/blogs/2007/11/30/volatile-almost-useless-for-multi-threaded-programming/

DS

[toc] | [prev] | [next] | [standalone]

#263

"curious" <curious.oskar58@gmail.com> wrote in message 
news:18fa2208-ba1c-42c5-bf1f-280e6cbc41e6@bl1g2000vbb.googlegroups.com...
> Hi All,
>
> I wanted to see the effects of false sharing and wrote small test
> program below:

[...]

IMVHO, a "decent test" should probably measure the effect that false sharing 
has on "reader threads". A simple scenario could be: thread A is a writer 
and threads B-N are readers. Thread A works on completely unrelated data wrt 
threads B-N; they should not interfere with each other... A simple setup in 
pseudo-code might be something like:
______________________________________________________
#define ITERS 100000000
#define WRITERS 1
#define READERS 4
#define WRITER_YIELD 100


static atomic_word_t g_state_1; // = 0
static atomic_word_t g_state_2; // = 0
static atomic_word_t g_writers_complete; // = 0


void writer_threads()
{
    for (unsigned i = 0; i < ITERS; ++i)
    {
        ATOMIC_FETCH_ADD(&g_state_1, 1);

        if (! (i % WRITER_YIELD)) thread_yield();
    }

    ATOMIC_FETCH_ADD(&g_writers_complete, 1);
}


void reader_threads()
{
    unsigned iterations = 0;

    while (ATOMIC_LOAD(&g_writers_complete) != WRITERS)
    {
        unsigned state = ATOMIC_LOAD(&g_state_2);

        ++iterations;
    }

    printf("reader_thread iterations: %u\n", iterations);
}
______________________________________________________




The writer threads atomic RMW operations on the `g_state_1' variable will 
probably be false shared with the reader threads load operations on 
`g_state_2'. This should have an impact on the number of iterations a reader 
thread can perform. Compare the performance on the program above with 
something like this:
______________________________________________________
#define ITERS 100000000
#define WRITERS 1
#define READERS 4
#define WRITER_YIELD 100
#define L2_CACHE 128
#define L2_CACHE_ATOMIC_WORD_PAD \
    (L2_CACHE - sizeof(atomic_word_t))


COMPILER_PADDING(OFF)
struct shared_state
{
    atomic_word_t state_1;
    char l2_cache_pad_1[L2_CACHE_ATOMIC_WORD_PAD];

    atomic_word_t state_2;
    char l2_cache_pad_2[L2_CACHE_ATOMIC_WORD_PAD];

    atomic_word_t writers_complete;
    char l2_cache_pad_3[L2_CACHE_ATOMIC_WORD_PAD];
};


// assume that the following is aligned on a L2_CACHE line boundary!!!
static COMPILER_ALIGN(L2_CACHE)
    struct shared_state g_shared_state = { 0 };


void writer_threads()
{
    for (unsigned i = 0; i < ITERS; ++i)
    {
        ATOMIC_FETCH_ADD(&g_shared_state.state_1, 1);

        if (! (i % WRITER_YIELD)) thread_yield();
    }

    ATOMIC_FETCH_ADD(&g_shared_state.writers_complete, 1);
}


void reader_threads()
{
    unsigned iterations = 0;

    while (ATOMIC_LOAD(&g_shared_state.writers_complete)
              != WRITERS)
    {
        unsigned state = ATOMIC_LOAD(&g_shared_state.state_2);

        ++iterations;
    }

    printf("reader_thread iterations: %u\n", iterations);
}
______________________________________________________



The latter program "should" be reporting higher reader thread iteration 
counts than the one that is crippled from false sharing.

Humm... 

[toc] | [prev] | [next] | [standalone]

#264

"Chris M. Thomasson" <cristom@charter.net> wrote in message 
news:D3SIp.1908$g12.189@newsfe20.iad...
> "curious" <curious.oskar58@gmail.com> wrote in message 
> news:18fa2208-ba1c-42c5-bf1f-280e6cbc41e6@bl1g2000vbb.googlegroups.com...
>> Hi All,
>>
>> I wanted to see the effects of false sharing and wrote small test
>> program below:
>
> [...]
>
> IMVHO, a "decent test" should probably measure the effect that false 
> sharing has on "reader threads". A simple scenario could be: thread A is a 
> writer and threads B-N are readers. Thread A works on completely unrelated 
> data wrt threads B-N; they should not interfere with each other... A 
> simple setup in pseudo-code might be something like:
> ______________________________________________________
[...]

> COMPILER_PADDING(OFF)
> struct shared_state
> {
>    atomic_word_t state_1;
>    char l2_cache_pad_1[L2_CACHE_ATOMIC_WORD_PAD];
>
>    atomic_word_t state_2;
>    char l2_cache_pad_2[L2_CACHE_ATOMIC_WORD_PAD];
>
>    atomic_word_t writers_complete;
>    char l2_cache_pad_3[L2_CACHE_ATOMIC_WORD_PAD];
> };

[...]

> void reader_threads()
> {
>    unsigned iterations = 0;
>
>    while (ATOMIC_LOAD(&g_shared_state.writers_complete)
>              != WRITERS)
>    {
>        unsigned state = ATOMIC_LOAD(&g_shared_state.state_2);
>
>        ++iterations;
>    }
>
>    printf("reader_thread iterations: %u\n", iterations);
> }
> ______________________________________________________

Perhaps `struct shared_state' should be defined as:
______________________________________________________
COMPILER_PADDING(OFF)
struct shared_state
{
    atomic_word_t state_1;
    char l2_cache_pad_1[L2_CACHE_ATOMIC_WORD_PAD];

    atomic_word_t state_2;
    /* char l2_cache_pad_2[L2_CACHE_ATOMIC_WORD_PAD]; */

    atomic_word_t writers_complete;
    char l2_cache_pad_3[L2_CACHE_ATOMIC_WORD_PAD * 2];
};
______________________________________________________


> The latter program "should" be reporting higher reader thread iteration 
> counts than the one that is crippled from false sharing.
>
> Humm... 

[toc] | [prev] | [next] | [standalone]

#265

"Chris M. Thomasson" <cristom@charter.net> wrote in message 
news:tVVIp.1449$z%.742@newsfe12.iad...
[...]

> Perhaps `struct shared_state' should be defined as:
> ______________________________________________________
> COMPILER_PADDING(OFF)
> struct shared_state
> {
>    atomic_word_t state_1;
>    char l2_cache_pad_1[L2_CACHE_ATOMIC_WORD_PAD];
>
>    atomic_word_t state_2;
>    /* char l2_cache_pad_2[L2_CACHE_ATOMIC_WORD_PAD]; */
>
>    atomic_word_t writers_complete;
>    char l2_cache_pad_3[L2_CACHE_ATOMIC_WORD_PAD * 2];
> };
> ______________________________________________________


Ummm.... That should be:
______________________________________________________
COMPILER_PADDING(OFF)
struct shared_state
{
    atomic_word_t state_1;
    char l2_cache_pad_1[L2_CACHE_ATOMIC_WORD_PAD];

    atomic_word_t state_2;
    atomic_word_t writers_complete;
    char l2_cache_pad_2[L2_CACHE_ATOMIC_WORD_PAD -
                                      sizeof(atomic_word_t)];
};
______________________________________________________


;^o 

[toc] | [prev] | [standalone]

Back to top | Article view | comp.programming.threads

csiph-web