Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.programming.threads > #207

Re: False sharing

Show key headers only | View raw

On 05/30/2011 01:26 PM, curious wrote:
> Hi All,
>
> I wanted to see the effects of false sharing and wrote small test
> program below:
>

False sharing happens when data reaches RAM (or cache at least). Here,
it probably doesn't (at least not for every loop).


>
> #include<pthread.h>
> #include<iostream>
>
> typedef volatile long work_t;
>

Note: volatile, not "atomic". People often misunderstand 'volatile' as
'always reach RAM'. It might not even reach all cache levels.

>
>
> struct WorkArea {
>    work_t __attribute__ ((aligned (128))) worker1; // making sure the
> first data item is alligned to cache line

Just as a small warning: older compilers didn't align values on stack,
even if they had the right attributes; silently.
Double check if the address is really aligned.


>    //char buffer[128];
>    work_t worker2;
>    WorkArea() : worker1(0), worker2(0) {}
> };
>
>
> void* worker_thread(void* data);
> volatile bool g_wait = true;
> int main() {
>    pthread_t thread1, thread2;
>    WorkArea wa;
>    int  iret1, iret2;
>    iret1 = pthread_create(&thread1, NULL, worker_thread,
> (void*)&wa.worker1 );
>    iret2 = pthread_create(&thread2, NULL, worker_thread,
> (void*)&wa.worker2);

You can change that even to wa.worker1, wouldn't make a huge difference
(real sharing). On my CPU this adds about 3%,w hen same test varies
by ~2%. Of course, the result is undefined then.

>    g_wait = false;
>    pthread_join( thread1, NULL);
>    pthread_join( thread2, NULL);
>    std::cout<<  "worker 1 "<<  wa.worker1<<  " worker2 "<<  wa.worker2
> <<  std::endl;
>
>    return 0;
> }
>
> void* worker_thread(void* data)
> {
>    while(g_wait){}

Don't...

>    work_t&work = *(work_t*)data;
>    for(long i =0; i<  10 * 1000L * 1000L * 1000L; ++i) {
>      work += i;

What happens here: compiler cannot just compute the result instead
of coding the loop, because of volatile. CPU however knows
nothing about 'volatile'. It only knows that this is regular RAM
(so such things as write-combining are on),
which is different than IO memory - then every write and read
would have to be performed (that's what volatile is for BTW).
The part of the CPU which loads and stores data core <-> cache
  will not do anything, because it is told 'wait, I need to
load it back and change again' - why would it store anything?
It maybe stores the value sometimes, maybe it doesn't.

CPUs today are way smarter than microcontrolers. They optimise.
If one writes two values into same memory address, only the second
one will be stored, for example.

However, the CPU still has to do addition. If you just store, the thing
is much simpler. Multiplication takes about 1.5x time for addition
when multiplying zeroes,
(interesting on its own). So: not RAM access takes time.
Division: 7x addition.

>    }

At this point the unit responsible for RAM gets told 'nothing more'
and stores for sure. Maybe it did store something before, maybe not.
(Ok: it probably did, but an order of magnitude fewer times than
the loop count). Still, the other thread might not see this immediately.


>    return data;
> }
>
> I expected it to run much faster when the char buffer[128] would not
> be commented out. However it seems that it does not affect the
> runtime. For both scenarios I get almost identical results.
>
> Typical results are all very close, here is one example
> real	0m24.405s
> user	0m47.910s
> sys	0m0.020s
>
> I run this on Intel(R) Core(TM)2 Duo CPU     P8700  @ 2.53GHz
>
> Could this be because shared L2 cache it does not suffer from the
> false sharing as easily?
>
> Another interesting observation is that if I change
> work += i to work = i ( so that it does not need to read the old work
> value) then it runs much faster:
> real	0m5.840s
> user	0m11.350s
> sys	0m0.020s
>
> Which kind of makes sense, but it would be great if someone could
> explain in terms of MESI how the two scenarios differ and why the huge
> difference in speed.

cmp and jump, plus: 4 ops vs. 2 ops, and no calculation,
maybe also the fact that there is no read-after-write
changes things too:
for (...) {
    work = work;
}
also takes about the same time as addition. Probably calculation
is hidden by some bubble in handling the dependencies.
  (Q6600, similar to a pair of slower than yours core2's glued together).


if I change the loop to
for (...) {
    __sync_fetch_and_add (&work, i);
}
... then there is a huge difference if the padding is there or
not (almost 5x), because of false sharing.

__sync_synchronize() in a loop also has a similar effect, though it
is faster than two threads thrashing the bus with LOCKs. Of course,
still much slower than your version (2x).
for (...) {
    work += i;
    __sync_synchronize ();
}

They all have a different purpose.

>
> Many Thanks!

HTH
Edek

PS. What I am not saying is that false sharing may never happen when
there is no atomicity or at least a fence. But if data is not stored, 
there is no false sharing. Store is forced by a barrier or atomicity,
otherwise will happen sooner or later, with the stress on _later_.

Back to comp.programming.threads | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

False sharing curious <curious.oskar58@gmail.com> - 2011-05-30 04:26 -0700
  Re: False sharing Edek <edek.pienkowski@gmail.com> - 2011-05-30 22:29 +0200
    Re: False sharing curious <curious.oskar58@gmail.com> - 2011-05-31 03:21 -0700
      Re: False sharing Marcel Müller <news.5.maazl@spamgourmet.com> - 2011-05-31 15:41 +0200
        Re: False sharing curious <curious.oskar58@gmail.com> - 2011-05-31 10:18 -0700
        Re: False sharing Edek <edek.pienkowski@gmail.com> - 2011-05-31 20:10 +0200
      Re: False sharing Edek <edek.pienkowski@gmail.com> - 2011-05-31 20:37 +0200
        Re: False sharing Edek <edek.pienkowski@gmail.com> - 2011-05-31 21:18 +0200
          Re: False sharing curious <curious.oskar58@gmail.com> - 2011-05-31 14:40 -0700
            Re: False sharing "Balog Pal" <pasa@lib.hu> - 2011-06-01 03:17 +0200
            Re: False sharing Edek <edek.pienkowski@gmail.com> - 2011-06-01 22:20 +0200
        Re: False sharing curious <curious.oskar58@gmail.com> - 2011-05-31 14:29 -0700
  Re: False sharing David Schwartz <davids@webmaster.com> - 2011-05-31 14:12 -0700
  Re: False sharing "Chris M. Thomasson" <cristom@charter.net> - 2011-06-11 15:34 -0700
    Re: False sharing "Chris M. Thomasson" <cristom@charter.net> - 2011-06-11 19:56 -0700
      Re: False sharing "Chris M. Thomasson" <cristom@charter.net> - 2011-06-11 21:07 -0700

csiph-web