Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.c > #130450 > unrolled thread

OpenMP and AVX2 Problem

Back to article view | Back to comp.lang.c

Contents

  OpenMP and AVX2 Problem "F. Russell" <fr@random.info> - 2018-05-12 15:09 +0000
    Re: OpenMP and AVX2 Problem Tim Prince <tprince@intelretiree.com> - 2018-05-12 11:33 -0400
    Re: OpenMP and AVX2 Problem Melzzzzz <Melzzzzz@zzzzz.com> - 2018-05-12 16:05 +0000
      Re: OpenMP and AVX2 Problem "F. Russell" <fr@random.info> - 2018-05-12 17:46 +0000
      Re: OpenMP and AVX2 Problem "F. Russell" <fr@random.info> - 2018-05-12 19:10 +0000
    Re: OpenMP and AVX2 Problem [SOLVED] "F. Russell" <fr@random.info> - 2018-05-12 19:42 +0000
      Re: OpenMP and AVX2 Problem [SOLVED] Keith Thompson <kst-u@mib.org> - 2018-05-12 14:04 -0700
    Re: OpenMP and AVX2 Problem scott@slp53.sl.home (Scott Lurndal) - 2018-05-13 17:47 +0000
    Re: OpenMP and AVX2 Problem Wouter Verhelst <w@uter.be> - 2018-05-13 09:57 +0200

#130450 — OpenMP and AVX2 Problem

I am using AVX2 intrinsics to speed up math calculations.
For example, using AVX2 can speed up the vector cross product,
C = A X B, by a factor of 4, which for large arrays of vectors
is a huge boost to performance.

I am now thinking of using openmp to create multi-threaded
routines where each processor, or thread, can execute the
same AVX2 instructions on a part of the array.  This should
speed thing up by at least another factor of four.

However, my initial attempts do not work.  In fact, using
openmp slows the calculations down by a factor of 5.
I must be doing something wrong but I don't know what.

Before I hit the references for some deep research on openmp
I thought I would first post my code example here.

The code is quite simple.  There is just a for loop
over an array of vectors using AVX2 intrinsics.  The
sample code is posted at the end.

Since AVX2 operates on 4 double FP values, the array
is an array of struct with each struct containing 4
double components.  The fourth component is not used
but needs to be there because the values must be 32-byte
aligned.

Using an array of size N=1,000,000 here are my results:

Run Time Single-Thread: 0.010427 seconds
Run Time Multi-Thread: 0.05072 seconds

The following code was compiled with gcc using the command:

gcc -O0 -march=native -fopenmp -o vectprod-parallel vectprod-parallel.c

I used "-O0" so that the compiler would not optimize the code
and only the multi-threading would make any diference.

==================
C code begin
==================
// vector cross product with avx2 intrinsics
// C = A X B

#include <stdio.h>
#include <stdlib.h>
#include <malloc.h>
#include <x86intrin.h>
#include <time.h>
#include <omp.h>

typedef struct {
	double vx;
	double vy;
	double vz;
	double vw;
} vector4;

int main()
{

int i, N=1e6;
double *res, seconds;
clock_t start, end;
vector4 *vectA, *vectB, *vectC;
__m256d a, b, c;

// allocate arrays of vector4 with 32-byte alignment
vectA = (vector4 *)memalign(32, N*sizeof(vector4));
vectB = (vector4 *)memalign(32, N*sizeof(vector4));
vectC = (vector4 *)memalign(32, N*sizeof(vector4));

// initialize arrays with test values
for(i=0; i<N; i++) {
(vectA+i)->vx=1.0; vectA[i].vy=2.0; (vectA+i)->vz=3.0;
(vectB+i)->vx=6.0; vectB[i].vy=4.0; (vectB+i)->vz=5.0;
}

start = clock();

// calculate with AVX2 over the array
for(i=0; i<N; i++ ) {

a = _mm256_load_pd((double *)(vectA+i));
b = _mm256_load_pd((double *)(vectB+i));

c = _mm256_permute4x64_pd
(
    _mm256_sub_pd
    (
        _mm256_mul_pd(a, _mm256_permute4x64_pd(b, _MM_SHUFFLE(3, 0, 2, 1))),
        _mm256_mul_pd(b, _mm256_permute4x64_pd(a, _MM_SHUFFLE(3, 0, 2, 1)))
    ),
    _MM_SHUFFLE(3, 0, 2, 1)
);
_mm256_store_pd((double *)(vectC+i), c);

}

end = clock();
seconds = (double)(end - start)/CLOCKS_PER_SEC;
printf("\n\nRun Time Single-Thread: %g seconds\n\n", seconds);

// Print test result of last vector: -2, 13, -8

res=(double *)(vectC+(N-1));
fprintf(stdout, "%g %g %g %g\n", res[0], res[1], res[2], res[3]);


// multi-threaded AVX2 vector product loop

start = clock();

// set up parallel loop with openmp

#pragma omp parallel for private(i, a, b, c) shared(vectA, vectB, vectC)
for(i=0; i<N; i++) {

a = _mm256_load_pd((double *)(vectA+i));
b = _mm256_load_pd((double *)(vectB+i));

c = _mm256_permute4x64_pd
(
    _mm256_sub_pd
    (
        _mm256_mul_pd(a, _mm256_permute4x64_pd(b, _MM_SHUFFLE(3, 0, 2, 1))),
        _mm256_mul_pd(b, _mm256_permute4x64_pd(a, _MM_SHUFFLE(3, 0, 2, 1)))
    ),
    _MM_SHUFFLE(3, 0, 2, 1)
);
_mm256_store_pd((double *)(vectC+i), c);

}

end = clock();
seconds = (double)(end - start)/CLOCKS_PER_SEC;
printf("\n\nRun Time Multi-Thread: %g seconds\n\n", seconds);


// Print test result of last vector: -2, 13, -8

res=(double *)(vectC+(N-1));
fprintf(stdout, "%g %g %g %g\n", res[0], res[1], res[2], res[3]);

free(vectA); free(vectB); free(vectC);
return(0);

}

=======================
End C code
=======================

[toc] | [next] | [standalone]

#130451

On 5/12/2018 11:09 AM, F. Russell wrote:
> I am using AVX2 intrinsics to speed up math calculations.
> For example, using AVX2 can speed up the vector cross product,
> C = A X B, by a factor of 4, which for large arrays of vectors
> is a huge boost to performance.
> 
> I am now thinking of using openmp to create multi-threaded
> routines where each processor, or thread, can execute the
> same AVX2 instructions on a part of the array.  This should
> speed thing up by at least another factor of four.
> 
> However, my initial attempts do not work.  In fact, using
> openmp slows the calculations down by a factor of 5.
> I must be doing something wrong but I don't know what.
> 
> Before I hit the references for some deep research on openmp
> I thought I would first post my code example here.
> 
> The code is quite simple.  There is just a for loop
> over an array of vectors using AVX2 intrinsics.  The
> sample code is posted at the end.
> 
> Since AVX2 operates on 4 double FP values, the array
> is an array of struct with each struct containing 4
> double components.  The fourth component is not used
> but needs to be there because the values must be 32-byte
> aligned.
> 
> Using an array of size N=1,000,000 here are my results:
> 
> Run Time Single-Thread: 0.010427 seconds
> Run Time Multi-Thread: 0.05072 seconds
> 
> The following code was compiled with gcc using the command:
> 
> gcc -O0 -march=native -fopenmp -o vectprod-parallel vectprod-parallel.c
> 
> I used "-O0" so that the compiler would not optimize the code
> and only the multi-threading would make any diference.
> 
> ==================
> C code begin
> ==================
> // vector cross product with avx2 intrinsics
> // C = A X B
> 
> #include <stdio.h>
> #include <stdlib.h>
> #include <malloc.h>
> #include <x86intrin.h>
> #include <time.h>
> #include <omp.h>
> 
> typedef struct {
> 	double vx;
> 	double vy;
> 	double vz;
> 	double vw;
> } vector4;
> 
> int main()
> {
> 
> int i, N=1e6;
> double *res, seconds;
> clock_t start, end;
> vector4 *vectA, *vectB, *vectC;
> __m256d a, b, c;
> 
> // allocate arrays of vector4 with 32-byte alignment
> vectA = (vector4 *)memalign(32, N*sizeof(vector4));
> vectB = (vector4 *)memalign(32, N*sizeof(vector4));
> vectC = (vector4 *)memalign(32, N*sizeof(vector4));
> 
> // initialize arrays with test values
> for(i=0; i<N; i++) {
> (vectA+i)->vx=1.0; vectA[i].vy=2.0; (vectA+i)->vz=3.0;
> (vectB+i)->vx=6.0; vectB[i].vy=4.0; (vectB+i)->vz=5.0;
> }
> 
> start = clock();
> 
> // calculate with AVX2 over the array
> for(i=0; i<N; i++ ) {
> 
> a = _mm256_load_pd((double *)(vectA+i));
> b = _mm256_load_pd((double *)(vectB+i));
> 
> c = _mm256_permute4x64_pd
> (
>     _mm256_sub_pd
>     (
>         _mm256_mul_pd(a, _mm256_permute4x64_pd(b, _MM_SHUFFLE(3, 0, 2, 1))),
>         _mm256_mul_pd(b, _mm256_permute4x64_pd(a, _MM_SHUFFLE(3, 0, 2, 1)))
>     ),
>     _MM_SHUFFLE(3, 0, 2, 1)
> );
> _mm256_store_pd((double *)(vectC+i), c);
> 
> }
> 
> end = clock();
> seconds = (double)(end - start)/CLOCKS_PER_SEC;
> printf("\n\nRun Time Single-Thread: %g seconds\n\n", seconds);
> 
> // Print test result of last vector: -2, 13, -8
> 
> res=(double *)(vectC+(N-1));
> fprintf(stdout, "%g %g %g %g\n", res[0], res[1], res[2], res[3]);
> 
> 
> // multi-threaded AVX2 vector product loop
> 
> start = clock();
> 
> // set up parallel loop with openmp
> 
> #pragma omp parallel for private(i, a, b, c) shared(vectA, vectB, vectC)
> for(i=0; i<N; i++) {
> 
> a = _mm256_load_pd((double *)(vectA+i));
> b = _mm256_load_pd((double *)(vectB+i));
> 
> c = _mm256_permute4x64_pd
> (
>     _mm256_sub_pd
>     (
>         _mm256_mul_pd(a, _mm256_permute4x64_pd(b, _MM_SHUFFLE(3, 0, 2, 1))),
>         _mm256_mul_pd(b, _mm256_permute4x64_pd(a, _MM_SHUFFLE(3, 0, 2, 1)))
>     ),
>     _MM_SHUFFLE(3, 0, 2, 1)
> );
> _mm256_store_pd((double *)(vectC+i), c);
> 
> }
> 
> end = clock();
> seconds = (double)(end - start)/CLOCKS_PER_SEC;
> printf("\n\nRun Time Multi-Thread: %g seconds\n\n", seconds);
> 
> 
> // Print test result of last vector: -2, 13, -8
> 
> res=(double *)(vectC+(N-1));
> fprintf(stdout, "%g %g %g %g\n", res[0], res[1], res[2], res[3]);
> 
> free(vectA); free(vectB); free(vectC);
> return(0);
> 
> }
> 
> =======================
> End C code
> =======================
> 
You are measuring total time summed up from all threads. The usual
expectation is for threading parallelism to reduce elapsed time at the
expense of using more threads.  If you are using 4 threads and pin them
all to different cores, and you have 99% L1 cache hit rate, you would
hope to reduce elapsed time, but you might still be increasing your
total time of all threads by a factor of 3. If you use 2 cores, hoping
that hyperthreads will give you a gain of more than 2x, you are likely
to be disappointed.  I know, it took me several months to confirm that
my laptop is dual core hyperthreaded, as there seemed to be an attempt
to mislead, in part by preventing the disabling of hyperthreads.

-- 
Tim Prince

[toc] | [prev] | [next] | [standalone]

#130454

On 2018-05-12, F. Russell <fr@random.info> wrote:
> I am using AVX2 intrinsics to speed up math calculations.
> For example, using AVX2 can speed up the vector cross product,
> C = A X B, by a factor of 4, which for large arrays of vectors
> is a huge boost to performance.
>
> I am now thinking of using openmp to create multi-threaded
> routines where each processor, or thread, can execute the
> same AVX2 instructions on a part of the array.  This should
> speed thing up by at least another factor of four.
>
> However, my initial attempts do not work.  In fact, using
> openmp slows the calculations down by a factor of 5.
> I must be doing something wrong but I don't know what.
>
> Before I hit the references for some deep research on openmp
> I thought I would first post my code example here.
>
> The code is quite simple.  There is just a for loop
> over an array of vectors using AVX2 intrinsics.  The
> sample code is posted at the end.
>
> Since AVX2 operates on 4 double FP values, the array
> is an array of struct with each struct containing 4
> double components.  The fourth component is not used
> but needs to be there because the values must be 32-byte
> aligned.
>
> Using an array of size N=1,000,000 here are my results:
>
> Run Time Single-Thread: 0.010427 seconds
> Run Time Multi-Thread: 0.05072 seconds
>
>
> end = clock();
> seconds = (double)(end - start)/CLOCKS_PER_SEC;
> printf("\n\nRun Time Single-Thread: %g seconds\n\n", seconds);
>
Hey, you are measuring clocks not time.
Replace that with clock_gettime or gettimofday or Windows equialent
and youl get:
~/News >>> ./a.out


Run Time Single-Thread: 151.231 milliseconds

-2 13 -8 0


Run Time Multi-Thread: 46.348 milliseconds

-2 13 -8 0

-- 
press any key to continue or any other to quit...

[toc] | [prev] | [next] | [standalone]

#130457

On Sat, 12 May 2018 16:05:53 +0000, Melzzzzz wrote:

>
> Replace that with clock_gettime or gettimofday
> 

Clock_gettime() measures ELAPSED time, or the time difference
of an external clock.  I don't want to measure ELAPSED time
because it may include other things on a multi-tasking system.

I want only the exact CPU time of my process and that is measured
with clock().

But I am not sure how multi-threading with clock() will behave
since I am only starting to investigate multi-threading.

[toc] | [prev] | [next] | [standalone]

#130460

On Sat, 12 May 2018 16:05:53 +0000, Melzzzzz wrote:

>
> Replace that with clock_gettime or gettimofday
>

I want actual CPU time.

With clock_gettime() there is an option:

"CLOCK_PROCESS_CPUTIME_ID (since Linux 2.6.12)
Per-process  CPU-time  clock  (measures CPU time consumed by all
threads in the process)."

clock_gettime(CLOCK_PROCESS_CPUTIME_ID, &timespec);

Using this I get:

Run Time Single-Thread: 10 msec
Run Time Multi-Thread: 45 msec


I can also use rusage(RUSAGE_SELF, rusestruct): 

"RUSAGE_SELF
Return resource usage statistics for the calling process,  which
is the sum of resources used by all threads in the process."

Using rusage(RUSAGE_SELF, rusestruct) I get:

Run Time Single-Thread: 10 msec
Run Time Multi-Thread: 43 msec

So measuring the actual CPU time used by all threads with
different methods gives the same results and this result
differs from what is expected.

What is the problem here?

[toc] | [prev] | [next] | [standalone]

#130463 — Re: OpenMP and AVX2 Problem [SOLVED]

On Sat, 12 May 2018 15:09:50 +0000, F. Russell wrote:

> 
> Run Time Single-Thread: 0.010427 seconds
> Run Time Multi-Thread: 0.05072 seconds
> 

I believe that I have found the reason for the discrepancy.

The functions clock(), clock_gettime(CLOCK_PROCESS_CPUTIME_ID),
and rusage(RUSAGE_SELF) are indeed the correct functions to
measure the CPU time.

But for a multi-threaded process these functions will return
the SUM of the CPU time used by all threads:

https://en.wikipedia.org/wiki/CPU_time

Thus, the time returned should be divided by the number of threads
to give a rough final value.

The number of threads can be specified with the command:

OMP_NUM_THREADS=N ./vectprod-parallel

The results are consistent:

OMP_NUM_THREADS=2 ./vectprod-parallel
Run Time Multi-Thread: 10 msec

OMP_NUM_THREADS=4 ./vectprod-parallel
Run Time Multi-Thread: 20 msec

OMP_NUM_THREADS=8 ./vectprod-parallel
Run Time Multi-Thread: 42 msec
(this value varies  a lot from 32 - 52) 

I believe that this is the correct explanation.  If anyone knows
better please enlighten us.

[toc] | [prev] | [next] | [standalone]

#130471 — Re: OpenMP and AVX2 Problem [SOLVED]

"F. Russell" <fr@random.info> writes:
> On Sat, 12 May 2018 15:09:50 +0000, F. Russell wrote:
>> Run Time Single-Thread: 0.010427 seconds
>> Run Time Multi-Thread: 0.05072 seconds
>
> I believe that I have found the reason for the discrepancy.
>
> The functions clock(), clock_gettime(CLOCK_PROCESS_CPUTIME_ID),
> and rusage(RUSAGE_SELF) are indeed the correct functions to
> measure the CPU time.
>
> But for a multi-threaded process these functions will return
> the SUM of the CPU time used by all threads:

If you're using Linux 2.6.12 or later, you can use either
CLOCK_PROCESS_CPUTIME_ID or CLOCK_THREAD_CPUTIME_ID.

This is not defined by the C standard.  comp.unix.programmer might be a
better place to discuss it.

-- 
Keith Thompson (The_Other_Keith) kst-u@mib.org  <http://www.ghoti.net/~kst>
Working, but not speaking, for JetHead Development, Inc.
"We must do something.  This is something.  Therefore, we must do this."
    -- Antony Jay and Jonathan Lynn, "Yes Minister"

[toc] | [prev] | [next] | [standalone]

#130496

"F. Russell" <fr@random.info> writes:
>I am using AVX2 intrinsics to speed up math calculations.
>For example, using AVX2 can speed up the vector cross product,
>C = A X B, by a factor of 4, which for large arrays of vectors
>is a huge boost to performance.
>
>I am now thinking of using openmp to create multi-threaded
>routines where each processor, or thread, can execute the
>same AVX2 instructions on a part of the array.  This should
>speed thing up by at least another factor of four.
>
>However, my initial attempts do not work.  In fact, using
>openmp slows the calculations down by a factor of 5.
>I must be doing something wrong but I don't know what.

My first guess would be cache contention.    Try ensuring that
all the data being operated on by a given thread is colocated
in memory (or at least 64-bytes apart from memory used by a
different thread).

[toc] | [prev] | [next] | [standalone]

#130593

On 12-05-18 17:09, F. Russell wrote:
> Using an array of size N=1,000,000 here are my results:
> 
> Run Time Single-Thread: 0.010427 seconds
> Run Time Multi-Thread: 0.05072 seconds

Use a bigger array. Those numbers are low enough that they're likely to
be skewed by overhead etc.

Try building something that takes 10 seconds or more when doing it
single threaded; once you've got that, compare against your OpenMP version.

If you can't come up with code that takes 10 seconds to run, then
spending the effort to do it multithreaded is probably not worth it.

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.c

csiph-web