Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #86525

Re: Parallelization of Python on GPU?

Show key headers only | View raw

On Thu, 2015-02-26 at 16:53 +0000, Sturla Molden wrote:
> GPU computing is great if you have the following:
> 
> 1. Your data structures are arrays floating point numbers.

It actually works equally great, if not better, for integers.

> 2. You have a data-parallel problem.

This is the biggest one, IMO. ^^^

> 3. You are happy with single precision.

NVidia GPUs have double-precision maths in hardware since compute
capability 1.2 (GTX 280).  That's ca. 2008.  In optimized CPU code, you
still get ~50% benefit going from double to single precision (it's
rarely ever that high, but 20-30% is commonplace in my experience of
optimized code).  It's admittedly a bigger hit on most GPUs, but there
are ways to work around it (e.g., fixed precision), and you can still do
double precision work where it's needed.  One of the articles I linked
previously demonstrates that a hybrid precision model (based on fixed
precision) provides exactly the same numerical stability as double
precision (which is much better than pure single precision) for that
application.

Double precision can often be avoided in many parts of a calculation,
using it only where those bits matter (like accumulators with
potentially small contributions, subtractions of two numbers of similar
magnitude, etc.).

> 4. You have time to code erything in CUDA or OpenCL.

This is the second biggest one, IMO. ^^^

> 5. You have enough video RAM to store your data.

Again, it can be worked around, but the frequent GPU->CPU xfers involved
if you can't fit everything on the GPU can be painstaking to limit its
potentially devastating effects on performance.

> 
> For Python the easiest solution is to use Numba Pro.

Agreed, although I've never actually tried PyCUDA before...

All the best,
Jason

Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Parallelization of Python on GPU? John Ladasky <john_ladasky@sbcglobal.net> - 2015-02-25 18:35 -0800
  Re: Parallelization of Python on GPU? Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2015-02-26 14:02 +1100
    Re: Parallelization of Python on GPU? John Ladasky <john_ladasky@sbcglobal.net> - 2015-02-25 20:01 -0800
    Re: Parallelization of Python on GPU? Jason Swails <jason.swails@gmail.com> - 2015-02-26 10:06 -0500
    Re: Parallelization of Python on GPU? Sturla Molden <sturla.molden@gmail.com> - 2015-02-26 16:53 +0000
    Re: Parallelization of Python on GPU? Terry Reedy <tjreedy@udel.edu> - 2015-02-26 12:16 -0500
    Re: Parallelization of Python on GPU? Jason Swails <jason.swails@gmail.com> - 2015-02-26 12:48 -0500
    Re: Parallelization of Python on GPU? Sturla Molden <sturla.molden@gmail.com> - 2015-02-26 22:10 +0100
    Re: Parallelization of Python on GPU? Jason Swails <jason.swails@gmail.com> - 2015-02-26 17:28 -0500
  Re: Parallelization of Python on GPU? Ethan Furman <ethan@stoneleaf.us> - 2015-02-25 19:03 -0800
  Re: Parallelization of Python on GPU? Ethan Furman <ethan@stoneleaf.us> - 2015-02-25 19:05 -0800
    Re: Parallelization of Python on GPU? John Ladasky <john_ladasky@sbcglobal.net> - 2015-02-25 21:53 -0800
      Re: Parallelization of Python on GPU? Christian Gollwitzer <auriocus@gmx.de> - 2015-02-27 19:55 +0100
  Re: Parallelization of Python on GPU? Jason Swails <jason.swails@gmail.com> - 2015-02-26 10:27 -0500
  Re: Parallelization of Python on GPU? Sturla Molden <sturla.molden@gmail.com> - 2015-02-26 16:40 +0000
    Re: Parallelization of Python on GPU? John Ladasky <john_ladasky@sbcglobal.net> - 2015-02-26 09:34 -0800
      Re: Parallelization of Python on GPU? Sturla Molden <sturla.molden@gmail.com> - 2015-02-26 21:54 +0100

csiph-web