Path: csiph.com!v102.xanadu-bbs.net!xanadu-bbs.net!feeder.erje.net!eu.feeder.erje.net!newsfeed.xs4all.nl!newsfeed4a.news.xs4all.nl!xs4all!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail Return-Path: X-Original-To: python-list@python.org Delivered-To: python-list@mail.python.org X-Spam-Status: OK 0.105 X-Spam-Level: * X-Spam-Evidence: '*H*': 0.79; '*S*': 0.00; 'subject:Python': 0.06; 'arrays': 0.09; '(unlike': 0.16; 'behave': 0.16; 'integers.': 0.16; 'pointers.': 0.16; 'resist': 0.16; 'rewritten': 0.16; 'to:name:python list': 0.16; 'vectors': 0.16; 'vertex': 0.16; 'well-known': 0.16; 'language': 0.16; 'wrote:': 0.18; 'thu,': 0.19; '(the': 0.22; 'feb': 0.22; '>>>': 0.22; 'email addr:gmail.com>': 0.22; 'references': 0.26; 'least': 0.26; 'header:In-Reply-To:1': 0.27; 'idea': 0.28; 'point': 0.28; '[1]': 0.29; 'array': 0.29; 'field,': 0.30; 'message-id:@mail.gmail.com': 0.30; 'work.': 0.31; 'card,': 0.31; 'gaming': 0.31; 'there.': 0.32; 'regular': 0.32; 'limitation': 0.33; 'computing': 0.35; 'but': 0.35; 'received:google.com': 0.35; 'science,': 0.36; 'subject:?': 0.36; 'should': 0.36; 'skip:& 10': 0.38; 'jason': 0.38; 'to:addr:python-list': 0.38; 'pm,': 0.38; "couldn't": 0.39; 'though,': 0.39; 'to:addr:python.org': 0.39; 'how': 0.40; 'algorithms': 0.60; 'most': 0.60; 'times': 0.62; 'great': 0.65; 'investment': 0.66; 'biggest': 0.67; '26,': 0.68; 'acts': 0.74; 'manner.': 0.74; '2015': 0.84; 'comparable': 0.84; 'cpu.': 0.84; 'faster...': 0.84; 'fragment': 0.84; 'gpgpu': 0.84; 'laid': 0.84; 'mistake': 0.91; 'promised': 0.91 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=8V3k51yOvhGn+mEO1fTJ3Q8gIrfkHsRY6ibYI8Q/Cvo=; b=r12ZeThOW6tqINrOOsyUtsrzGeVWKw+OMOdLM6ClArk3SiXGyJ2tCtDS12zn7Xd4sx 1INo0WML6GG8pvD790zcnC8Iw7JIfo9OZBwNlAJ/5DkL9OQ5sT4jojy0PR/kbjquAJGs eV2tL+pSVGyq6C93fQCkD16MwcyCfaVHIkRwxE3eiFrptBgmLW6B/zZnyIvXBQG3TXZX Z5ov3BGzEzVAVEoSbMh4GRQ2VfKmOXWOK9AFjWCNtvaWkAEiY1H4TzpL47kfz+hacnC9 bXdS+R8M0hnhRiTk85t5rp9RZbdpljhDW8brHV7hjEHtDS7hlW35vgAyksiWCs5qM2Q2 YX+Q== MIME-Version: 1.0 X-Received: by 10.170.191.199 with SMTP id i190mr10663700yke.69.1424989736117; Thu, 26 Feb 2015 14:28:56 -0800 (PST) In-Reply-To: References: <82642f3a-49e8-4982-b135-66ffc04d67d9@googlegroups.com> <54ee8ce2$0$11109$c3e8da3@news.astraweb.com> <1424963166.30927.73.camel@gmail.com> <1915907417446661989.682673sturla.molden-gmail.com@news.gmane.org> <1424972883.30927.138.camel@gmail.com> Date: Thu, 26 Feb 2015 17:28:56 -0500 Subject: Re: Parallelization of Python on GPU? From: Jason Swails To: python list Content-Type: multipart/alternative; boundary=001a11c1044e07ab0d051005478c X-BeenThere: python-list@python.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: General discussion list for the Python programming language List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Newsgroups: comp.lang.python Message-ID: Lines: 101 NNTP-Posting-Host: 2001:888:2000:d::a6 X-Trace: 1424989738 news.xs4all.nl 2952 [2001:888:2000:d::a6]:41342 X-Complaints-To: abuse@xs4all.nl Xref: csiph.com comp.lang.python:86552 --001a11c1044e07ab0d051005478c Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On Thu, Feb 26, 2015 at 4:10 PM, Sturla Molden wrote: > On 26/02/15 18:48, Jason Swails wrote: > >> On Thu, 2015-02-26 at 16:53 +0000, Sturla Molden wrote: >> >>> GPU computing is great if you have the following: >>> >>> 1. Your data structures are arrays floating point numbers. >>> >> >> It actually works equally great, if not better, for integers. >> > > Right, but not complicated data structures with a lot of references or > pointers. It requires data are laid out in regular arrays, and then it ac= ts > on these arrays in a data-parallel manner. It is designed to process > vertices in parallel for computer graphics, and that is a limitation whic= h > is always there. It is not a CPU with 1024 cores. It is a "floating point > monster" which can process 1024 vectors in parallel. You write a tiny > kernel in a C-like language (CUDA, OpenCL) to process one vector, and the= n > it will apply the kernel to all the vectors in an array of vectors. It is > very comparable to how GLSL and Direct3D vertex and fragment shaders work= . > (The reason for which should be obvious.) The GPU is actually great for a > lot of things in science, but it is not a CPU. The biggest mistake in the > GPGPU hype is the idea that the GPU will behave like a CPU with many core= s. Very well summarized. At least in my field, though, it is well-known that GPUs are not 'uber-fast CPUs'. Algorithms have been redesigned, programs rewritten to take advantage of their architecture. It has been a *massive* investment of time and resources, but (unlike the Xeon Phi coprocessor [1]) has reaped most of its promised rewards. =E2=80=8B--Jason [1] I couldn't resist the jab. At several times the cost of the top of the line NVidia gaming card, the GPU is about 15-20x faster... --001a11c1044e07ab0d051005478c Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable

<= /div>

On Thu, Feb = 26, 2015 at 4:10 PM, Sturla Molden <sturla.molden@gmail.com><= /span> wrote:
On 26/02/1= 5 18:48, Jason Swails wrote:
On Thu, 2015-02-26 at 16:53 +0000, Sturla Molden wrote:
GPU computing is great if you have the following:

1. Your data structures are arrays floating point numbers.

It actually works equally great, if not better, for integers.

 Right, but not complicated data structures with a lot of references or poin= ters. It requires data are laid out in regular arrays, and then it acts on = these arrays in a data-parallel manner. It is designed to process vertices = in parallel for computer graphics, and that is a limitation which is always= there. It is not a CPU with 1024 cores. It is a "floating point monst= er" which can process 1024 vectors in parallel. You write a tiny kerne= l in a C-like language (CUDA, OpenCL) to process one vector, and then it wi= ll apply the kernel to all the vectors in an array of vectors. It is very c= omparable to how GLSL and Direct3D vertex and fragment shaders work. (The r= eason for which should be obvious.) The GPU is actually great for a lot of = things in science, but it is not a CPU. The biggest mistake in the GPGPU hy= pe is the idea that the GPU will behave like a CPU with many cores.

Very well summarized.=C2=A0 At least in my field, thou= gh, it is well-known that GPUs are not 'uber-fast CPUs'.=C2=A0 Algo= rithms have been redesigned, programs rewritten to take advantage of their = architecture.=C2=A0 It has been a *massive* investment of time and resource= s, but (unlike the Xeon Phi coprocessor [1]) has reaped most of its promise= d rewards.

=E2=80=8B--Jason

[1] I couldn't resist the jab.=C2=A0 At severa= l times the cost of the top of the line NVidia gaming card, the GPU is abou= t 15-20x faster...
--001a11c1044e07ab0d051005478c--