Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #4085

Re: client-server parallellised number crunching

Show all headers | View raw

On Tue, Apr 26, 2011 at 1:20 PM, Chris Angelico <rosuav@gmail.com> wrote:
> But question: Why are you doing major number crunching in Python? On
> your quad-core machine, recode in C and see if you can do the whole
> job without bothering the unreliable boxen at all.

The reason is very simple.  I cannot afford the time to code it in C.
Furthermore, the work is research and the system is experimental, 
making the legibility of the code paramount.

On Tue, 26 Apr 2011 13:33:25 -0700, Dan Stromberg
  <drsalists@gmail.com> wrote:
:  I'd suggest that rewriting an entire software system in C because of
:  one inner loop, is overkill.  'better to rewrite just the inner loop
:  (if needed after profiling), and leave the rest in Python.

Well, that's the other reason.  The most intense number crunching in
this part of the project is basic vector and matrix operations done in 
numpy.  AFAIU that means I use exactly the same libraries underneath as
I would have done in C.  Please correct me if I am wrong.

I could run a profiler and squeeze out some performance by coding
another couple of components in C, but I cannot afford the programming
time whereas I can afford to waste the CPU cycles.  And once I get the
client/server parallellisation working, I shall be able to reuse it at
negligible cost on other subproblems, whereas the profiling and C
reimplementation would cost almost as much time for every subsystem.

Does that answer your question, Chris?


-- 
:-- Hans Georg

Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar

Thread

client-server parallellised number crunching Hans Georg Schaathun <georg@schaathun.net> - 2011-04-26 20:55 +0100
  Re: client-server parallellised number crunching Chris Angelico <rosuav@gmail.com> - 2011-04-27 06:20 +1000
  Re: client-server parallellised number crunching Dan Stromberg <drsalists@gmail.com> - 2011-04-26 13:31 -0700
  Re: client-server parallellised number crunching Dan Stromberg <drsalists@gmail.com> - 2011-04-26 13:33 -0700
    Re: client-server parallellised number crunching Hans Georg Schaathun <hg@schaathun.net> - 2011-04-26 21:47 +0100
      Re: client-server parallellised number crunching Chris Angelico <rosuav@gmail.com> - 2011-04-27 07:07 +1000
  Re: client-server parallellised number crunching Chris Angelico <rosuav@gmail.com> - 2011-04-27 06:35 +1000
  Re: client-server parallellised number crunching geremy condra <debatem1@gmail.com> - 2011-04-26 14:31 -0700
    Re: client-server parallellised number crunching Hans Georg Schaathun <georg@schaathun.net> - 2011-04-27 06:58 +0100
      Re: client-server parallellised number crunching geremy condra <debatem1@gmail.com> - 2011-04-26 23:54 -0700
        Re: client-server parallellised number crunching Hans Georg Schaathun <georg@schaathun.net> - 2011-04-27 10:57 +0100
  Re: client-server parallellised number crunching Thomas Rachel <nutznetz-0c1b6768-bfa9-48d5-a470-7603bd3aa915@spamschutz.glglgl.de> - 2011-04-27 11:35 +0200
    Re: client-server parallellised number crunching Hans Georg Schaathun <hg@schaathun.net> - 2011-04-27 13:21 +0100
      Re: client-server parallellised number crunching Chris Angelico <rosuav@gmail.com> - 2011-04-27 23:35 +1000
        Re: client-server parallellised number crunching Hans Georg Schaathun <hg@schaathun.net> - 2011-04-27 15:15 +0100
          Re: client-server parallellised number crunching Chris Angelico <rosuav@gmail.com> - 2011-04-28 00:58 +1000
            Re: client-server parallellised number crunching Hans Georg Schaathun <hg@schaathun.net> - 2011-04-27 19:28 +0100

csiph-web