Re: client-server parallellised number crunching

From	Hans Georg Schaathun <hg@schaathun.net>
Newsgroups	comp.lang.python
Subject	Re: client-server parallellised number crunching
Date	2011-04-27 19:28 +0100
Organization	University of Bergen
Message-ID	<mq3m88-jf1.ln1@svn.schaathun.net> (permalink)
References	(1 earlier) <ip8o0q$p4f$1@r03.glglgl.eu> <1cel88-0d3.ln1@svn.schaathun.net> <mailman.895.1303911309.9059.python-list@python.org> <90ll88-1h3.ln1@svn.schaathun.net> <mailman.897.1303916305.9059.python-list@python.org>

Show all headers | View raw

On Thu, 28 Apr 2011 00:58:22 +1000, Chris Angelico
  <rosuav@gmail.com> wrote:
:  thousand threads? a couple of million? In Python, it'll probably end
:  up pretty similar; chances are you won't be taking much advantage of
:  multiple CPUs/cores (because the threads will all be waiting for
:  socket read, or the single thread will mostly be waiting in select()),
:  so it's mainly a resource usage issue. Probably worth testing with
:  your actual code.

For my own application, the performance issue is rather negligible.
I don't have more than about 50 idle CPU-s which I can access easily,
and even if I had, it would always stop at 6-7000 functions calls
to evaluate.  In the current test runs using 4 clients and one master
on a quad-core, the master never uses more than around 7% of a core,
and that includes some simple post-processing as well as com's. 

In short, my philosophy is that it works, so why change it?

But, I am aware that some more technical adept programmers think
otherwise, and I am quite happy with that :-)

-- 
:-- Hans Georg

Back to comp.lang.python | Previous | Next — Previous in thread | Find similar

Thread

client-server parallellised number crunching Hans Georg Schaathun <georg@schaathun.net> - 2011-04-26 20:55 +0100
  Re: client-server parallellised number crunching Chris Angelico <rosuav@gmail.com> - 2011-04-27 06:20 +1000
  Re: client-server parallellised number crunching Dan Stromberg <drsalists@gmail.com> - 2011-04-26 13:31 -0700
  Re: client-server parallellised number crunching Dan Stromberg <drsalists@gmail.com> - 2011-04-26 13:33 -0700
    Re: client-server parallellised number crunching Hans Georg Schaathun <hg@schaathun.net> - 2011-04-26 21:47 +0100
      Re: client-server parallellised number crunching Chris Angelico <rosuav@gmail.com> - 2011-04-27 07:07 +1000
  Re: client-server parallellised number crunching Chris Angelico <rosuav@gmail.com> - 2011-04-27 06:35 +1000
  Re: client-server parallellised number crunching geremy condra <debatem1@gmail.com> - 2011-04-26 14:31 -0700
    Re: client-server parallellised number crunching Hans Georg Schaathun <georg@schaathun.net> - 2011-04-27 06:58 +0100
      Re: client-server parallellised number crunching geremy condra <debatem1@gmail.com> - 2011-04-26 23:54 -0700
        Re: client-server parallellised number crunching Hans Georg Schaathun <georg@schaathun.net> - 2011-04-27 10:57 +0100
  Re: client-server parallellised number crunching Thomas Rachel <nutznetz-0c1b6768-bfa9-48d5-a470-7603bd3aa915@spamschutz.glglgl.de> - 2011-04-27 11:35 +0200
    Re: client-server parallellised number crunching Hans Georg Schaathun <hg@schaathun.net> - 2011-04-27 13:21 +0100
      Re: client-server parallellised number crunching Chris Angelico <rosuav@gmail.com> - 2011-04-27 23:35 +1000
        Re: client-server parallellised number crunching Hans Georg Schaathun <hg@schaathun.net> - 2011-04-27 15:15 +0100
          Re: client-server parallellised number crunching Chris Angelico <rosuav@gmail.com> - 2011-04-28 00:58 +1000
            Re: client-server parallellised number crunching Hans Georg Schaathun <hg@schaathun.net> - 2011-04-27 19:28 +0100

csiph-web