Re: Speed ups for a disk IO bound machine

From	David Brown <david.brown@hesbynett.no>
Newsgroups	comp.os.linux.hardware
Subject	Re: Speed ups for a disk IO bound machine
Date	2016-12-16 09:39 +0100
Organization	A noiseless patient Spider
Message-ID	<o3095o$alo$1@dont-email.me> (permalink)
References	(2 earlier) <eb8ounFe48qU1@mid.individual.net> <o2obe6$ogv$1@dont-email.me> <ebe26hFmogdU1@mid.individual.net> <o2u1eg$er$1@dont-email.me> <ebgl1uFbeu0U1@mid.individual.net>

Show all headers | View raw

On 15/12/16 23:50, Peter Chant wrote:
> On 12/15/2016 12:15 PM, David Brown wrote:
> 
>>>
>>> I think my lamp stack is somewhat atypical.  I'm storing numerical data
>>> in it and doing some calcs on that.  Might be good for storage but calcs
>>> in python and php are not optimal.  However, this is partly historic and
>>> partly convenience and rework would be a major pita.
>>
>> For Python, you can look at numpy and scipy for serious calculations -
>> if you can work with your data as homogeneous arrays then numpy will do
>> the calculations in fast C libraries rather than interpreted Python.
>> Also look at pypy or psyco as ways to speed up Python code.  You may
>> also find that if you have heavy Python pages you are better using
>> Twisted as a webserver rather than Apache so that you work entirely
>> within the one Python process rather than starting and stopping
>> processes all the time.
>>
> 
> At this stage the rewrite is probally not worthwhile.  I'm not sure this
> bit is necessarily the bottleneck anyway.  Investigation required.

If you are using 32-bit Python 2, psyco can be a great solution.
Install it, and then add this to your Python script:

	import psyco
	psyco.full()

That's all you need, and psyco will do JIT compilation of functions that
are run often.

As a project, psyco is now "dead" - PyPy is the successor for newer
Python systems.  But it can be a very easy way to speed up some kinds of
code quite significantly.

Back to comp.os.linux.hardware | Previous | Next — Previous in thread | Find similar | Unroll thread

Thread

Speed ups for a disk IO bound machine Peter Chant <pete@petezilla.co.uk> - 2016-12-07 22:41 +0000
  Re: Speed ups for a disk IO bound machine Roger Blake <rogblake@iname.invalid> - 2016-12-08 16:44 +0000
    Re: Speed ups for a disk IO bound machine Peter Chant <pete@petezilla.co.uk> - 2016-12-10 11:24 +0000
      Re: Speed ups for a disk IO bound machine Roger Blake <rogblake@iname.invalid> - 2016-12-11 01:34 +0000
        Re: Speed ups for a disk IO bound machine Peter Chant <pete@petezilla.co.uk> - 2016-12-11 22:07 +0000
  Re: Speed ups for a disk IO bound machine Piergiorgio Sartor <piergiorgio.sartor.this.should.not.be.used@nexgo.REMOVETHIS.de> - 2016-12-10 12:43 +0100
    Re: Speed ups for a disk IO bound machine Peter Chant <pete@petezilla.co.uk> - 2016-12-10 20:44 +0000
      Re: Speed ups for a disk IO bound machine Piergiorgio Sartor <piergiorgio.sartor.this.should.not.be.used@nexgo.REMOVETHIS.de> - 2016-12-10 23:05 +0100
        Re: Speed ups for a disk IO bound machine Peter Chant <pete@petezilla.co.uk> - 2016-12-11 11:54 +0000
          Re: Speed ups for a disk IO bound machine Piergiorgio Sartor <piergiorgio.sartor.this.should.not.be.used@nexgo.REMOVETHIS.de> - 2016-12-11 14:10 +0100
            Re: Speed ups for a disk IO bound machine Peter Chant <pete@petezilla.co.uk> - 2016-12-11 22:46 +0000
              Re: Speed ups for a disk IO bound machine Piergiorgio Sartor <piergiorgio.sartor.this.should.not.be.used@nexgo.REMOVETHIS.de> - 2016-12-12 19:22 +0100
            Re: Speed ups for a disk IO bound machine David Brown <david.brown@hesbynett.no> - 2016-12-13 10:17 +0100
              Re: Speed ups for a disk IO bound machine Peter Chant <pete@petezilla.co.uk> - 2016-12-14 20:03 +0000
                Re: Speed ups for a disk IO bound machine David Brown <david.brown@hesbynett.no> - 2016-12-15 12:55 +0100
                Re: Speed ups for a disk IO bound machine Peter Chant <pete@petezilla.co.uk> - 2016-12-15 23:18 +0000
                Re: Speed ups for a disk IO bound machine David Brown <david.brown@hesbynett.no> - 2016-12-16 09:34 +0100
        Re: Speed ups for a disk IO bound machine David Brown <david.brown@hesbynett.no> - 2016-12-13 09:57 +0100
  Re: Speed ups for a disk IO bound machine David Brown <david.brown@hesbynett.no> - 2016-12-12 09:11 +0100
    Re: Speed ups for a disk IO bound machine Peter Chant <pete@petezilla.co.uk> - 2016-12-12 23:08 +0000
      Re: Speed ups for a disk IO bound machine David Brown <david.brown@hesbynett.no> - 2016-12-13 09:29 +0100
        Re: Speed ups for a disk IO bound machine Peter Chant <pete@petezilla.co.uk> - 2016-12-14 23:16 +0000
          Re: Speed ups for a disk IO bound machine David Brown <david.brown@hesbynett.no> - 2016-12-15 13:15 +0100
            Re: Speed ups for a disk IO bound machine Peter Chant <pete@petezilla.co.uk> - 2016-12-15 22:50 +0000
              Re: Speed ups for a disk IO bound machine David Brown <david.brown@hesbynett.no> - 2016-12-16 09:39 +0100

csiph-web