Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.os.linux.hardware > #3263

Re: Speed ups for a disk IO bound machine

Show all headers | View raw

On 12/15/2016 12:15 PM, David Brown wrote:

>>
>> I think my lamp stack is somewhat atypical.  I'm storing numerical data
>> in it and doing some calcs on that.  Might be good for storage but calcs
>> in python and php are not optimal.  However, this is partly historic and
>> partly convenience and rework would be a major pita.
> 
> For Python, you can look at numpy and scipy for serious calculations -
> if you can work with your data as homogeneous arrays then numpy will do
> the calculations in fast C libraries rather than interpreted Python.
> Also look at pypy or psyco as ways to speed up Python code.  You may
> also find that if you have heavy Python pages you are better using
> Twisted as a webserver rather than Apache so that you work entirely
> within the one Python process rather than starting and stopping
> processes all the time.
> 

At this stage the rewrite is probally not worthwhile.  I'm not sure this
bit is necessarily the bottleneck anyway.  Investigation required.


>>
>> Given that there is little free now the numbers I quoted earlier might
>> not have been representative.   I did not see a noticeable difference
>> between 4 & 8 GB therefore I'd not considered more ram.  However, if it
>> is really likely to make a big difference and with 16GB of DDR2 going
>> for between £15 and £45 on ebay then some research is warranted.
> 
> I have seen extra ram make an impressive difference to speed.  Not long
> ago a fellow developer here thought he needed a new graphics card at
> about £500 because his current £300 one was too slow for the 3D
> rendering he was doing.  But £30 more ram doubled the speed of the
> system, while a new graphics card would have made little difference.
> 

The lack of noticable difference put me off.  But a little careful
ebaying might bump me to 16 for a reasonable price.


> It is often more efficient to have /tmp on tmpfs and let it spill out
> into swap, than to have the /tmp directly on the disk.
> 

That is good to know for the odd occasion.  Hmm.  I've just noticed a
tiny swap hit with 7% free physical memory as I write this.  First ever,
that I have noticed!


> A newer cpu and motherboard may not seem useful from the viewpoint of
> processor power, but they will have better throughput on the I/O and
> faster native SATA.
> 

I've a cheap as chips PCIe SATA 3 controller waiting to go in.  I'm
curious as to weather it would make a difference to the HDDs, it should
not.  Anyway, it should help the SSD or SSDs.

Back to comp.os.linux.hardware | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Speed ups for a disk IO bound machine Peter Chant <pete@petezilla.co.uk> - 2016-12-07 22:41 +0000
  Re: Speed ups for a disk IO bound machine Roger Blake <rogblake@iname.invalid> - 2016-12-08 16:44 +0000
    Re: Speed ups for a disk IO bound machine Peter Chant <pete@petezilla.co.uk> - 2016-12-10 11:24 +0000
      Re: Speed ups for a disk IO bound machine Roger Blake <rogblake@iname.invalid> - 2016-12-11 01:34 +0000
        Re: Speed ups for a disk IO bound machine Peter Chant <pete@petezilla.co.uk> - 2016-12-11 22:07 +0000
  Re: Speed ups for a disk IO bound machine Piergiorgio Sartor <piergiorgio.sartor.this.should.not.be.used@nexgo.REMOVETHIS.de> - 2016-12-10 12:43 +0100
    Re: Speed ups for a disk IO bound machine Peter Chant <pete@petezilla.co.uk> - 2016-12-10 20:44 +0000
      Re: Speed ups for a disk IO bound machine Piergiorgio Sartor <piergiorgio.sartor.this.should.not.be.used@nexgo.REMOVETHIS.de> - 2016-12-10 23:05 +0100
        Re: Speed ups for a disk IO bound machine Peter Chant <pete@petezilla.co.uk> - 2016-12-11 11:54 +0000
          Re: Speed ups for a disk IO bound machine Piergiorgio Sartor <piergiorgio.sartor.this.should.not.be.used@nexgo.REMOVETHIS.de> - 2016-12-11 14:10 +0100
            Re: Speed ups for a disk IO bound machine Peter Chant <pete@petezilla.co.uk> - 2016-12-11 22:46 +0000
              Re: Speed ups for a disk IO bound machine Piergiorgio Sartor <piergiorgio.sartor.this.should.not.be.used@nexgo.REMOVETHIS.de> - 2016-12-12 19:22 +0100
            Re: Speed ups for a disk IO bound machine David Brown <david.brown@hesbynett.no> - 2016-12-13 10:17 +0100
              Re: Speed ups for a disk IO bound machine Peter Chant <pete@petezilla.co.uk> - 2016-12-14 20:03 +0000
                Re: Speed ups for a disk IO bound machine David Brown <david.brown@hesbynett.no> - 2016-12-15 12:55 +0100
                Re: Speed ups for a disk IO bound machine Peter Chant <pete@petezilla.co.uk> - 2016-12-15 23:18 +0000
                Re: Speed ups for a disk IO bound machine David Brown <david.brown@hesbynett.no> - 2016-12-16 09:34 +0100
        Re: Speed ups for a disk IO bound machine David Brown <david.brown@hesbynett.no> - 2016-12-13 09:57 +0100
  Re: Speed ups for a disk IO bound machine David Brown <david.brown@hesbynett.no> - 2016-12-12 09:11 +0100
    Re: Speed ups for a disk IO bound machine Peter Chant <pete@petezilla.co.uk> - 2016-12-12 23:08 +0000
      Re: Speed ups for a disk IO bound machine David Brown <david.brown@hesbynett.no> - 2016-12-13 09:29 +0100
        Re: Speed ups for a disk IO bound machine Peter Chant <pete@petezilla.co.uk> - 2016-12-14 23:16 +0000
          Re: Speed ups for a disk IO bound machine David Brown <david.brown@hesbynett.no> - 2016-12-15 13:15 +0100
            Re: Speed ups for a disk IO bound machine Peter Chant <pete@petezilla.co.uk> - 2016-12-15 22:50 +0000
              Re: Speed ups for a disk IO bound machine David Brown <david.brown@hesbynett.no> - 2016-12-16 09:39 +0100

csiph-web