Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.os.linux.hardware > #3260

Re: Speed ups for a disk IO bound machine

Show key headers only | View raw

On 12/13/2016 08:29 AM, David Brown wrote:
> On 13/12/16 00:08, Peter Chant wrote:
>> On 12/12/2016 08:11 AM, David Brown wrote:
>>
>>> My first thought is to ask what the machine is being used for.  It is
>>> not normal to have apache/php/mariadb and firefox/browsing on the same
>>> machine.  Usually you have either a server (and therefore no desktop
>>> processes), or a desktop (and therefore no server processes of
>>> particular relevance).  So what are you doing with the system?  What is
>>> it that is using the I/O ?  Is the machine really too slow, and what do
>>> you /feel/ is slow on it?
>>>
>>
>> Home desktop / server.  I have mariadb on it and the easierst way for a
>> front end for some stuff I am doing seemed to be a lamp stack.
> 
> Do you mean you are running server an active webserver on the same
> system you are using for browsing, development, games, email, etc.?
> That is a poor setup, for efficiency, reliability, and security.  Of
> course, economics can be a factor - but if you can afford to be playing
> around with SSDs and multiple disks, then you should also consider if
> you should have a separate machine for the server.  A small Intel NUC
> with a single disk is likely to be good enough for your LAMP stack and
> mediatomb server, leaving your desktop free to be a desktop.
> 

I think my lamp stack is somewhat atypical.  I'm storing numerical data
in it and doing some calcs on that.  Might be good for storage but calcs
in python and php are not optimal.  However, this is partly historic and
partly convenience and rework would be a major pita.

However, if I get slowdowns on this fairly elderly machine and a six
core cpu and 8GB of ram then with albeit a newer generation CPU I don't
see a NUC being much faster, though I admit I've never got my ands on
one.  Plus there is not room for the two hdds plus the ssd OS disk.
Using this machine as the server and the nuc as the desktop would make
more sense.

I did think about this in the past, or getting a nice laptop / docking
station combination and a server setup.

The system is fairly responsive right now but I am not hitting the HDDs
right now using thunderbird as I type this.  This is with duperemove
hitting the HDDs hard and I'd not consider doing anything else that hit
the HDDs.  Incidentally application data is now 72% of physical memory
and disk cache 24-25% with the remainder few % free.


>>
>> Also have mediatomb on it and serve files to mpd on a raspberry pi.
>> Mediatomb seems to churn the disk sometimes.  Not installed right now.
>>
>>
>>> My second thought is that often the best way to improve I/O performance
>>> is to add more ram.  Is that a possibility with your hardware?
>>>
>>
>> I've got 8GB and about 40% acts as a disk cache.  I could try hunting
>> down 16GB on ebay but I paid full price a year or so ago for the 8GB.
>> Looking at kinfocentre right now I have 19% of my memory free, 35% in
>> disk cache and 44% application data.  So would adding more ram do
>> anything but add to free memory?
>>
> 
> Yes, adding memory will make a /serious/ difference when you are trying
> to work as a server and a desktop - /if/ you really are having
> performance issues with I/O.  But to be honest, I don't think you /are/
> having I/O performance issues - I suspect you just think you are.  If
> you have a lot of free memory (and 19% is quite a lot, unless you have
> just stopped a large process) then you are not actually doing a lot of
> I/O, because disk data is cached in ram whenever there is /any/ free ram.
> 

Given that there is little free now the numbers I quoted earlier might
not have been representative.   I did not see a noticeable difference
between 4 & 8 GB therefore I'd not considered more ram.  However, if it
is really likely to make a big difference and with 16GB of DDR2 going
for between £15 and £45 on ebay then some research is warranted.

> With more ram, writes go faster because they stay in memory for longer
> and don't get flushed to disk as often.  Reads go faster because it is
> much more likely that the data is already in memory.  You also have the
> option of putting /tmp on tmpfs to speed up processes that use a lot of
> temporary files.

I've put /tmp on tmpfs before.  I have /dev/shm on tmpfs at the moment
as part of slackware's default config.  Generally I've abandoned this
when compiling packages filled up /tmp and I ran out of tmp space.
Generally a failure to clean up /tmp, but some packages are large and
have a lot of dependencies.

> 
> But again, I would strongly suggest you try to identify what /feels/
> slow, and consider how you use the machine.  What are you doing in
> parallel?  What sort of serving are you actually doing, and is it
> running in parallel with desktop usage?  Why do you think your I/O is slow?
> 

I used the term 'IO bound' as I've seen the HDDs hit 80-90% for long
periods yet CPU usage has been relatively low.  So to me IO was the
limiting factor.  Going out and spending lots of cash (not cache!) on
the latest i7 + motherboard and memory therefore I assume would not
improve the user experience whereas speeding up the existing IO would,
if possible.

The lamp load above is likely excessive but I have seen slowdowns before
with this machine.  Sometimes btrfs seems to build up a backlog of stuff
to do (btrfs cleaner, transactions etc) for a while after doing
something disk intensive.  But I've noticed this less lately.  Btrfs has
not got a reputation for being slow although odd and specific cases do
show up on the mailing list from time to time.  I'm not planning on
swapping file systems unless to another with subvolumes and probably
snapshots as subvolumes have let me organise things in a much more
logical and efficient manner since I have started using them.

I have a nagging feeling that something just is not right.  However, I
need to benchmark.  I also have had a cheap two interface SATAIII card.
If there is something odd with the disk interface (can't see what) maybe
that will shake it out.  It should allow the SSD to function to its
potential anyway, so it is not a bad idea.

Unfortunately the slightly higher range 4 port SATA III PCIe x2 cards
seem limited right now, I'd have to go up quite a notch in price to
eight port / SAS cards and I'm starting to through reasonable sums of
money at an elderly mobo / processor / ram combination with no assured
outcome.  However, cheap improvements and especially improvements with
existing kit are definitely work pursuing.

Back to comp.os.linux.hardware | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Speed ups for a disk IO bound machine Peter Chant <pete@petezilla.co.uk> - 2016-12-07 22:41 +0000
  Re: Speed ups for a disk IO bound machine Roger Blake <rogblake@iname.invalid> - 2016-12-08 16:44 +0000
    Re: Speed ups for a disk IO bound machine Peter Chant <pete@petezilla.co.uk> - 2016-12-10 11:24 +0000
      Re: Speed ups for a disk IO bound machine Roger Blake <rogblake@iname.invalid> - 2016-12-11 01:34 +0000
        Re: Speed ups for a disk IO bound machine Peter Chant <pete@petezilla.co.uk> - 2016-12-11 22:07 +0000
  Re: Speed ups for a disk IO bound machine Piergiorgio Sartor <piergiorgio.sartor.this.should.not.be.used@nexgo.REMOVETHIS.de> - 2016-12-10 12:43 +0100
    Re: Speed ups for a disk IO bound machine Peter Chant <pete@petezilla.co.uk> - 2016-12-10 20:44 +0000
      Re: Speed ups for a disk IO bound machine Piergiorgio Sartor <piergiorgio.sartor.this.should.not.be.used@nexgo.REMOVETHIS.de> - 2016-12-10 23:05 +0100
        Re: Speed ups for a disk IO bound machine Peter Chant <pete@petezilla.co.uk> - 2016-12-11 11:54 +0000
          Re: Speed ups for a disk IO bound machine Piergiorgio Sartor <piergiorgio.sartor.this.should.not.be.used@nexgo.REMOVETHIS.de> - 2016-12-11 14:10 +0100
            Re: Speed ups for a disk IO bound machine Peter Chant <pete@petezilla.co.uk> - 2016-12-11 22:46 +0000
              Re: Speed ups for a disk IO bound machine Piergiorgio Sartor <piergiorgio.sartor.this.should.not.be.used@nexgo.REMOVETHIS.de> - 2016-12-12 19:22 +0100
            Re: Speed ups for a disk IO bound machine David Brown <david.brown@hesbynett.no> - 2016-12-13 10:17 +0100
              Re: Speed ups for a disk IO bound machine Peter Chant <pete@petezilla.co.uk> - 2016-12-14 20:03 +0000
                Re: Speed ups for a disk IO bound machine David Brown <david.brown@hesbynett.no> - 2016-12-15 12:55 +0100
                Re: Speed ups for a disk IO bound machine Peter Chant <pete@petezilla.co.uk> - 2016-12-15 23:18 +0000
                Re: Speed ups for a disk IO bound machine David Brown <david.brown@hesbynett.no> - 2016-12-16 09:34 +0100
        Re: Speed ups for a disk IO bound machine David Brown <david.brown@hesbynett.no> - 2016-12-13 09:57 +0100
  Re: Speed ups for a disk IO bound machine David Brown <david.brown@hesbynett.no> - 2016-12-12 09:11 +0100
    Re: Speed ups for a disk IO bound machine Peter Chant <pete@petezilla.co.uk> - 2016-12-12 23:08 +0000
      Re: Speed ups for a disk IO bound machine David Brown <david.brown@hesbynett.no> - 2016-12-13 09:29 +0100
        Re: Speed ups for a disk IO bound machine Peter Chant <pete@petezilla.co.uk> - 2016-12-14 23:16 +0000
          Re: Speed ups for a disk IO bound machine David Brown <david.brown@hesbynett.no> - 2016-12-15 13:15 +0100
            Re: Speed ups for a disk IO bound machine Peter Chant <pete@petezilla.co.uk> - 2016-12-15 22:50 +0000
              Re: Speed ups for a disk IO bound machine David Brown <david.brown@hesbynett.no> - 2016-12-16 09:39 +0100

csiph-web