Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.java.programmer > #12302

Re: O.T. optimising file placement

Show all headers | View raw

On Fri, 24 Feb 2012 10:50:39 -0800, Lew wrote:

> Martin Gregorie wrote:
>> Unlike some, I take a good deal of interest in what my machines are up
>> to, so I was quoting what I see using top on my Linux systems. During
>> normal operation there is very little activity on my laptop except from
>> the programs I'm actively using unless, as you say, logwatch/smartd/
>> rkhunter/updatedb get run by atd, but on a reasonably quick machine
>> they don't run for long.
>>
>> Of course, the house server is a different case, since it has several
>> 24/7 services on it, but again its only heavy, continuous disk activity
>> is overnight when it runs backups/logwatch/smartd/updatedb. Apart from
>> that requests that wake up Postfix/Spamassassin/Apache/or ftpd/sshd are
>> pretty sporadic and the disk LED flashes are best described as
>> intermittent.
> 
> Sounds like disk optimizations would help that system.
>
Probably not - they are all cron jobs and hence get run sequentially.
 
>> The longest continuously busy time on either machine is during backups
>> and even there there precious little contention since rsync or tar+gzip
>> since the only stuff being written to the disk its reading from are
>> backup logs. Same applies to software update sessions. To the best of
>> my knowledge (and watching top) none of yum, rpm, tar, gzip or rsync
>> are multi-threaded: rsync is probably using poll() based async i/o but
>> from top and observed behaviour none of the others seem to do that. In
>> fact the only long-running programs on my systems that I know to be
>> multi- threaded are Apache, Postgres, SA and Postfix.
> 
> In your particular case you have no need of optimization of your disk
> processes. You don't mention it but by omission I will grant you that
> virtual memory on your system does not seriously contend for disk
> either.
>
Well spotted. My type of load almost never swaps. That was the case with 
the old 512 MB RAM box and is double true with its replacement (4 GB 
RAM), but that still doesn't stop me setting swap space at twice RAM.

In fact the only program I have that does use gobs on RAM is a JavaMail + 
Postgres app and I'm not sure if its a problem due to JavaMail's queueing 
or if I've got overly long lived Object instances. Tracking this down in 
on my to-do list. All I know at present is that the same program using 
the same JVM uses gobs more RAM on the new machine (which is 6 times 
faster as well as having 8x more RAM), so it might simply be a case of 
persuading the GC to run more often. 

> But a typical consumer scenario is to listen to a stream while
> surfing the web on Windows with several chat windows open, causing
> multiple disk IO ops on a constant basis of themselves and also putting
> pressure on virtual memory. Even such a single-user system can benefit
> from elevator seeking and on-disk buffers.
>
I'm not saying head movement optimisation is a bad thing, just that it 
can be difficult to get enough queued requests for it to work without a 
large population of active processes that all do a lot of disk accesses.

You may well be right about the typical consumer setup: I lack any 
experience that: all I understand is the pattern that my own use pattern 
generates. However, I would point out that streamed music or video may 
never touch the disk (though of course a torrent will). The amount of 
disk i/o due to chat/IM/Twitter/web browsers may be less that we'd expect 
because its (a) very bursty and (b) disk i/o time is vastly outweighed by 
human reading and typing time.
  
> Consider also that burstiness of demand does not argue against the need
> for optimization, really. During bursts the optimization helps, and a
> user might complain if their disks got weird once an hour.
>
Sure, but the user's activity scan and resulting interaction with one 
program at a time, which may well be single threaded, for a few minutes 
before switching to another. This tends to produce widely separated 
bursts of i/o from one or two processes.
 
> Regardless, if you don't need optimization why worry? It's like the Pope
> comparing brands of condoms.
> 
Like it! 

> Again, we don't excoriate the value of optimizations by citing examples
> where optimization isn't needed. We evaluate optimizations by how useful
> they are when they are needed.
>
I wasn't intending to do that, having seen just how well head scheduling 
works. I merely intended to point out that there are corner cases where 
such algorithms don't help - but are not a hindrance either.


-- 
martin@   | Martin Gregorie
gregorie. | Essex, UK
org       |

Back to comp.lang.java.programmer | Previous | Next — Previous in thread | Next in thread | Find similar

Thread

O.T. optimising file placement Roedy Green <see_website@mindprod.com.invalid> - 2012-02-21 09:59 -0800
  Re: O.T. optimising file placement Daniel Pitts <newsgroup.nospam@virtualinfinity.net> - 2012-02-21 18:05 -0800
  Re: O.T. optimising file placement Jeff Higgins <jeff@invalid.invalid> - 2012-02-21 21:41 -0500
    Re: O.T. optimising file placement Jeff Higgins <jeff@invalid.invalid> - 2012-02-21 23:23 -0500
      Re: O.T. optimising file placement Roedy Green <see_website@mindprod.com.invalid> - 2012-02-22 11:15 -0800
        Re: O.T. optimising file placement Jeff Higgins <jeff@invalid.invalid> - 2012-02-22 16:00 -0500
          Re: O.T. optimising file placement Jeff Higgins <jeff@invalid.invalid> - 2012-02-22 16:03 -0500
            Re: O.T. optimising file placement Gene Wirchenko <genew@ocis.net> - 2012-02-22 14:52 -0800
              Re: O.T. optimising file placement Jeff Higgins <jeff@invalid.invalid> - 2012-02-22 18:24 -0500
                Re: O.T. optimising file placement Daniel Pitts <newsgroup.nospam@virtualinfinity.net> - 2012-02-22 17:10 -0800
                Re: O.T. optimising file placement Gene Wirchenko <genew@ocis.net> - 2012-02-22 20:02 -0800
        Re: O.T. optimising file placement Lew <noone@lewscanon.com> - 2012-02-22 14:27 -0800
          Re: O.T. optimising file placement Martin Gregorie <martin@address-in-sig.invalid> - 2012-02-23 01:31 +0000
            Re: O.T. optimising file placement Arne Vajhøj <arne@vajhoej.dk> - 2012-02-22 21:40 -0500
              Re: O.T. optimising file placement Martin Gregorie <martin@address-in-sig.invalid> - 2012-02-23 23:16 +0000
                Re: O.T. optimising file placement Patricia Shanahan <pats@acm.org> - 2012-02-23 16:22 -0800
                Re: O.T. optimising file placement Patricia Shanahan <pats@acm.org> - 2012-02-23 21:45 -0800
                Re: O.T. optimising file placement Martin Gregorie <martin@address-in-sig.invalid> - 2012-02-24 20:21 +0000
                Re: O.T. optimising file placement Arne Vajhøj <arne@vajhoej.dk> - 2012-02-24 20:31 -0500
                Re: O.T. optimising file placement Martin Gregorie <martin@address-in-sig.invalid> - 2012-02-25 13:15 +0000
            Re: O.T. optimising file placement Lew <noone@lewscanon.com> - 2012-02-23 16:39 -0800
              Re: O.T. optimising file placement Martin Gregorie <martin@address-in-sig.invalid> - 2012-02-24 01:49 +0000
                Re: O.T. optimising file placement Lew <noone@lewscanon.com> - 2012-02-24 10:50 -0800
                Re: O.T. optimising file placement Martin Gregorie <martin@address-in-sig.invalid> - 2012-02-24 23:59 +0000
              Re: O.T. optimising file placement Arne Vajhøj <arne@vajhoej.dk> - 2012-02-24 20:25 -0500
    Re: O.T. optimising file placement Roedy Green <see_website@mindprod.com.invalid> - 2012-02-22 10:54 -0800

csiph-web