Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]


Groups > comp.lang.java.programmer > #12235 > unrolled thread

O.T. optimising file placement

Started byRoedy Green <see_website@mindprod.com.invalid>
First post2012-02-21 09:59 -0800
Last post2012-02-22 10:54 -0800
Articles 20 on this page of 26 — 8 participants

Back to article view | Back to comp.lang.java.programmer


Contents

  O.T. optimising file placement Roedy Green <see_website@mindprod.com.invalid> - 2012-02-21 09:59 -0800
    Re: O.T. optimising file placement Daniel Pitts <newsgroup.nospam@virtualinfinity.net> - 2012-02-21 18:05 -0800
    Re: O.T. optimising file placement Jeff Higgins <jeff@invalid.invalid> - 2012-02-21 21:41 -0500
      Re: O.T. optimising file placement Jeff Higgins <jeff@invalid.invalid> - 2012-02-21 23:23 -0500
        Re: O.T. optimising file placement Roedy Green <see_website@mindprod.com.invalid> - 2012-02-22 11:15 -0800
          Re: O.T. optimising file placement Jeff Higgins <jeff@invalid.invalid> - 2012-02-22 16:00 -0500
            Re: O.T. optimising file placement Jeff Higgins <jeff@invalid.invalid> - 2012-02-22 16:03 -0500
              Re: O.T. optimising file placement Gene Wirchenko <genew@ocis.net> - 2012-02-22 14:52 -0800
                Re: O.T. optimising file placement Jeff Higgins <jeff@invalid.invalid> - 2012-02-22 18:24 -0500
                  Re: O.T. optimising file placement Daniel Pitts <newsgroup.nospam@virtualinfinity.net> - 2012-02-22 17:10 -0800
                  Re: O.T. optimising file placement Gene Wirchenko <genew@ocis.net> - 2012-02-22 20:02 -0800
          Re: O.T. optimising file placement Lew <noone@lewscanon.com> - 2012-02-22 14:27 -0800
            Re: O.T. optimising file placement Martin Gregorie <martin@address-in-sig.invalid> - 2012-02-23 01:31 +0000
              Re: O.T. optimising file placement Arne Vajhøj <arne@vajhoej.dk> - 2012-02-22 21:40 -0500
                Re: O.T. optimising file placement Martin Gregorie <martin@address-in-sig.invalid> - 2012-02-23 23:16 +0000
                  Re: O.T. optimising file placement Patricia Shanahan <pats@acm.org> - 2012-02-23 16:22 -0800
                  Re: O.T. optimising file placement Patricia Shanahan <pats@acm.org> - 2012-02-23 21:45 -0800
                    Re: O.T. optimising file placement Martin Gregorie <martin@address-in-sig.invalid> - 2012-02-24 20:21 +0000
                  Re: O.T. optimising file placement Arne Vajhøj <arne@vajhoej.dk> - 2012-02-24 20:31 -0500
                    Re: O.T. optimising file placement Martin Gregorie <martin@address-in-sig.invalid> - 2012-02-25 13:15 +0000
              Re: O.T. optimising file placement Lew <noone@lewscanon.com> - 2012-02-23 16:39 -0800
                Re: O.T. optimising file placement Martin Gregorie <martin@address-in-sig.invalid> - 2012-02-24 01:49 +0000
                  Re: O.T. optimising file placement Lew <noone@lewscanon.com> - 2012-02-24 10:50 -0800
                    Re: O.T. optimising file placement Martin Gregorie <martin@address-in-sig.invalid> - 2012-02-24 23:59 +0000
                Re: O.T. optimising file placement Arne Vajhøj <arne@vajhoej.dk> - 2012-02-24 20:25 -0500
      Re: O.T. optimising file placement Roedy Green <see_website@mindprod.com.invalid> - 2012-02-22 10:54 -0800

Page 1 of 2  [1] 2  Next page →


#12235 — O.T. optimising file placement

FromRoedy Green <see_website@mindprod.com.invalid>
Date2012-02-21 09:59 -0800
SubjectO.T. optimising file placement
Message-ID<3gm7k7lgroqn30g0nd7lqgsqo60hq02em1@4ax.com>
Let's say I had an SSD. see http://mindprod.com/bgloss/ssd.html Lets
say I had a list of my most active files, some of which were data
files and some were windows system files.

Is there a way to move these files to SSD in a way that Java, the OS,
Windows etc act as if it did not notice they had moved? Can something
be done with those symbolic links?  Is there a disk driver that uses
the SSD transparently as a cache for the most active files or
clusters?
-- 
Roedy Green Canadian Mind Products
http://mindprod.com
One of the most useful comments you can put in a program is 
"If you change this, remember to change ?XXX? too".
 

[toc] | [next] | [standalone]


#12242

FromDaniel Pitts <newsgroup.nospam@virtualinfinity.net>
Date2012-02-21 18:05 -0800
Message-ID<P3Y0r.9875$yb.4477@newsfe20.iad>
In reply to#12235
On 2/21/12 9:59 AM, Roedy Green wrote:
> Let's say I had an SSD. see http://mindprod.com/bgloss/ssd.html Lets
> say I had a list of my most active files, some of which were data
> files and some were windows system files.
>
> Is there a way to move these files to SSD in a way that Java, the OS,
> Windows etc act as if it did not notice they had moved? Can something
> be done with those symbolic links?  Is there a disk driver that uses
> the SSD transparently as a cache for the most active files or
> clusters?

I seem to recall hearing about a way to tell windows to use SSD as a 
kind of smart cache, but I haven't played around with it.

[toc] | [prev] | [next] | [standalone]


#12244

FromJeff Higgins <jeff@invalid.invalid>
Date2012-02-21 21:41 -0500
Message-ID<ji1kln$k5a$1@dont-email.me>
In reply to#12235
On 02/21/2012 12:59 PM, Roedy Green wrote:
> Let's say I had an SSD. see http://mindprod.com/bgloss/ssd.html Lets
> say I had a list of my most active files, some of which were data
> files and some were windows system files.
>
> Is there a way to move these files to SSD in a way that Java, the OS,
> Windows etc act as if it did not notice they had moved? Can something
> be done with those symbolic links?  Is there a disk driver that uses
> the SSD transparently as a cache for the most active files or
> clusters?
<http://en.wikipedia.org/wiki/ReadyBoost>
<http://en.wikipedia.org/wiki/Smart_Response_Technology>
<http://www.highpoint-tech.com/USA_new/series_RC3240X8.htm>

[toc] | [prev] | [next] | [standalone]


#12245

FromJeff Higgins <jeff@invalid.invalid>
Date2012-02-21 23:23 -0500
Message-ID<ji1qjm$drr$1@dont-email.me>
In reply to#12244
On 02/21/2012 09:41 PM, Jeff Higgins wrote:
> On 02/21/2012 12:59 PM, Roedy Green wrote:
>> Let's say I had an SSD. see http://mindprod.com/bgloss/ssd.html Lets
>> say I had a list of my most active files, some of which were data
>> files and some were windows system files.
>>
>> Is there a way to move these files to SSD in a way that Java, the OS,
>> Windows etc act as if it did not notice they had moved? Can something
>> be done with those symbolic links? Is there a disk driver that uses
>> the SSD transparently as a cache for the most active files or
>> clusters?
> <http://en.wikipedia.org/wiki/ReadyBoost>
> <http://en.wikipedia.org/wiki/Smart_Response_Technology>
> <http://www.highpoint-tech.com/USA_new/series_RC3240X8.htm>
>
<http://en.wikipedia.org/wiki/Hybrid_drive>
<http://en.wikipedia.org/wiki/Vista_IO_technologies#ReadyDrive>
<http://www.ocztechnology.com/synapse-faq>
etc. ... giyf

[toc] | [prev] | [next] | [standalone]


#12250

FromRoedy Green <see_website@mindprod.com.invalid>
Date2012-02-22 11:15 -0800
Message-ID<f6fak7d0tf9tva9lqh0jtd6imjgm8u5oa3@4ax.com>
In reply to#12245
On Tue, 21 Feb 2012 23:23:14 -0500, Jeff Higgins
<jeff@invalid.invalid> wrote, quoted or indirectly quoted someone who
said :

><http://en.wikipedia.org/wiki/Hybrid_drive>

Back before the Internet, I was pushing for what I call "Marthaing"
drives. We might get them any year now.  The idea is a microprocessor
inside the hard disk in the background keeps reordering the tracks so
that the most frequently used ones are near the outer rim.
It would have a delayed write cache built in.  Today it would be
easier to implement since you could use a  non-volatile fast SSD to
store the map of logical to physical track numbers and a fat delayed
write. In the background the disk moves infrequently used tracks from
the outer rim toward the center in order to free up slots for writing.

People are willing to pay double or triple for 7200 RPM to 10,000 RPM.
With that sort of budget, used instead for internal cleverness, RAM
and SSD you might be able to get considerably better performance.
-- 
Roedy Green Canadian Mind Products
http://mindprod.com
One of the most useful comments you can put in a program is 
"If you change this, remember to change ?XXX? too".
 

[toc] | [prev] | [next] | [standalone]


#12251

FromJeff Higgins <jeff@invalid.invalid>
Date2012-02-22 16:00 -0500
Message-ID<ji3l0j$63h$1@dont-email.me>
In reply to#12250
On 02/22/2012 02:15 PM, Roedy Green wrote:
>
> Back before the Internet, I was pushing for what I call "Marthaing"
> drives. We might get them any year now.
Who is Martha? Back before the Internet I was advocating for squeezable 
catsup bottles. We have'em, but I haven't got a dime for'em. :(

[toc] | [prev] | [next] | [standalone]


#12252

FromJeff Higgins <jeff@invalid.invalid>
Date2012-02-22 16:03 -0500
Message-ID<ji3l7c$7l3$1@dont-email.me>
In reply to#12251
On 02/22/2012 04:00 PM, Jeff Higgins wrote:
> On 02/22/2012 02:15 PM, Roedy Green wrote:
>>
>> Back before the Internet, I was pushing for what I call "Marthaing"
>> drives. We might get them any year now.
> Who is Martha? Back before the Internet I was advocating for squeezable
> catsup bottles. We have'em, but I haven't got a dime for'em. :(
>
>
<http://uncyclopedia.wikia.com/wiki/Ketchup_v._Catsup>

[toc] | [prev] | [next] | [standalone]


#12257

FromGene Wirchenko <genew@ocis.net>
Date2012-02-22 14:52 -0800
Message-ID<8csak7h6a6a0ubhh9lo1ld7ocgf5onvgc0@4ax.com>
In reply to#12252
On Wed, 22 Feb 2012 16:03:41 -0500, Jeff Higgins
<jeff@invalid.invalid> wrote:

>On 02/22/2012 04:00 PM, Jeff Higgins wrote:
>> On 02/22/2012 02:15 PM, Roedy Green wrote:
>>>
>>> Back before the Internet, I was pushing for what I call "Marthaing"
>>> drives. We might get them any year now.
>> Who is Martha? Back before the Internet I was advocating for squeezable
>> catsup bottles. We have'em, but I haven't got a dime for'em. :(

><http://uncyclopedia.wikia.com/wiki/Ketchup_v._Catsup>

     I noticed that the tone is not as academic as Wikipedia.

Sincerely,

Gene Wirchenko

[toc] | [prev] | [next] | [standalone]


#12259

FromJeff Higgins <jeff@invalid.invalid>
Date2012-02-22 18:24 -0500
Message-ID<ji3ten$pmt$1@dont-email.me>
In reply to#12257
On 02/22/2012 05:52 PM, Gene Wirchenko wrote:
> On Wed, 22 Feb 2012 16:03:41 -0500, Jeff Higgins
> <jeff@invalid.invalid>  wrote:
>
>> On 02/22/2012 04:00 PM, Jeff Higgins wrote:
>>> On 02/22/2012 02:15 PM, Roedy Green wrote:
>>>>
>>>> Back before the Internet, I was pushing for what I call "Marthaing"
>>>> drives. We might get them any year now.
>>> Who is Martha? Back before the Internet I was advocating for squeezable
>>> catsup bottles. We have'em, but I haven't got a dime for'em. :(
>
>> <http://uncyclopedia.wikia.com/wiki/Ketchup_v._Catsup>
>
>       I noticed that the tone is not as academic as Wikipedia.
>
Yep ;)
"Tomato ketchup is a pseudoplastic — or "shear thinning" substance — 
which can make it difficult to pour from a glass bottle."

[toc] | [prev] | [next] | [standalone]


#12261

FromDaniel Pitts <newsgroup.nospam@virtualinfinity.net>
Date2012-02-22 17:10 -0800
Message-ID<3mg1r.11230$yb.3048@newsfe20.iad>
In reply to#12259
On 2/22/12 3:24 PM, Jeff Higgins wrote:
> On 02/22/2012 05:52 PM, Gene Wirchenko wrote:
>> On Wed, 22 Feb 2012 16:03:41 -0500, Jeff Higgins
>> <jeff@invalid.invalid> wrote:
>>
>>> On 02/22/2012 04:00 PM, Jeff Higgins wrote:
>>>> On 02/22/2012 02:15 PM, Roedy Green wrote:
>>>>>
>>>>> Back before the Internet, I was pushing for what I call "Marthaing"
>>>>> drives. We might get them any year now.
>>>> Who is Martha? Back before the Internet I was advocating for squeezable
>>>> catsup bottles. We have'em, but I haven't got a dime for'em. :(
>>
>>> <http://uncyclopedia.wikia.com/wiki/Ketchup_v._Catsup>
>>
>> I noticed that the tone is not as academic as Wikipedia.
>>
> Yep ;)
> "Tomato ketchup is a pseudoplastic — or "shear thinning" substance —
> which can make it difficult to pour from a glass bottle."
>
Edible Non Newtonian fluids FTW

[toc] | [prev] | [next] | [standalone]


#12264

FromGene Wirchenko <genew@ocis.net>
Date2012-02-22 20:02 -0800
Message-ID<uhebk7pg7grvp5p7cnjs34t4pu01mem6mm@4ax.com>
In reply to#12259
On Wed, 22 Feb 2012 18:24:06 -0500, Jeff Higgins
<jeff@invalid.invalid> wrote:

>On 02/22/2012 05:52 PM, Gene Wirchenko wrote:
>> On Wed, 22 Feb 2012 16:03:41 -0500, Jeff Higgins
>> <jeff@invalid.invalid>  wrote:
>>
>>> On 02/22/2012 04:00 PM, Jeff Higgins wrote:
>>>> On 02/22/2012 02:15 PM, Roedy Green wrote:
>>>>>
>>>>> Back before the Internet, I was pushing for what I call "Marthaing"
>>>>> drives. We might get them any year now.
>>>> Who is Martha? Back before the Internet I was advocating for squeezable
>>>> catsup bottles. We have'em, but I haven't got a dime for'em. :(
>>
>>> <http://uncyclopedia.wikia.com/wiki/Ketchup_v._Catsup>
>>
>>       I noticed that the tone is not as academic as Wikipedia.
>>
>Yep ;)
>"Tomato ketchup is a pseudoplastic — or "shear thinning" substance — 
>which can make it difficult to pour from a glass bottle."

     "Would you like fries with that?"

Sincerely,

Gene Wirchenko

[toc] | [prev] | [next] | [standalone]


#12256

FromLew <noone@lewscanon.com>
Date2012-02-22 14:27 -0800
Message-ID<ji3q44$le1$1@news.albasani.net>
In reply to#12250
On 02/22/2012 11:15 AM, Roedy Green wrote:
> On Tue, 21 Feb 2012 23:23:14 -0500, Jeff Higgins
> <jeff@invalid.invalid>  wrote, quoted or indirectly quoted someone who
> said :
>
>> <http://en.wikipedia.org/wiki/Hybrid_drive>
>
> Back before the Internet, I was pushing for what I call "Marthaing"
> drives. We might get them any year now.  The idea is a microprocessor
> inside the hard disk in the background keeps reordering the tracks so
> that the most frequently used ones are near the outer rim.

That won't help much.

> It would have a delayed write cache built in.  Today it would be
> easier to implement since you could use a  non-volatile fast SSD to
> store the map of logical to physical track numbers and a fat delayed
> write. In the background the disk moves infrequently used tracks from
> the outer rim toward the center in order to free up slots for writing.
>
> People are willing to pay double or triple for 7200 RPM to 10,000 RPM.
> With that sort of budget, used instead for internal cleverness, RAM
> and SSD you might be able to get considerably better performance.

Modern hard drives, pretty much all of them, have a buffer and microprocessor 
as part of the hardware. We're not going to get any "Marthaing" as you 
describe it (wherever the heck /that/ term came from) because what they're 
already doing is already so effective.

What they mostly do is collect read and write requests and combine them in 
elevator-seek order, along with full-track readahead. This optimizes disk 
access for single sweeps of the drive heads. The on-drive buffer also holds 
enough data for most reads and writes, overtaking any advantage that any 
(perforce extremely slow) physical re-ordering of the tracks could accomplish.

-- 
Lew
Honi soit qui mal y pense.
http://upload.wikimedia.org/wikipedia/commons/c/cf/Friz.jpg

[toc] | [prev] | [next] | [standalone]


#12262

FromMartin Gregorie <martin@address-in-sig.invalid>
Date2012-02-23 01:31 +0000
Message-ID<ji44tm$j5o$3@localhost.localdomain>
In reply to#12256
On Wed, 22 Feb 2012 14:27:17 -0800, Lew wrote:

> Modern hard drives, pretty much all of them, have a buffer and
> microprocessor as part of the hardware. We're not going to get any
> "Marthaing" as you describe it (wherever the heck /that/ term came from)
> because what they're already doing is already so effective.
> 
> What they mostly do is collect read and write requests and combine them
> in elevator-seek order, along with full-track readahead. This optimizes
> disk access for single sweeps of the drive heads.
>
Agreed, and a mainframe OS I was using in the early '70s (ICL's George 3) 
was doing it back then and very effective it is too for speeding up disk 
access. Back in the day it pushed the speed of the 2800 rpm, 60 MB 
washing-machine sized disk drives up from around 8 accesses/sec to 
something like 20-30 per sec.

However, its ineffective unless there are many active processes 
simultaneously requesting disk i/o. If all the requests come from one 
single threaded process then it can't optimize head movement because 
there's never more than one pending request at a time. I know this is 
reduction ad absurdam, but it does make the point that a small active 
process population is unlikely to be optimised as well as a large one. 
This is relevant today for allmost all single-user workstations 
regardless of whether they are running Windows, Linux or OS X. Since the 
majority of applications run on these machines are single threaded, about 
the only time you have more than one process accessing the disk is when 
the user is hammering away at a task, be it wordprocessing, spread-sheet, 
browser or IDE and the mail reader, sitting in the background, finds some 
mail waiting.  
 
> The on-drive buffer
> also holds enough data for most reads and writes, overtaking any
> advantage that any (perforce extremely slow) physical re-ordering of the
> tracks could accomplish.
>
Yep, the on-drive buffer will almost always be capable of holding several 
physical tracks and, in addition, on a *NIX system anyway, all RAM not 
occupied by running processes and their data will contain disk buffers.


-- 
martin@   | Martin Gregorie
gregorie. | Essex, UK
org       |

[toc] | [prev] | [next] | [standalone]


#12263

FromArne Vajhøj <arne@vajhoej.dk>
Date2012-02-22 21:40 -0500
Message-ID<4f45a713$0$291$14726298@news.sunsite.dk>
In reply to#12262
On 2/22/2012 8:31 PM, Martin Gregorie wrote:
> On Wed, 22 Feb 2012 14:27:17 -0800, Lew wrote:
>> Modern hard drives, pretty much all of them, have a buffer and
>> microprocessor as part of the hardware. We're not going to get any
>> "Marthaing" as you describe it (wherever the heck /that/ term came from)
>> because what they're already doing is already so effective.
>>
>> What they mostly do is collect read and write requests and combine them
>> in elevator-seek order, along with full-track readahead. This optimizes
>> disk access for single sweeps of the drive heads.
>>
> Agreed, and a mainframe OS I was using in the early '70s (ICL's George 3)
> was doing it back then and very effective it is too for speeding up disk
> access. Back in the day it pushed the speed of the 2800 rpm, 60 MB
> washing-machine sized disk drives up from around 8 accesses/sec to
> something like 20-30 per sec.
>
> However, its ineffective unless there are many active processes
> simultaneously requesting disk i/o. If all the requests come from one
> single threaded process then it can't optimize head movement because
> there's never more than one pending request at a time. I know this is
> reduction ad absurdam, but it does make the point that a small active
> process population is unlikely to be optimised as well as a large one.
> This is relevant today for allmost all single-user workstations
> regardless of whether they are running Windows, Linux or OS X. Since the
> majority of applications run on these machines are single threaded, about
> the only time you have more than one process accessing the disk is when
> the user is hammering away at a task, be it wordprocessing, spread-sheet,
> browser or IDE and the mail reader, sitting in the background, finds some
> mail waiting.

Most OS'es support async IO.

Arne

[toc] | [prev] | [next] | [standalone]


#12287

FromMartin Gregorie <martin@address-in-sig.invalid>
Date2012-02-23 23:16 +0000
Message-ID<ji6hcc$8j1$3@localhost.localdomain>
In reply to#12263
On Wed, 22 Feb 2012 21:40:19 -0500, Arne Vajhøj wrote:

> 
> Most OS'es support async IO. 
>
Yes, I know, but its not relevant to a single-threaded process since its 
logic generally requires it to wait for a read or write to complete 
before it continues[1]. Hence my comment that this prevents head movement 
being optimized unless a lot of processes are active because there's only 
one outstanding IOP per process.

[1] unless you're deliberately doing async i/o using poll() or 
    select() (in C) or nio (in Java), in which case the process is often
    best regarded as a half-way house between single and multi-threaded
    logic.


-- 
martin@   | Martin Gregorie
gregorie. | Essex, UK
org       |

[toc] | [prev] | [next] | [standalone]


#12288

FromPatricia Shanahan <pats@acm.org>
Date2012-02-23 16:22 -0800
Message-ID<AO6dnfwe0sjXRdvSnZ2dnUVZ_uydnZ2d@earthlink.com>
In reply to#12287
On 2/23/2012 3:16 PM, Martin Gregorie wrote:
> On Wed, 22 Feb 2012 21:40:19 -0500, Arne Vajhøj wrote:
>
>>
>> Most OS'es support async IO.
>>
> Yes, I know, but its not relevant to a single-threaded process since its
> logic generally requires it to wait for a read or write to complete
> before it continues[1]. Hence my comment that this prevents head movement
> being optimized unless a lot of processes are active because there's only
> one outstanding IOP per process.
>
> [1] unless you're deliberately doing async i/o using poll() or
>      select() (in C) or nio (in Java), in which case the process is often
>      best regarded as a half-way house between single and multi-threaded
>      logic.
>
>

Last time I looked at operating system internals, there was a lot of
prefetch logic that could generate multiple parallel requests from a
one-at-a-time sequence.

Patricia

[toc] | [prev] | [next] | [standalone]


#12291

FromPatricia Shanahan <pats@acm.org>
Date2012-02-23 21:45 -0800
Message-ID<dKmdnbUGUZqfudrSnZ2dnUVZ_g6dnZ2d@earthlink.com>
In reply to#12287
On 2/23/2012 3:16 PM, Martin Gregorie wrote:
> On Wed, 22 Feb 2012 21:40:19 -0500, Arne Vajhøj wrote:
>
>>
>> Most OS'es support async IO.
>>
> Yes, I know, but its not relevant to a single-threaded process since its
> logic generally requires it to wait for a read or write to complete
> before it continues[1]. Hence my comment that this prevents head movement
> being optimized unless a lot of processes are active because there's only
> one outstanding IOP per process.
>
> [1] unless you're deliberately doing async i/o using poll() or
>      select() (in C) or nio (in Java), in which case the process is often
>      best regarded as a half-way house between single and multi-threaded
>      logic.
>
>

There are some exceptions to this. For example, if you are reading a
file sequentially, the OS may prefetch blocks you have not yet
requested, and have multiple reads outstanding as a result.

Depending on the OS and how the IO is being handled, a write may appear
to be complete from the program's point of view once the data has been
copied to a kernel buffer. The OS may be writing out modified blocks,
including swap space blocks, at any time.

Patricia

[toc] | [prev] | [next] | [standalone]


#12297

FromMartin Gregorie <martin@address-in-sig.invalid>
Date2012-02-24 20:21 +0000
Message-ID<ji8rgu$s68$1@localhost.localdomain>
In reply to#12291
On Thu, 23 Feb 2012 21:45:33 -0800, Patricia Shanahan wrote:

> On 2/23/2012 3:16 PM, Martin Gregorie wrote:
>> On Wed, 22 Feb 2012 21:40:19 -0500, Arne Vajhøj wrote:
>>
>>
>>> Most OS'es support async IO.
>>>
>> Yes, I know, but its not relevant to a single-threaded process since
>> its logic generally requires it to wait for a read or write to complete
>> before it continues[1]. Hence my comment that this prevents head
>> movement being optimized unless a lot of processes are active because
>> there's only one outstanding IOP per process.
>>
>> [1] unless you're deliberately doing async i/o using poll() or
>>      select() (in C) or nio (in Java), in which case the process is
>>      often best regarded as a half-way house between single and
>>      multi-threaded logic.
>>
>>
>>
> There are some exceptions to this. For example, if you are reading a
> file sequentially, the OS may prefetch blocks you have not yet
> requested, and have multiple reads outstanding as a result.
>
Fair point, and I've seen blinding speed from reads where the disk 
drivers used track reads, but it still doesn't affect my point that 
there's still only one I/O request in the queue per active single 
threaded process. Head movement optimisation is simply sidestepped in 
this case.

> Depending on the OS and how the IO is being handled, a write may appear
> to be complete from the program's point of view once the data has been
> copied to a kernel buffer. The OS may be writing out modified blocks,
> including swap space blocks, at any time.
> 
Again agreed: its fair to regard a write as complete from the program's 
POV as soon as it can reread the block/record - something that many 
indexed sequential access schemes need to do to re-establish a 'current 
record' pointer.


-- 
martin@   | Martin Gregorie
gregorie. | Essex, UK
org       |

[toc] | [prev] | [next] | [standalone]


#12308

FromArne Vajhøj <arne@vajhoej.dk>
Date2012-02-24 20:31 -0500
Message-ID<4f4839ed$0$294$14726298@news.sunsite.dk>
In reply to#12287
On 2/23/2012 6:16 PM, Martin Gregorie wrote:
> On Wed, 22 Feb 2012 21:40:19 -0500, Arne Vajhøj wrote:
>> Most OS'es support async IO.
>>
> Yes, I know, but its not relevant to a single-threaded process since its
> logic generally requires it to wait for a read or write to complete
> before it continues[1]. Hence my comment that this prevents head movement
> being optimized unless a lot of processes are active because there's only
> one outstanding IOP per process.
>
> [1] unless you're deliberately doing async i/o using poll() or
>      select() (in C) or nio (in Java), in which case the process is often
>      best regarded as a half-way house between single and multi-threaded
>      logic.

I am talking about deliberate not accidental async IO.

And you maybe consider it half singlethreaded half multithreaded,
but when there is only one thread it is usually just called
single threaded.

Arne

[toc] | [prev] | [next] | [standalone]


#12314

FromMartin Gregorie <martin@address-in-sig.invalid>
Date2012-02-25 13:15 +0000
Message-ID<jiamtr$bco$1@localhost.localdomain>
In reply to#12308
On Fri, 24 Feb 2012 20:31:25 -0500, Arne Vajhøj wrote:

> On 2/23/2012 6:16 PM, Martin Gregorie wrote:
>> On Wed, 22 Feb 2012 21:40:19 -0500, Arne Vajhøj wrote:
>>> Most OS'es support async IO.
>>>
>> Yes, I know, but its not relevant to a single-threaded process since
>> its logic generally requires it to wait for a read or write to complete
>> before it continues[1]. Hence my comment that this prevents head
>> movement being optimized unless a lot of processes are active because
>> there's only one outstanding IOP per process.
>>
>> [1] unless you're deliberately doing async i/o using poll() or
>>      select() (in C) or nio (in Java), in which case the process is
>>      often best regarded as a half-way house between single and
>>      multi-threaded logic.
> 
> I am talking about deliberate not accidental async IO.
> 
> And you maybe consider it half singlethreaded half multithreaded,
> but when there is only one thread it is usually just called single
> threaded.
>
Yes, I agree that it is technically single threaded programming and 
therefore dodges a lot of problems associated with threading (especially 
in C).

Nonetheless most of the async i/o I've seen in C ends up treating 
transfers to and from a channel as a connected set of transfers, i.e. 
some context is associated with the channel and is kept distinct from 
context associated with other channels. The use of context implies having 
dedicated data storage for that channel, so from some viewpoints the 
logic of the program is indistinguishable from that used in a threaded 
approach. I can't comment about nio in Java since I haven't yet needed to 
use it.

Anyway, that's what I meant by the mongrel 'half-threaded' term.
   

-- 
martin@   | Martin Gregorie
gregorie. | Essex, UK
org       |

[toc] | [prev] | [next] | [standalone]


Page 1 of 2  [1] 2  Next page →

Back to top | Article view | comp.lang.java.programmer


csiph-web