Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.java.help > #2380 > unrolled thread

FileOutputStream questions

Back to article view | Back to comp.lang.java.help

Contents

  FileOutputStream questions Roedy Green <see_website@mindprod.com.invalid> - 2012-12-25 06:36 -0800
    Re: FileOutputStream questions markspace <markspace@nospam.nospam> - 2012-12-25 07:54 -0800
      Re: FileOutputStream questions Roedy Green <see_website@mindprod.com.invalid> - 2012-12-26 01:30 -0800
    Re: FileOutputStream questions Jukka Lahtinen <jtfjdehf@hotmail.com.invalid> - 2012-12-25 18:32 +0200
    Re: FileOutputStream questions Knute Johnson <nospam@knutejohnson.com> - 2012-12-25 11:45 -0800
      Re: FileOutputStream questions Roedy Green <see_website@mindprod.com.invalid> - 2012-12-26 01:32 -0800
        Re: FileOutputStream questions Knute Johnson <nospam@knutejohnson.com> - 2012-12-26 09:19 -0800
    Re: FileOutputStream questions Roedy Green <see_website@mindprod.com.invalid> - 2012-12-25 13:17 -0800
      Re: FileOutputStream questions Knute Johnson <nospam@knutejohnson.com> - 2012-12-25 14:05 -0800
        Re: FileOutputStream questions Roedy Green <see_website@mindprod.com.invalid> - 2012-12-26 01:35 -0800
        Re: FileOutputStream questions markspace <markspace@nospam.nospam> - 2012-12-26 07:46 -0800
      Re: FileOutputStream questions Stanimir Stamenkov <s7an10@netscape.net> - 2013-01-01 02:36 +0200

#2380 — FileOutputStream questions

I have just realised I do not understand some basic things about
unbuffered FileOutputStream.

I used them in two ways, wrapping in a BufferedOutputStream or
BufferedWriter, or writing an entire file in one i/o without
buffering.

If you write single bytes at a time, will you trigger physical I/O on
every byte, or is there some small buffer in there anyway?

I have experimented with flush during file write. The file size stays
at 0 until I close, at least to DIR. 

What I want is to log bytes that will be largely recoverable even if
the program terminates unexpectedly without closing. Flush does not do
it.  Close/reopen periodically seem at bit heavy handed. Is there
something I am missing?
-- 
Roedy Green Canadian Mind Products http://mindprod.com
Students who hire or con others to do their homework are as foolish 
as couch potatoes who hire others to go to the gym for them. 

[toc] | [next] | [standalone]

#2381

On 12/25/2012 6:36 AM, Roedy Green wrote:

> I have experimented with flush during file write. The file size stays
> at 0 until I close, at least to DIR.

What's DIR?  The directory?

[toc] | [prev] | [next] | [standalone]

#2389

On Tue, 25 Dec 2012 07:54:07 -0800, markspace
<markspace@nospam.nospam> wrote, quoted or indirectly quoted someone
who said :

>What's DIR?  The directory?

the Windows command.exe/TCMD.exe intrinsic DIR directory command.

If I type dir somefile.bin during the generation of somefile.bin with
a FileOutputStream I see size 0 until close.
-- 
Roedy Green Canadian Mind Products http://mindprod.com
Students who hire or con others to do their homework are as foolish 
as couch potatoes who hire others to go to the gym for them. 

[toc] | [prev] | [next] | [standalone]

#2382

Roedy Green <see_website@mindprod.com.invalid> writes:

> I have just realised I do not understand some basic things about
> unbuffered FileOutputStream.
> I used them in two ways, wrapping in a BufferedOutputStream or
> BufferedWriter, or writing an entire file in one i/o without
> buffering.
> If you write single bytes at a time, will you trigger physical I/O on
> every byte, or is there some small buffer in there anyway?

I suppose that even if you use a non-buffering java class for file
writing, the operating system may still buffer the output. AFAIK, at
least Linux does that.

> I have experimented with flush during file write. The file size stays
> at 0 until I close, at least to DIR. 

The directory entry is probably not fully written or updated before you
close the fle. As long as the file is open for writing, the operating
system does not know wheter you will still be writing to it.

-- 
Jukka Lahtinen

[toc] | [prev] | [next] | [standalone]

#2383

On 12/25/2012 6:36 AM, Roedy Green wrote:
> I have just realised I do not understand some basic things about
> unbuffered FileOutputStream.
>
> I used them in two ways, wrapping in a BufferedOutputStream or
> BufferedWriter, or writing an entire file in one i/o without
> buffering.
>
> If you write single bytes at a time, will you trigger physical I/O on
> every byte, or is there some small buffer in there anyway?
>
> I have experimented with flush during file write. The file size stays
> at 0 until I close, at least to DIR.
>
> What I want is to log bytes that will be largely recoverable even if
> the program terminates unexpectedly without closing. Flush does not do
> it.  Close/reopen periodically seem at bit heavy handed. Is there
> something I am missing?
>

I use a buffered output stream with the file opened in append mode to 
keep some log files.  I flush those after every write.

"If the intended destination of this stream is an abstraction provided 
by the underlying operating system, for example a file, then flushing 
the stream guarantees only that bytes previously written to the stream 
are passed to the operating system for writing; it does not guarantee 
that they are actually written to a physical device such as a disk drive."

I haven't had any problems with the above getting it to write 
immediately.  All of this has been on WinXP which I assume you probably 
are with the mention of DIR.

Under Windows, even the OS has a write buffer.  I don't know what causes 
it to commit or if a Java stream flush is passed through.  I just looked 
at the source code and OutputStream.flush() doesn't do anything.

-- 

Knute Johnson

[toc] | [prev] | [next] | [standalone]

#2390

On Tue, 25 Dec 2012 11:45:30 -0800, Knute Johnson
<nospam@knutejohnson.com> wrote, quoted or indirectly quoted someone
who said :

>
>I use a buffered output stream with the file opened in append mode to 
>keep some log files.  I flush those after every write.

Have you ever aborted or powered off to see how much you lose?

I suspect flush is not really much help.

See http://mindprod.com/jgloss/flush.html for my findings.
-- 
Roedy Green Canadian Mind Products http://mindprod.com
Students who hire or con others to do their homework are as foolish 
as couch potatoes who hire others to go to the gym for them. 

[toc] | [prev] | [next] | [standalone]

#2396

On 12/26/2012 1:32 AM, Roedy Green wrote:
> On Tue, 25 Dec 2012 11:45:30 -0800, Knute Johnson
> <nospam@knutejohnson.com> wrote, quoted or indirectly quoted someone
> who said :
>
>>
>> I use a buffered output stream with the file opened in append mode to
>> keep some log files.  I flush those after every write.
>
> Have you ever aborted or powered off to see how much you lose?

No but I have copied the file while it was open and it didn't appear to 
lose any data.

> I suspect flush is not really much help.

The buffers are flushed anyway.

-- 

Knute Johnson

[toc] | [prev] | [next] | [standalone]

#2384

On Tue, 25 Dec 2012 06:36:29 -0800, Roedy Green
<see_website@mindprod.com.invalid> wrote, quoted or indirectly quoted
someone who said :

>What I want is to log bytes that will be largely recoverable even if
>the program terminates unexpectedly without closing. Flush does not do
>it.  Close/reopen periodically seem at bit heavy handed. Is there
>something I am missing?

I discovered this 

                fos.flush();
                fos.getFD().sync();

Which acts like close/reopen.
-- 
Roedy Green Canadian Mind Products http://mindprod.com
Students who hire or con others to do their homework are as foolish 
as couch potatoes who hire others to go to the gym for them. 

[toc] | [prev] | [next] | [standalone]

#2385

On 12/25/2012 1:17 PM, Roedy Green wrote:
> On Tue, 25 Dec 2012 06:36:29 -0800, Roedy Green
> <see_website@mindprod.com.invalid> wrote, quoted or indirectly quoted
> someone who said :
>
>> What I want is to log bytes that will be largely recoverable even if
>> the program terminates unexpectedly without closing. Flush does not do
>> it.  Close/reopen periodically seem at bit heavy handed. Is there
>> something I am missing?
>
> I discovered this
>
>                  fos.flush();
>                  fos.getFD().sync();
>
> Which acts like close/reopen.
>

I never saw that before and it has been there since almost the 
beginning.  Post back if that works for you please, cause I'm really 
curious now.

-- 

Knute Johnson

[toc] | [prev] | [next] | [standalone]

#2391

On Tue, 25 Dec 2012 14:05:52 -0800, Knute Johnson
<nospam@knutejohnson.com> wrote, quoted or indirectly quoted someone
who said :

>>
>
>I never saw that before and it has been there since almost the 
>beginning.  Post back if that works for you please, cause I'm really 
>curious now.

It does work, exactly as you would expect.  When I flush sync, I can
see a burst of disk activity, and when I look with dir, the file has
grown. There is not much point in getting bytes on disk if the length
in the directory is not updated too unless you were prepared to do
some low level recovery after a crash, or unless you allocated your
space, then closed, the opened, then just used flush, keeping track of
length by your own internal means.

I remember doing this sort of thing for database recovery back in the
mainframe days.
-- 
Roedy Green Canadian Mind Products http://mindprod.com
Students who hire or con others to do their homework are as foolish 
as couch potatoes who hire others to go to the gym for them. 

[toc] | [prev] | [next] | [standalone]

#2394

On 12/25/2012 2:05 PM, Knute Johnson wrote:

> I never saw that before and it has been there since almost the
> beginning.  Post back if that works for you please, cause I'm really
> curious now.

Googled for "java flush doesn't work", third result:

<http://stackoverflow.com/questions/11829922/logback-file-appender-doesnt-flush-immediately>

No, I didn't know about it either.

[toc] | [prev] | [next] | [standalone]

#2401

Tue, 25 Dec 2012 13:17:14 -0800, /Roedy Green/:

> I discovered this
>
>                  fos.flush();
>                  fos.getFD().sync();
>
> Which acts like close/reopen.

You may also checkout the following thread from the Xerces-J Users 
mailing list:

"Does LSSerailzer creates a thread of its own." 
<http://mail-archives.apache.org/mod_mbox/xerces-j-users/200708.mbox/browser>

The following reply suggest a newer Java NIO API - 
FileChannel.force(true), for the same purpose:

http://mail-archives.apache.org/mod_mbox/xerces-j-users/200708.mbox/%3C46D3FBFA.8060104%40netscape.net%3E

http://docs.oracle.com/javase/6/docs/api/java/nio/channels/FileChannel.html#force%28boolean%29

And the following appears just the same as your finding:

http://mail-archives.apache.org/mod_mbox/xerces-j-users/200708.mbox/%3C46D59744.2010505%40netscape.net%3E

-- 
Stanimir

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.java.help

csiph-web