Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.programming > #2856

Re: Do integrity checks by file comparison really work?

Show all headers | View raw

On Wed, 16 Jan 2013 11:38:54 +0100, Thomas B <thomasb@nospam.it>
wrote:

>Thanks to everybody who has replied.
>
>[...]
>
>On 13/01/2013 12.04, Robert Wessel wrote:
>
>> It also has
>> a issue that you might not detect a write error if the read to that
>> device is satisfied from the original buffer use to write.  Also, you
>> might now have gotten the buffers written at all, if the OS is being
>> somewhat lazy about writing dirty buffers to disk.
>
>Yes, I was aware of this.
>
>> But so long as you're on different volumes, and you arrange to discard
>> all data in disk cache before you do the compare (which may require
>> powering off the system and restarting it in some cases), you should
>> be fine.  Or if you can ask the OS to address a file with physical
>> reads at all times.
>
>I don't understand what you are saying about "different volumes". Which 
>would be the problem when comparing two files on the same volume?
>
>You wrote: "I've never seen an OS that would share file buffers between 
>two files like that", so I suppose this should be the same for files on 
>the same volume.


Some OS's/filesystems can do a copy with the equivalent of a *nix hard
link, and then do a copy-on-write procedure if the file is ever
modified.  In that case you may have only a single copy of the file.
It can't do that to you across volumes.


>Regarding discarding/flushing the disk cache, should a simple system 
>reboot (instead of powering off) do the job, with most "normal" hard disks?


Usually.  Depending on how the drive is configured, it may still have
data in (device) cache not yet written to disk.  The OS should force a
cache flush on the device when it shuts down (on SCSI, a
Synchronize-Cache, for example).  But a reboot without a clean
shutdown might not flush things in all cases.  Higher end devices
would have non-volatile write buffers, which makes the issue mostly
moot (unless there's a time limit on the non-volatility, and you leave
the drive powered down for more than that).


>> OTOH, as a practical matter, this is silly, hard disks and file system
>> caches are reliable enough that the above doesn't really present and
>> meaningful exposure in the vast majority of cases.  And if you're that
>> worried, you should be using some removable media anyway.
>
>I do backups both to an internal hard drive, and to external USB hard 
>drives. Why are you suggesting to use some removable media (do you mean 
>a USB hard drive, or optical disks, or other?). I don't understand what 
>you mean, which is the advantage.


An external USB drive is fine.  It certainly counts as removable.  And
the OS should flush buffers when you dismount it properly.

Back to comp.programming | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Do integrity checks by file comparison really work? Thomas B <thomasb@nospam.it> - 2013-01-08 13:04 +0100
  Re: Do integrity checks by file comparison really work? Robert Wessel <robertwessel2@yahoo.com> - 2013-01-13 05:04 -0600
    Re: Do integrity checks by file comparison really work? Mark F <mark53916@gmail.com> - 2013-01-13 09:26 -0500
      Re: Do integrity checks by file comparison really work? Mark F <mark53916@gmail.com> - 2013-01-13 09:40 -0500
    Re: Do integrity checks by file comparison really work? Thomas B <thomasb@nospam.it> - 2013-01-16 11:38 +0100
      Re: Do integrity checks by file comparison really work? Robert Wessel <robertwessel2@yahoo.com> - 2013-01-18 15:07 -0600
        Re: Do integrity checks by file comparison really work? Ian Collins <ian-news@hotmail.com> - 2013-01-19 17:12 +1300
          Re: Do integrity checks by file comparison really work? Robert Wessel <robertwessel2@yahoo.com> - 2013-01-18 22:37 -0600
            Re: Do integrity checks by file comparison really work? Ian Collins <ian-news@hotmail.com> - 2013-01-19 18:36 +1300
        Re: Do integrity checks by file comparison really work? Thomas B <thomasb@nospam.it> - 2013-01-19 14:06 +0100
          Re: Do integrity checks by file comparison really work? Robert Wessel <robertwessel2@yahoo.com> - 2013-01-21 03:40 -0600
            Re: Do integrity checks by file comparison really work? Thomas B <thomasb@nospam.it> - 2013-01-21 17:41 +0100
              Re: Do integrity checks by file comparison really work? Robert Wessel <robertwessel2@yahoo.com> - 2013-01-21 15:28 -0600
  Re: Do integrity checks by file comparison really work? Willem <willem@turtle.stack.nl> - 2013-01-14 18:23 +0000
    Re: Do integrity checks by file comparison really work? Thomas B <thomasb@nospam.it> - 2013-01-16 16:46 +0100
      Re: Do integrity checks by file comparison really work? Robert Wessel <robertwessel2@yahoo.com> - 2013-01-18 15:09 -0600

csiph-web