Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.programming > #2814

Re: Do integrity checks by file comparison really work?

Show key headers only | View raw

Thomas B wrote:
) Hello,
)
) I'm sorry if this post is not really programming-related, but I thought 
) that programmers could know about it.
)
) I use a program (Beyond Compare) which has a Folder Compare module, with 
) a function for doing a binary comparison between the files inside two 
) folders. I use it to check the integrity of my backups, by comparing my 
) main hard disk files, with the ones on the backup hard disks. If two 
) files get recognized as being different, then one of them should be 
) corrupt (if they didn't get modified "normally").
)
) My question is: will this method work to detect real file corruptions? 

Yes, it should.

) Will it detect any kind of corruption, both copy corruptions, and "bit 
) rot" corruptions? Will there be no problem regarding read cache?

Well, if you just wrote it, it might remain in the cache and not get read
from disk.  But not what you meant below.

) I mean: 
) if the first file to compare gets read from disk, and is kept in a read 
) cache (by Windows or by the hard disk), will the second file get read 
) from the same cache?

No.

) Do Windows and hard disks have some kind of 
) procedure to detect if a file to read from disk is already available in 
) cache, even if it is in a different folder than the "original" one? 

No.

) Perhaps some kind of file-checksum-system which decides that the files 
) are same? (And this system would not notice if the file to compare is 
) corrupt). If this would be true, then integrity checks by file 
) comparison would not work, because in practice the same file would be 
) read twice (first from disk, and then from cache), instead of reading 
) both the two files to be compared from disk.

The only thing that could happen (but not on Windows) is that the file
system decides that two files are the same and then uses the same disk
space to store them in.  Which also means that if the disk gets corrupted,
both files are bad.

) I have already done test by manually "corrupting" files (changing 
) slightly the contents, while keeping size and timestamp the same), and 
) it works (the files get recognized as different). But I'm not sure if it 
) will work also with "real" corrupt files.

If you read immediately after writing, it might be read out of the write
cache.  It could even be that it hasn't actually been written to disk yet(*).

Other than that, there are no issues.


*) Some newer file systems keep small files in the write cache for a longer
   time, because it anticipates that those are temporary files which will
   be deleted soon anyway.  So they never make it to disk.


SaSW, Willem
-- 
Disclaimer: I am in no way responsible for any of the statements
            made in the above text. For all I know I might be
            drugged or something..
            No I'm not paranoid. You all think I'm paranoid, don't you !
#EOT

Back to comp.programming | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Do integrity checks by file comparison really work? Thomas B <thomasb@nospam.it> - 2013-01-08 13:04 +0100
  Re: Do integrity checks by file comparison really work? Robert Wessel <robertwessel2@yahoo.com> - 2013-01-13 05:04 -0600
    Re: Do integrity checks by file comparison really work? Mark F <mark53916@gmail.com> - 2013-01-13 09:26 -0500
      Re: Do integrity checks by file comparison really work? Mark F <mark53916@gmail.com> - 2013-01-13 09:40 -0500
    Re: Do integrity checks by file comparison really work? Thomas B <thomasb@nospam.it> - 2013-01-16 11:38 +0100
      Re: Do integrity checks by file comparison really work? Robert Wessel <robertwessel2@yahoo.com> - 2013-01-18 15:07 -0600
        Re: Do integrity checks by file comparison really work? Ian Collins <ian-news@hotmail.com> - 2013-01-19 17:12 +1300
          Re: Do integrity checks by file comparison really work? Robert Wessel <robertwessel2@yahoo.com> - 2013-01-18 22:37 -0600
            Re: Do integrity checks by file comparison really work? Ian Collins <ian-news@hotmail.com> - 2013-01-19 18:36 +1300
        Re: Do integrity checks by file comparison really work? Thomas B <thomasb@nospam.it> - 2013-01-19 14:06 +0100
          Re: Do integrity checks by file comparison really work? Robert Wessel <robertwessel2@yahoo.com> - 2013-01-21 03:40 -0600
            Re: Do integrity checks by file comparison really work? Thomas B <thomasb@nospam.it> - 2013-01-21 17:41 +0100
              Re: Do integrity checks by file comparison really work? Robert Wessel <robertwessel2@yahoo.com> - 2013-01-21 15:28 -0600
  Re: Do integrity checks by file comparison really work? Willem <willem@turtle.stack.nl> - 2013-01-14 18:23 +0000
    Re: Do integrity checks by file comparison really work? Thomas B <thomasb@nospam.it> - 2013-01-16 16:46 +0100
      Re: Do integrity checks by file comparison really work? Robert Wessel <robertwessel2@yahoo.com> - 2013-01-18 15:09 -0600

csiph-web