Groups | Search | Server Info | Keyboard shortcuts | Login | Register

Groups > comp.os.linux.development.system > #657

Re: shred or scrub

Show all headers | View raw

On 27/04/14 13:59, crankypuss wrote:
> On 04/26/2014 08:30 AM, David Brown wrote:
>
>> I think you are failing to understand that there are multiple layers
>> here.
>
>> If the media does not support erasing or overwriting, or if it makes it
>> hard to do so reliably, then /no/ filesystem can help.
>
>> The device is not lying - it is just that no one claims that harddrives
>> (or other media) work the way you apparently think they do.  Once you
>> realise that your idea of harddisks as a simple array of directly
>> addressable sectors is oversimplifying, and completely wrong for flash
>> devices, then you will understand the issues here.
>
>> Nonsense.  You are imagining rules about what /you/ think device
>> manufacturers should do - but they are your rules, and neither device
>> manufacturers nor other users play by those rules.  There are no hard
>> and fast dividing lines here.
>
>> Reliability has no remote connection with your desire for 100%
>> erasability.
>
>> See earlier in this post for the critical issue of what a filesystem
>> actually /is/, and how it is independent of the media.
>>
>> A combination of filesystem and media may give you close to the 100%
>> data erasability that you are looking for, although it still assumes
>> there are no failures in hardware and software along the way.  Old hard
>> disks did not have relocatable sectors, and thus no risk of storing data
>> in places that were not addressable by the filesystem - such disks could
>> be used with a filesystem that supports scrubbing.
>
> It seems to me that we are talking at cross-purposes here; we are
> largely stating the same meanings but expressing them in different ways,
> and the syntax of our expressions is making their semantics seem more
> different than it is.
>
> You seem to think I'm saying that hardware makers should conform to my
> views of how hardware should work, but what I'm saying is something
> different, I'm saying that the software on the client system needs to
> ascertain the nature of the subsystem that serves it files and behave
> accordingly.  Maybe that will be ascertained by looking up the device
> identification presented by the device (they do sometimes fib about what
> they are when new hardware is introduced to support existing interface
> models) then finding its characteristics in some device information
> database, or it might be ascertained by monitoring the behavior of the
> device; eventually, we are likely to reach the point where only
> behavior-monitoring is useful, bestcase.
>

This would ruin the flexibility, testability, and modularisation of 
filesystems which are dependent on a separation of layers.  A filesystem 
should /not/ know about the block device under it - it should know only 
about the set of operations provided by that device (such as to read or 
write sectors of data, and flush queues of writes or otherwise report 
back that a write has "hit the metal").  There are a few other things 
that filesystems ask about in order to better optimise accesses, such as 
whether it is an SSD-class or harddisk-class device (to know about head 
movements), and stripe sizes (if it is a raid device).

Trying to have something like a database of devices leads to broken 
systems - it makes testing infeasible (as you need to test every 
filesystem with every device type), it leads to bugs and security holes 
(as everything is needlessly complicated, and cannot be tested), and 
endless user frustration (as they are unable to use the filesystem of 
their choice on the device of their choice, as that combination is not 
in the database).

There are good reasons why operating systems are built in layers.

> However.  One thing I am saying is that if an operating system claims to
> support file-scrubbing on a "filesystem" that does not support
> file-scrubbing, then that operating system is broken.

I don't disagree with that.  But since no operating system (that I know 
of) and no filesystem (that I know of) claim to support file scrubbing, 
it is a non-issue.

> If it allows a
> file that needs to be placed only on a "scrub-able" filesystem to be
> placed on any other type of filesystem, it is broken.  And if it does
> not provide interfaces that allow the programmer to identify the file as
> "scrub-able" versus "plain-old-data" when it is being created, then it
> is not exactly broken, but it definitely lame.

/True/ scrubbing is not possible, and is not supported by any system (as 
I have explained).

You can get /reasonable/ scrubbing on some filesystems, if the 
underlying media is a harddisk.  Such scrubs are often good enough.

>
> So, disclaimers that shred/scrub are only useful on this or that kind of
> filesystem are inherently faulty; at a minimum the shred/scrub
> operations should be returning a blatant failure indication when issued
> against a file residing on an unsupported filesystem (and never having
> used them, much less read their code, I don't know if that is the case
> or not, but as I recall the OP didn't mention their failing, only the
> warning not to trust them on this or that filesystem).

Note that the original poster is a half-wit troll (just look him up in 
the Usenet archives if you are not familiar with him), so don't pay any 
attention to what he has said.  He accidentally triggered an interesting 
discussion, but we would do best to ignore his actual post.

The "shred" manual page says fairly clearly what it does - it writes 
over a file one or more times "in order to make it harder for even very 
expensive hardware probing to recover the data".  And it makes clear the 
critical assumption - that the filesystem overwrites data in place.  It 
does not claim to completely delete all the data - it just claims to 
make it harder to recover, and only if the filesystem works in a 
particular way.

And even these claims are being made by a /program/ - not the operating 
system or the filesystem.

>
> Now, granted there are not a preponderance of devices available today
> that support the concept of scrubbing, but that does not mean that
> encryption as we know it today is the answer.  An interim answer, yes, I
> suppose, but not a real answer.  Passphrase-oriented protection
> mechanisms are only useful if the passphrase cannot be guessed (through
> cleverness or a brute-force attack) and if the owner of the data cannot
> be dragged away to some cellar where he is thumped-on or waterboarded or
> whatever until he gives up the passphrase.

There /is/ a way to handle this, and it is done today - either in 
software or in hardware (you can get disk drives with this feature built 
in).  You need two passwords - one is known to the user, and one is 
generated randomly on creation of the filesystem (or when the harddisk 
is first locked).  The user password is used to encrypt the random 
password, and this random password is stored in the disk's controller or 
in some other special way.  All data on the disk is encrypted using the 
random password.  So to be able to read something off the disk, the user 
must first present the user password in order to decode the real 
encryption password.  And you can safely do a 100% erasure of the disk 
by deleting the random password.  In the case of the secure hard disk, 
this password has never left the drive's electronics.  Even the toughest 
rubber-hose cryptoanalysis cannot recover the drive's contents once the 
random password is deleted, because the user never knew it.

>
> No, at present I have nothing better to offer than the passphrase
> concept, but if someone calls you on the phone and claims to be your
> wife or a close friend, it won't take you very long to identify the
> person as a fraud; I think it is the mechanisms involved therein that
> will provide a better answer than passphrase protection.
>
> And I think that operating systems should place more emphasis on who is
> allowed to access "files" than whether whoever it actually is has the
> proper passphrase.  I think that instead of continuing with the old
> philosophy of a "filesystem" implemented as an integral part of a
> monolithic operating system, we need to lean more toward the concept of
> device-handlers as servers, so that the increasingly complex world of
> cloud servers and suchlike does not continue to be handled as a "special
> case" but in the same was as local devices are handled.
>
> One thing that still amazes me is the way linux handles passwords, such
> as the omnipotent 'root' password.  Some GUI screen is presented and the
> user enters the password, with nothing whatsoever to indicate that the
> password request is not bogus.  There are better ways to handle that
> kind of thing imo.  Whatever, enjoy your pancakes.

Back to comp.os.linux.development.system | Previous | Next — Previous in thread | Next in thread | Find similar

Thread

shred or scrub "Bill Cunningham" <nospam@nspam.invalid> - 2014-04-16 18:17 -0400
  Re: shred or scrub crankypuss <crankypuss@nomail.invalid> - 2014-04-17 04:19 -0600
    Re: shred or scrub "Bill Cunningham" <nospam@nspam.invalid> - 2014-04-18 22:30 -0400
      Re: shred or scrub Jasen Betts <jasen@xnet.co.nz> - 2014-04-19 07:42 +0000
        Re: shred or scrub Richard Kettlewell <rjk@greenend.org.uk> - 2014-04-19 10:04 +0100
      Re: shred or scrub crankypuss <crankypuss@nomail.invalid> - 2014-04-19 02:15 -0600
      Re: shred or scrub Rainer Weikusat <rweikusat@mobileactivedefense.com> - 2014-04-19 23:05 +0100
        Re: shred or scrub crankypuss <crankypuss@nomail.invalid> - 2014-04-20 02:47 -0600
          Re: shred or scrub John Hasler <jhasler@newsguy.com> - 2014-04-20 07:56 -0500
            Re: shred or scrub crankypuss <crankypuss@nomail.invalid> - 2014-04-21 03:51 -0600
              Re: shred or scrub Jasen Betts <jasen@xnet.co.nz> - 2014-04-21 11:50 +0000
                Re: shred or scrub crankypuss <crankypuss@nomail.invalid> - 2014-04-21 06:14 -0600
            Re: shred or scrub "Bill Cunningham" <nospam@nspam.invalid> - 2014-04-21 18:44 -0400
          Re: shred or scrub Rainer Weikusat <rweikusat@mobileactivedefense.com> - 2014-04-21 13:24 +0100
            Re: shred or scrub crankypuss <crankypuss@nomail.invalid> - 2014-04-22 04:10 -0600
              Re: shred or scrub Rainer Weikusat <rweikusat@mobileactivedefense.com> - 2014-04-22 14:39 +0100
  Re: shred or scrub Kristof Provost <kristof@codepro.be> - 2014-04-17 13:15 +0000
    Re: shred or scrub John Hasler <jhasler@newsguy.com> - 2014-04-17 09:40 -0500
      Re: shred or scrub Kristof Provost <kristof@codepro.be> - 2014-04-18 14:40 +0000
    Re: shred or scrub crankypuss <crankypuss@nomail.invalid> - 2014-04-18 02:12 -0600
      Re: shred or scrub David Brown <david.brown@hesbynett.no> - 2014-04-18 11:49 +0200
        Re: shred or scrub crankypuss <crankypuss@nomail.invalid> - 2014-04-18 09:59 -0600
          Re: shred or scrub David Brown <david.brown@hesbynett.no> - 2014-04-21 16:14 +0200
            Re: shred or scrub crankypuss <crankypuss@nomail.invalid> - 2014-04-22 04:22 -0600
              Re: shred or scrub David Brown <david.brown@hesbynett.no> - 2014-04-23 00:06 +0200
                Re: shred or scrub crankypuss <crankypuss@nomail.invalid> - 2014-04-23 05:50 -0600
                Re: shred or scrub David Brown <david.brown@hesbynett.no> - 2014-04-24 22:46 +0200
                Re: shred or scrub crankypuss <crankypuss@nomail.invalid> - 2014-04-25 03:57 -0600
                Re: shred or scrub Rainer Weikusat <rweikusat@mobileactivedefense.com> - 2014-04-25 19:14 +0100
                Re: shred or scrub crankypuss <crankypuss@nomail.invalid> - 2014-04-26 04:02 -0600
                Re: shred or scrub Rainer Weikusat <rweikusat@mobileactivedefense.com> - 2014-04-27 21:26 +0100
                Re: shred or scrub crankypuss <crankypuss@nomail.invalid> - 2014-04-28 03:27 -0600
                Re: shred or scrub Rainer Weikusat <rweikusat@mobileactivedefense.com> - 2014-04-28 12:17 +0100
                Re: shred or scrub Rainer Weikusat <rweikusat@mobileactivedefense.com> - 2014-04-28 13:01 +0100
                Re: shred or scrub crankypuss <crankypuss@nomail.invalid> - 2014-04-29 02:50 -0600
                UNIX(*)/ Linux history & system design (was: shred or scrub) Rainer Weikusat <rweikusat@mobileactivedefense.com> - 2014-05-05 21:31 +0100
                Re: UNIX(*)/ Linux history & system design crankypuss <crankypuss@nomail.invalid> - 2014-05-05 16:02 -0600
                Re: UNIX(*)/ Linux history & system design David Brown <david.brown@hesbynett.no> - 2014-05-06 01:17 +0200
                Re: UNIX(*)/ Linux history & system design crankypuss <crankypuss@nomail.invalid> - 2014-05-06 01:46 -0600
                Re: UNIX(*)/ Linux history & system design Rainer Weikusat <rweikusat@mobileactivedefense.com> - 2014-05-06 15:09 +0100
                Re: UNIX(*)/ Linux history & system design crankypuss <crankypuss@nomail.invalid> - 2014-05-06 23:47 -0600
                Re: UNIX(*)/ Linux history & system design Rainer Weikusat <rweikusat@mobileactivedefense.com> - 2014-05-07 16:23 +0100
                Re: UNIX(*)/ Linux history & system design crankypuss <crankypuss@nomail.invalid> - 2014-05-07 10:51 -0600
                Re: UNIX(*)/ Linux history & system design Jerry Peters <jerry@example.invalid> - 2014-05-07 20:25 +0000
                Re: UNIX(*)/ Linux history & system design crankypuss <crankypuss@nomail.invalid> - 2014-05-08 03:50 -0600
                Re: UNIX(*)/ Linux history & system design Jerry Peters <jerry@example.invalid> - 2014-05-08 20:24 +0000
                Re: UNIX(*)/ Linux history & system design crankypuss <crankypuss@nomail.invalid> - 2014-05-09 02:23 -0600
                Re: UNIX(*)/ Linux history & system design Rainer Weikusat <rweikusat@mobileactivedefense.com> - 2014-05-09 18:36 +0100
                Re: UNIX(*)/ Linux history & system design Rainer Weikusat <rweikusat@mobileactivedefense.com> - 2014-05-09 21:24 +0100
                Re: UNIX(*)/ Linux history & system design Rainer Weikusat <rweikusat@mobileactivedefense.com> - 2014-05-07 22:01 +0100
                Re: UNIX(*)/ Linux history & system design crankypuss <crankypuss@nomail.invalid> - 2014-05-08 03:37 -0600
                Re: UNIX(*)/ Linux history & system design Rainer Weikusat <rweikusat@mobileactivedefense.com> - 2014-05-08 14:02 +0100
                Re: UNIX(*)/ Linux history & system design crankypuss <crankypuss@nomail.invalid> - 2014-05-09 02:56 -0600
                Re: UNIX(*)/ Linux history & system design David Brown <david.brown@hesbynett.no> - 2014-05-07 00:15 +0200
                Re: UNIX(*)/ Linux history & system design crankypuss <crankypuss@nomail.invalid> - 2014-05-07 00:32 -0600
                Re: UNIX(*)/ Linux history & system design Jorgen Grahn <grahn+nntp@snipabacken.se> - 2014-05-07 08:47 +0000
                Re: UNIX(*)/ Linux history & system design crankypuss <crankypuss@nomail.invalid> - 2014-05-07 10:59 -0600
                Re: UNIX(*)/ Linux history & system design Rainer Weikusat <rweikusat@mobileactivedefense.com> - 2014-05-06 14:35 +0100
                Re: shred or scrub David Brown <david.brown@hesbynett.no> - 2014-04-26 16:30 +0200
                Re: shred or scrub crankypuss <crankypuss@nomail.invalid> - 2014-04-27 05:59 -0600
                Re: shred or scrub Rainer Weikusat <rweikusat@mobileactivedefense.com> - 2014-04-27 20:15 +0100
                Re: shred or scrub crankypuss <crankypuss@nomail.invalid> - 2014-04-28 03:29 -0600
                Re: shred or scrub Rainer Weikusat <rweikusat@mobileactivedefense.com> - 2014-04-28 12:06 +0100
                Re: shred or scrub David Brown <david.brown@hesbynett.no> - 2014-04-27 21:41 +0200
                Re: shred or scrub crankypuss <crankypuss@nomail.invalid> - 2014-04-28 04:03 -0600
                Re: shred or scrub Richard Kettlewell <rjk@greenend.org.uk> - 2014-04-28 16:44 +0100
                Re: shred or scrub David Brown <david.brown@hesbynett.no> - 2014-04-28 23:39 +0200
      Re: shred or scrub John Hasler <jhasler@newsguy.com> - 2014-04-18 07:37 -0500
        Re: shred or scrub crankypuss <crankypuss@nomail.invalid> - 2014-04-18 10:16 -0600
          Re: shred or scrub John Hasler <jhasler@newsguy.com> - 2014-04-18 12:01 -0500
      Re: shred or scrub Kristof Provost <kristof@codepro.be> - 2014-04-18 14:42 +0000
  Re: shred or scrub David Brown <david.brown@hesbynett.no> - 2014-04-17 16:41 +0200

csiph-web