Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > comp.os.linux.development.system > #618 > unrolled thread
| Started by | "Bill Cunningham" <nospam@nspam.invalid> |
|---|---|
| First post | 2014-04-16 18:17 -0400 |
| Last post | 2014-04-17 16:41 +0200 |
| Articles | 12 on this page of 72 — 10 participants |
Back to article view | Back to comp.os.linux.development.system
shred or scrub "Bill Cunningham" <nospam@nspam.invalid> - 2014-04-16 18:17 -0400
Re: shred or scrub crankypuss <crankypuss@nomail.invalid> - 2014-04-17 04:19 -0600
Re: shred or scrub "Bill Cunningham" <nospam@nspam.invalid> - 2014-04-18 22:30 -0400
Re: shred or scrub Jasen Betts <jasen@xnet.co.nz> - 2014-04-19 07:42 +0000
Re: shred or scrub Richard Kettlewell <rjk@greenend.org.uk> - 2014-04-19 10:04 +0100
Re: shred or scrub crankypuss <crankypuss@nomail.invalid> - 2014-04-19 02:15 -0600
Re: shred or scrub Rainer Weikusat <rweikusat@mobileactivedefense.com> - 2014-04-19 23:05 +0100
Re: shred or scrub crankypuss <crankypuss@nomail.invalid> - 2014-04-20 02:47 -0600
Re: shred or scrub John Hasler <jhasler@newsguy.com> - 2014-04-20 07:56 -0500
Re: shred or scrub crankypuss <crankypuss@nomail.invalid> - 2014-04-21 03:51 -0600
Re: shred or scrub Jasen Betts <jasen@xnet.co.nz> - 2014-04-21 11:50 +0000
Re: shred or scrub crankypuss <crankypuss@nomail.invalid> - 2014-04-21 06:14 -0600
Re: shred or scrub "Bill Cunningham" <nospam@nspam.invalid> - 2014-04-21 18:44 -0400
Re: shred or scrub Rainer Weikusat <rweikusat@mobileactivedefense.com> - 2014-04-21 13:24 +0100
Re: shred or scrub crankypuss <crankypuss@nomail.invalid> - 2014-04-22 04:10 -0600
Re: shred or scrub Rainer Weikusat <rweikusat@mobileactivedefense.com> - 2014-04-22 14:39 +0100
Re: shred or scrub Kristof Provost <kristof@codepro.be> - 2014-04-17 13:15 +0000
Re: shred or scrub John Hasler <jhasler@newsguy.com> - 2014-04-17 09:40 -0500
Re: shred or scrub Kristof Provost <kristof@codepro.be> - 2014-04-18 14:40 +0000
Re: shred or scrub crankypuss <crankypuss@nomail.invalid> - 2014-04-18 02:12 -0600
Re: shred or scrub David Brown <david.brown@hesbynett.no> - 2014-04-18 11:49 +0200
Re: shred or scrub crankypuss <crankypuss@nomail.invalid> - 2014-04-18 09:59 -0600
Re: shred or scrub David Brown <david.brown@hesbynett.no> - 2014-04-21 16:14 +0200
Re: shred or scrub crankypuss <crankypuss@nomail.invalid> - 2014-04-22 04:22 -0600
Re: shred or scrub David Brown <david.brown@hesbynett.no> - 2014-04-23 00:06 +0200
Re: shred or scrub crankypuss <crankypuss@nomail.invalid> - 2014-04-23 05:50 -0600
Re: shred or scrub David Brown <david.brown@hesbynett.no> - 2014-04-24 22:46 +0200
Re: shred or scrub crankypuss <crankypuss@nomail.invalid> - 2014-04-25 03:57 -0600
Re: shred or scrub Rainer Weikusat <rweikusat@mobileactivedefense.com> - 2014-04-25 19:14 +0100
Re: shred or scrub crankypuss <crankypuss@nomail.invalid> - 2014-04-26 04:02 -0600
Re: shred or scrub Rainer Weikusat <rweikusat@mobileactivedefense.com> - 2014-04-27 21:26 +0100
Re: shred or scrub crankypuss <crankypuss@nomail.invalid> - 2014-04-28 03:27 -0600
Re: shred or scrub Rainer Weikusat <rweikusat@mobileactivedefense.com> - 2014-04-28 12:17 +0100
Re: shred or scrub Rainer Weikusat <rweikusat@mobileactivedefense.com> - 2014-04-28 13:01 +0100
Re: shred or scrub crankypuss <crankypuss@nomail.invalid> - 2014-04-29 02:50 -0600
UNIX(*)/ Linux history & system design (was: shred or scrub) Rainer Weikusat <rweikusat@mobileactivedefense.com> - 2014-05-05 21:31 +0100
Re: UNIX(*)/ Linux history & system design crankypuss <crankypuss@nomail.invalid> - 2014-05-05 16:02 -0600
Re: UNIX(*)/ Linux history & system design David Brown <david.brown@hesbynett.no> - 2014-05-06 01:17 +0200
Re: UNIX(*)/ Linux history & system design crankypuss <crankypuss@nomail.invalid> - 2014-05-06 01:46 -0600
Re: UNIX(*)/ Linux history & system design Rainer Weikusat <rweikusat@mobileactivedefense.com> - 2014-05-06 15:09 +0100
Re: UNIX(*)/ Linux history & system design crankypuss <crankypuss@nomail.invalid> - 2014-05-06 23:47 -0600
Re: UNIX(*)/ Linux history & system design Rainer Weikusat <rweikusat@mobileactivedefense.com> - 2014-05-07 16:23 +0100
Re: UNIX(*)/ Linux history & system design crankypuss <crankypuss@nomail.invalid> - 2014-05-07 10:51 -0600
Re: UNIX(*)/ Linux history & system design Jerry Peters <jerry@example.invalid> - 2014-05-07 20:25 +0000
Re: UNIX(*)/ Linux history & system design crankypuss <crankypuss@nomail.invalid> - 2014-05-08 03:50 -0600
Re: UNIX(*)/ Linux history & system design Jerry Peters <jerry@example.invalid> - 2014-05-08 20:24 +0000
Re: UNIX(*)/ Linux history & system design crankypuss <crankypuss@nomail.invalid> - 2014-05-09 02:23 -0600
Re: UNIX(*)/ Linux history & system design Rainer Weikusat <rweikusat@mobileactivedefense.com> - 2014-05-09 18:36 +0100
Re: UNIX(*)/ Linux history & system design Rainer Weikusat <rweikusat@mobileactivedefense.com> - 2014-05-09 21:24 +0100
Re: UNIX(*)/ Linux history & system design Rainer Weikusat <rweikusat@mobileactivedefense.com> - 2014-05-07 22:01 +0100
Re: UNIX(*)/ Linux history & system design crankypuss <crankypuss@nomail.invalid> - 2014-05-08 03:37 -0600
Re: UNIX(*)/ Linux history & system design Rainer Weikusat <rweikusat@mobileactivedefense.com> - 2014-05-08 14:02 +0100
Re: UNIX(*)/ Linux history & system design crankypuss <crankypuss@nomail.invalid> - 2014-05-09 02:56 -0600
Re: UNIX(*)/ Linux history & system design David Brown <david.brown@hesbynett.no> - 2014-05-07 00:15 +0200
Re: UNIX(*)/ Linux history & system design crankypuss <crankypuss@nomail.invalid> - 2014-05-07 00:32 -0600
Re: UNIX(*)/ Linux history & system design Jorgen Grahn <grahn+nntp@snipabacken.se> - 2014-05-07 08:47 +0000
Re: UNIX(*)/ Linux history & system design crankypuss <crankypuss@nomail.invalid> - 2014-05-07 10:59 -0600
Re: UNIX(*)/ Linux history & system design Rainer Weikusat <rweikusat@mobileactivedefense.com> - 2014-05-06 14:35 +0100
Re: shred or scrub David Brown <david.brown@hesbynett.no> - 2014-04-26 16:30 +0200
Re: shred or scrub crankypuss <crankypuss@nomail.invalid> - 2014-04-27 05:59 -0600
Re: shred or scrub Rainer Weikusat <rweikusat@mobileactivedefense.com> - 2014-04-27 20:15 +0100
Re: shred or scrub crankypuss <crankypuss@nomail.invalid> - 2014-04-28 03:29 -0600
Re: shred or scrub Rainer Weikusat <rweikusat@mobileactivedefense.com> - 2014-04-28 12:06 +0100
Re: shred or scrub David Brown <david.brown@hesbynett.no> - 2014-04-27 21:41 +0200
Re: shred or scrub crankypuss <crankypuss@nomail.invalid> - 2014-04-28 04:03 -0600
Re: shred or scrub Richard Kettlewell <rjk@greenend.org.uk> - 2014-04-28 16:44 +0100
Re: shred or scrub David Brown <david.brown@hesbynett.no> - 2014-04-28 23:39 +0200
Re: shred or scrub John Hasler <jhasler@newsguy.com> - 2014-04-18 07:37 -0500
Re: shred or scrub crankypuss <crankypuss@nomail.invalid> - 2014-04-18 10:16 -0600
Re: shred or scrub John Hasler <jhasler@newsguy.com> - 2014-04-18 12:01 -0500
Re: shred or scrub Kristof Provost <kristof@codepro.be> - 2014-04-18 14:42 +0000
Re: shred or scrub David Brown <david.brown@hesbynett.no> - 2014-04-17 16:41 +0200
Page 4 of 4 — ← Prev page 1 2 3 [4]
| From | Rainer Weikusat <rweikusat@mobileactivedefense.com> |
|---|---|
| Date | 2014-04-27 20:15 +0100 |
| Message-ID | <871twishw0.fsf@sable.mobileactivedefense.com> |
| In reply to | #655 |
crankypuss <crankypuss@nomail.invalid> writes: > On 04/26/2014 08:30 AM, David Brown wrote: [...] > You seem to think I'm saying that hardware makers should conform to my > views of how hardware should work, but what I'm saying is something > different, I'm saying that the software on the client system needs to > ascertain the nature of the subsystem that serves it files and behave > accordingly. This is not possible. A 'filesystem' is an implementation of an abstract 'user interface' providing certain operation which operates on a block device, itself an abstract interface to 'all kinds of things providing it'. This can be something a primitive as a 'floppy disk drive' where the OS block driver actually has to do stuff like turn the motor on and off to something as sophisticated as a 'distributed and replicated [network] block device', ie a cluster of servers implementing a certain protocol whose 'user interfaces' happens to be 'the block device user interface', http://www.drbd.org/home/what-is-drbd/ How filesystems and 'block device thingies' are combined is generally and administrative descision and even 'ordinary harddisks' are actually servers connected to local high-speed (or 'not so high-speed') bus which are operated by exchanging protocol requests and replies with them.
[toc] | [prev] | [next] | [standalone]
| From | crankypuss <crankypuss@nomail.invalid> |
|---|---|
| Date | 2014-04-28 03:29 -0600 |
| Message-ID | <ljl71g$rbi$2@dont-email.me> |
| In reply to | #656 |
On 04/27/2014 01:15 PM, Rainer Weikusat wrote: > crankypuss <crankypuss@nomail.invalid> writes: >> On 04/26/2014 08:30 AM, David Brown wrote: > > [...] > >> You seem to think I'm saying that hardware makers should conform to my >> views of how hardware should work, but what I'm saying is something >> different, I'm saying that the software on the client system needs to >> ascertain the nature of the subsystem that serves it files and behave >> accordingly. > > This is not possible. A 'filesystem' is an implementation of an abstract > 'user interface' providing certain operation which operates on a block > device, itself an abstract interface to 'all kinds of things providing > it'. This can be something a primitive as a 'floppy disk drive' where > the OS block driver actually has to do stuff like turn the motor on and > off to something as sophisticated as a 'distributed and replicated > [network] block device', ie a cluster of servers implementing a certain > protocol whose 'user interfaces' happens to be 'the block device user > interface', > > http://www.drbd.org/home/what-is-drbd/ > > How filesystems and 'block device thingies' are combined is generally > and administrative descision and even 'ordinary harddisks' are actually > servers connected to local high-speed (or 'not so high-speed') bus which > are operated by exchanging protocol requests and replies with them. > Thank you for your opinion.
[toc] | [prev] | [next] | [standalone]
| From | Rainer Weikusat <rweikusat@mobileactivedefense.com> |
|---|---|
| Date | 2014-04-28 12:06 +0100 |
| Message-ID | <878uqpww66.fsf@sable.mobileactivedefense.com> |
| In reply to | #660 |
crankypuss <crankypuss@nomail.invalid> writes: > On 04/27/2014 01:15 PM, Rainer Weikusat wrote: >> crankypuss <crankypuss@nomail.invalid> writes: >>> On 04/26/2014 08:30 AM, David Brown wrote: >> >> [...] >> >>> You seem to think I'm saying that hardware makers should conform to my >>> views of how hardware should work, but what I'm saying is something >>> different, I'm saying that the software on the client system needs to >>> ascertain the nature of the subsystem that serves it files and behave >>> accordingly. >> >> This is not possible. A 'filesystem' is an implementation of an abstract >> 'user interface' providing certain operation which operates on a block >> device, itself an abstract interface to 'all kinds of things providing >> it'. This can be something a primitive as a 'floppy disk drive' where >> the OS block driver actually has to do stuff like turn the motor on and >> off to something as sophisticated as a 'distributed and replicated >> [network] block device', ie a cluster of servers implementing a certain >> protocol whose 'user interfaces' happens to be 'the block device user >> interface', >> >> http://www.drbd.org/home/what-is-drbd/ >> >> How filesystems and 'block device thingies' are combined is generally >> and administrative descision and even 'ordinary harddisks' are actually >> servers connected to local high-speed (or 'not so high-speed') bus which >> are operated by exchanging protocol requests and replies with them. >> > > Thank you for your opinion. This is not 'my opinion'. It's a summary of some facts about the system we're talking about (and some other things which happen to be related to the topic, ie, that current 'hard disks' are actually servers providing an interface supposed to be compatible to some protocol definition).
[toc] | [prev] | [next] | [standalone]
| From | David Brown <david.brown@hesbynett.no> |
|---|---|
| Date | 2014-04-27 21:41 +0200 |
| Message-ID | <ljjmi2$a8n$1@dont-email.me> |
| In reply to | #655 |
On 27/04/14 13:59, crankypuss wrote: > On 04/26/2014 08:30 AM, David Brown wrote: > >> I think you are failing to understand that there are multiple layers >> here. > >> If the media does not support erasing or overwriting, or if it makes it >> hard to do so reliably, then /no/ filesystem can help. > >> The device is not lying - it is just that no one claims that harddrives >> (or other media) work the way you apparently think they do. Once you >> realise that your idea of harddisks as a simple array of directly >> addressable sectors is oversimplifying, and completely wrong for flash >> devices, then you will understand the issues here. > >> Nonsense. You are imagining rules about what /you/ think device >> manufacturers should do - but they are your rules, and neither device >> manufacturers nor other users play by those rules. There are no hard >> and fast dividing lines here. > >> Reliability has no remote connection with your desire for 100% >> erasability. > >> See earlier in this post for the critical issue of what a filesystem >> actually /is/, and how it is independent of the media. >> >> A combination of filesystem and media may give you close to the 100% >> data erasability that you are looking for, although it still assumes >> there are no failures in hardware and software along the way. Old hard >> disks did not have relocatable sectors, and thus no risk of storing data >> in places that were not addressable by the filesystem - such disks could >> be used with a filesystem that supports scrubbing. > > It seems to me that we are talking at cross-purposes here; we are > largely stating the same meanings but expressing them in different ways, > and the syntax of our expressions is making their semantics seem more > different than it is. > > You seem to think I'm saying that hardware makers should conform to my > views of how hardware should work, but what I'm saying is something > different, I'm saying that the software on the client system needs to > ascertain the nature of the subsystem that serves it files and behave > accordingly. Maybe that will be ascertained by looking up the device > identification presented by the device (they do sometimes fib about what > they are when new hardware is introduced to support existing interface > models) then finding its characteristics in some device information > database, or it might be ascertained by monitoring the behavior of the > device; eventually, we are likely to reach the point where only > behavior-monitoring is useful, bestcase. > This would ruin the flexibility, testability, and modularisation of filesystems which are dependent on a separation of layers. A filesystem should /not/ know about the block device under it - it should know only about the set of operations provided by that device (such as to read or write sectors of data, and flush queues of writes or otherwise report back that a write has "hit the metal"). There are a few other things that filesystems ask about in order to better optimise accesses, such as whether it is an SSD-class or harddisk-class device (to know about head movements), and stripe sizes (if it is a raid device). Trying to have something like a database of devices leads to broken systems - it makes testing infeasible (as you need to test every filesystem with every device type), it leads to bugs and security holes (as everything is needlessly complicated, and cannot be tested), and endless user frustration (as they are unable to use the filesystem of their choice on the device of their choice, as that combination is not in the database). There are good reasons why operating systems are built in layers. > However. One thing I am saying is that if an operating system claims to > support file-scrubbing on a "filesystem" that does not support > file-scrubbing, then that operating system is broken. I don't disagree with that. But since no operating system (that I know of) and no filesystem (that I know of) claim to support file scrubbing, it is a non-issue. > If it allows a > file that needs to be placed only on a "scrub-able" filesystem to be > placed on any other type of filesystem, it is broken. And if it does > not provide interfaces that allow the programmer to identify the file as > "scrub-able" versus "plain-old-data" when it is being created, then it > is not exactly broken, but it definitely lame. /True/ scrubbing is not possible, and is not supported by any system (as I have explained). You can get /reasonable/ scrubbing on some filesystems, if the underlying media is a harddisk. Such scrubs are often good enough. > > So, disclaimers that shred/scrub are only useful on this or that kind of > filesystem are inherently faulty; at a minimum the shred/scrub > operations should be returning a blatant failure indication when issued > against a file residing on an unsupported filesystem (and never having > used them, much less read their code, I don't know if that is the case > or not, but as I recall the OP didn't mention their failing, only the > warning not to trust them on this or that filesystem). Note that the original poster is a half-wit troll (just look him up in the Usenet archives if you are not familiar with him), so don't pay any attention to what he has said. He accidentally triggered an interesting discussion, but we would do best to ignore his actual post. The "shred" manual page says fairly clearly what it does - it writes over a file one or more times "in order to make it harder for even very expensive hardware probing to recover the data". And it makes clear the critical assumption - that the filesystem overwrites data in place. It does not claim to completely delete all the data - it just claims to make it harder to recover, and only if the filesystem works in a particular way. And even these claims are being made by a /program/ - not the operating system or the filesystem. > > Now, granted there are not a preponderance of devices available today > that support the concept of scrubbing, but that does not mean that > encryption as we know it today is the answer. An interim answer, yes, I > suppose, but not a real answer. Passphrase-oriented protection > mechanisms are only useful if the passphrase cannot be guessed (through > cleverness or a brute-force attack) and if the owner of the data cannot > be dragged away to some cellar where he is thumped-on or waterboarded or > whatever until he gives up the passphrase. There /is/ a way to handle this, and it is done today - either in software or in hardware (you can get disk drives with this feature built in). You need two passwords - one is known to the user, and one is generated randomly on creation of the filesystem (or when the harddisk is first locked). The user password is used to encrypt the random password, and this random password is stored in the disk's controller or in some other special way. All data on the disk is encrypted using the random password. So to be able to read something off the disk, the user must first present the user password in order to decode the real encryption password. And you can safely do a 100% erasure of the disk by deleting the random password. In the case of the secure hard disk, this password has never left the drive's electronics. Even the toughest rubber-hose cryptoanalysis cannot recover the drive's contents once the random password is deleted, because the user never knew it. > > No, at present I have nothing better to offer than the passphrase > concept, but if someone calls you on the phone and claims to be your > wife or a close friend, it won't take you very long to identify the > person as a fraud; I think it is the mechanisms involved therein that > will provide a better answer than passphrase protection. > > And I think that operating systems should place more emphasis on who is > allowed to access "files" than whether whoever it actually is has the > proper passphrase. I think that instead of continuing with the old > philosophy of a "filesystem" implemented as an integral part of a > monolithic operating system, we need to lean more toward the concept of > device-handlers as servers, so that the increasingly complex world of > cloud servers and suchlike does not continue to be handled as a "special > case" but in the same was as local devices are handled. > > One thing that still amazes me is the way linux handles passwords, such > as the omnipotent 'root' password. Some GUI screen is presented and the > user enters the password, with nothing whatsoever to indicate that the > password request is not bogus. There are better ways to handle that > kind of thing imo. Whatever, enjoy your pancakes.
[toc] | [prev] | [next] | [standalone]
| From | crankypuss <crankypuss@nomail.invalid> |
|---|---|
| Date | 2014-04-28 04:03 -0600 |
| Message-ID | <ljl90k$8mm$1@dont-email.me> |
| In reply to | #657 |
On 04/27/2014 01:41 PM, David Brown wrote: > On 27/04/14 13:59, crankypuss wrote: >> On 04/26/2014 08:30 AM, David Brown wrote: >> >>> I think you are failing to understand that there are multiple layers >>> here. >> >>> If the media does not support erasing or overwriting, or if it makes it >>> hard to do so reliably, then /no/ filesystem can help. >> >>> The device is not lying - it is just that no one claims that harddrives >>> (or other media) work the way you apparently think they do. Once you >>> realise that your idea of harddisks as a simple array of directly >>> addressable sectors is oversimplifying, and completely wrong for flash >>> devices, then you will understand the issues here. >> >>> Nonsense. You are imagining rules about what /you/ think device >>> manufacturers should do - but they are your rules, and neither device >>> manufacturers nor other users play by those rules. There are no hard >>> and fast dividing lines here. >> >>> Reliability has no remote connection with your desire for 100% >>> erasability. >> >>> See earlier in this post for the critical issue of what a filesystem >>> actually /is/, and how it is independent of the media. >>> >>> A combination of filesystem and media may give you close to the 100% >>> data erasability that you are looking for, although it still assumes >>> there are no failures in hardware and software along the way. Old hard >>> disks did not have relocatable sectors, and thus no risk of storing data >>> in places that were not addressable by the filesystem - such disks could >>> be used with a filesystem that supports scrubbing. >> >> It seems to me that we are talking at cross-purposes here; we are >> largely stating the same meanings but expressing them in different ways, >> and the syntax of our expressions is making their semantics seem more >> different than it is. >> >> You seem to think I'm saying that hardware makers should conform to my >> views of how hardware should work, but what I'm saying is something >> different, I'm saying that the software on the client system needs to >> ascertain the nature of the subsystem that serves it files and behave >> accordingly. Maybe that will be ascertained by looking up the device >> identification presented by the device (they do sometimes fib about what >> they are when new hardware is introduced to support existing interface >> models) then finding its characteristics in some device information >> database, or it might be ascertained by monitoring the behavior of the >> device; eventually, we are likely to reach the point where only >> behavior-monitoring is useful, bestcase. >> > > This would ruin the flexibility, testability, and modularisation of > filesystems which are dependent on a separation of layers. A filesystem > should /not/ know about the block device under it - it should know only > about the set of operations provided by that device (such as to read or > write sectors of data, and flush queues of writes or otherwise report > back that a write has "hit the metal"). There are a few other things > that filesystems ask about in order to better optimise accesses, such as > whether it is an SSD-class or harddisk-class device (to know about head > movements), and stripe sizes (if it is a raid device). > > Trying to have something like a database of devices leads to broken > systems - it makes testing infeasible (as you need to test every > filesystem with every device type), it leads to bugs and security holes > (as everything is needlessly complicated, and cannot be tested), and > endless user frustration (as they are unable to use the filesystem of > their choice on the device of their choice, as that combination is not > in the database). > > There are good reasons why operating systems are built in layers. Talking across one another some more, implementation layers versus abstraction layers, so it goes. >> However. One thing I am saying is that if an operating system claims to >> support file-scrubbing on a "filesystem" that does not support >> file-scrubbing, then that operating system is broken. > > I don't disagree with that. But since no operating system (that I know > of) and no filesystem (that I know of) claim to support file scrubbing, > it is a non-issue. Whatever, I've written a filesystem that would support scrubbing, but it's been obsolete for longer than most posters here have been alive. Over the years we learn things, and sometimes we see that what we thought were "abstractions" weren't really "the" abstraction we thought they were because of the things we've learned. Some people grasp a new level of abstraction and others don't, and whether it's because they aren't capable of further abstraction, or because they have vested interests in existing ones, isn't much relevant. >> If it allows a >> file that needs to be placed only on a "scrub-able" filesystem to be >> placed on any other type of filesystem, it is broken. And if it does >> not provide interfaces that allow the programmer to identify the file as >> "scrub-able" versus "plain-old-data" when it is being created, then it >> is not exactly broken, but it definitely lame. > > /True/ scrubbing is not possible, and is not supported by any system (as > I have explained). > > You can get /reasonable/ scrubbing on some filesystems, if the > underlying media is a harddisk. Such scrubs are often good enough. I suppose that whether one considers the "involate" attribute to mean one thing, or another, can make a difference. Back to "levels" again, oh my. >> So, disclaimers that shred/scrub are only useful on this or that kind of >> filesystem are inherently faulty; at a minimum the shred/scrub >> operations should be returning a blatant failure indication when issued >> against a file residing on an unsupported filesystem (and never having >> used them, much less read their code, I don't know if that is the case >> or not, but as I recall the OP didn't mention their failing, only the >> warning not to trust them on this or that filesystem). > > Note that the original poster is a half-wit troll (just look him up in > the Usenet archives if you are not familiar with him), so don't pay any > attention to what he has said. Even trolls occasionally stumble across an interesting point. > He accidentally triggered an interesting > discussion, but we would do best to ignore his actual post. > > The "shred" manual page says fairly clearly what it does - it writes > over a file one or more times "in order to make it harder for even very > expensive hardware probing to recover the data". And it makes clear the > critical assumption - that the filesystem overwrites data in place. It > does not claim to completely delete all the data - it just claims to > make it harder to recover, and only if the filesystem works in a > particular way. > > And even these claims are being made by a /program/ - not the operating > system or the filesystem. I'm going to carefully restrain myself here so as not to open the can of worms associated with the question of where an "operating system" begins and ends, whether the "operating system" is comprised of the kernel-and-below, or all-provided-interfaces (which includes commands as well as libs, and which pipeline-lovers could argue includes command-output as well). >> Now, granted there are not a preponderance of devices available today >> that support the concept of scrubbing, but that does not mean that >> encryption as we know it today is the answer. An interim answer, yes, I >> suppose, but not a real answer. Passphrase-oriented protection >> mechanisms are only useful if the passphrase cannot be guessed (through >> cleverness or a brute-force attack) and if the owner of the data cannot >> be dragged away to some cellar where he is thumped-on or waterboarded or >> whatever until he gives up the passphrase. > > There /is/ a way to handle this, and it is done today - either in > software or in hardware (you can get disk drives with this feature built > in). You need two passwords - one is known to the user, and one is > generated randomly on creation of the filesystem (or when the harddisk > is first locked). The user password is used to encrypt the random > password, and this random password is stored in the disk's controller or > in some other special way. All data on the disk is encrypted using the > random password. So to be able to read something off the disk, the user > must first present the user password in order to decode the real > encryption password. It isn't entirely clear whether what you are describing is dual-key cryptography like PGP uses, or something simpler and not truly multiplicative. Whatever level of encryption suits you is fine with me, it's a whole bunch easier to encrypt something than it is to decrypt it, there are not-very-difficult techniques that must make the NSA crazy. However, my point was that instead of using sloppy access protocols that make encryption essential, perhaps we should put more attention into the access protocol and leave data encryption as a private matter. I'm more interested in moving my own codebase forward another tiny step than I am in winning the perpetual debate; it's a more useful activity for all concerned imo.
[toc] | [prev] | [next] | [standalone]
| From | Richard Kettlewell <rjk@greenend.org.uk> |
|---|---|
| Date | 2014-04-28 16:44 +0100 |
| Message-ID | <wwvoazlihlh.fsf@l1AntVDjLrnP7Td3DQJ8ynzIq3lJMueXf87AxnpFoA.invalid> |
| In reply to | #657 |
David Brown <david.brown@hesbynett.no> writes: > The "shred" manual page says fairly clearly what it does - it writes > over a file one or more times "in order to make it harder for even > very expensive hardware probing to recover the data". And it makes > clear the critical assumption - that the filesystem overwrites data in > place. It does not claim to completely delete all the data - it just > claims to make it harder to recover, and only if the filesystem works > in a particular way. The supposed need for multiple writes would be more convincing if it actually identified the ‘very expensive hardware’ supposedly capable of retrieving data overwritten even once. Usually this stuff turns out to reflect recovery mechanisms that might have worked on disks from two decades ago, but are completely irrelevant to today’s hardware. > There /is/ a way to handle this, and it is done today - either in > software or in hardware (you can get disk drives with this feature > built in). You need two passwords - one is known to the user, and one > is generated randomly on creation of the filesystem (or when the > harddisk is first locked). The user password is used to encrypt the > random password, and this random password is stored in the disk's > controller or in some other special way. All data on the disk is > encrypted using the random password. So to be able to read something > off the disk, the user must first present the user password in order > to decode the real encryption password. And you can safely do a 100% > erasure of the disk by deleting the random password. In the case of > the secure hard disk, this password has never left the drive's > electronics. Even the toughest rubber-hose cryptoanalysis cannot > recover the drive's contents once the random password is deleted, > because the user never knew it. Encryption uses keys, not passwords. What you are almost describing is deriving a key encryption key from a passphrase and using that to decrypt a bulk data confidentiality key. As well as supporting passphrase revocation (supposing the encrypted data confidentiality key can successfully be erased), this is necessary to allow changing passphrases in a practical way: if the data confidentiality key were directly derived from the passphrase then changing passphrase would require re-encrypting the entire block device. It also allows multiple passphrases to be supported. -- http://www.greenend.org.uk/rjk/
[toc] | [prev] | [next] | [standalone]
| From | David Brown <david.brown@hesbynett.no> |
|---|---|
| Date | 2014-04-28 23:39 +0200 |
| Message-ID | <ljmhpu$soi$1@dont-email.me> |
| In reply to | #665 |
On 28/04/14 17:44, Richard Kettlewell wrote: > David Brown <david.brown@hesbynett.no> writes: >> The "shred" manual page says fairly clearly what it does - it writes >> over a file one or more times "in order to make it harder for even >> very expensive hardware probing to recover the data". And it makes >> clear the critical assumption - that the filesystem overwrites data in >> place. It does not claim to completely delete all the data - it just >> claims to make it harder to recover, and only if the filesystem works >> in a particular way. > > The supposed need for multiple writes would be more convincing if it > actually identified the ‘very expensive hardware’ supposedly capable of > retrieving data overwritten even once. > > Usually this stuff turns out to reflect recovery mechanisms that might > have worked on disks from two decades ago, but are completely irrelevant > to today’s hardware. You are absolutely right. The multi-pass shredding idea is based on a theoretical concept from a single academic paper - there has not been a single example of how it might be implemented in practice, and plenty of good arguments (theoretical and practical) showing that reading erased data is impossible. However, some half-wits in various state-run departments (of various states) decided that an absurd set of passes must be run for data to be "securely erased" (forgetting that things like sector relocation can leave data on the disk without the user knowing it) - thus tools like "shred" support multiple passes in order to appease pencil-pushers who don't understand the technology. There exists "very expensive hardware" (and software) capable of recovering a surprising amount of "deleted" data from disks, including disks that have been physically damaged. They cannot recover data if it has actually been overwritten, but they /can/ recover data from logically deleted files, relocated sectors, etc., and they /can/ recover data from flash drives if the sector has not been erased. > >> There /is/ a way to handle this, and it is done today - either in >> software or in hardware (you can get disk drives with this feature >> built in). You need two passwords - one is known to the user, and one >> is generated randomly on creation of the filesystem (or when the >> harddisk is first locked). The user password is used to encrypt the >> random password, and this random password is stored in the disk's >> controller or in some other special way. All data on the disk is >> encrypted using the random password. So to be able to read something >> off the disk, the user must first present the user password in order >> to decode the real encryption password. And you can safely do a 100% >> erasure of the disk by deleting the random password. In the case of >> the secure hard disk, this password has never left the drive's >> electronics. Even the toughest rubber-hose cryptoanalysis cannot >> recover the drive's contents once the random password is deleted, >> because the user never knew it. > > Encryption uses keys, not passwords. What you are almost describing is > deriving a key encryption key from a passphrase and using that to > decrypt a bulk data confidentiality key. Yes, I was a little lax with the terminology. > > As well as supporting passphrase revocation (supposing the encrypted > data confidentiality key can successfully be erased), this is necessary > to allow changing passphrases in a practical way: if the data > confidentiality key were directly derived from the passphrase then > changing passphrase would require re-encrypting the entire block device. > > It also allows multiple passphrases to be supported. > I hadn't thought of these advantages (they are not directly relevant here), but you are of course correct.
[toc] | [prev] | [next] | [standalone]
| From | John Hasler <jhasler@newsguy.com> |
|---|---|
| Date | 2014-04-18 07:37 -0500 |
| Message-ID | <87r44u7r0m.fsf@thumper.dhh.gt.org> |
| In reply to | #623 |
crankypuss writes: > That seems very messed up. Some people are more interested in preserving their data than in destroying it. http://en.wikipedia.org/wiki/ZFS#Copy-on-write_transactional_model -- John Hasler jhasler@newsguy.com Dancing Horse Hill Elmwood, WI USA
[toc] | [prev] | [next] | [standalone]
| From | crankypuss <crankypuss@nomail.invalid> |
|---|---|
| Date | 2014-04-18 10:16 -0600 |
| Message-ID | <lirj58$721$1@dont-email.me> |
| In reply to | #625 |
On 04/18/2014 06:37 AM, John Hasler wrote: > crankypuss writes: >> That seems very messed up. > > Some people are more interested in preserving their data than in > destroying it. > http://en.wikipedia.org/wiki/ZFS#Copy-on-write_transactional_model > I'm a big fan of preserving the data I've gone through the effort of constructing. On the other hand I'm not a believer in encrypted filesystems for that same reason, I not only want the data to stay put, I want to be able to get at it, and I've reached the age where it isn't clear to me whether I'll remember all the things I now know at some unspecified time in the future. Given the state of the medical arts and the unknown diseases mutating toward us, it isn't obvious that *anyone* should simply believe they'll always remember what they now know, and writing down passwords is not a great idea either. As for copy-on-write, it seems like the most straightforward way to build an effective versioned filesystem, and knowing what has changed since one last "blessed" (committed/whatever) his system is a real good thing. Still, if I delete a file, I want it gone, and I don't want residual copies laying around to be possible sources of mischief. Maybe it's just something that I haven't fully thought out. As for being able to shred/scrub the file, if I'm not mistaken one can achieve that on any filesystem by setting the file's attributes to include "inviolate". It isn't often-used except by some boot-loaders that have to know where the second-level loader lives, but it does exist.
[toc] | [prev] | [next] | [standalone]
| From | John Hasler <jhasler@newsguy.com> |
|---|---|
| Date | 2014-04-18 12:01 -0500 |
| Message-ID | <87y4z2607a.fsf@thumper.dhh.gt.org> |
| In reply to | #629 |
crankypuss writes: > I've reached the age where it isn't clear to me whether I'll remember > all the things I now know at some unspecified time in the future. > Given the state of the medical arts and the unknown diseases mutating > toward us, it isn't obvious that *anyone* should simply believe > they'll always remember what they now know, and writing down passwords > is not a great idea either. Writing passwords down is often a great idea: this is an example. If you store that data that you want to be able to destroy on an encrypted filesystem and write down (but do not memorize) the password you can utterly destroy it by burning that piece of paper. You are then no worse off (and arguably better off) than you would be had you left the file unencrypted on a filesystem where you knew for sure that you could totally delete it. On Bruce Schneier's advice I write down all my passwords. I am quite confident in my ability to secure a single little book. -- John Hasler jhasler@newsguy.com Dancing Horse Hill Elmwood, WI USA
[toc] | [prev] | [next] | [standalone]
| From | Kristof Provost <kristof@codepro.be> |
|---|---|
| Date | 2014-04-18 14:42 +0000 |
| Message-ID | <slrnll2ea9.2p2.kristof@vega.codepro.be> |
| In reply to | #623 |
On 2014-04-18, crankypuss <crankypuss@nomail.invalid> wrote: > On 04/17/2014 07:15 AM, Kristof Provost wrote: >> That's because there's no way to guarantee that the file system will >> write the new data over the same block as the old data. In fact, in >> log-structured file systems (like ZFS, but not ext3/4) the file system >> will deliberately not do this. > > That seems very messed up. Aside from the data safety point it also makes implementing snapshots really easy and efficient. Kristof
[toc] | [prev] | [next] | [standalone]
| From | David Brown <david.brown@hesbynett.no> |
|---|---|
| Date | 2014-04-17 16:41 +0200 |
| Message-ID | <liop7h$c4m$1@dont-email.me> |
| In reply to | #618 |
On 17/04/14 00:17, Bill Cunningham wrote: > I am using ext4 on my linux. I'm not quite sure of the difference in it > and ext3 but anyway; the shred man page says with the ext3 filesystem shred > cannot be guaranteed to work. What about scrub? What's the best way to > eliminate a file? > Use "rm". Unless you are hiding state secrets or the accounts for a drug empire, it is good enough for most purposes. It is very hard to "undelete" files on ext3 or ext4 - so you can be confident that no one is going to do it without extremely good reason. If you really want to keep things safe from your paranoid delusions, use an encrypted file system. "Shred" your files by forgetting the password.
[toc] | [prev] | [standalone]
Page 4 of 4 — ← Prev page 1 2 3 [4]
Back to top | Article view | comp.os.linux.development.system
csiph-web