Re: [sot] file times / file metadata

From	cross@spitfire.i.gajendra.net (Dan Cross)
Newsgroups	comp.lang.c
Subject	Re: [sot] file times / file metadata
Date	2017-11-28 16:59 +0000
Organization	PANIX Public Access Internet and UNIX, NYC
Message-ID	<ovk4la$7di$1@reader2.panix.com> (permalink)
References	<5a91718b-b272-4cdf-adfd-8f6777851b23@googlegroups.com> <626274f7-ae91-4eab-9e55-cae6facbcd5d@googlegroups.com> <ov4l1f$v6e$1@jstuckle.eternal-september.org> <ov61fa$dgk$1@dont-email.me>

Show all headers | View raw

In article <ov61fa$dgk$1@dont-email.me>,
David Brown  <david.brown@hesbynett.no> wrote:
>With a COW copy on btrfs, zfs, lvm, or other filessystem/block devices
>with such features, you can /delete/ the original and the copy is still
>there.  That is, at heart, what most people think of as "making a copy".

I suppose this means that a filesystem that uses block-level
deduplication on content-addressed storage is incapable of
holding a copy of a file, since the data blocks are stored in
exactly one place.  (Note for those who don't recognize it: that
was sarcasm.)

This came up when Plan 9 moved the filesystem to the Venti
storage server.  Venti uses content-addressible storage: blocks
are hashed using a cryptographically-secure hashing algorithm,
and a mapping from hash to block is kept in an index volume.
When writing to the store, an attempt is made to look up the
block's hash in the index; if an entry already exists, one
assumes the block data is already present and returns success.
Otherwise, a block is allocated, the data are written to the
newly-allocated block, and the (hash, block address) pair is
added to the index.

The fear was block-corruption on the storage device would lead
to data loss due to lack of redundancy.  The response was to
recognize that the beauty of the scheme lay in the hashing
mechanism forming a mapping between the filesystem and some sort
of durable media; that could be a mirror or other RAID-style
configuration, real-time cross-site replication, etc.  That is,
redunancy was provided below the content-addressable storage
layer, allowing a site to provide distributed durability
guarantees to minimize the probability of loss in the event of
component failure.

Note that this latter can be one way a "file" can be "copied"
without being "read": when the data are originally written, a
copy is sent to some remote service for archival storage.  There
is no read operation since it would be unnecessary: the data is
already available to the archiver as a byproduct of the write
operation.

	- Dan C.

Back to comp.lang.c | Previous | Next — Previous in thread | Next in thread | Find similar

Thread

[sot] file times / file metadata fir <profesor.fir@gmail.com> - 2017-11-21 11:50 -0800
  Re: [sot] file times / file metadata "Pascal J. Bourguignon" <pjb@informatimago.com> - 2017-11-21 21:05 +0100
    Re: [sot] file times / file metadata Ian Collins <ian-news@hotmail.com> - 2017-11-22 09:15 +1300
    Re: [sot] file times / file metadata fir <profesor.fir@gmail.com> - 2017-11-21 12:23 -0800
      Re: [sot] file times / file metadata fir <profesor.fir@gmail.com> - 2017-11-21 12:42 -0800
        Re: [sot] file times / file metadata fir <profesor.fir@gmail.com> - 2017-11-21 13:01 -0800
        Re: [sot] file times / file metadata "Pascal J. Bourguignon" <pjb@informatimago.com> - 2017-11-21 22:04 +0100
          Re: [sot] file times / file metadata fir <profesor.fir@gmail.com> - 2017-11-21 13:17 -0800
          Re: [sot] file times / file metadata fir <profesor.fir@gmail.com> - 2017-11-21 13:22 -0800
            Re: [sot] file times / file metadata "Pascal J. Bourguignon" <pjb@informatimago.com> - 2017-11-21 23:23 +0100
              Re: [sot] file times / file metadata Ian Collins <ian-news@hotmail.com> - 2017-11-22 11:34 +1300
              Re: [sot] file times / file metadata Lew Pitcher <lew.pitcher@digitalfreehold.ca> - 2017-11-21 17:34 -0500
              Re: [sot] file times / file metadata fir <profesor.fir@gmail.com> - 2017-11-21 16:16 -0800
                Re: [sot] file times / file metadata "Pascal J. Bourguignon" <pjb@informatimago.com> - 2017-11-22 01:44 +0100
                Re: [sot] file times / file metadata Ian Collins <ian-news@hotmail.com> - 2017-11-22 14:10 +1300
                Re: [sot] file times / file metadata gordonb.eiy78@burditt.org (Gordon Burditt) - 2017-11-28 01:55 -0600
          Re: [sot] file times / file metadata David Brown <david.brown@hesbynett.no> - 2017-11-22 11:07 +0100
            Re: [sot] file times / file metadata Jerry Stuckle <jstucklex@attglobal.net> - 2017-11-22 09:13 -0500
              Re: [sot] file times / file metadata David Brown <david.brown@hesbynett.no> - 2017-11-22 15:50 +0100
                Re: [sot] file times / file metadata jameskuyper@verizon.net - 2017-11-22 07:08 -0800
                Re: [sot] file times / file metadata David Brown <david.brown@hesbynett.no> - 2017-11-22 16:22 +0100
                Re: [sot] file times / file metadata bartc <bc@freeuk.com> - 2017-11-22 15:37 +0000
                Re: [sot] file times / file metadata bartc <bc@freeuk.com> - 2017-11-22 15:43 +0000
                Re: [sot] file times / file metadata Jerry Stuckle <jstucklex@attglobal.net> - 2017-11-22 15:01 -0500
                Re: [sot] file times / file metadata Ian Collins <ian-news@hotmail.com> - 2017-11-23 17:52 +1300
                Re: [sot] file times / file metadata Jerry Stuckle <jstucklex@attglobal.net> - 2017-11-23 14:12 -0500
                Re: [sot] file times / file metadata Ian Collins <ian-news@hotmail.com> - 2017-11-24 08:28 +1300
                Re: [sot] file times / file metadata Jerry Stuckle <jstucklex@attglobal.net> - 2017-11-24 08:52 -0500
                Re: [sot] file times / file metadata David Brown <david.brown@hesbynett.no> - 2017-11-23 09:39 +0100
                Re: [sot] file times / file metadata Jerry Stuckle <jstucklex@attglobal.net> - 2017-11-23 14:17 -0500
                Re: [sot] file times / file metadata cross@spitfire.i.gajendra.net (Dan Cross) - 2017-11-28 16:44 +0000
                Re: [sot] file times / file metadata Jerry Stuckle <jstucklex@attglobal.net> - 2017-11-28 14:23 -0500
                Re: [sot] file times / file metadata cross@spitfire.i.gajendra.net (Dan Cross) - 2017-11-28 19:35 +0000
                Re: [sot] file times / file metadata Jerry Stuckle <jstucklex@attglobal.net> - 2017-11-30 15:54 -0500
                Re: [sot] file times / file metadata cross@spitfire.i.gajendra.net (Dan Cross) - 2017-11-30 21:43 +0000
                Re: [sot] file times / file metadata Jerry Stuckle <jstucklex@attglobal.net> - 2017-12-04 11:42 -0500
                Re: [sot] file times / file metadata cross@spitfire.i.gajendra.net (Dan Cross) - 2017-12-04 19:35 +0000
                Re: [sot] file times / file metadata Keith Thompson <kst-u@mib.org> - 2017-12-04 11:53 -0800
                Re: [sot] file times / file metadata cross@spitfire.i.gajendra.net (Dan Cross) - 2017-12-04 20:11 +0000
                Re: [sot] file times / file metadata Jerry Stuckle <jstucklex@attglobal.net> - 2017-12-04 15:03 -0500
                Re: [sot] file times / file metadata scott@slp53.sl.home (Scott Lurndal) - 2017-11-28 19:42 +0000
                Re: [sot] file times / file metadata cross@spitfire.i.gajendra.net (Dan Cross) - 2017-11-28 20:13 +0000
                Re: [sot] file times / file metadata Jerry Stuckle <jstucklex@attglobal.net> - 2017-11-30 15:57 -0500
                Re: [sot] file times / file metadata scott@slp53.sl.home (Scott Lurndal) - 2017-11-30 21:17 +0000
                Re: [sot] file times / file metadata Jerry Stuckle <jstucklex@attglobal.net> - 2017-12-04 11:56 -0500
                Re: [sot] file times / file metadata scott@slp53.sl.home (Scott Lurndal) - 2017-12-04 18:36 +0000
                Re: [sot] file times / file metadata Jerry Stuckle <jstucklex@attglobal.net> - 2017-12-04 15:07 -0500
                Re: [sot] file times / file metadata scott@slp53.sl.home (Scott Lurndal) - 2017-12-04 22:27 +0000
                Re: [sot] file times / file metadata Jerry Stuckle <jstucklex@attglobal.net> - 2017-12-04 17:57 -0500
                Re: [sot] file times / file metadata David Brown <david.brown@hesbynett.no> - 2017-12-05 08:50 +0100
                Re: [sot] file times / file metadata Jerry Stuckle <jstucklex@attglobal.net> - 2017-12-05 08:49 -0500
                Re: [sot] file times / file metadata David Brown <david.brown@hesbynett.no> - 2017-12-05 15:17 +0100
                Re: [sot] file times / file metadata Jerry Stuckle <jstucklex@attglobal.net> - 2017-12-05 16:32 -0500
                Re: [sot] file times / file metadata scott@slp53.sl.home (Scott Lurndal) - 2017-12-05 15:29 +0000
                Re: [sot] file times / file metadata Jerry Stuckle <jstucklex@attglobal.net> - 2017-12-05 16:37 -0500
                Re: [sot] file times / file metadata scott@slp53.sl.home (Scott Lurndal) - 2017-12-06 01:26 +0000
                Re: [sot] file times / file metadata Jerry Stuckle <jstucklex@attglobal.net> - 2017-12-06 12:38 -0500
                Re: [sot] file times / file metadata scott@slp53.sl.home (Scott Lurndal) - 2017-12-06 01:27 +0000
                Re: [sot] file times / file metadata jameskuyper@verizon.net - 2017-11-28 09:35 -0800
                Re: [sot] file times / file metadata Ian Collins <ian-news@hotmail.com> - 2017-11-29 08:24 +1300
                Re: [sot] file times / file metadata Ian Collins <ian-news@hotmail.com> - 2017-11-29 08:27 +1300
                Re: [sot] file times / file metadata Jerry Stuckle <jstucklex@attglobal.net> - 2017-11-28 14:29 -0500
                Re: [sot] file times / file metadata David Brown <david.brown@hesbynett.no> - 2017-11-29 09:51 +0100
                Re: [sot] file times / file metadata Jerry Stuckle <jstucklex@attglobal.net> - 2017-11-30 15:59 -0500
                Re: [sot] file times / file metadata David Brown <david.brown@hesbynett.no> - 2017-11-29 10:00 +0100
                Re: [sot] file times / file metadata "James R. Kuyper" <jameskuyper@verizon.net> - 2017-11-29 09:11 -0500
                Re: [sot] file times / file metadata David Brown <david.brown@hesbynett.no> - 2017-11-29 15:28 +0100
                Re: [sot] file times / file metadata "James R. Kuyper" <jameskuyper@verizon.net> - 2017-11-29 11:32 -0500
                Re: [sot] file times / file metadata James Kuyper <jameskuyper@verizon.net> - 2017-11-23 15:05 -0500
                Re: [sot] file times / file metadata David Brown <david.brown@hesbynett.no> - 2017-11-24 10:20 +0100
                Re: [sot] file times / file metadata Jerry Stuckle <jstucklex@attglobal.net> - 2017-11-24 09:05 -0500
                Re: [sot] file times / file metadata Robert Wessel <robertwessel2@yahoo.com> - 2017-11-24 22:50 -0600
                Re: [sot] file times / file metadata cross@spitfire.i.gajendra.net (Dan Cross) - 2017-11-28 16:59 +0000
                Re: [sot] file times / file metadata Ian Collins <ian-news@hotmail.com> - 2017-11-29 08:26 +1300
                Re: [sot] file times / file metadata David Brown <david.brown@hesbynett.no> - 2017-11-29 10:10 +0100
                Re: [sot] file times / file metadata cross@spitfire.i.gajendra.net (Dan Cross) - 2017-11-29 21:01 +0000
                Re: [sot] file times / file metadata David Brown <david.brown@hesbynett.no> - 2017-11-29 23:13 +0100
                Re: [sot] file times / file metadata cross@spitfire.i.gajendra.net (Dan Cross) - 2017-11-30 17:26 +0000
              Re: [sot] file times / file metadata fir <profesor.fir@gmail.com> - 2017-11-22 08:00 -0800
              Re: [sot] file times / file metadata "Pascal J. Bourguignon" <pjb@informatimago.com> - 2017-11-22 17:11 +0100
            Re: [sot] file times / file metadata "Pascal J. Bourguignon" <pjb@informatimago.com> - 2017-11-22 16:51 +0100
              Re: [sot] file times / file metadata scott@slp53.sl.home (Scott Lurndal) - 2017-11-22 16:36 +0000

csiph-web