Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.os.linux.misc > #1222 > unrolled thread

Archive 20GB?

Back to article view | Back to comp.os.linux.misc

Contents

  Archive 20GB? Stan Bischof <stan@worldbadminton.com> - 2011-05-23 19:42 +0000
    Re: Archive 20GB? Kevin the Drummer <nobody@cosgroves.us> - 2011-05-23 22:41 -0500
      Re: Archive 20GB? The Natural Philosopher <tnp@invalid.invalid> - 2011-05-24 11:00 +0100
        Re: Archive 20GB? Mark <i@dontgetlotsofspamanymore.invalid> - 2011-05-25 10:32 +0100
    Re: Archive 20GB? David Brown <david.brown@removethis.hesbynett.no> - 2011-05-24 08:05 +0200
    Re: Archive 20GB? Stan Bischof <stan@worldbadminton.com> - 2011-05-24 18:58 +0000
      Re: Archive 20GB? Mark <i@dontgetlotsofspamanymore.invalid> - 2011-05-25 10:32 +0100
        Re: Archive 20GB? Tony <tony@darkstorm.invalid> - 2011-05-25 15:24 +0100
          Re: Archive 20GB? Rui Maciel <rui.maciel@gmail.com> - 2011-05-25 15:40 +0100
            Re: Archive 20GB? The Natural Philosopher <tnp@invalid.invalid> - 2011-05-25 17:05 +0100
              Re: Archive 20GB? Rui Maciel <rui.maciel@gmail.com> - 2011-05-25 17:55 +0100
              Re: Archive 20GB? Kevin the Drummer <nobody@cosgroves.us> - 2011-05-29 16:56 -0500
            Re: Archive 20GB? Robert Riches <spamtrap42@jacob21819.net> - 2011-05-26 03:19 +0000
        Re: Archive 20GB? Keith Keller <kkeller-usenet@wombat.san-francisco.ca.us> - 2011-05-25 07:25 -0700
          Re: Archive 20GB? Jean-David Beyer <jeandavid8@verizon.net> - 2011-05-25 11:26 -0400
          Re: Archive 20GB? Mark <i@dontgetlotsofspamanymore.invalid> - 2011-05-25 16:34 +0100
            Re: Archive 20GB? Jean-David Beyer <jeandavid8@verizon.net> - 2011-05-25 11:56 -0400
              Re: Archive 20GB? TomB <tommy.bongaerts@gmail.com> - 2011-05-25 20:19 +0200
            Re: Archive 20GB? Keith Keller <kkeller-usenet@wombat.san-francisco.ca.us> - 2011-05-25 10:23 -0700
          Re: Archive 20GB? The Natural Philosopher <tnp@invalid.invalid> - 2011-05-25 16:58 +0100
          Re: Archive 20GB? Grant Edwards <invalid@invalid.invalid> - 2011-05-25 16:13 +0000

Page 1 of 2  [1] 2  Next page →

#1222 — Archive 20GB?

Howdy

Have need of archiving around 20GB with a million
or so files to DVD. DVD is _not_
local so need to create files that can be later moved
to machine with DVD writer and then written. Network
mount is not feasible in this case.

In old days this would be a split tarball  ( tar piped into split )
with small enough chunks to write to DVD.

RHEL 5.6 system

Is there a better way to write the archives? Ideally one
in which I don't have to recreate the entire tarball
to dig out one archived file, but also one which will be
readable in 5 or 10 years?

thanks
Stan

[toc] | [next] | [standalone]

#1225

Stan Bischof <stan@worldbadminton.com> wrote:
>  Have need of archiving around 20GB with a million
>  or so files to DVD. DVD is _not_
>  local so need to create files that can be later moved
>  to machine with DVD writer and then written. Network
>  mount is not feasible in this case.
> 
>  In old days this would be a split tarball  ( tar piped into split )
>  with small enough chunks to write to DVD.
> 
>  RHEL 5.6 system
> 
>  Is there a better way to write the archives? Ideally one
>  in which I don't have to recreate the entire tarball
>  to dig out one archived file, but also one which will be
>  readable in 5 or 10 years?

If it were me, then I'd put it onto a hard drive and lock away
the hard drive.  40-80GB hard drives are dirt cheap.  Buy 2 or 3.
Write to all of them.  Keep them in different time zones in case
of disaster.  Maybe even write to a 32GB SD card.

Good luck....

-- 
PLEASE post a SUMMARY of the answer(s) to your question(s)!
Unless otherwise noted, the statements herein reflect my personal
opinions and not those of any organization with which I may be affiliated.

[toc] | [prev] | [next] | [standalone]

#1228

Kevin the Drummer wrote:
> Stan Bischof <stan@worldbadminton.com> wrote:
>>  Have need of archiving around 20GB with a million
>>  or so files to DVD. DVD is _not_
>>  local so need to create files that can be later moved
>>  to machine with DVD writer and then written. Network
>>  mount is not feasible in this case.
>>
>>  In old days this would be a split tarball  ( tar piped into split )
>>  with small enough chunks to write to DVD.
>>
>>  RHEL 5.6 system
>>
>>  Is there a better way to write the archives? Ideally one
>>  in which I don't have to recreate the entire tarball
>>  to dig out one archived file, but also one which will be
>>  readable in 5 or 10 years?
> 
> If it were me, then I'd put it onto a hard drive and lock away
> the hard drive.  40-80GB hard drives are dirt cheap.  Buy 2 or 3.
> Write to all of them.  Keep them in different time zones in case
> of disaster.  Maybe even write to a 32GB SD card.
> 
> Good luck....
> 
My conclusion too. There were so many issues in archiving to DVD I gave 
up and archived to another disk.

[toc] | [prev] | [next] | [standalone]

#1241

On Tue, 24 May 2011 11:00:02 +0100, The Natural Philosopher
<tnp@invalid.invalid> wrote:

>Kevin the Drummer wrote:
>> Stan Bischof <stan@worldbadminton.com> wrote:
>>>  Have need of archiving around 20GB with a million
>>>  or so files to DVD. DVD is _not_
>>>  local so need to create files that can be later moved
>>>  to machine with DVD writer and then written. Network
>>>  mount is not feasible in this case.
>>>
>>>  In old days this would be a split tarball  ( tar piped into split )
>>>  with small enough chunks to write to DVD.
>>>
>>>  RHEL 5.6 system
>>>
>>>  Is there a better way to write the archives? Ideally one
>>>  in which I don't have to recreate the entire tarball
>>>  to dig out one archived file, but also one which will be
>>>  readable in 5 or 10 years?
>> 
>> If it were me, then I'd put it onto a hard drive and lock away
>> the hard drive.  40-80GB hard drives are dirt cheap.  Buy 2 or 3.
>> Write to all of them.  Keep them in different time zones in case
>> of disaster.  Maybe even write to a 32GB SD card.
>> 
>> Good luck....
>> 
>My conclusion too. There were so many issues in archiving to DVD I gave 
>up and archived to another disk.

DVDR & RW are very poor as an archive medium as they don't last very
long.  

With backing up or achiving quantity is the key.  i.e. Make /many/
different copies on different media and in different places.
-- 
(\__/)  M.
(='.'=) Due to the amount of spam posted via googlegroups and
(")_(") their inaction to the problem. I am blocking some articles
posted from there.  If you wish your postings to be seen by
everyone you will need use a different method of posting.

[toc] | [prev] | [next] | [standalone]

#1227

On 23/05/11 21:42, Stan Bischof wrote:
> Howdy
>
> Have need of archiving around 20GB with a million
> or so files to DVD. DVD is _not_
> local so need to create files that can be later moved
> to machine with DVD writer and then written. Network
> mount is not feasible in this case.
>
> In old days this would be a split tarball  ( tar piped into split )
> with small enough chunks to write to DVD.
>
> RHEL 5.6 system
>
> Is there a better way to write the archives? Ideally one
> in which I don't have to recreate the entire tarball
> to dig out one archived file, but also one which will be
> readable in 5 or 10 years?
>
> thanks
> Stan

Presumably you've already thought of this, is the tarball still too big 
when you've bzip'ed it?

And can you use double-layer DVD's that are something like 7.5 GB?

[toc] | [prev] | [next] | [standalone]

#1239

Thanks for the various hints. I ended up sticking
with ttried and true: tar and DVD since the odds of
reading those in 10 years are much higher than
most other solutions.

Stan

[toc] | [prev] | [next] | [standalone]

#1242

On 24 May 2011 18:58:15 GMT, Stan Bischof <stan@worldbadminton.com>
wrote:

>Thanks for the various hints. I ended up sticking
>with ttried and true: tar and DVD since the odds of
>reading those in 10 years are much higher than
>most other solutions.

I would estimate the odds of being able to read a DVDR or RW after 10
years to be very small.
-- 
(\__/)  M.
(='.'=) Due to the amount of spam posted via googlegroups and
(")_(") their inaction to the problem. I am blocking some articles
posted from there.  If you wish your postings to be seen by
everyone you will need use a different method of posting.

[toc] | [prev] | [next] | [standalone]

#1243

On 25/05/2011 10:32, Mark wrote:
> On 24 May 2011 18:58:15 GMT, Stan Bischof<stan@worldbadminton.com>
> wrote:
>
>> Thanks for the various hints. I ended up sticking
>> with ttried and true: tar and DVD since the odds of
>> reading those in 10 years are much higher than
>> most other solutions.
>
> I would estimate the odds of being able to read a DVDR or RW after 10
> years to be very small.

Estimate based on any evidence? Or a gut feel?

-- 
Tony Evans
I'm trying to revive uk.media.films - why not join me there?

[ anything below this line wasn't written by me ]

[toc] | [prev] | [next] | [standalone]

#1245

Tony wrote:

> Estimate based on any evidence? Or a gut feel?

I've had a couple of boxes of CD-Rs fail after being stored for about 6 
years.  These weren't cheap chinese store knock-off CD-Rs.  They were 
stored in air-tight containers and away from sunlight, which is supposed 
to be the right way of storing optical media.  Granted, not all 50 CD-Rs 
failed, but a considerable number of them failed, and I was left without 
any way to recover that data after 6 years.

This case was a personal anecdote which can't possibly be generalized to 
every case where someone stored information on CD-Rs, let alone DVDs.  
Yet, this is also a bit beyond the "gut feeling" territory.


Rui Maciel

[toc] | [prev] | [next] | [standalone]

#1250

Rui Maciel wrote:
> Tony wrote:
> 
>> Estimate based on any evidence? Or a gut feel?
> 
> I've had a couple of boxes of CD-Rs fail after being stored for about 6 
> years.  These weren't cheap chinese store knock-off CD-Rs.  They were 
> stored in air-tight containers and away from sunlight, which is supposed 
> to be the right way of storing optical media.  Granted, not all 50 CD-Rs 
> failed, but a considerable number of them failed, and I was left without 
> any way to recover that data after 6 years.
> 
> This case was a personal anecdote which can't possibly be generalized to 
> every case where someone stored information on CD-Rs, let alone DVDs.  
> Yet, this is also a bit beyond the "gut feeling" territory.
> 
> 
> Rui Maciel

that is also my worry.

By the time I had costed out what keeping 10GB of data would cost on an 
other medium, plus the hassle of it, I simply stuck a second disk on the 
server...

If you are really paranoid, put a third disk on a machine somewhere 
else.And trickle the data across the internet to it.

[toc] | [prev] | [next] | [standalone]

#1252

The Natural Philosopher wrote:

> that is also my worry.
> 
> By the time I had costed out what keeping 10GB of data would cost on an
> other medium, plus the hassle of it, I simply stuck a second disk on the
> server...
> 
> If you are really paranoid, put a third disk on a machine somewhere
> else.And trickle the data across the internet to it.

Indeed.  Nowadays, HDs tend to go for around 0.10 euros/MB.  Although DVD-
Rs tend to go around half that much, in the end the cost difference will 
amount to peanuts.  If we factor in how inconvenient it is to use DVDs to 
store data when compared with simply pluging in a HD then there isn't much 
left in favour of DVDs.

There is also the option to rely on any form of flash memory.  It may be 
more expensive than HDs and DVD-Rs in terms of price per GB but it appears 
to be much more reliable than the other options in this sort of 
application.


Rui Maciel

[toc] | [prev] | [next] | [standalone]

#1276

The Natural Philosopher <tnp@invalid.invalid> wrote:
>  If you are really paranoid, put a third disk on a machine somewhere 
>  else.And trickle the data across the internet to it.

I backup two machines.  I mirror them to each other, and then
rsync the full data of each to their own external hard drive.
One machine has its external drive cycled with an off-site copy.
I have not yet sent my data to a storage cloud.  I don't consider
most Internet entities to be stable, though maybe some are.

-- 
PLEASE post a SUMMARY of the answer(s) to your question(s)!
Unless otherwise noted, the statements herein reflect my personal
opinions and not those of any organization with which I may be affiliated.

[toc] | [prev] | [next] | [standalone]

#1258

On 2011-05-25, Rui Maciel <rui.maciel@gmail.com> wrote:
> Tony wrote:
>
>> Estimate based on any evidence? Or a gut feel?
>
> I've had a couple of boxes of CD-Rs fail after being stored for about 6 
> years.  These weren't cheap chinese store knock-off CD-Rs.  They were 
> stored in air-tight containers and away from sunlight, which is supposed 
> to be the right way of storing optical media.  Granted, not all 50 CD-Rs 
> failed, but a considerable number of them failed, and I was left without 
> any way to recover that data after 6 years.
>
> This case was a personal anecdote which can't possibly be generalized to 
> every case where someone stored information on CD-Rs, let alone DVDs.  
> Yet, this is also a bit beyond the "gut feeling" territory.
>
>
> Rui Maciel

While I don't remember the exact source, I have read that
DVD+/-Rs use a different, much more stable, dye than CD-Rs and
should last much longer.

-- 
Robert Riches
spamtrap42@jacob21819.net
(Yes, that is one of my email addresses.)

[toc] | [prev] | [next] | [standalone]

#1244

On 2011-05-25, Mark <i@dontgetlotsofspamanymore.invalid> wrote:
>
> I would estimate the odds of being able to read a DVDR or RW after 10
> years to be very small.

Why do you say that?  We can still read 3.5" floppies, after all.  If
you've still got the hardware you could probably still read a 5.25"
floppy even.

In 10 years we won't be able to read floppy disks, but it's hard to
imagine not being able to read DVD media in that time.

--keith

-- 
kkeller-usenet@wombat.san-francisco.ca.us
(try just my userid to email me)
AOLSFAQ=http://www.therockgarden.ca/aolsfaq.txt
see X- headers for PGP signature information

[toc] | [prev] | [next] | [standalone]

#1246

Keith Keller wrote:
> On 2011-05-25, Mark <i@dontgetlotsofspamanymore.invalid> wrote:
>> I would estimate the odds of being able to read a DVDR or RW after 10
>> years to be very small.
> 
> Why do you say that?  We can still read 3.5" floppies, after all.  If
> you've still got the hardware you could probably still read a 5.25"
> floppy even.

I do not know about that. After replacing several floppy drives, I just
took them out of my computers. When I get brand new floppy discs,
perhaps 25% of them cannot be formatted. Others can be formatted, but if
I try to compare what I wrote on them with the originals, I got failures
right away. In the old days 5 1/4 and 3 1/2 floppies worked quite well,
but by 2005 or so, they were useless.
> 
> In 10 years we won't be able to read floppy disks, but it's hard to
> imagine not being able to read DVD media in that time.

I have been able to read CD-R disks 5 years later. But I do not know if
I have written enough of them to have statistically significant results.
I guess I do know: my experience is probably not statistically significant.

I do my backups on VXA digital tapes using VXA-2 tape drives. I can
certainly read them easily a year later, and probably much longer. I
have been using magnetic tape since the IBM 7-track tapes and they have
all worked perfectly many years after writing them. Of course, I doubt
you could get an IBM 7-track compatible tape drive these days. I have
used the 4mm DAT tapes, but they were hopeless. I used some cartridge
tapes that hooked up to floppy controllers (QIC?) and they worked for
ears too, until I could not get drive belts (o-rings) for the drive. I
switched to a VXA-1 tape drive in about 2002 after giving up on the DAT
drive, and upgraded to VXA-2 drives somewhat later.
> 
> --keith
> 


-- 
  .~.  Jean-David Beyer          Registered Linux User 85642.
  /V\  PGP-Key: 9A2FC99A         Registered Machine   241939.
 /( )\ Shrewsbury, New Jersey    http://counter.li.org
 ^^-^^ 11:15:01 up 39 days, 14:33, 3 users, load average: 4.96, 4.95, 4.98

[toc] | [prev] | [next] | [standalone]

#1247

On Wed, 25 May 2011 07:25:14 -0700, Keith Keller
<kkeller-usenet@wombat.san-francisco.ca.us> wrote:

>On 2011-05-25, Mark <i@dontgetlotsofspamanymore.invalid> wrote:
>>
>> I would estimate the odds of being able to read a DVDR or RW after 10
>> years to be very small.
>
>Why do you say that?  We can still read 3.5" floppies, after all.  If
>you've still got the hardware you could probably still read a 5.25"
>floppy even.

Totally different technology.  Floppy disks are magnetic & DVDs are
optical.

>In 10 years we won't be able to read floppy disks, but it's hard to
>imagine not being able to read DVD media in that time.

It's easy to imagine.  The dyes used in CD & DVD media decay over
time.  In a good environment they can be made to last longer but I
wouldn't rely on any dye or film based optical media lasting.  I have
had several fail after a very short time myself and this is not
uncommon.
-- 
(\__/)  M.
(='.'=) Due to the amount of spam posted via googlegroups and
(")_(") their inaction to the problem. I am blocking some articles
posted from there.  If you wish your postings to be seen by
everyone you will need use a different method of posting.

[toc] | [prev] | [next] | [standalone]

#1248

Mark wrote:
> On Wed, 25 May 2011 07:25:14 -0700, Keith Keller
> <kkeller-usenet@wombat.san-francisco.ca.us> wrote:
> 
>> On 2011-05-25, Mark <i@dontgetlotsofspamanymore.invalid> wrote:
>>> I would estimate the odds of being able to read a DVDR or RW after 10
>>> years to be very small.
>> Why do you say that?  We can still read 3.5" floppies, after all.  If
>> you've still got the hardware you could probably still read a 5.25"
>> floppy even.
> 
> Totally different technology.  Floppy disks are magnetic & DVDs are
> optical.
> 
>> In 10 years we won't be able to read floppy disks, but it's hard to
>> imagine not being able to read DVD media in that time.
> 
> It's easy to imagine.  The dyes used in CD & DVD media decay over
> time.  In a good environment they can be made to last longer but I
> wouldn't rely on any dye or film based optical media lasting.  I have
> had several fail after a very short time myself and this is not
> uncommon.

 Decades ago, someone well known in the computer business, but I forget
just who, remarked

Data should be stored magnetically, as God intended.

-- 
  .~.  Jean-David Beyer          Registered Linux User 85642.
  /V\  PGP-Key: 9A2FC99A         Registered Machine   241939.
 /( )\ Shrewsbury, New Jersey    http://counter.li.org
 ^^-^^ 11:55:01 up 39 days, 15:13, 3 users, load average: 4.63, 4.77, 4.95

[toc] | [prev] | [next] | [standalone]

#1254

On 2011-05-25, the following emerged from the brain of Jean-David Beyer:
>
>  Decades ago, someone well known in the computer business, but I forget
> just who, remarked
>
> Data should be stored magnetically, as God intended.

Amen to that. Nothing beats a good tape back-up.

-- 
"Programming today is a race between software engineers striving to
build bigger and better idiot-proof programs, and the Universe trying to
produce bigger and better idiots. So far, the Universe is winning."
	~ Rick Cook

[toc] | [prev] | [next] | [standalone]

#1253

On 2011-05-25, Mark <i@dontgetlotsofspamanymore.invalid> wrote:
> On Wed, 25 May 2011 07:25:14 -0700, Keith Keller
><kkeller-usenet@wombat.san-francisco.ca.us> wrote:
>
>>In 10 years we won't be able to read floppy disks, but it's hard to
>>imagine not being able to read DVD media in that time.
>
> It's easy to imagine.  The dyes used in CD & DVD media decay over
> time.  In a good environment they can be made to last longer but I
> wouldn't rely on any dye or film based optical media lasting.  I have
> had several fail after a very short time myself and this is not
> uncommon.

Bah, I apologize for completely misunderstanding--I read your comment as
"DVD drives will no longer work", not "DVD media will not work".

--keith


-- 
kkeller-usenet@wombat.san-francisco.ca.us
(try just my userid to email me)
AOLSFAQ=http://www.therockgarden.ca/aolsfaq.txt
see X- headers for PGP signature information

[toc] | [prev] | [next] | [standalone]

#1249

Keith Keller wrote:
> On 2011-05-25, Mark <i@dontgetlotsofspamanymore.invalid> wrote:
>> I would estimate the odds of being able to read a DVDR or RW after 10
>> years to be very small.
> 
> Why do you say that?  We can still read 3.5" floppies, after all.  If
> you've still got the hardware you could probably still read a 5.25"
> floppy even.
> 
> In 10 years we won't be able to read floppy disks, but it's hard to
> imagine not being able to read DVD media in that time.
> 
> --keith
> 
media degradation.

[toc] | [prev] | [next] | [standalone]

Page 1 of 2  [1] 2  Next page →

Back to top | Article view | comp.os.linux.misc

csiph-web