Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.compression > #1025 > unrolled thread

I'm looking for a program that will go through a number of files...

Back to article view | Back to comp.compression

Contents

  I'm looking for a program that will go through a number of files... jules Gilbert <jules.stocks@gmail.com> - 2012-02-09 01:14 -0800
    Re: I'm looking for a program that will go through a number of files... stan <smoore@exis.net> - 2012-02-09 07:59 -0500
      Re: I'm looking for a program that will go through a number of files... jules Gilbert <jules.stocks@gmail.com> - 2012-02-09 10:08 -0800
    Re: I'm looking for a program that will go through a number of files... Alex Mizrahi <alex.mizrahi@gmail.com> - 2012-02-09 15:54 +0200
      Re: I'm looking for a program that will go through a number of files... jules Gilbert <jules.stocks@gmail.com> - 2012-02-09 10:10 -0800
      Re: I'm looking for a program that will go through a number of files... Thomas Richter <thor@math.tu-berlin.de> - 2012-02-09 20:36 +0100

#1025 — I'm looking for a program that will go through a number of files...

Fully decompressed, I have (this is an estimate only!,) about 20-30 TB
of files, some, .gz, some .bz2, some .tar. some .arj, you get the
idea.  I think the largest file is, say, around 250GB.

And these files are, in some cases, duplicates of others.  Or almost
duplicates, but for a few diff's.

Also, I want to apply certain rules, such as always deleting a file
that is "*.core" (which is a UN*X style core dump.)

I want a program that will rummage through my files, organizing, etc.
I intend to use the dar program.  That's "D-ar", a French built and
managed open-source tool built to replace tar but with much greater
functionality.  I have the source and am attempting to extend it, at
least for internal use.

I've purchased one of those new WD external drives, called a "My Book"
Essential.  (I think that's the name, don't have the box handy.)  In
fact I got that drive expecting to use it with my laptop but
discovered that USB 3.0 is pretty new and had to purchase a PCI card
converter.  No big deal, just be careful if you're thinking of getting
one.

I've been backing up for years and years and now have thousands
of ,tar files.  And want help from a program in reorganizing these
bunches of files.  Does such a program exist??

[toc] | [next] | [standalone]

#1026

jules Gilbert wrote:
> Fully decompressed, I have (this is an estimate only!,) about 20-30 TB
> of files, some, .gz, some .bz2, some .tar. some .arj, you get the
> idea.  I think the largest file is, say, around 250GB.

The first step would seem t clearly define the problem.

> I want a program that will rummage through my files, organizing, etc.
> I intend to use the dar program.  That's "D-ar", a French built and
> managed open-source tool built to replace tar but with much greater
> functionality.  I have the source and am attempting to extend it, at
> least for internal use.

The tar utility has been used for backups for many years, if this dar
is better why not ask people who use dar? IOW why would anyone here
know anything about it?

> I've been backing up for years and years and now have thousands
> of ,tar files.  And want help from a program in reorganizing these
> bunches of files.  Does such a program exist??

It's very difficult to imagine any program could read your mind to
determine the changes you want. Your vague description would seem to
call for a simple shell script but I can't imagine why you would think
this is the place to discuss shell scripts or backups.

For the record, why not just apply your secret sauce and shrink all
your stuff down to fit in on a floppy? Your system works right?

[toc] | [prev] | [next] | [standalone]

#1034

On Feb 9, 7:59 am, stan <smo...@exis.net> wrote:
> jules Gilbert wrote:
> > Fully decompressed, I have (this is an estimate only!,) about 20-30 TB
> > of files, some, .gz, some .bz2, some .tar. some .arj, you get the
> > idea.  I think the largest file is, say, around 250GB.
>
> The first step would seem t clearly define the problem.
>
> > I want a program that will rummage through my files, organizing, etc.
> > I intend to use the dar program.  That's "D-ar", a French built and
> > managed open-source tool built to replace tar but with much greater
> > functionality.  I have the source and am attempting to extend it, at
> > least for internal use.
>
> The tar utility has been used for backups for many years, if this dar
> is better why not ask people who use dar? IOW why would anyone here
> know anything about it?
>
> > I've been backing up for years and years and now have thousands
> > of ,tar files.  And want help from a program in reorganizing these
> > bunches of files.  Does such a program exist??
>
> It's very difficult to imagine any program could read your mind to
> determine the changes you want. Your vague description would seem to
> call for a simple shell script but I can't imagine why you would think
> this is the place to discuss shell scripts or backups.
>
> For the record, why not just apply your secret sauce and shrink all
> your stuff down to fit in on a floppy? Your system works right?

Oh no, apparently you don't know that several large firms who's
business involves IT storage solutions have developed or purchased
(for their customer's,) programs that go through their clients files
doing things like, seeing two nearly identical files, represent the
older, but the newer is replaced by a link to the older plus the
necessary diff's.

You're not up to date.  What I was looking for (I agree, I didn't say
this specifically,) is an open-source program that does this.  Or is
it too soon?

[toc] | [prev] | [next] | [standalone]

#1027

> I've been backing up for years and years and now have thousands
> of ,tar files.  And want help from a program in reorganizing these
> bunches of files.  Does such a program exist??

UNIX shell such as bash or zsh might help, I guess, it won't think for you.

 > Also, I want to apply certain rules, such as always deleting a file
 > that is "*.core" (which is a UN*X style core dump.)

In zsh it is

rm **/*.core

Simple, eh?

[toc] | [prev] | [next] | [standalone]

#1035

On Feb 9, 8:54 am, Alex Mizrahi <alex.mizr...@gmail.com> wrote:
> > I've been backing up for years and years and now have thousands
> > of ,tar files.  And want help from a program in reorganizing these
> > bunches of files.  Does such a program exist??
>
> UNIX shell such as bash or zsh might help, I guess, it won't think for you.
>
>  > Also, I want to apply certain rules, such as always deleting a file
>  > that is "*.core" (which is a UN*X style core dump.)
>
> In zsh it is
>
> rm **/*.core
>
> Simple, eh?

that's really cool!  I'm going to load that on my laptop today.  Thank
you.

[toc] | [prev] | [next] | [standalone]

#1036

On 09.02.2012 14:54, Alex Mizrahi wrote:
>> I've been backing up for years and years and now have thousands
>> of ,tar files. And want help from a program in reorganizing these
>> bunches of files. Does such a program exist??
>
> UNIX shell such as bash or zsh might help, I guess, it won't think for you.
>
>  > Also, I want to apply certain rules, such as always deleting a file
>  > that is "*.core" (which is a UN*X style core dump.)
>
> In zsh it is
>
> rm **/*.core
>
> Simple, eh?

And in a standard POSIX environment, it is

find . -name "*.core" -exec rm \{\} \;

but what is the relation to compression? This is just a simple trivial 
exercise in the POSIX toolchain.

[toc] | [prev] | [standalone]

Back to top | Article view | comp.compression

csiph-web