Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.user > #270519 > unrolled thread

Backup.

Back to article view | Back to linux.debian.user

Contents

  Backup. peter@easthope.ca - 2024-06-27 21:10 +0200
    Re: Backup. eben@gmx.us - 2024-06-27 22:00 +0200
      Re: Backup. eben@gmx.us - 2024-06-28 05:20 +0200
      Re: Backup. peter@easthope.ca - 2024-06-30 17:10 +0200
        Re: Backup. eben@gmx.us - 2024-06-30 18:10 +0200
    Re: Backup. Jerome BENOIT <sphericaltriangle@rezozer.net> - 2024-06-27 22:20 +0200
      Re: Backup. peter@easthope.ca - 2024-06-30 16:50 +0200
    Re: Backup. "Thomas Schmitt" <scdbackup@gmx.net> - 2024-06-27 22:50 +0200
      Re: Backup. CHRIS M <CHRIS@CWM030.COM> - 2024-06-27 23:00 +0200
      Re: Backup. peter@easthope.ca - 2024-06-28 20:00 +0200
        Re: Backup. "Thomas Schmitt" <scdbackup@gmx.net> - 2024-06-28 23:40 +0200
          Re: Backup. peter@easthope.ca - 2024-06-30 15:40 +0200
            Re: Backup. "Thomas Schmitt" <scdbackup@gmx.net> - 2024-07-02 14:20 +0200
      Re: Backup. peter@easthope.ca - 2025-04-06 16:20 +0200
    Re: Backup. peter@easthope.ca - 2024-06-30 16:00 +0200
      Re: Backup. Michael Kjörling <c9bc136c6063@ewoof.net> - 2024-06-30 16:10 +0200
      Re: Backup. Andy Smith <andy@strugglers.net> - 2024-06-30 16:30 +0200
        Re: Backup. <tomas@tuxteam.de> - 2024-06-30 16:40 +0200
        Re: Backup peter@easthope.ca - 2024-06-30 17:40 +0200
          Re: Backup Andy Smith <andy@strugglers.net> - 2024-06-30 17:50 +0200
      Re: Backup. Anssi Saari <anssi.saari@debian-user.mail.kapsi.fi> - 2024-06-30 17:30 +0200

Page 1 of 2  [1] 2  Next page →

#270519 — Backup.

Hi,

My working data is in a directory we can refer to as A. A is on a 
removable flash store.  "du -hs /home/me/A" reports 3.0G.  I want a 
reliable backup of most files A/*.

I created a directory "Backup" on the HDD and apply this shell 
function whenever motivated.

Backup() { \
  if [ $# -gt 1 ]; then
    echo "Too many arguments.";
  else
    echo "0 or 1 arguments are OK.";
    source="/home/me/A/*";
    echo "source is $source.";
    if [ $# -eq 0 ]; then
      echo "0 arguments is OK.";
      destination=/home/me/Backup;
      echo "destination is $destination.";
    else
      echo "1 argument is OK.";
      destination=/home/me/$1;
      echo "destination is $destination.";
    fi;
      echo "Executing sync and rsync.";
      sync;
      rsync \
      --exclude='Trap*' \
      --exclude='*.mp3' \
      --exclude='*.mp4' \
      -auv $source $destination ;
     /bin/ls -ld $destination/MailMessages;
     printf "du -hs $destination => ";
     du -hs $destination;
  fi;
}

When the flash store fails, work since the last execution of Backup 
can be lost.

In case the Backup directory on the HDD is lost or I want to see an 
old file not current in A, I want backups of Backup.  This function is 
applied every week or two to write to a DVD.

FilesToDVD () { \
  printf "Insert open or new DVD-R.";
  read t;
  startPath=$PWD;
  echo "startPath is $startPath";
  source=/home/me/Backup;
  echo "source is $source";
  cd $source;
  xorriso -for_backup -dev /dev/sr0 \
          -update_r . / \
          -commit \
          -toc -check_md5 failure -- \
          -eject all ;
  cd $startPath ;
  echo "  xorriso -dev /dev/sr0 -toc            ";
  echo "  mount -o sbsector=nnnnnn  /dev/sr0 /mnt/iso            "; }
  
Finding a file as it existed months or years ago can be tedious.  For 
example, find A/MailMessages as it was at 2023.02.07.  Otherwise the 
backup system works well.

Now I have a pair of 500 GB external USB drives.  Large compared to my 
working data of ~3 GB.  Please suggest improvements to my backup 
system by exploiting these drives.  I can imagine a complete copy of A 
onto an external drive for each backup; but with most files in A not 
changing during the backup interval, that is inefficient.

Thanks,            ... Peter E.

-- 
VoIP:   +1 604 670 0140
work: https://en.wikibooks.org/wiki/User:PeterEasthope

[toc] | [next] | [standalone]

#270521

On 6/27/24 14:23, peter@easthope.ca wrote:

> Finding a file as it existed months or years ago can be tedious.  For
> example, find A/MailMessages as it was at 2023.02.07.  Otherwise the
> backup system works well.

On one computer I use rsync to do what appear to be complete backups, only
files identical to the previous run are hard-linked.  This appears to fail
sometimes (not sure under what conditions), so I wrote scripts to find
identical files and link them.  I think the scripts are too complex for my
pea brain to debug them successfully, so there are errors.  But they seem to
work.

> I can imagine a complete copy of A onto an external drive for each
> backup; but with most files in A not changing during the backup interval,
> that is inefficient.

When rsync works, that is exactly what it fixes.

When I boot the file server (possibly today but definitely tomorrow) I'll
post my backup script.

--
Most people don't even know what sysadmins do, but trust me,
if they all took a lunch break at the same time they wouldn't
make it to the deli before you ran out of bullets protecting
your canned goods from roving bands of mutants. -- Peter Welch

[toc] | [prev] | [next] | [standalone]

#270531

On 6/27/24 15:52, eben@gmx.us wrote:
 > When I boot the file server (possibly today but definitely tomorrow) I'll
 > post my backup script.

OK, it's pretty long so I won't post the whole thing, but the important
lines are

rsyncoptions="--archive --progress --verbose --recursive"

"$rsync" $rsyncoptions --hard-links --link-dest "$previous" "$source" "$current"

I have it create destination dirs like

/backup/2024-01-04_22:26:35/
/backup/2024-01-12_20:21:02/
/backup/2024-01-20_23:19:11/
/backup/2024-01-26_23:04:31/
/backup/2024-02-01_22:13:38/
/backup/2024-02-14_00:12:37/
/backup/2024-03-25_23:52:52/
/backup/2024-05-06_22:47:31/
/backup/2024-05-12_22:08:31/
/backup/2024-06-12_23:39:44/

so $previous and $current are two of those, and $source is /files .

--
"Never go off on tangents, which are lines that intersect a [circle] at
only one point and were discovered by Euclid, who lived in the [4th C
BC], which was an era dominated by the Goths, who lived in what we now
know as Poland." - from the Nov. 1998 issue of _Infosystems Executive_.

[toc] | [prev] | [next] | [standalone]

#270659

    From: eben@gmx.us
    Date: Thu, 27 Jun 2024 15:52:44 -0400
> On one computer I use rsync ...

See reply to Eduardo.

Thx,               ... P.

-- 
VoIP:   +1 604 670 0140
work: https://en.wikibooks.org/wiki/User:PeterEasthope

[toc] | [prev] | [next] | [standalone]

#270666

On 6/30/24 10:42, peter@easthope.ca wrote:
 >      From: eben@gmx.us
 >      Date: Thu, 27 Jun 2024 15:52:44 -0400
 >> On one computer I use rsync ...
 >
 > See reply to Eduardo.

Ah, you mean this one:

On 6/30/24 10:36, peter@easthope.ca wrote:
 >      From: Eduardo M KALINOWSKI <eduardo@kalinowski.com.br>
 >      Date: Thu, 27 Jun 2024 16:06:18 -0300
 >> rnapshot
 >
 >>From https://rsnapshot.org/
 >> rsnapshot is a filesystem snapshot utility ...
 >
 > Rather than a snapshot of the extant file system, I want to keep a
 > history of the files in the file system.

rsync does do individual files.  Not sure what it's on about.  I can do
cp /backup/2024-06-25_13:55:24/x/y/z .
(assuming I did a backup then).

--
Scientist A: A matterbaby is a very unstable particle.
Scientist B: What's a matterbaby?
Scientist A: I'm doing fine honey, how you doing? -- mrshowrules on Fark

[toc] | [prev] | [next] | [standalone]

#270522

Hello,
why did you not use something as backup2l ?
Best wishes,
Jerome


On 27/06/2024 20:23, peter@easthope.ca wrote:
> Hi,
> 
> My working data is in a directory we can refer to as A. A is on a
> removable flash store.  "du -hs /home/me/A" reports 3.0G.  I want a
> reliable backup of most files A/*.
> 
> I created a directory "Backup" on the HDD and apply this shell
> function whenever motivated.
> 
> Backup() { \
>    if [ $# -gt 1 ]; then
>      echo "Too many arguments.";
>    else
>      echo "0 or 1 arguments are OK.";
>      source="/home/me/A/*";
>      echo "source is $source.";
>      if [ $# -eq 0 ]; then
>        echo "0 arguments is OK.";
>        destination=/home/me/Backup;
>        echo "destination is $destination.";
>      else
>        echo "1 argument is OK.";
>        destination=/home/me/$1;
>        echo "destination is $destination.";
>      fi;
>        echo "Executing sync and rsync.";
>        sync;
>        rsync \
>        --exclude='Trap*' \
>        --exclude='*.mp3' \
>        --exclude='*.mp4' \
>        -auv $source $destination ;
>       /bin/ls -ld $destination/MailMessages;
>       printf "du -hs $destination => ";
>       du -hs $destination;
>    fi;
> }
> 
> When the flash store fails, work since the last execution of Backup
> can be lost.
> 
> In case the Backup directory on the HDD is lost or I want to see an
> old file not current in A, I want backups of Backup.  This function is
> applied every week or two to write to a DVD.
> 
> FilesToDVD () { \
>    printf "Insert open or new DVD-R.";
>    read t;
>    startPath=$PWD;
>    echo "startPath is $startPath";
>    source=/home/me/Backup;
>    echo "source is $source";
>    cd $source;
>    xorriso -for_backup -dev /dev/sr0 \
>            -update_r . / \
>            -commit \
>            -toc -check_md5 failure -- \
>            -eject all ;
>    cd $startPath ;
>    echo "  xorriso -dev /dev/sr0 -toc            ";
>    echo "  mount -o sbsector=nnnnnn  /dev/sr0 /mnt/iso            "; }
>    
> Finding a file as it existed months or years ago can be tedious.  For
> example, find A/MailMessages as it was at 2023.02.07.  Otherwise the
> backup system works well.
> 
> Now I have a pair of 500 GB external USB drives.  Large compared to my
> working data of ~3 GB.  Please suggest improvements to my backup
> system by exploiting these drives.  I can imagine a complete copy of A
> onto an external drive for each backup; but with most files in A not
> changing during the backup interval, that is inefficient.
> 
> Thanks,            ... Peter E.
> 

[toc] | [prev] | [next] | [standalone]

#270657

    From: Jerome BENOIT <sphericaltriangle@rezozer.net>
    Date: Thu, 27 Jun 2024 21:53:44 +0200
> why did you not use something as backup2l ?

Unaware of it.

From: https://github.com/gkiefer/backup2l
> The restore function allows to easily restore the state of the file 
> system or arbitrary directories/files of previous points in time. 
> ...
> An integrated split-and-collect function allows to comfortably 
> transfer all or selected archives to a set of CDs or other removable 
> media.

Appears relevant.  To catch subtleties, will need to work with it. Two 
phases of workflow may be required.

(1) backup2l records filesystem to rewritable medium such as HDD.

(2) Copy to optical medium.  Each copy may require a fresh optical 
disk. Thomas Schmidt's xorriso appends data to a disk.  Medium is used 
more efficiently..

Thanks,                   ... P.

-- 
VoIP:   +1 604 670 0140
work: https://en.wikibooks.org/wiki/User:PeterEasthope

[toc] | [prev] | [next] | [standalone]

#270525

Hi,

peter@easthope.ca wrote:
> This function is applied every week or two to write to a DVD.
>  xorriso -for_backup -dev /dev/sr0 \
>          -update_r . / \
>          -commit \
>          -toc -check_md5 failure -- \
>          -eject all ;
>
> Finding a file as it existed months or years ago can be tedious.

You could give the backups volume ids which tell the date.

         -volid BOB_"$(date '+%Y_%m_%d_%H%M%S')"

(BOB = Backup Of Backup :))
This would also make it possible to verify that the medium is either an
appendable BOB or blank. Before -dev you would insert:

         -assert_volid 'BOB_*' fatal

As a whole:

  xorriso -for_backup \
          -assert_volid 'BOB_*' fatal \
          -dev /dev/sr0 \
          -volid BOB_"$(date '+%Y_%m_%d_%H%M%S')" \
          -update_r . / \
          -commit \
          -toc -check_md5 failure -- \
          -eject all

Then -toc can later tell the backup date.
I do daily backups with "HOME_" instead of "BOB_":

  Media current: BD-RE
  ...
  ISO session  :   1 ,        32 ,   2129457s , HOME_2024_02_10_152530
  ISO session  :   2 ,   2129504 ,     26203s , HOME_2024_02_11_151813
  ...
  ISO session  : 138 ,   8090176 ,     34330s , HOME_2024_06_26_152446
  ISO session  : 139 ,   8124512 ,     50534s , HOME_2024_06_27_152907
  Media summary: 139 sessions, 8172847 data blocks, 15.6g data, 7643m free

which gives me the opportunity to guess the volume id by date:

  sudo osirrox -mount /dev/sr4 volid '*_2024_04_30_*' /mnt/iso

With less regular dates it would be helpful if the user could wish for
"youngest_before_given_date". I will ponder ...

For now you would have to apply manual work or your own shell magic based
on the output of
  xorriso -indev /dev/sr0 -toc
in order to find the appropriate sbsector number, unless you can describe
the date string by a shell parser expression.


> Now I have a pair of 500 GB external USB drives.  Large compared to my
> working data of ~3 GB.  Please suggest improvements to my backup
> system by exploiting these drives.  I can imagine a complete copy of A
> onto an external drive for each backup; but with most files in A not
> changing during the backup interval, that is inefficient.

You could let xorriso do a daily update.
E.g. on a file of the mounted read-write filesystem on the external disk

  destination=/mnt/backup_disk/xorriso_daily.iso
  ...
  xorriso ... \
          -dev "$destination" \
          ...
          -not_leaf 'Trap*' \
          -not_leaf '*.mp3' \
          -not_leaf '*.mp4' \
          -update_r "$source" / \
          ...

(-eject all would be futile. :))

You could easily have one or more ISOs with history and one or more rsync
mirror trees in the same filesystem. It is always good to keep one valid
backup untouched when the other gets endangered by writing.

(If xorriso -update_r is too slow compared to rsync, consider xorriso
command -disk_dev_ino as in the man page example.)

You could also dedicate the whole plain disk device without any prepared
partitions or filesystems (i do this with USB sticks):
  xorriso ... -dev stdio:/dev/sdd ...
or a partition without prepared filesystem:
  xorriso ... -dev stdio:/dev/sdd1 ...
In the beginning you would have to pseudo-blank the device file, so that
xorriso does not find data in its first few kB which would let it declare
it as "closed":
  xorriso -outdev stdio:/dev/sdd -blank fast
(dd-ing 64 KiB of /dev/zero would have the same effect.)


Have a nice day :)

Thomas

[toc] | [prev] | [next] | [standalone]

#270527

On Thursday 27 June 2024 03:49:03 PM (-05:00), Thomas Schmitt wrote:

> Hi,
> 
> peter@easthope.ca wrote:
> > This function is applied every week or two to write to a DVD.
> >  xorriso -for_backup -dev /dev/sr0 \
> >          -update_r . / \
> >          -commit \
> >          -toc -check_md5 failure -- \
> >          -eject all ;
> >
> > Finding a file as it existed months or years ago can be tedious.
> 
> You could give the backups volume ids which tell the date.
> 
>          -volid BOB_"$(date '+%Y_%m_%d_%H%M%S')"
> 
> (BOB = Backup Of Backup :))
> This would also make it possible to verify that the medium is either an
> appendable BOB or blank. Before -dev you would insert:
> 



< SNIP >

Did anyone else get this reply twice?

Thanks,
Chris
-- 
Sent with Vivaldi Mail. Download Vivaldi for free at vivaldi.com

[toc] | [prev] | [next] | [standalone]

#270552

Hello Thomas & all,

    From: "Thomas Schmitt" <scdbackup@gmx.net>
    Date: Thu, 27 Jun 2024 22:49:10 +0200
> You could give the backups volume ids which tell the date.

Thanks.  I should have added that when you mentioned a few 
years ago.

> This would also make it possible to verify that the medium is either an
> appendable BOB or blank. Before -dev you would insert:
> 
>          -assert_volid 'BOB_*' fatal

Prevents me from appending to a DVD from another backup system. But I 
need to add it when beginning a blank DVD.

Thanks.  Will be sure that works before unwrapping one of the 500 GB 
drives.

                               ... P.

-- 
VoIP:   +1 604 670 0140
work: https://en.wikibooks.org/wiki/User:PeterEasthope

[toc] | [prev] | [next] | [standalone]

#270572

Hi,

peter@easthope.ca wrote:
> >   xorriso -for_backup -dev /dev/sr0 \
> > Finding a file as it existed months or years ago can be tedious

I wrote:
> > You could give the backups volume ids which tell the date.

peter@easthope.ca wrote:
> Thanks.  I should have added that when you mentioned a few
> years ago.

I am working on a solution for your non-unique volume id situation
by optionally referring to modification timestamps.
A new command -toc_info_type can switch -toc away from showing volume ids:

  $ xorriso -indev /dev/sr0  -toc_info_type mtime  -toc
  xorriso 1.5.7 : RockRidge filesystem manipulator, libburnia project.
  ...
  Media current: DVD+RW
  ...
  Volume id    : 'HOME_Z_2024_06_27_225526'
  ...
  TOC layout   : Idx ,  sbsector ,       Size , Modification Time
  ISO session  :   1 ,        32 ,   1240808s , 2024.06.20.232334
  ISO session  :   2 ,   1240864 ,     29797s , 2024.06.21.220651
  ISO session  :   3 ,   1270688 ,     20484s , 2024.06.23.225019
  ISO session  :   4 ,   1291200 ,     28928s , 2024.06.24.224429
  ISO session  :   5 ,   1320128 ,     21352s , 2024.06.25.223943
  ISO session  :   6 ,   1341504 ,     30352s , 2024.06.26.223934
  ISO session  :   7 ,   1371872 ,     29023s , 2024.06.27.225617
  Media summary: 7 sessions, 1400744 data blocks, 2736m data, 1746m free

This is a zisofs compressed backup which happens every evening except
saturdays.
Note the time difference between 2024_06_27_225526 and 2024.06.27.225617.
These 51 seconds where spent between program start and begin of writing.

This program enhancement is already committed to git.
In a few days there will be a new GNU xorriso 1.5.7 tarball, which is
easy to build and to test without any danger of frankendebianing.
I'll give you a note.

Currently i am working on a mount gesture which shall automate the task
of picking the youngest backup before a given time point:

   -mount /dev/sr0 not_after 2024.06.22.120000 /mnt/iso

I think "not_after" is a concise way to say "younger_or_exactly_that_age".
Up to the next release i could take better proposals. :))

(I am having my fun with time zones. The ISO 9660 timestamps can have a
time zone from their maker and the computer running xorriso has a local
time zone. Argh ...)


> >          -assert_volid 'BOB_*' fatal

> Prevents me from appending to a DVD from another backup system. But I
> need to add it when beginning a blank DVD.

You may use it also on a DVD which already has sessions with non-unique
ids. This needs two editing operations in your script.

Begin by adding the volume id command to the backup-of-backup run:

  -volid BOB_"$(date '+%Y_%m_%d_%H%M%S')"

Then do an add-on backup to the DVD with the boring ids.

After this session add the check command:

  -assert_volid 'BOB_*' fatal

In the further backup runs, the DVD is supposed to pass this check.


Have a nice day :)

Thomas

[toc] | [prev] | [next] | [standalone]

#270650

    From: "Thomas Schmitt" <scdbackup@gmx.net>
    Date: Fri, 28 Jun 2024 23:35:31 +0200
> I am working on a solution for your non-unique volume id situation
> by optionally referring to modification timestamps.
> A new command -toc_info_type can switch -toc away from showing volume ids:
> 
>   $ xorriso -indev /dev/sr0  -toc_info_type mtime  -toc
>   xorriso 1.5.7 : RockRidge filesystem manipulator, libburnia project.
>   ...
>   Media current: DVD+RW
>   ...
>   Volume id    : 'HOME_Z_2024_06_27_225526'
>   ...
>   TOC layout   : Idx ,  sbsector ,       Size , Modification Time
>   ISO session  :   1 ,        32 ,   1240808s , 2024.06.20.232334
>   ISO session  :   2 ,   1240864 ,     29797s , 2024.06.21.220651
>   ISO session  :   3 ,   1270688 ,     20484s , 2024.06.23.225019
>   ISO session  :   4 ,   1291200 ,     28928s , 2024.06.24.224429
>   ISO session  :   5 ,   1320128 ,     21352s , 2024.06.25.223943
>   ISO session  :   6 ,   1341504 ,     30352s , 2024.06.26.223934
>   ISO session  :   7 ,   1371872 ,     29023s , 2024.06.27.225617
>   Media summary: 7 sessions, 1400744 data blocks, 2736m data, 1746m free
> 
> This is a zisofs compressed backup which happens every evening except
> saturdays.
> Note the time difference between 2024_06_27_225526 and 2024.06.27.225617.
> These 51 seconds where spent between program start and begin of writing.
> 
> This program enhancement is already committed to git.
> In a few days there will be a new GNU xorriso 1.5.7 tarball, which is
> easy to build and to test without any danger of frankendebianing.

Thanks Thomas.  Ideally I should find time to follow your suggestions  
but already overcommitted to volunteer activities.  I might have to wait 
until -toc_info_type is in a Debian release.

Thx,                  ... P.

-- 
VoIP:   +1 604 670 0140
work: https://en.wikibooks.org/wiki/User:PeterEasthope

[toc] | [prev] | [next] | [standalone]

#270733

Hi,

peter@easthope.ca wrote:
> Thanks Thomas.  Ideally I should find time to follow your suggestions
> but already overcommitted to volunteer activities.  I might have to wait
> until -toc_info_type is in a Debian release.

Yeah. Everybody has to follow the own timetable.


So without any implied expectation of a test report by you:

I uploaded the new development snapshot

   https://www.gnu.org/software/xorriso/xorriso-1.5.7.tar.gz

with the new time oriented table-of-content features.

They are for situations where the usually shown volume ids are fewly
informative (like "ISOIMAGE" in each session) but also if it is difficult
to create a search expression for the desired volume id when mounting.

Time oriented table-of-content:

  $ xorriso -indev /dev/sr0 -toc_info_type mtime -toc
  ...
  TOC layout   : Idx ,  sbsector ,       Size , Modification Time
  ISO session  :   1 ,        32 ,   1240808s , 2024.06.20.232334
  ISO session  :   2 ,   1240864 ,     29797s , 2024.06.21.220651
  ISO session  :   3 ,   1270688 ,     20484s , 2024.06.23.225019
  ISO session  :   4 ,   1291200 ,     28928s , 2024.06.24.224429
  ISO session  :   5 ,   1320128 ,     21352s , 2024.06.25.223943
  ISO session  :   6 ,   1341504 ,     30352s , 2024.06.26.223934
  ISO session  :   7 ,   1371872 ,     29023s , 2024.06.27.225617
  ISO session  :   8 ,   1400896 ,     31463s , 2024.06.28.232751
  ISO session  :   9 ,   1432384 ,     34008s , 2024.06.30.223451
  ISO session  :  10 ,   1466400 ,     20329s , 2024.07.01.220901
  Media summary: 10 sessions, 1486544 data blocks, 2903m data, 1579m free

Creating mount command lines by time constraints:

  $ xorriso -mount_cmd /dev/sr0 not_after 2024.06.26.060000 /mnt/iso
  ...
  mount -t iso9660 -o nodev,noexec,nosuid,ro,sbsector=1320128 '/dev/sr0' '/mnt/iso'

The defined time constraints are:
  "at_time", "before", "after", "not_before", "not_after"

Time can be given in various formats, among them 'June 26' and
'June 26 2024'. See man xorriso, command -alter_date and "Examples of
input timestrings". If you become too perky, expect rejections like:
  xorriso : SORRY : -mount: Cannot decode timestring 'Rapunzel'

Complaints about unfulfillable time constraints will tell the internally
used seconds since 1970:
  $ xorriso -mount_cmd /dev/sr0 before "June 01 2024" /mnt/iso
  ...
  libisoburn: FAILURE : Failed to find "before" "=1717192800"

"=" is the time_t indicator for xorriso.
Program "date" digests the string if "=" gets replaced by "@":
  $ date -d @1717192800
  Sat Jun  1 00:00:00 CEST 2024


Have a nice day :)

Thomas

[toc] | [prev] | [next] | [standalone]

#278900

    From: "Thomas Schmitt" <scdbackup@gmx.net>
    Date: Thu, 27 Jun 2024 22:49:10 +0200
> You could give the backups volume ids which tell the date.
> 
>          -volid BOB_"$(date '+%Y_%m_%d_%H%M%S')"
> ...

Just ran this shell function with no difficulty evident.

FilesToHDD () { \
  source=/home/root/Backup;
  echo "source is $source";
  destination=stdio:/dev/disk/by-id/usb-SEAGATE_ST3500830A_10000E000D959403-0:0;
  echo "destination is $destination";
  xorriso -for_backup \
          -dev "$destination" \
          -assert_volid 'BOB.*' fatal \
          -volid BOB."$(date '+%Y.%m.%d.%H:%M:%S')" \
          -update_r "$source" / \
          -commit \
          -toc \
          -check_md5 failure -- \
          -rollback_end ; }
          
/dev/disk/by-id/... is too long but avoids ambiguity.

> You could easily have one or more ISOs with history and one or more 
> rsync mirror trees in the same filesystem. It is always good to keep 
> one valid backup untouched when the other gets endangered by 
> writing.

Will have two backups at each of two sites.

> (If xorriso -update_r is too slow compared to rsync, consider 
> xorriso command -disk_dev_ino as in the man page example.)

Thanks.  No complaint about speed.

    From: "Thomas Schmitt" <scdbackup@gmx.net>
    Date: Mon, 24 Mar 2025 08:48:34 +0100
    Message-id: <21203323233134896576@scdbackup.webframe.org>
> You could use command -rollback_end to refrain from writing:
> 
>   xorriso ...the.desired.commands... -rollback_end
> 
> This will perform the commands but then just end the program run
> instead of writing the result and thus reading all the content of
> the files which were mapped into the ISO.

The shell function above has -rollback_end and delivered output.
-commit overrides -rollback_end?  -rollback_end should replace or 
precede -commit?

 > I have a classification of my
> system disks in startup file /etc/opt/xorriso/rc :
> 
>   -drive_class banned   '/dev/sda*'
>   -drive_class banned   '/dev/sdb*'
>   -drive_class harmless /dev/null
 
Did that.  Thanks.  Incidentally, "-drive_class banned   '/dev/sda*'" 
as first option in the command gave an error message.

Your manual page is excellent.  Thanks.   Some of the details you 
mentioned might fit in examples.

Thx,                 ... p.

-- 
VoIP:   +1 604 670 0140
work: en.wikibooks.org/wiki/User:PeterEasthope

[toc] | [prev] | [next] | [standalone]

#270652

    From: "Thomas Schmitt" <scdbackup@gmx.net>
    Date: Fri, 28 Jun 2024 23:35:31 +0200
> You could give the backups volume ids which tell the date.
> 
>          -volid BOB_"$(date '+%Y_%m_%d_%H%M%S')"
> 
> (BOB = Backup Of Backup :))

I'm beginning to learn Git.  So I wonder about another approach where 
files are in a local Git repository.  That would allow  tracing the 
history of any file.  A backup of the extant repository would still be 
necessary.  

I don't know the software well enough to compare the two approaches.

Thx,                  ... P.

-- 
VoIP:   +1 604 670 0140
work: https://en.wikibooks.org/wiki/User:PeterEasthope

[toc] | [prev] | [next] | [standalone]

#270654

On 30 Jun 2024 06:40 -0700, from peter@easthope.ca:
>> You could give the backups volume ids which tell the date.
>> 
>>          -volid BOB_"$(date '+%Y_%m_%d_%H%M%S')"
>> 
>> (BOB = Backup Of Backup :))
> 
> I'm beginning to learn Git.  So I wonder about another approach where 
> files are in a local Git repository.  That would allow  tracing the 
> history of any file.  A backup of the extant repository would still be 
> necessary.  

That sounds a lot like etckeeper, except on a larger scale.

-- 
Michael Kjörling                     🔗 https://michael.kjorling.se
“Remember when, on the Internet, nobody cared that you were a dog?”

[toc] | [prev] | [next] | [standalone]

#270655

Hello,

On Sun, Jun 30, 2024 at 06:40:03AM -0700, peter@easthope.ca wrote:
> I'm beginning to learn Git.  So I wonder about another approach where 
> files are in a local Git repository.  That would allow  tracing the 
> history of any file.  A backup of the extant repository would still be 
> necessary.  
> 
> I don't know the software well enough to compare the two approaches.

Git has some properties that are desirable for general backup
purposes, but also some fairly huge downsides. For example:

- It's not efficient or performant for storing large binary files.
  As a result, several extensions and external programs around git
  exist for getting large binary files into git. Trying to use git
  for general purpose backups will run up against this unless you
  never want to back up large binary files.

- Git stores full (compressed) copies of every version of every
  file. Most backup solutions do better on space.

- Git has no built in way to purge old content. It keeps it all. A
  typical requirement for backup software is to have a bounded limit
  on the oldest versions that will be kept, and quite often there
  are more complex requirements such as "keep daily copies for a
  month, week;y copies for 6 months, monthly copies for 6 years"
  etc. Very hard to do with git.

My first thought when I read the post that started this thread was,

    "What is this person doing? If the goal is to have a real world
    project to learn some programming techniques and have fun, fair
    enough, but if the goal here is to have a decent backup scheme
    why are they not using any of the existing excellent solutions
    that have thought of and solved so many of the problems in this
    space?"

That did not seem like it would be a welcome response at the time so
I held my tongue, but if you are now thinking of looking in to using
git for the purpose, I think it's a wrong step, and in saying so I
might as well say the other as well.

Just use borgbackup, restic, amanda or even rsnapshot (ancient but
still functional).

Unless you are wanting to have a first hand learning experience
about why you should just use borgbackup, restic, amanda or even
rsnapshot.

(Also I think learning about git is best done by using it for what
it was designed for: managing source code in git.)

Thanks,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting

[toc] | [prev] | [next] | [standalone]

#270656

[Multipart message — attachments visible in raw view] — view raw

On Sun, Jun 30, 2024 at 02:21:45PM +0000, Andy Smith wrote:
> Hello,

[...]

> Git has some properties that are desirable for general backup
> purposes, but also some fairly huge downsides. For example:
> 
> - It's not efficient or performant for storing large binary files.

[...]

Plus, it doesn't record file ownership (for general backup,
this *is* important).

I'm a fan of rsync. If you want to keep more than one generation,
--link-dest or --compare-dest, as has been stated elsewhere in
this thread, are your friends.

Cheers
-- 
t

[toc] | [prev] | [next] | [standalone]

#270661 — Re: Backup

    From: Andy Smith <andy@strugglers.net>
    Date: Sun, 30 Jun 2024 14:21:45 +0000
> What is this person doing?

Keeping a historical backup by an efficient method. 
https://lists.debian.org/debian-user/2024/06/msg00780.html

> Just use borgbackup, restic, amanda or even rsnapshot (ancient but
> still functional).
 
Ref. URL above.  I've used xorriso for several years.  If you advise 
against it, please explain why.

Thx,                             ... P.


-- 
VoIP:   +1 604 670 0140
work: https://en.wikibooks.org/wiki/User:PeterEasthope

[toc] | [prev] | [next] | [standalone]

#270664 — Re: Backup

On Sun, Jun 30, 2024 at 08:19:54AM -0700, peter@easthope.ca wrote:
>     From: Andy Smith <andy@strugglers.net>
>     Date: Sun, 30 Jun 2024 14:21:45 +0000
> > What is this person doing?
> 
> Keeping a historical backup by an efficient method. 

While refusing to look into any modern backup system designed by
experienced people for that express goal, and instead writing tomes
of navel gazing to debian-user. It's not my cup of tea, but have
fun; as this and other threads demonstrate you will still find a
large number of willing playmates.

Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting

[toc] | [prev] | [next] | [standalone]

Page 1 of 2  [1] 2  Next page →

Back to top | Article view | linux.debian.user

csiph-web