Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.user > #263081 > unrolled thread

How to use dmsetuup?

Back to article view | Back to linux.debian.user

Contents

  How to use dmsetuup? gene heskett <gheskett@shentel.net> - 2023-11-03 17:30 +0100
    Re: How to use dmsetuup? gene heskett <gheskett@shentel.net> - 2023-11-03 17:50 +0100
    Re: How to use dmsetuup? Stefan Monnier <monnier@iro.umontreal.ca> - 2023-11-03 18:10 +0100
    Re: How to use dmsetuup? David Christensen <dpchrist@holgerdanske.com> - 2023-11-03 22:50 +0100
      Re: How to use dmsetuup? gene heskett <gheskett@shentel.net> - 2023-11-04 01:00 +0100
        Re: How to use dmsetuup? David Christensen <dpchrist@holgerdanske.com> - 2023-11-04 01:50 +0100
    Re: How to use dmsetuup? "Andrew M.A. Cater" <amacater@einval.com> - 2023-11-03 23:20 +0100
      Re: How to use dmsetuup? gene heskett <gheskett@shentel.net> - 2023-11-04 01:10 +0100
      Re: How to use dmsetuup? gene heskett <gheskett@shentel.net> - 2023-11-04 12:50 +0100
        Re: How to use dmsetuup? <tomas@tuxteam.de> - 2023-11-04 14:50 +0100
          Re: How to use dmsetuup? gene heskett <gheskett@shentel.net> - 2023-11-04 18:30 +0100
            Re: How to use dmsetuup? tomas@tuxteam.de - 2023-11-04 19:40 +0100
              Re: How to use dmsetuup? debian-user@howorth.org.uk - 2023-11-04 21:50 +0100
            Re: How to use dmsetuup? David Christensen <dpchrist@holgerdanske.com> - 2023-11-04 22:40 +0100
              Re: How to use dmsetuup? gene heskett <gheskett@shentel.net> - 2023-11-04 23:30 +0100
                Re: How to use dmsetuup? David Christensen <dpchrist@holgerdanske.com> - 2023-11-05 00:40 +0100
                  Re: How to use dmsetuup? yxcv@vienna.at - 2023-11-05 01:10 +0100
                  Re: How to use dmsetuup? gene heskett <gheskett@shentel.net> - 2023-11-05 02:00 +0100
                    Re: How to use dmsetuup? David Christensen <dpchrist@holgerdanske.com> - 2023-11-05 04:20 +0100
                      Re: How to use dmsetuup? gene heskett <gheskett@shentel.net> - 2023-11-05 05:10 +0100
                        Re: How to use dmsetuup? David Christensen <dpchrist@holgerdanske.com> - 2023-11-05 07:50 +0100
                          Re: How to use dmsetuup? gene heskett <gheskett@shentel.net> - 2023-11-05 10:50 +0100
                            Re: How to use dmsetuup? David Christensen <dpchrist@holgerdanske.com> - 2023-11-05 11:30 +0100
                              Re: How to use dmsetuup? gene heskett <gheskett@shentel.net> - 2023-11-05 16:30 +0100
                  Re: How to use dmsetuup? "Thomas Schmitt" <scdbackup@gmx.net> - 2023-11-05 09:10 +0100
                    Re: How to use dmsetuup? David Christensen <dpchrist@holgerdanske.com> - 2023-11-05 21:20 +0100
                      Re: How to use dmsetuup? debian-user@howorth.org.uk - 2023-11-05 21:50 +0100
                        Re: How to use dmsetuup? David Christensen <dpchrist@holgerdanske.com> - 2023-11-05 23:30 +0100
                      Re: How to use dmsetuup? "Thomas Schmitt" <scdbackup@gmx.net> - 2023-11-05 23:20 +0100
                        Re: How to use dmsetuup? David Christensen <dpchrist@holgerdanske.com> - 2023-11-05 23:50 +0100
                          Re: How to use dmsetuup? "Thomas Schmitt" <scdbackup@gmx.net> - 2023-11-06 08:40 +0100
                    xorriso and SIGTERM/SIGINT handler (was: Re: How to use dmsetuup?) Max Nikulin <manikulin@gmail.com> - 2023-11-06 05:30 +0100
                      Re: xorriso and SIGTERM/SIGINT handler "Thomas Schmitt" <scdbackup@gmx.net> - 2023-11-06 09:00 +0100
          Re: How to use dmsetuup? gene heskett <gheskett@shentel.net> - 2023-11-04 23:10 +0100
    Re: How to use dmsetuup? Andy Smith <andy@strugglers.net> - 2023-11-04 10:40 +0100
      Re: How to use dmsetuup? gene heskett <gheskett@shentel.net> - 2023-11-04 13:30 +0100
    Re: How to use dmsetuup? Franco Martelli <martellif67@gmail.com> - 2023-11-06 16:50 +0100
      Re: How to use dmsetuup? gene heskett <gheskett@shentel.net> - 2023-11-06 17:30 +0100
      Re: How to use dmsetuup? Tom Dial <tddial@comcast.net> - 2023-11-08 00:50 +0100
        Re: How to use dmsetuup? gene heskett <gheskett@shentel.net> - 2023-11-08 01:30 +0100
          Re: How to use dmsetuup? David Christensen <dpchrist@holgerdanske.com> - 2023-11-08 06:10 +0100
          Re: How to use dmsetuup? <tomas@tuxteam.de> - 2023-11-08 06:40 +0100
            Re: How to use dmsetuup? gene heskett <gheskett@shentel.net> - 2023-11-08 11:30 +0100
              Re: How to use dmsetuup? "Andrew M.A. Cater" <amacater@einval.com> - 2023-11-08 19:10 +0100
                Re: How to use dmsetuup? jeremy ardley <jeremy.ardley@gmail.com> - 2023-11-08 23:00 +0100
              Re: How to use dmsetuup? Tom Dial <tddial@comcast.net> - 2023-11-09 02:10 +0100
              Re: How to use dmsetuup? David Christensen <dpchrist@holgerdanske.com> - 2023-11-11 01:50 +0100
                Re: How to use dmsetuup? gene heskett <gheskett@shentel.net> - 2023-11-11 04:30 +0100
                  Hardware for a back up server? [WAS Re: How to use dmsetuup?] "Andrew M.A. Cater" <amacater@einval.com> - 2023-11-11 18:00 +0100
                    Re: Hardware for a back up server? [WAS Re: How to use dmsetuup?] Stefan Monnier <monnier@iro.umontreal.ca> - 2023-11-11 18:10 +0100
                      Re: Hardware for a back up server? [WAS Re: How to use dmsetuup?] Pocket <pocket@columbus.rr.com> - 2023-11-11 18:40 +0100
                        Re: Hardware for a back up server? [WAS Re: How to use dmsetuup?] Stefan Monnier <monnier@iro.umontreal.ca> - 2023-11-11 19:50 +0100
                          Re: Hardware for a back up server? [WAS Re: How to use dmsetuup?] Pocket <pocket@columbus.rr.com> - 2023-11-11 21:50 +0100
                            Re: Hardware for a back up server? [WAS Re: How to use dmsetuup?] gene heskett <gheskett@shentel.net> - 2023-11-11 22:40 +0100
                          Re: Hardware for a back up server? [WAS Re: How to use dmsetuup?] Jeffrey Walton <noloader@gmail.com> - 2023-11-11 22:10 +0100
                            Re: Hardware for a back up server? [WAS Re: How to use dmsetuup?] Jeffrey Walton <noloader@gmail.com> - 2023-11-12 00:30 +0100
                            Re: Hardware for a back up server? [WAS Re: How to use dmsetuup?] Dan Ritter <dsr@randomstring.org> - 2023-11-12 00:40 +0100
                    Re: Hardware for a back up server? [WAS Re: How to use dmsetuup?] fxkl47BF@protonmail.com - 2023-11-11 18:30 +0100
                    Re: Hardware for a back up server? [WAS Re: How to use dmsetuup?] David Christensen <dpchrist@holgerdanske.com> - 2023-11-12 01:10 +0100
                      Re: Hardware for a back up server? [WAS Re: How to use dmsetuup?] Stefan Monnier <monnier@iro.umontreal.ca> - 2023-11-12 05:40 +0100
                        Re: Hardware for a back up server? [WAS Re: How to use dmsetuup?] Anssi Saari <anssi.saari@debian-user.mail.kapsi.fi> - 2023-11-13 11:40 +0100
                          Re: Hardware for a back up server? [WAS Re: How to use dmsetuup?] Stefan Monnier <monnier@iro.umontreal.ca> - 2023-11-13 14:00 +0100
                      Re: Hardware for a back up server? [WAS Re: How to use dmsetuup?] Andy Smith <andy@strugglers.net> - 2023-11-12 14:20 +0100
                        Re: Hardware for a back up server? [WAS Re: How to use dmsetuup?] David Christensen <dpchrist@holgerdanske.com> - 2023-11-12 19:40 +0100
                      Re: Hardware for a back up server? [WAS Re: How to use dmsetuup?] "Andrew M.A. Cater" <amacater@einval.com> - 2023-11-12 18:20 +0100
                        Re: Hardware for a back up server? [WAS Re: How to use dmsetuup?] David Christensen <dpchrist@holgerdanske.com> - 2023-11-12 20:30 +0100
                          Re: Hardware for a back up server? [WAS Re: How to use dmsetuup?] Stefan Monnier <monnier@iro.umontreal.ca> - 2023-11-12 21:20 +0100
                            Re: Hardware for a back up server? [WAS Re: How to use dmsetuup?] <tomas@tuxteam.de> - 2023-11-13 06:40 +0100
                              Linux supprt (was: Hardware for a back up server? [WAS Re: How to use dmsetuup?]) Stefan Monnier <monnier@iro.umontreal.ca> - 2023-11-13 13:50 +0100
                                Re: Linux supprt (was: Hardware for a back up server? [WAS Re: How to  use dmsetuup?]) Larry Martell <larry.martell@gmail.com> - 2023-11-13 16:00 +0100
                                  Re: Linux supprt Stefan Monnier <monnier@iro.umontreal.ca> - 2023-11-13 16:40 +0100
                                    Re: Linux supprt Nicholas Geovanis <nickgeovanis@gmail.com> - 2023-11-13 22:50 +0100
                                Re: Linux supprt John Hasler <john@sugarbit.com> - 2023-11-13 19:00 +0100
                                  Re: Linux supprt <tomas@tuxteam.de> - 2023-11-13 19:40 +0100
                                    Re: Linux supprt Nicholas Geovanis <nickgeovanis@gmail.com> - 2023-11-14 18:40 +0100
                                      Re: Linux supprt <tomas@tuxteam.de> - 2023-11-14 18:50 +0100
                            Re: Hardware for a back up server? [WAS Re: How to use dmsetuup?] Joe <joe@jretrading.com> - 2023-11-13 15:10 +0100
          Re: How to use dmsetuup? Tom Dial <tddial@comcast.net> - 2023-11-09 01:50 +0100
            Re: How to use dmsetuup? Andy Smith <andy@strugglers.net> - 2023-11-09 16:20 +0100
              Re: How to use dmsetuup? debian-user@howorth.org.uk - 2023-11-09 17:50 +0100

Page 2 of 4 — ← Prev page 1 [2] 3 4  Next page →

#263140

On 11/4/23 21:05, gene heskett wrote:
> On 11/4/23 23:15, David Christensen wrote:
>> On 11/4/23 17:55, gene heskett wrote:
>>> FWIW the rw's I have and that continue to work, are Sony DVD+RW, well 
>>> over 5 years old now. I understand there is a DVD-RW but I've no 
>>> experience with them.  Today my objection is the size. In comparison 
>>> to a system driving 3d printers with gcode from Cura-5.4 that is not 
>>> rolled up into subroutine loops, I have some of the more complex and 
>>> large parts part files that will not fit on a dvd. So it simply 
>>> impractical for me to back up to a measly 4.7Gig dvd.
>>
>> That's why they invented Blu-ray:
>>
>>      https://en.wikipedia.org/wiki/Blu-ray
>>
>>      25 GB (single-layer)
>>      50, 66 GB (dual-layer)
>>      100, 128 GB (BDXL)
>>
> Shudder. Anything mechanical can be destroyed by a smoke particle 100x 
> to small to be seen with a good eye. I am a CET & Electronics in general 
> I understand the physics of, and in electronics the only thing moving is 
> a few electrons here or there. As long as the voltage does not force an 
> electron thru the oxide layer that is the capacitors insulation, forming 
> a leakage path that avalanches thru the oxide film and essentially 
> destroys the device, there is no physical reason that it will not 
> continue to do its jobs for hundreds or thousands of years. It will be 
> external environmental effects that will eventually reach the chip and 
> byproducts of the humidity let in by the breach of the package sealing 
> that finally destroys it.
> 
> The size of a bit that is detectable on a disk is determined by the 
> wavelegth of the light reading that bit, cd's were designed with the IR 
> lasers of the day, which emmit light in the 1100 nanometer range. Far 
> infrared IOW. DVD's were made possible with a shorter visible light 
> laser, then blue rays got that down to abut 400 nanometers. The next gen 
> of those will have a uv laser  but we'll have to invent it first. But 
> part of that problem is that decent optical glass for the lenses does 
> not pass UV to a usable amount. Plastic lets it blast on thru but can we 
> make plastic lenses that precisely for the price bleeding edge users 
> will pay? IDK.


Interesting tangent.


The point I was trying to make is that  proper disaster preparedness 
involves defenses in depth.  AFAIK your data and your backups are on the 
same computer and you have no other recent backups or archives.  If 
true, then, as you already know, the computer is a single point of 
failure that could destroy both data and backups.


And, now you are touching HBA's, touching drives, and issuing root 
commands that are in direct proximity to your data and backups.  As you 
already know, human error is the most common failure mode.  I am worried 
that you are going to make a mistake and suffer a data disaster (partial 
or total).  That is why I suggested that you give the Asus a rest and 
build a backup server now.  If you then trash the Asus, recovery will be 
possible.  A duplicate set of backups is wise in case something happens 
to the primary backups (notably, human error during recovery).


David

[toc] | [prev] | [next] | [standalone]

#263147

On 11/5/23 01:46, David Christensen wrote:
> On 11/4/23 21:05, gene heskett wrote:
>> On 11/4/23 23:15, David Christensen wrote:
>>> On 11/4/23 17:55, gene heskett wrote:
>>>> FWIW the rw's I have and that continue to work, are Sony DVD+RW, 
>>>> well over 5 years old now. I understand there is a DVD-RW but I've 
>>>> no experience with them.  Today my objection is the size. In 
>>>> comparison to a system driving 3d printers with gcode from Cura-5.4 
>>>> that is not rolled up into subroutine loops, I have some of the more 
>>>> complex and large parts part files that will not fit on a dvd. So it 
>>>> simply impractical for me to back up to a measly 4.7Gig dvd.
>>>
>>> That's why they invented Blu-ray:
>>>
>>>      https://en.wikipedia.org/wiki/Blu-ray
>>>
>>>      25 GB (single-layer)
>>>      50, 66 GB (dual-layer)
>>>      100, 128 GB (BDXL)
>>>
>> Shudder. Anything mechanical can be destroyed by a smoke particle 100x 
>> to small to be seen with a good eye. I am a CET & Electronics in 
>> general I understand the physics of, and in electronics the only thing 
>> moving is a few electrons here or there. As long as the voltage does 
>> not force an electron thru the oxide layer that is the capacitors 
>> insulation, forming a leakage path that avalanches thru the oxide film 
>> and essentially destroys the device, there is no physical reason that 
>> it will not continue to do its jobs for hundreds or thousands of 
>> years. It will be external environmental effects that will eventually 
>> reach the chip and byproducts of the humidity let in by the breach of 
>> the package sealing that finally destroys it.
>>
>> The size of a bit that is detectable on a disk is determined by the 
>> wavelegth of the light reading that bit, cd's were designed with the 
>> IR lasers of the day, which emmit light in the 1100 nanometer range. 
>> Far infrared IOW. DVD's were made possible with a shorter visible 
>> light laser, then blue rays got that down to abut 400 nanometers. The 
>> next gen of those will have a uv laser  but we'll have to invent it 
>> first. But part of that problem is that decent optical glass for the 
>> lenses does not pass UV to a usable amount. Plastic lets it blast on 
>> thru but can we make plastic lenses that precisely for the price 
>> bleeding edge users will pay? IDK.
> 
> 
> Interesting tangent.
> 
> 
> The point I was trying to make is that  proper disaster preparedness 
> involves defenses in depth.  AFAIK your data and your backups are on the 
> same computer and you have no other recent backups or archives.  If 
> true, then, as you already know, the computer is a single point of 
> failure that could destroy both data and backups.
> 
> 
> And, now you are touching HBA's, touching drives, and issuing root 
> commands that are in direct proximity to your data and backups.  As you 
> already know, human error is the most common failure mode.  I am worried 
> that you are going to make a mistake and suffer a data disaster (partial 
> or total).  That is why I suggested that you give the Asus a rest and 
> build a backup server now.  If you then trash the Asus, recovery will be 
> possible.  A duplicate set of backups is wise in case something happens 
> to the primary backups (notably, human error during recovery).
> 
> 
> David
> 
> .
Amanda is now been able the use big disk storage for a couple decades, 
separate from the computers main drive. These disks then contain 
tarball, compressed if possible, that can be read on a rebuilt machine 
with a bare metal install for recovery.  One must be fam with how it 
works, and w/o its database but it can be done. I'm also into 3d 
printers, and that has made me fam with the arm64 sbc cards such as the 
bananapi-m5 which has 4 ub3 ports on a 2GHz 4 core cpu. Startech makes a 
usb3 to sata adapter that can do 500M/sec to an SSD. I am doing it on an 
rpi4b.  So I am tempted to build my own NAS by using all 4 of those 
ports to hook 4 of these 2T gigastones up as a 4G raid10. Run it 
headless by an ssh login, setup amanda server on it, setup amanda-client 
on the rest, setup amanda to do any compression on the clients which are 
fast enough to do it and let the pi handle the actual storage, including 
its database which makes a recovery a matter to telling it which file 
and how old. I normally setup for 60 to 90 days of retention. It won't 
be fast but it will be isolated from anything that fails on the rest of 
my net,

Fast is relative, running on an older mobo in this machines original 
incarnation, it backed up itself and 4 other machines on my local net in 
30 to 45 minute sessions everynight. Using a separate drive, but that 
drive, one of two 2T seagates went off line forever at about 6 weeks 
runtime, followed 3 days later by its twin which was the boot drive in 
this machine, leading to a 22 install disaster because the installer at 
the time was or still is busted.

Buster and bullseye both worked great, bookworm has been a disaster from 
the gitgo for me. I absolutely own every byte of /home/gene, but 
something is getting in the way that didn't for buster or bullseye, and 
I have a hard time believing I am the only one on the planet with such a 
problem. The evidence says I am. opening a write requester to select 
where the file is to be put takes from 30 seconds to 5 minutes, once its 
drawn on screen, things are instant. Where is that delay? And better 
yet, how do I fix it?

I can recall, 35 years ago, running os9 on a trs-80 color computer, a 
mini unix at the time that could run on a 64k machine, would get laggy 
when the floppy disk was getting full because it was scanning the file 
allocation table for open space, but those lags weren't anything like 
this.  Is the sheer size of the system creating a similar problem?
So my first experiment will be to move /home off the raid to a single 2T 
SSD. Unfortunately my partner for the last 34 years in the crime of 
marriage has passed, so I don't have anyone to tell, "here, hold my beer 
and watch this". ;o)>

I just gotta get off my bum and do it.  At 89 yo, my body doesn't always 
want to do what my brain tells it to. Next project is finding the bottom 
of a rafter and drilling a hole to drive a screw eye into the ceiling so 
I can pick an 81 lb 3d printer up, swing it over a table and set it 
down. Hopefully yet today. I've drug in everything but the stepladder to 
drill the hole and install the skyhook.

Thanks all.

Cheers, Gene Heskett.
-- 
"There are four boxes to be used in defense of liberty:
  soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author, 1940)
If we desire respect for the law, we must first make the law respectable.
  - Louis D. Brandeis

[toc] | [prev] | [next] | [standalone]

#263149

On 11/5/23 01:47, gene heskett wrote:
> On 11/5/23 01:46, David Christensen wrote:
>> I am 
>> worried that you are going to make a mistake and suffer a data 
>> disaster (partial or total).  That is why I suggested that you give 
>> the Asus a rest and build a backup server now.

> I'm also into 3d 
> printers, and that has made me fam with the arm64 sbc cards such as the 
> bananapi-m5 which has 4 ub3 ports on a 2GHz 4 core cpu. Startech makes a 
> usb3 to sata adapter that can do 500M/sec to an SSD. I am doing it on an 
> rpi4b.  So I am tempted to build my own NAS by using all 4 of those 
> ports to hook 4 of these 2T gigastones up as a 4G raid10. Run it 
> headless by an ssh login, setup amanda server on it, setup amanda-client 
> on the rest, setup amanda to do any compression on the clients which are 
> fast enough to do it and let the pi handle the actual storage, including 
> its database which makes a recovery a matter to telling it which file 
> and how old. I normally setup for 60 to 90 days of retention. It won't 
> be fast but it will be isolated from anything that fails on the rest of 
> my net,


Wow!  I got Gene to consider my suggestion!  :-)


David

[toc] | [prev] | [next] | [standalone]

#263153

On 11/5/23 05:28, David Christensen wrote:
> On 11/5/23 01:47, gene heskett wrote:
>> On 11/5/23 01:46, David Christensen wrote:
>>> I am worried that you are going to make a mistake and suffer a data 
>>> disaster (partial or total).  That is why I suggested that you give 
>>> the Asus a rest and build a backup server now.
> 
>> I'm also into 3d printers, and that has made me fam with the arm64 sbc 
>> cards such as the bananapi-m5 which has 4 ub3 ports on a 2GHz 4 core 
>> cpu. Startech makes a usb3 to sata adapter that can do 500M/sec to an 
>> SSD. I am doing it on an rpi4b.  So I am tempted to build my own NAS 
>> by using all 4 of those ports to hook 4 of these 2T gigastones up as a 
>> 4G raid10. Run it headless by an ssh login, setup amanda server on it, 
>> setup amanda-client on the rest, setup amanda to do any compression on 
>> the clients which are fast enough to do it and let the pi handle the 
>> actual storage, including its database which makes a recovery a matter 
>> to telling it which file and how old. I normally setup for 60 to 90 
>> days of retention. It won't be fast but it will be isolated from 
>> anything that fails on the rest of my net,
> 
> 
> Wow!  I got Gene to consider my suggestion!  :-)
> 
> 
> David

After I wrote that in the not so wee hour of the morning, I went to one 
of my bpi's and verified that amanda is in the ubuntu jammy arm64 
repo's.  I'll have to get a few more of those drives and build a box. Or 
perhaps hide the bpi in a used drive cage. But it looks doable to me.

Why? Just because I can.  I'm the same guy whose running a 1500 lb, 80+ 
yo Sheldon lathe, using LinuxCNC with an rpi3b, now an rpi4b, for 7 or 
so years. I wanted to see if it could be done and it works so well I've 
had no urge to redo it with more conventional wintel hardware like my 
other 3 machines. LinuxCNC has taught that lathe many new dance steps it 
could not do when it left Chicago in the late 1940's and corrected for 
13 thou of bed wear in front of the chuck.  Cuts threads, imperial or 
metric, even tapered threads I have invented, Anything that 2 small 
motors moving the tool in micron or better synchronized in both 
directions can do, it Just Does.

All I have to do is muster up the giddyup to do it. a 89 yo, that is 
most assuredly getting harder.

Take care & stay well everybody.

Cheers, Gene Heskett.
-- 
"There are four boxes to be used in defense of liberty:
  soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author, 1940)
If we desire respect for the law, we must first make the law respectable.
  - Louis D. Brandeis

[toc] | [prev] | [next] | [standalone]

#263144

Hi,

Gene Heskett wrote:
> > I have 3 100 disk spindles of dvd's bought years ago, that are
> > no longer recognized in any of the 4 or 5 dvd writers I have, but one box
> > of rewritables about the same age, stored n a light tight cardboard box,
> > will likely outlast me.

Unwritten write-once media can indeed get unusable when exposed to light
over a long time. But i have not heard of written DVD getting bad from
indirect light. (Direct sun light is deadly for many things.)


> > Lesson learnt, do not use optical media for long term storage unless
> > stored in tin boxes like AOL gave away billions of 20 years or more ago.

I have a little locker for my media stock. But a substantial number of
my written media are not completely protected from stray light.


David Christensen wrote:
> I have been burning archive DVD-R discs for ~14 years and storing them in a
> drawer (e.g. darkness).  I checked the oldest just now and it reads okay.

That's my experience too. I check by MD5 which are stored on the medium
together with the data. If a medium turns out unreadable then in nearly
all cases directly after burning.
Lesson learnt: Never overwrite the two youngest backups.


> I have heard of CD discs disintegrating if the lacquer is scratched:
> https://en.wikipedia.org/wiki/Disc_rot

This never hit me. But i keep my media away from high moisture, corroding
chemicals, and abrasive substances.


> I have heard that RW media has a shorter lifespan than R media.

Not to my experience. I have 4x CD-RW from 2002 (*), DVD+RW from 2004, and
BD-RE from 2008 which still work for backups. From time to time a medium
dies during writing. This seems not to be closely related to age, though.

(*) I have older 2x CD-RW, but no burner any more which would accept them.
    They can be still read.


> Does anyone have experience with M-Disc media?

Only by reports of libburn users which say that M-Disc works like the
other media. Their durability can hardly be evaluated, given that normal
media seem to last for more than 2 decades without much problem.


I think the better alternative for archived data would be to make several
identical copies and to checkread them in intervals of a few years. As
soon as one of the copies shows problems, make new copies of the healthy
ones. Payload checksums on the medium give extra trust, although i only
once in my life watched a DVD giving bad data without an SCSI error.
That was reproducible with only a single drive. All others either read
the affected 32 KiB chunk correctly or threw error. Having more than one
drive helps a lot when the read quality is on the edge.

A sincere rescue effort would look like:

  xorriso -outdev /dev/sr0 \
          -check_media use=outdev \
                       what=disc \
                       time_limit=7200 \
                       data_to="$HOME"/sr0.image \
                       sector_map="$HOME"/sr0.sector_map \
                       --

After 7200 seconds the attempt will end even if not finished.
If you want to abort earlier, do not press Ctrl+C but rather do
  touch /var/opt/xorriso/do_abort_check_media

The disc image will emerge as file "$HOME"/sr0.image .
The file "$HOME"/sr0.sector_map will record which blocks could be read
without SCSI error. If the run is repeated, then only the missing blocks
will be attempted to be read.
One may repeat with the same drive or better with different ones.
The file "$HOME"/sr0.image and "$HOME"/sr0.sector_map may be carried to
other computers to continue the rescue attempt with their drives.
If there are other partly damaged identical copies of the archive medium,
then one may use them too with the same sr0.image and sr0.sector_map
files.

If the image is an ISO 9660 filesystem made by xorriso with option
-for_backup and thus contains MD5s, then one may check the resulting
image by:

  xorriso -for_backup -indev "$HOME"/sr0.image -check_media --

Any protest would indicate that the rescue attempt was not successful.
If the direcory tree is undamaged, one may check the data files by

  xorriso -for_backup -indev "$HOME"/sr0.image -check_md5_r sorry / --

to get the paths of files with damaged content.

Other than with most rescue tools, the read operations of xorriso on an
optical drive will be done by direct SCSI commands, not by the Linux block
layer. This has the advantage that error messages are more specific than
just "i/o error" and that no inappropriate reading ahead will happen. Such
reading ahead causes the old TAO CD bug of Linux which gave birth to
interesting urban legends about a "fuzzy end" of data CDs.


Have a nice day :)

Thomas

[toc] | [prev] | [next] | [standalone]

#263156

On 11/5/23 01:04, Thomas Schmitt wrote:
> David Christensen wrote:
>> I have been burning archive DVD-R discs for ~14 years and storing them in a
>> drawer (e.g. darkness).  I checked the oldest just now and it reads okay.
> 
> That's my experience too. 


Okay.


> I check by MD5 which are stored on the medium
> together with the data. If a medium turns out unreadable then in nearly
> all cases directly after burning.


Adding checksum file(s) to the contents burned to disc is an important 
step that should not be omitted -- checksum files are a simple and 
direct way to validate the integrity of disc contents.  Without checksum 
files, a computer with working applications is required to read and 
validate the specific file format(s).  I suspect tar(1), gzip(1) and 
ccrypt(1) do this (?), but you are lost with plain text files.


> Lesson learnt: Never overwrite the two youngest backups.


I try to use the term "backup" to mean a data copying process whereby 
older data is overwritten by newer data.


I try to use the term "archive' to mean a data copying process whereby 
the copy is never modified or erased.


>> I have heard of CD discs disintegrating if the lacquer is scratched:
>> https://en.wikipedia.org/wiki/Disc_rot
> 
> This never hit me. But i keep my media away from high moisture, corroding
> chemicals, and abrasive substances.


I expect CD disc rot occurs when discs are accidentally scratched or 
when discs are treated badly (e.g. stored loose; not in a case).


>> I have heard that RW media has a shorter lifespan than R media.
> 
> Not to my experience. I have 4x CD-RW from 2002 (*), DVD+RW from 2004, and
> BD-RE from 2008 which still work for backups.


Okay.


> From time to time a medium
> dies during writing. This seems not to be closely related to age, though.
> 
> (*) I have older 2x CD-RW, but no burner any more which would accept them.
>      They can be still read.


The DVD+-RW in my primary workstation started producing intermittent 
write failures a few months ago.  This quickly progressed to consistent 
write failures and consistent read failures.  I cleaned the lens today, 
and now it reads okay.  I will find out if it writes okay the next time 
I burn a disc.


>> Does anyone have experience with M-Disc media?
> 
> Only by reports of libburn users which say that M-Disc works like the
> other media. Their durability can hardly be evaluated, given that normal
> media seem to last for more than 2 decades without much problem.


Okay.


> I think the better alternative for archived data would be to make several
> identical copies and to checkread them in intervals of a few years. As
> soon as one of the copies shows problems, make new copies of the healthy
> ones. Payload checksums on the medium give extra trust, 


The optical media could be a single point of failure.  If I remove two 
DVD-R discs from the same case this month, burn them with identical 
contents, verify the checksums, store one disc locally, store the other 
disc off-site, and verify checksums once per year in the future, I could 
find that both discs fail in the same year.


So, I both burn my monthly archive encrypted tarballs to DVD-R and I 
keep the original on RAID, which is included in my backup process.


> although i only
> once in my life watched a DVD giving bad data without an SCSI error.
> That was reproducible with only a single drive. All others either read
> the affected 32 KiB chunk correctly or threw error. Having more than one
> drive helps a lot when the read quality is on the edge.


Interesting.  My WAG is that there was a marginal/ ambiguous dot on disc 
(?).


> A sincere rescue effort would look like:
> 
>    xorriso -outdev /dev/sr0 \
>            -check_media use=outdev \
>                         what=disc \
>                         time_limit=7200 \
>                         data_to="$HOME"/sr0.image \
>                         sector_map="$HOME"/sr0.sector_map \
>                         --
> 
> After 7200 seconds the attempt will end even if not finished.
> If you want to abort earlier, do not press Ctrl+C but rather do
>    touch /var/opt/xorriso/do_abort_check_media
> 
> The disc image will emerge as file "$HOME"/sr0.image .
> The file "$HOME"/sr0.sector_map will record which blocks could be read
> without SCSI error. If the run is repeated, then only the missing blocks
> will be attempted to be read.
> One may repeat with the same drive or better with different ones.
> The file "$HOME"/sr0.image and "$HOME"/sr0.sector_map may be carried to
> other computers to continue the rescue attempt with their drives.
> If there are other partly damaged identical copies of the archive medium,
> then one may use them too with the same sr0.image and sr0.sector_map
> files.
> 
> If the image is an ISO 9660 filesystem made by xorriso with option
> -for_backup and thus contains MD5s, then one may check the resulting
> image by:
> 
>    xorriso -for_backup -indev "$HOME"/sr0.image -check_media --
> 
> Any protest would indicate that the rescue attempt was not successful.
> If the direcory tree is undamaged, one may check the data files by
> 
>    xorriso -for_backup -indev "$HOME"/sr0.image -check_md5_r sorry / --
> 
> to get the paths of files with damaged content.
> 
> Other than with most rescue tools, the read operations of xorriso on an
> optical drive will be done by direct SCSI commands, not by the Linux block
> layer. This has the advantage that error messages are more specific than
> just "i/o error" and that no inappropriate reading ahead will happen. Such
> reading ahead causes the old TAO CD bug of Linux which gave birth to
> interesting urban legends about a "fuzzy end" of data CDs.


xorriso(1) is a killer app.  :-)


I seem to recall a post by you that indicated *BSD lacked the features 
needed for good optical drive/ media/ format support.  Has this 
improved?  Can I get xorriso(1) on FreeBSD?


David

[toc] | [prev] | [next] | [standalone]

#263157

David Christensen <dpchrist@holgerdanske.com> wrote:
> On 11/5/23 01:04, Thomas Schmitt wrote:

> > Lesson learnt: Never overwrite the two youngest backups.  
> 
> I try to use the term "backup" to mean a data copying process whereby 
> older data is overwritten by newer data.
> 
> I try to use the term "archive' to mean a data copying process
> whereby the copy is never modified or erased.

You're entitled to do that I suppose, but I don't suppose most other
people do. They separate the words by their meanings and purpose.
Backups are intended for use recovering information that has become
lost. Archives are places to keep information for long term storage.

So your definition of archive is correct. But your definition of backup
isn't. It's perfectly reasonable to have more than one version or age
of backup, but it's also perfectly reasonable to erase them at some
chosen age or version.

It is perfectly reasonable to discuss 'the two youngest backups', IMHO.

[toc] | [prev] | [next] | [standalone]

#263161

On 11/5/23 12:46, debian-user@howorth.org.uk wrote:
> David Christensen <dpchrist@holgerdanske.com> wrote:
>> On 11/5/23 01:04, Thomas Schmitt wrote:
> 
>>> Lesson learnt: Never overwrite the two youngest backups.
>>
>> I try to use the term "backup" to mean a data copying process whereby
>> older data is overwritten by newer data.
>>
>> I try to use the term "archive' to mean a data copying process
>> whereby the copy is never modified or erased.
> 
> You're entitled to do that I suppose, but I don't suppose most other
> people do. They separate the words by their meanings and purpose.
> Backups are intended for use recovering information that has become
> lost. Archives are places to keep information for long term storage.
> 
> So your definition of archive is correct. But your definition of backup
> isn't. It's perfectly reasonable to have more than one version or age
> of backup, but it's also perfectly reasonable to erase them at some
> chosen age or version.
> 
> It is perfectly reasonable to discuss 'the two youngest backups', IMHO.


English is ambiguous.  "Backup" and "archive" can both be used as nouns 
and/or verbs, depending upon context.  This makes communication hard, 
especially for non-native speakers.


I agree that the primary purpose of backup (verb when performed, noun 
for the media containing one copy of the data) is for recovery (verb 
when performed, noun for the result).


I agree that a backup (singular noun for the media) can be archived 
(verb), thereby becoming an archive (singular noun for the media) in an 
archive (collective noun for all the media and singular noun for a place).


The differentiating factor is whether or not the media containing the 
copy is intended for ongoing re-use.  I re-use my backup RAID's and 
HDD's.  I cannot re-use my archive DVD-R's.


David

[toc] | [prev] | [next] | [standalone]

#263160

Hi,

David Christensen wrote:
> Adding checksum file(s) to the contents burned to disc is an important step
> that should not be omitted

I let xorriso compute and store the checksums in a non-file block range
at the end of the ISO filesystem. Each file gets an AAIP attribute which
points to an MD5 in this checksum array.
The user only has to issue the xorriso command -for_backup or -md5 "on".


I wrote:
> > i only once in my life watched a DVD giving bad data without an SCSI
> > error.

David Christensen wrote:
> Interesting.  My WAG is that there was a marginal/ ambiguous dot on disc (?).

The astonishing fact is not the damaged data chunk but the drive's failure
to recognize the damage. There are substantial parity data wrapped around
each DVD "ECC Block" which the drive may use for error detection and
possibly for correction. If i count correctly in MMC-5 Figure 26, then
32 KiB payload data get added 192 * 10 + 15 * 182 = 4650 bytes of parity
data. ECMA-337 (about DVD+RW) mentions in 13.1 two more checksums with
6 bytes per 2048 bytes of payload data.

With such much of redundancy it is highly unlikely that an alteration
stays undetected or that an error correction yields a wrong result.
Nevertheless in this special combination of medium and drive the error was
not reported or corrected by the drive but rather a wrong data chunk was
handed out.

At that occasion an MD5 recorded by xorriso indicated the error.
Comparing the read results on several drives showed that it was about a
single ECC block of 32 KiB payload.


> I seem to recall a post by you that indicated *BSD lacked the features
> needed for good optical drive/ media/ format support.

I have difficulties to remember ...

> Can I get xorriso(1) on FreeBSD?

I don't have a FreeBSD test machine any more. So we have to rely on the
web ... xorriso seems to be available in the versions as in Sid (1.5.6)
and Buster (1.5.0):
  https://www.freshports.org/sysutils/xorriso/
GNU xorriso should compile on FreeBSD out of the box:
  https://www.gnu.org/software/xorriso/#download
Older versions available on
  https://ftp.gnu.org/gnu/xorriso
User experience reports are welcome.


Have a nice day :)

Thomas

[toc] | [prev] | [next] | [standalone]

#263163

On 11/5/23 14:16, Thomas Schmitt wrote:
> David Christensen wrote:
>> Adding checksum file(s) to the contents burned to disc is an important step
>> that should not be omitted
> 
> I let xorriso compute and store the checksums in a non-file block range
> at the end of the ISO filesystem. Each file gets an AAIP attribute which
> points to an MD5 in this checksum array.
> The user only has to issue the xorriso command -for_backup or -md5 "on".


Are there tools other than xorriso(1) that can create a compatible 
checksum?  Read the checksum?


My approach is *.md5 and *.sha256 sister files for each archive 
encrypted tarball file.


>> Thomas Schmitt wrote:
>>> i only once in my life watched a DVD giving bad data without an SCSI
>>> error.
>>
>> Interesting.  My WAG is that there was a marginal/ ambiguous dot on disc (?).
> 
> The astonishing fact is not the damaged data chunk but the drive's failure
> to recognize the damage. There are substantial parity data wrapped around
> each DVD "ECC Block" which the drive may use for error detection and
> possibly for correction. If i count correctly in MMC-5 Figure 26, then
> 32 KiB payload data get added 192 * 10 + 15 * 182 = 4650 bytes of parity
> data. ECMA-337 (about DVD+RW) mentions in 13.1 two more checksums with
> 6 bytes per 2048 bytes of payload data.
> 
> With such much of redundancy it is highly unlikely that an alteration
> stays undetected or that an error correction yields a wrong result.
> Nevertheless in this special combination of medium and drive the error was
> not reported or corrected by the drive but rather a wrong data chunk was
> handed out.
> 
> At that occasion an MD5 recorded by xorriso indicated the error.
> Comparing the read results on several drives showed that it was about a
> single ECC block of 32 KiB payload.


I was thinking opto-electronics -- phototransistor and analog-to-digital 
conversion -- but you are right: the error detection and error 
correction math should be the same on all the drives and they all should 
have caught a bad bit (or combination of bad bits).  Then again, 
implementing algorithms from standards is non-trivial; bugs are not 
uncommon.  Perhaps the cause was a combination of both.  But, your 
xorriso(1) MD5 checksum added one more layer of defense and that saved 
the day.


>> I seem to recall a post by you that indicated *BSD lacked the features
>> needed for good optical drive/ media/ format support.
> 
> I have difficulties to remember ...
> 
>> Can I get xorriso(1) on FreeBSD?
> 
> I don't have a FreeBSD test machine any more. So we have to rely on the
> web ... xorriso seems to be available in the versions as in Sid (1.5.6)
> and Buster (1.5.0):
>    https://www.freshports.org/sysutils/xorriso/
> GNU xorriso should compile on FreeBSD out of the box:
>    https://www.gnu.org/software/xorriso/#download
> Older versions available on
>    https://ftp.gnu.org/gnu/xorriso
> User experience reports are welcome.


Bingo!

2023-11-05 14:39:58 dpchrist@f3 ~
$ freebsd-version -kru ; uname -a
12.4-RELEASE-p6
12.4-RELEASE-p6
12.4-RELEASE-p6
FreeBSD f3.tracy.holgerdanske.com 12.4-RELEASE-p6 FreeBSD 
12.4-RELEASE-p6 GENERIC  amd64

2023-11-05 14:40:05 dpchrist@f3 ~
$ pkg search xorriso
xorriso-1.5.6                  ISO image manipulation tool based on 
Libburnia


David

[toc] | [prev] | [next] | [standalone]

#263175

Hi,

David Christensen wrote:
> Are there tools other than xorriso(1) that can create a compatible checksum?
> Read the checksum?

Not yet. The data format is documented in
  https://dev.lovelyhq.com/libburnia/libisofs/raw/branch/master/doc/checksums.txt
For the general concept of AAIP attributes see
  https://dev.lovelyhq.com/libburnia/libisofs/raw/branch/master/doc/susp_aaip_2_0.txt
For the exact format of attributes "isofs.ca" and "isofs.cx" see
  https://dev.lovelyhq.com/libburnia/libisofs/raw/branch/master/doc/susp_aaip_isofs_names.txt

The implementation of this format in another program would be some work.
It would be much easier to link with libisoburn and to use its xorriso
API. Each xorriso command can be performed by a C function call. See
in /usr/include/libisoburn/xorriso.h the functions Xorriso_option_*().
Further there is Xorriso_interpreter() which performs commands and their
parameters given as text arguments. Xorriso_execute_option() splits a text
line into commands and parameters and performs them.

The way to get the MD5s of data files is an -exec action of command -find.
The MD5s must have been loaded at -indev time. So before -indev one has to
perform -for_backup or -md5 "on":

  xorriso -for_backup -indev /dev/sr0 -find / -exec get_md5 --

yields on stdout md5sum compatible lines of all MD5 equipped data files:

  bd8d516f33262f7d8ef3bf952729e671  /my/first_file
  90ae421ded24f03d6b7ae4d5bfdd41e9  /my/second_file
  ...

For the MD5 of just one particular file let -find start at that file

  md5_line=$( xorriso -for_backup -indev /dev/sr0 \
                      -find /my/second_file -exec get_md5 -- 2>/dev/null )


> My approach is *.md5 and *.sha256 sister files for each archive encrypted
> tarball file.

As we can see with xorriso's MD5s and the parity checksums of the medium,
it is always good to have own checksums (although i deem SHA256 overdone
unless protection against malicious manipulations is needed).


> implementing algorithms
> from standards is non-trivial; bugs are not uncommon.

Especially when the specs are sparse with describing the exact algorithm
and use nomenclature from the checksum community.
ECMA-337 (DVD+RW) lists four algorithms, which use three different
polynomials and a notation "RS(X,Y,Z)" which i don't know.

I once had to explore the checksum algorithms of ECMA-130 (CD-ROM)
annex A and B. Other than with DVD and BD, the CD commands of MMC let the
user supply own error correction data when a raw write mode is selected.
See the comments at the beginning of
  https://dev.lovelyhq.com/libburnia/libburn/raw/branch/master/libburn/ecma130ab.c
(which i don't really understand 14 years after i wrote them down).


Have a nice day :)

Thomas

[toc] | [prev] | [next] | [standalone]

#263170 — xorriso and SIGTERM/SIGINT handler (was: Re: How to use dmsetuup?)

On 05/11/2023 15:04, Thomas Schmitt wrote:
> If you want to abort earlier, do not press Ctrl+C but rather do
>    touch /var/opt/xorriso/do_abort_check_media

I do not have an optical drive around for last years, so feel free to 
ignore my question. Are there obstacles making implementation of proper 
SIGINT and SIGTERM signals handler prohibitively difficult? Ctrl+C is a 
common part of UI familiar to the most of users. There are should be 
serious reasons if it is necessary to teach them to touch an application 
specific file.

[toc] | [prev] | [next] | [standalone]

#263177 — Re: xorriso and SIGTERM/SIGINT handler

Hi,

Max Nikulin wrote:
> Are there obstacles making implementation of proper SIGINT and
> SIGTERM signals handler prohibitively difficult? Ctrl+C is a common part of
> UI familiar to the most of users. There are should be serious reasons if it
> is necessary to teach them to touch an application specific file.

15 years after the implementation of -check_media this is an interesting
question. I dimly remember to have introduced the abort file because
Ctrl+C was too rough.
Maybe this snippet from man xorriso explains the motivation of my
past self:

  -check_media_defaults [option [option ...]] --
         ...
         abort_file=disk_path  gives the path of the file which may abort
         a scan run. Abort happens if the file exists and  its  mtime  is
         not  older  than  the  start  time of the run. Use shell command
         "touch" to trigger this.  Other than  an  aborted  program  run,
         this  will  report the tested and untested blocks and go on with
         running xorriso.

I imagined the rescue attempts to happen in xorriso dialog mode.
After aborting an attempt with sr0, one could use the same xorriso run to
make an attempt with other min_lba= , max_lba= limits or with drive sr1.


Have a nice day :)

Thomas

[toc] | [prev] | [next] | [standalone]

#263127

On 11/4/23 09:45, tomas@tuxteam.de wrote:
> On Sat, Nov 04, 2023 at 07:46:09AM -0400, gene heskett wrote:
> 
> [...]
> 
>> I'v got to the above point but the first example that looked good created a
>> 100% allocated, no free space "homevol"
>> So I used gparted to delete the partitions & reformat them to ext4 again,
>> pvcreated them again an vgcreateded it again, getting:
> 
> Sorry, Gene -- I fear you have it backwards. The LVM stuff happens *below*
> the file system: you first add physical volumes (PV) to a volume group
> (think a "pool"). From that you "cut out" logical volumes (LV), which
> are just bunches of blocks which show themselves to the OS as "block
> devices" (/dev/mapper/foo, typically).
> 
> On top of that you can put a file system (as you can on any "bunch of blocks",
> i.e. a block device or file).
> 
> Perhaps having a mental model the docs make more sense.
> 
> Cheers

Cheers, Gene Heskett.
-- 
"There are four boxes to be used in defense of liberty:
  soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author, 1940)
If we desire respect for the law, we must first make the law respectable.
  - Louis D. Brandeis

[toc] | [prev] | [next] | [standalone]

#263112

Hi Gene,

On Fri, Nov 03, 2023 at 12:27:19PM -0400, gene heskett wrote:
> Thanks for help with dmsetup.

dmsetup is very much the wrong approach for you - it's too
low-level.

LVM alone is probably not the best idea either. For your use case as
I understand it, mdraid in RAID1 or RAID10 is probably the best
solution.

I regret I am not able to assist you with the problems you have with
your existing RAID10. Changing it for just LVM, or trying to do it
"by hand" with dmsetup are likely to be mistakes however.

Maybe it is time to just buy a "black box" NAS device and make all
this someone else's problem in return for money. It's not the way I
go, but you have had a lot of trouble getting your own mdraid to
work.

Thanks,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting

[toc] | [prev] | [next] | [standalone]

#263115

On 11/4/23 05:39, Andy Smith wrote:
> Hi Gene,
> 
> On Fri, Nov 03, 2023 at 12:27:19PM -0400, gene heskett wrote:
>> Thanks for help with dmsetup.
> 
> dmsetup is very much the wrong approach for you - it's too
> low-level.
> 
> LVM alone is probably not the best idea either. For your use case as
> I understand it, mdraid in RAID1 or RAID10 is probably the best
> solution.
> 
> I regret I am not able to assist you with the problems you have with
> your existing RAID10. Changing it for just LVM, or trying to do it
> "by hand" with dmsetup are likely to be mistakes however.
> 
I'm afraid I have to agree. I've easily gotten to the last example Andy 
Cater gave me, but these red hat sourced man pages are full of very 
copious examples while being totally opaque as to what the examples do. 
And that led to my creating a 3.7T lvm that had no free space. Wash 
rinse, repeat. Frustration is too mild a word.

> Maybe it is time to just buy a "black box" NAS device and make all
> this someone else's problem in return for money. It's not the way I
> go, but you have had a lot of trouble getting your own mdraid to
> work.

Good advice maybe, Andy Smith, thank you, but that puts it all at the 
mercy of a $5 cable.  Not exactly my cup of tea.  Since I'm into 
designing stuff in OpenSCAD, 3d printing the output, made into gcode 
with Cura, and none of the common tools for making gcode to drive the 
printers, or the printers own firmware, has learned how to roll up the 
code into repetitive loops, instead generating step by step printer 
instructions, so even the simplest part is half a gig of g-code. So the 
data is growing like a cancer. Really complex parts might be 50 gigs of 
g-code and takes the printer a week or more to make.

G-code, properly written is like a .pdf, its the best compression we 
have. I write it by hand for linuxcnc and have one 90 LOC file that 
takes the machine 3 days to run. Remove the comments and it fits on one 
side of one sheet of paper.

Take care & stay well, Andy.
> Thanks,
> Andy
> 

Cheers, Gene Heskett.
-- 
"There are four boxes to be used in defense of liberty:
  soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author, 1940)
If we desire respect for the law, we must first make the law respectable.
  - Louis D. Brandeis

[toc] | [prev] | [next] | [standalone]

#263189

On 03/11/23 at 17:27, gene heskett wrote:
> Greetings all;
> As usual, the man page may as well be written in swahili. The NDE 
> syndrome, meaning No D-----d Examples.
> 
> I have those 2 2T SSD's with a gpt partition table on both, allocated as 
> sdc1 and sdk1, formatted to ext4, named and labeled as lvm1 and lvm2.
> Temp mounted as sdc1 and sdk1 to /mnt/lvm1 and /mnt/lvm2
> 
> How do I create a single managed volume of labels lvm1 and lvm2 of these 
> to make a single volume that I can then rsynch /home to it, then switch 
> fstab to mount it as /home on a reboot?
> 

How about to use debian-installer: burn the dvd image of Bookworm 12.2, 
put into the DVD drive then reboot the system. You have to choose 
"Expert Install" and it's all menu driven from RAID device creation to 
LVM logical device and logical volume names.
I don't know if you can do that from debian-installer rescue disk mode.

HTH
kinds regards

-- 
Franco Martelli

[toc] | [prev] | [next] | [standalone]

#263193

On 11/6/23 10:48, Franco Martelli wrote:
> On 03/11/23 at 17:27, gene heskett wrote:
>> Greetings all;
>> As usual, the man page may as well be written in swahili. The NDE 
>> syndrome, meaning No D-----d Examples.
>>
>> I have those 2 2T SSD's with a gpt partition table on both, allocated 
>> as sdc1 and sdk1, formatted to ext4, named and labeled as lvm1 and lvm2.
>> Temp mounted as sdc1 and sdk1 to /mnt/lvm1 and /mnt/lvm2
>>
>> How do I create a single managed volume of labels lvm1 and lvm2 of 
>> these to make a single volume that I can then rsynch /home to it, then 
>> switch fstab to mount it as /home on a reboot?
>>
> 
> How about to use debian-installer: burn the dvd image of Bookworm 12.2, 
> put into the DVD drive then reboot the system. You have to choose 
> "Expert Install" and it's all menu driven from RAID device creation to 
> LVM logical device and logical volume names.
> I don't know if you can do that from debian-installer rescue disk mode.
> 
> HTH
> kinds regards
> 
All entirely possible, unless you forget to unplug everything usb that 
isn't keyboard or mouse. BTDT, 22+ times.

Cheers, Gene Heskett.
-- 
"There are four boxes to be used in defense of liberty:
  soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author, 1940)
If we desire respect for the law, we must first make the law respectable.
  - Louis D. Brandeis

[toc] | [prev] | [next] | [standalone]

#263241

On 11/6/23 08:47, Franco Martelli wrote:
> On 03/11/23 at 17:27, gene heskett wrote:
>> Greetings all;
>> As usual, the man page may as well be written in swahili. The NDE syndrome, meaning No D-----d Examples.
>>
>> I have those 2 2T SSD's with a gpt partition table on both, allocated as sdc1 and sdk1, formatted to ext4, named and labeled as lvm1 and lvm2.
>> Temp mounted as sdc1 and sdk1 to /mnt/lvm1 and /mnt/lvm2
>>
>> How do I create a single managed volume of labels lvm1 and lvm2 of these to make a single volume that I can then rsynch /home to it, then switch fstab to mount it as /home on a reboot?

You do not put a file system on the partitions you are using as LVM physical volumes. And you do not mount them.

The rough procedure is

Create LVM physical volumes on raw disk partitions using pvcreate  (or lvm pvcreate) e. g.,

pvcreate /dev/sdc1
pvcreate /dev/sdk1

This gives you two physical volumes to use to create one or two volume groups
Create an LVM volume group using vgcreate (or lvm vgcreate), e. g.,

vgcreate home-vg sdc1 sdk1

This gives you a volume group named "home-vg" with 4.4 TB raw storage in which you can create one or more logical volumes.
Create the logical volumes you want. It appears you want only one, to mounted at /home. For instance,

lvcreate --size 1024G -n home-volume home-vg

will create a 1 TB logical volume, represented under dev by /dev/home-vg/home-volume

Put a file system on the logical volume in the normal way, such  as:

mkfs -t ext4 /dev/home-vg/home-volume

mount the new volume (and put it in /etc/fstab for mounting at boot:

mount /dev/home-vg/home-volume /home

Doing this probably will not give you what you want. (For instance, if I remember right, the entire logical volume would, in this case, wind up on the first-named physical volume vgcreate command.) The man pages for lvm and its subcommands offer a lot of options for things like storage allocation between/among multiple physical volumes that make up a volume group, the size of allocation units, such things as RAID level, and a large number of other properties. You probably know what you want, and from what I've seen on this list seem quite able to fish it up out of the man pages, some of which have usefully suggestive examples.

OTOH, I would recommend ZFS for this based on experience with LVM and ZFS in both commercial (e. g., HP-UX and SolaRIS) and Linux environments. Both have learning curves that I would judge comparable, both are flexible and fairly easy to manage, and both are or can be highly resilient. On the whole, though, I prefer ZFS.

Regards,
Tom Dial

>>
> 
> How about to use debian-installer: burn the dvd image of Bookworm 12.2, put into the DVD drive then reboot the system. You have to choose "Expert Install" and it's all menu driven from RAID device creation to LVM logical device and logical volume names.
> I don't know if you can do that from debian-installer rescue disk mode.
> 
> HTH
> kinds regards
> 

[toc] | [prev] | [next] | [standalone]

#263243

On 11/7/23 18:42, Tom Dial wrote:
> 
> 
> On 11/6/23 08:47, Franco Martelli wrote:
>> On 03/11/23 at 17:27, gene heskett wrote:
>>> Greetings all;
>>> As usual, the man page may as well be written in swahili. The NDE 
>>> syndrome, meaning No D-----d Examples.
>>>
>>> I have those 2 2T SSD's with a gpt partition table on both, allocated 
>>> as sdc1 and sdk1, formatted to ext4, named and labeled as lvm1 and lvm2.
>>> Temp mounted as sdc1 and sdk1 to /mnt/lvm1 and /mnt/lvm2
>>>
>>> How do I create a single managed volume of labels lvm1 and lvm2 of 
>>> these to make a single volume that I can then rsynch /home to it, 
>>> then switch fstab to mount it as /home on a reboot?
> 
> You do not put a file system on the partitions you are using as LVM 
> physical volumes. And you do not mount them.
> 
What do I do if a gpt partition table has already been made and an ext4 
system is already installed? IOW just how "bare" a disk is needed? Is 
writing a null gpt sufficient?

Thank you Tom.

> The rough procedure is
> 
> Create LVM physical volumes on raw disk partitions using pvcreate  (or 
> lvm pvcreate) e. g.,
> 
> pvcreate /dev/sdc1
> pvcreate /dev/sdk1
> 
> This gives you two physical volumes to use to create one or two volume 
> groups
> Create an LVM volume group using vgcreate (or lvm vgcreate), e. g.,
> 
> vgcreate home-vg sdc1 sdk1
> 
> This gives you a volume group named "home-vg" with 4.4 TB raw storage in 
> which you can create one or more logical volumes.
> Create the logical volumes you want. It appears you want only one, to 
> mounted at /home. For instance,
> 
> lvcreate --size 1024G -n home-volume home-vg
> 
> will create a 1 TB logical volume, represented under dev by 
> /dev/home-vg/home-volume
> 
> Put a file system on the logical volume in the normal way, such  as:
> 
> mkfs -t ext4 /dev/home-vg/home-volume
> 
> mount the new volume (and put it in /etc/fstab for mounting at boot:
> 
> mount /dev/home-vg/home-volume /home
> 
> Doing this probably will not give you what you want. (For instance, if I 
> remember right, the entire logical volume would, in this case, wind up 
> on the first-named physical volume vgcreate command.) The man pages for 
> lvm and its subcommands offer a lot of options for things like storage 
> allocation between/among multiple physical volumes that make up a volume 
> group, the size of allocation units, such things as RAID level, and a 
> large number of other properties. You probably know what you want, and 
> from what I've seen on this list seem quite able to fish it up out of 
> the man pages, some of which have usefully suggestive examples.
> 
> OTOH, I would recommend ZFS for this based on experience with LVM and 
> ZFS in both commercial (e. g., HP-UX and SolaRIS) and Linux 
> environments. Both have learning curves that I would judge comparable, 
> both are flexible and fairly easy to manage, and both are or can be 
> highly resilient. On the whole, though, I prefer ZFS.
> 
> Regards,
> Tom Dial
> 
>>>
>>
>> How about to use debian-installer: burn the dvd image of Bookworm 
>> 12.2, put into the DVD drive then reboot the system. You have to 
>> choose "Expert Install" and it's all menu driven from RAID device 
>> creation to LVM logical device and logical volume names.
>> I don't know if you can do that from debian-installer rescue disk mode.
>>
>> HTH
>> kinds regards
>>
> 
> .

Cheers, Gene Heskett.
-- 
"There are four boxes to be used in defense of liberty:
  soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author, 1940)
If we desire respect for the law, we must first make the law respectable.
  - Louis D. Brandeis

[toc] | [prev] | [next] | [standalone]

Page 2 of 4 — ← Prev page 1 [2] 3 4  Next page →

Back to top | Article view | linux.debian.user

csiph-web