Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.user > #178621 > unrolled thread

Re: Some help with dd backing up into an iso

Back to article view | Back to linux.debian.user

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: Some help with dd backing up into an iso "Thomas Schmitt" <scdbackup@gmx.net> - 2017-03-09 18:00 +0100
    Re: Some help with dd backing up into an iso | now xorriso help GiaThnYgeia <GiaThnYgeia@openmailbox.org> - 2017-03-09 19:30 +0100
      Re: Some help with dd backing up into an iso "Thomas Schmitt" <scdbackup@gmx.net> - 2017-03-09 19:50 +0100
        Re: Some help with dd backing up into an iso GiaThnYgeia <GiaThnYgeia@openmailbox.org> - 2017-03-11 18:50 +0100
          Re: Some help with dd backing up into an iso "Thomas Schmitt" <scdbackup@gmx.net> - 2017-03-12 09:10 +0100
            Re: Some help with dd backing up into an iso GiaThnYgeia <GiaThnYgeia@openmailbox.org> - 2017-03-12 14:20 +0100
              Re: Some help with dd backing up into an iso "Thomas Schmitt" <scdbackup@gmx.net> - 2017-03-12 15:10 +0100
                Re: Some help with dd backing up into an iso GiaThnYgeia <GiaThnYgeia@openmailbox.org> - 2017-03-13 01:00 +0100
                  Re: Some help with dd backing up into an iso David Wright <deblis@lionunicorn.co.uk> - 2017-03-13 04:30 +0100
                  Re: Some help with dd backing up into an iso "Thomas Schmitt" <scdbackup@gmx.net> - 2017-03-13 09:20 +0100
                    Re: Some help with dd backing up into an iso GiaThnYgeia <GiaThnYgeia@openmailbox.org> - 2017-03-13 13:20 +0100
                      Re: Some help with dd backing up into an iso Greg Wooledge <wooledg@eeg.ccf.org> - 2017-03-13 14:00 +0100
                        Re: Some help with dd backing up into an iso "Thomas Schmitt" <scdbackup@gmx.net> - 2017-03-13 14:30 +0100
                    Re: Some help with dd backing up into an iso GiaThnYgeia <GiaThnYgeia@openmailbox.org> - 2017-03-13 14:10 +0100
                      Re: Some help with dd backing up into an iso "Thomas Schmitt" <scdbackup@gmx.net> - 2017-03-13 14:40 +0100
                        Re: Some help with dd backing up into an iso "Thomas Schmitt" <scdbackup@gmx.net> - 2017-03-13 17:50 +0100
                          Re: Some help with dd backing up into an iso GiaThnYgeia <GiaThnYgeia@openmailbox.org> - 2017-03-13 19:40 +0100
                            Re: Some help with dd backing up into an iso "Thomas Schmitt" <scdbackup@gmx.net> - 2017-03-13 21:10 +0100
            Disabling automount, and mounting/ unmounting the "old way" David Christensen <dpchrist@holgerdanske.com> - 2017-03-12 21:20 +0100
              Re: Disabling automount, and mounting/ unmounting the "old way" "Thomas Schmitt" <scdbackup@gmx.net> - 2017-03-12 22:00 +0100

#178621 — Re: Some help with dd backing up into an iso

Hi,

i forgot to adapt my xorriso example from a few days ago:

  xorriso \
  -for_backup \
  -outdev usb_part1.iso \
  -map /mnt/usb-Kingston_DataTraveler_3.0_08606E69C773BFC06965007B-0:0-part1 /

Note that the last "/" is not a misspelled "\" but the path to the
upcomming ISO's root directory. The "\" tell the shell that the command
line goes on in the next input line.

This will create the (probably large) file usb_part1.iso, which you
can mount (as superuser) by:

  mkdir /mnt/usb_part1_iso
  mount -o loop /where/it/is/usb_part1.iso /mnt/usb_part1_iso

with "/where/it/is" replaced by the absolute path to the ISO image file.
Then the file tree copy should show up under /mnt/usb_part1_iso .

If you have a DVD drive, with e.g. address /dev/sr0 and a DVD+RW in it,
then you may write directly to the DVD:

  xorriso \
  -for_backup \
  -outdev /dev/sr0 \
  -blank as_needed \
  -map /mnt/usb-Kingston_DataTraveler_3.0_08606E69C773BFC06965007B-0:0-part1 /

"-blank as_needed" will enable overwriting of old content on the DVD+RW
or blank a DVD-RW. With DVD+RW or formatted DVD-RW, it will be fast.
With unformatted DVD-RW it will last as long as a full write run.
DVD-R or DVD+R are usable if not yet written by other burn runs.


Have a nice day :)

Thomas

[toc] | [next] | [standalone]

#178623 — Re: Some help with dd backing up into an iso | now xorriso help

Thomas Schmitt:
> Hi,
> 
> i forgot to adapt my xorriso example from a few days ago:
> 
>   xorriso \
>   -for_backup \
>   -outdev usb_part1.iso \
>   -map /mnt/usb-Kingston_DataTraveler_3.0_08606E69C773BFC06965007B-0:0-part1 /

Ok, my drive has grown from 1,7gB to about 2GB since the last try with
DD which produced an image equal to the drive, 7,7gB and took for ever.

This run in a second and produced an iso that is about half a MB.  Is
something wrong?

-rw-r--r--  1 user     458752 Mar  9 00:08 usb_part1.iso


> 
> Note that the last "/" is not a misspelled "\" but the path to the
> upcomming ISO's root directory. The "\" tell the shell that the command
> line goes on in the next input line.
> 
> This will create the (probably large) file usb_part1.iso, which you
> can mount (as superuser) by:
> 
>   mkdir /mnt/usb_part1_iso
>   mount -o loop /where/it/is/usb_part1.iso /mnt/usb_part1_iso
> 
> with "/where/it/is" replaced by the absolute path to the ISO image file.
> Then the file tree copy should show up under /mnt/usb_part1_iso .
> 
> If you have a DVD drive, with e.g. address /dev/sr0 and a DVD+RW in it,
> then you may write directly to the DVD:
> 
>   xorriso \
>   -for_backup \
>   -outdev /dev/sr0 \
>   -blank as_needed \
>   -map /mnt/usb-Kingston_DataTraveler_3.0_08606E69C773BFC06965007B-0:0-part1 /
> 
> "-blank as_needed" will enable overwriting of old content on the DVD+RW
> or blank a DVD-RW. With DVD+RW or formatted DVD-RW, it will be fast.
> With unformatted DVD-RW it will last as long as a full write run.
> DVD-R or DVD+R are usable if not yet written by other burn runs.
> 
> 
> Have a nice day :)
> 
> Thomas
> 

-- 
 "The most violent element in society is ignorance" rEG

[toc] | [prev] | [next] | [standalone]

#178625

Hi,

GiaThnYgeia wrote:
> Is something wrong?  
> -rw-r--r--  1 user     458752 Mar  9 00:08 usb_part1.iso

That's much too small for any backup with substance.
What do you get from

  xorriso -indev usb_part1.iso -find / -exec lsdl --

Is the USB stick content visible underneath
  /mnt/usb-Kingston_DataTraveler_3.0_08606E69C773BFC06965007B-0:0-part1 
at all ?

If the mount point was really a link, one would have to enable link
following at least for files which are given as program arguments:

  xorriso \
  -for_backup \
  -follow default:param \
  -outdev usb_part1.iso \
  -map /mnt/usb-Kingston_DataTraveler_3.0_08606E69C773BFC06965007B-0:0-part1 /

But with default -follow settings you would have gotten an error message:
  xorriso : FAILURE : Source '/mnt/usb-...' is not a directory. Target '' would be.

So i assume that the USB stick content was simply not available under
the /mnt/usb-... directory.


Have a nice day :)

Thomas

[toc] | [prev] | [next] | [standalone]

#178685

I am nearly giving up, can't understand what I am doing wrong or what I
should be doing.

Thomas Schmitt:
> Hi,
> 
> GiaThnYgeia wrote:
>> Is something wrong?  
>> -rw-r--r--  1 user     458752 Mar  9 00:08 usb_part1.iso
> 
> That's much too small for any backup with substance.
> What do you get from
> 
>   xorriso -indev usb_part1.iso -find / -exec lsdl --

$ xorriso -indev sid1.iso -find / -exec lsdl --
xorriso 1.4.6 : RockRidge filesystem manipulator, libburnia project.

xorriso : NOTE : Loading ISO image tree from LBA 0
Drive current: -indev 'sid1.iso'
Media current: stdio file, overwriteable
Media status : is written , is appendable
Media summary: 1 session, 27 data blocks, 54.0k data,  458g free
Volume id    : 'ISOIMAGE'
drwxr-xr-x    1 0        0               0 Nov 24 11:14 '/'

On media this shows the 2nd system partition as a directory
/media/user/1340a59d-7c08-4257-a81d-9cb8ef707c0e



> 
> Is the USB stick content visible underneath
>   /mnt/usb-Kingston_DataTraveler_3.0_08606E69C773BFC06965007B-0:0-part1 
> at all ?
> 
> If the mount point was really a link, one would have to enable link
> following at least for files which are given as program arguments:
> 
>   xorriso \
>   -for_backup \
>   -follow default:param \
>   -outdev usb_part1.iso \
>   -map /mnt/usb-Kingston_DataTraveler_3.0_08606E69C773BFC06965007B-0:0-part1 /

xorriso 1.4.6 : RockRidge filesystem manipulator, libburnia project.

Drive current: -outdev 'siduct1.iso'
Media current: stdio file, overwriteable
Media status : is written , is appendable
Media summary: 1 session, 27 data blocks, 54.0k data,  458g free
xorriso : FAILURE : Cannot determine attributes of source file
'/media/user/DebonUSB
/usb-Kingston_DataTraveler_3.0_08606E69C773BFC06965007B-0:0-part1' : No
such file or directory



> But with default -follow settings you would have gotten an error message:
>   xorriso : FAILURE : Source '/mnt/usb-...' is not a directory. Target '' would be.
> 
> So i assume that the USB stick content was simply not available under
> the /mnt/usb-... directory.
> 
> 
> Have a nice day :)
> Thomas

You have a great one too!
I;m going back to try more dd options

-- 
 "The most violent element in society is ignorance" rEG

[toc] | [prev] | [next] | [standalone]

#178707

Hi,

GiaThnYgeia wrote:
> $ xorriso -indev sid1.iso -find / -exec lsdl --
> ...
> drwxr-xr-x    1 0        0               0 Nov 24 11:14 '/'

This explains why the ISO is so small. No files in it.
(The size consists mainly of the traditional 300 KB of padding at the
 end of the image.)


> On media this shows the 2nd system partition as a directory
> /media/user/1340a59d-7c08-4257-a81d-9cb8ef707c0e

Last time you showed it, it was as empty as the ISO.
What do you get from

  ls -ld /media/user/1340a59d-7c08-4257-a81d-9cb8ef707c0e


> >   xorriso -for_backup -follow default:param ...

> xorriso : FAILURE : Cannot determine attributes of source file
> '/media/user/DebonUSB
> /usb-Kingston_DataTraveler_3.0_08606E69C773BFC06965007B-0:0-part1' : No
> such file or directory

Very strange file address. It is probably the result of link following.
Thus my request to do "ls -ld".

Is the line break between "DebonUSB" and "/usb-Kingston" visible on the
terminal screen, too ? Or is it an artefact of copy+paste ?

(I am very happy that i disabled automounting on my system. Life becomes
 so clear and straightforward if one does it the old way.)


Have a nice day :)

Thomas

[toc] | [prev] | [next] | [standalone]

#178720

I am getting a little frustrated as neither dd or xorriso work for me as
I wanted.  With the dd and bzip2 combination I got an image really fast
(compared to dd if=.. of=.. ) but when I tried to restore it
dd bs=1M if=/dev/sdb | bzip2 >imagefile
bunzip2 imagefile | dd of=/dev/sdb

it unzipped the imagefile into an uncompress file and burned it ...
although I am not sure what I mixed up in the filenames it restored an
earlier on...

bunzip2 imagefile  -f -t -v | dd of=/dev/sdb

I think this option retains the original compressed image and show what
is doing, although to me the -v is meaningless.

Thomas Schmitt:
> Hi,
> 
> GiaThnYgeia wrote:
>> $ xorriso -indev sid1.iso -find / -exec lsdl --
>> ...
>> drwxr-xr-x    1 0        0               0 Nov 24 11:14 '/'
>> On media this shows the 2nd system partition as a directory
>> /media/user/1340a59d-7c08-4257-a81d-9cb8ef707c0e
> 
> Last time you showed it, it was as empty as the ISO.
> What do you get from
> 
>   ls -ld /media/user/1340a59d-7c08-4257-a81d-9cb8ef707c0e

I changed the volume name to sid

$ ls -ld /media/user/sid
drwxr-xr-x 1 user user 2048 Mar  5 20:30 /media/user/sid
$ ls -ld /mnt
$ ls /mnt
usb-Kingston_DataTraveler_3.0_08606E69C773BFC06965007B-0:0-part1
$ ls -ld /mnt/u*
drwxr-xr-x 2 root root 4096 Nov 24 11:14
/mnt/usb-Kingston_DataTraveler_3.0_08606E69C773BFC06965007B-0:0-part1
$ ls -ld /media/user/sid
drwxr-xr-x 1 user user 2048 Mar  5 20:30 /media/user/sid
$



> 
> 
>>>   xorriso -for_backup -follow default:param ...
> 
>> xorriso : FAILURE : Cannot determine attributes of source file
>> '/media/user/DebonUSB
>> /usb-Kingston_DataTraveler_3.0_08606E69C773BFC06965007B-0:0-part1' : No
>> such file or directory
> 
> Very strange file address. It is probably the result of link following.
> Thus my request to do "ls -ld".

It seems as if it is hardware created and can't be changed

> Is the line break between "DebonUSB" and "/usb-Kingston" visible on the
> terminal screen, too ? Or is it an artefact of copy+paste ?

I changed it so it is easier to deal with

> (I am very happy that i disabled automounting on my system. Life becomes
>  so clear and straightforward if one does it the old way.)

Being protective of the setup I have on the little disk I am trying to
restore its image in an identical disk (manufacturer and size)
Only when I get convinced that the restored image is identical (looks
and function) will I be able to go ahead to do more.  No I am worried
I'll ruin all the work and I will not have a last safe image to restore to.

> Have a nice day :)
> Thomas

You have a better one

katkat

-- 
 "The most violent element in society is ignorance" rEG

[toc] | [prev] | [next] | [standalone]

#178722

Hi,

i wrote:
> > bunzip2 <usbfilename.img | dd of=/dev/sdb

GiaThnYgeia wrote:
> bunzip2 imagefile | dd of=/dev/sdb

The small but decisive difference is the "<" in my example.

My example gives bunzip2 no file path, so that it begins to read from
standard input and writes to standard output. bunzip2's standard
input is redirected from file "imagefile". The standard output of
bunzip2 is then piped to standard input of dd, which due to lack of
option "if=" reads it. dd option "of=" then directs the data to /dev/sdb,
the USB stick's device file.

Your command gives bunzip2 the file path "imagefile" which means that
you want this file to be uncompressed. Any standard output from bunzip2
is piped to to dd which would copy it to /dev/sdb.
But i guess there were no bytes written to bunzip's standard output.


> it unzipped the imagefile into an uncompress file

The man page iof bzip2 says:
"bunzip2 (or bzip2 -d) decompresses all specified  files.
 ...
 As  with  compression, supplying no filenames causes decompression from
 standard input to standard output."


> and burned it ...

The file ? The USB stick ?
Not with flames and smoke, i hope ...


> bunzip2 imagefile  -f -t -v | dd of=/dev/sdb

Well, option -t means:
  -t --test
      Check  integrity  of the specified file(s), but don't decompress
      them.  This really performs a  trial  decompression  and  throws
      away the result.

So this run did nothing, i suppose.

In what state is "imagefile" now ? Compressed ? Uncompressed ? Ashes ?
(It is preferrable to marke bzip2'ed files by suffix .bz2, which bunzip2
 will remove when replacing the compressed file by the decompressed file.)


> $ ls -ld /mnt/u*
> drwxr-xr-x 2 root root 4096 Nov 24 11:14
> /mnt/usb-Kingston_DataTraveler_3.0_08606E69C773BFC06965007B-0:0-part1

I simply have no clue why "xorriso -follow param" would then complain
about a (one single) file with a newline character in its path:
  /media/user/DebonUSB
  /usb-Kingston_DataTraveler_3.0_08606E69C773BFC06965007B-0:0-part1


> >> xorriso : FAILURE : Cannot determine attributes of source file
> >> '/media/user/DebonUSB
> >> /usb-Kingston_DataTraveler_3.0_08606E69C773BFC06965007B-0:0-part1'

> > Is the line break between "DebonUSB" and "/usb-Kingston" visible on the
> > terminal screen, too ? Or is it an artefact of copy+paste ?

> I changed it so it is easier to deal with

This is not an answer to my question.
Is the reported address a single line
  /media/user/DebonUSB/usb-Kingston_DataTraveler_3.0_08606E69C773BFC06965007B-0:0-part1
or is it reported as two lines:
  /media/user/DebonUSB
  /usb-Kingston_DataTraveler_3.0_08606E69C773BFC06965007B-0:0-part1
?

The reason why i ask is that i wonder from where xorriso has this
strange two-line path.
It would be explainable if you had given it to xorriso command "-map".

(The theory that it might be a symbolic link effect is dead meanwhile.)

---------------------------------------------------------------------

New approach to get to a mount point of the stick:

If you do

  find /media/user/sid | less

while the stick is plugged in, do you see all the many file paths from
the stick ?
(No need to count them all. Just check whether it looks roughly like
the expected content.)

If so, then try that address instead of the lengthy one:

  xorriso \
  -for_backup \
  -outdev usb_part1.iso \
  -map /media/user/sid /


Have a nice day :)

Thomas

[toc] | [prev] | [next] | [standalone]

#178737

Thomas Schmitt:
> Hi,
> i wrote:
>>> bunzip2 <usbfilename.img | dd of=/dev/sdb
> 
> GiaThnYgeia wrote:
>> bunzip2 imagefile | dd of=/dev/sdb
> 
> The small but decisive difference is the "<" in my example.

My fault, I thought it was brackets to remind me to enter my own
filename and the second one was missing ;)

> My example gives bunzip2 no file path, so that it begins to read from
> standard input and writes to standard output. bunzip2's standard
> input is redirected from file "imagefile". The standard output of
> bunzip2 is then piped to standard input of dd, which due to lack of
> option "if=" reads it. dd option "of=" then directs the data to /dev/sdb,
> the USB stick's device file.

Makes perfect sense now!

> Your command gives bunzip2 the file path "imagefile" which means that
> you want this file to be uncompressed. Any standard output from bunzip2
> is piped to to dd which would copy it to /dev/sdb.
> But i guess there were no bytes written to bunzip's standard output.

I tried to improvise to get it to work in between "sessions" and I
couldn't see what it was doing so I can correct it.  A long time to wait
for 0 output.

>> it unzipped the imagefile into an uncompress file
> 
> The man page iof bzip2 says:
> "bunzip2 (or bzip2 -d) decompresses all specified  files.

Going back and forth in help documents I realized this although I have
learned to not assume much.

>  As  with  compression, supplying no filenames causes decompression from
>  standard input to standard output."

...aka screen dump?

>> and burned it ...
> 
> The file ? The USB stick ?
> Not with flames and smoke, i hope ...

It got really close to being recycled ... :)

>> bunzip2 imagefile  -f -t -v | dd of=/dev/sdb
> 
> Well, option -t means:
>   -t --test
>       Check  integrity  of the specified file(s), but don't decompress
>       them.  This really performs a  trial  decompression  and  throws
>       away the result.
> 
> So this run did nothing, i suppose.

Yeap!

> In what state is "imagefile" now ? Compressed ? Uncompressed ? Ashes ?
> (It is preferrable to marke bzip2'ed files by suffix .bz2, which bunzip2
>  will remove when replacing the compressed file by the decompressed file.)

Now it is at the state of being all thrown to trashcan (too big for it
so it gets deleted permanently) and a new project will begin sometime
next week ....

>> $ ls -ld /mnt/u*
>> drwxr-xr-x 2 root root 4096 Nov 24 11:14
>> /mnt/usb-Kingston_DataTraveler_3.0_08606E69C773BFC06965007B-0:0-part1
> 
> I simply have no clue why "xorriso -follow param" would then complain
> about a (one single) file with a newline character in its path:
>   /media/user/DebonUSB
>   /usb-Kingston_DataTraveler_3.0_08606E69C773BFC06965007B-0:0-part1

I think it was that dreadful Calamares installer that came with this sid
distro that locks onto the disk and prevents ovewriting.
But when I tried to install it in a small partition of the disk and used
its installation option of encrypting .... guess what?  It took the
passphrase and when it was done I entered it like there was no
encryption at all ...  I thought it would go somehow in the grub menu
and ask me for a pass to boot it, it ruined my grub, it would boot up
directly not giving me an option for the other installations, and go
straight into the login screen.

But the usb was so hard locked that gparted would erase its partition,
change the file system, rechange and repartition, and the thing was
still there.  So I burned a debian installation disk on it, then
repaired everything on hd, and deleted any evidence (I think) that
siduction even came close to the system.

>>>> xorriso : FAILURE : Cannot determine attributes of source file
>>>> '/media/user/DebonUSB
>>>> /usb-Kingston_DataTraveler_3.0_08606E69C773BFC06965007B-0:0-part1'
> 
>>> Is the line break between "DebonUSB" and "/usb-Kingston" visible on the
>>> terminal screen, too ? Or is it an artefact of copy+paste ?
>> I changed it so it is easier to deal with
> 
> This is not an answer to my question.
> Is the reported address a single line
>   /media/user/DebonUSB/usb-Kingston_DataTraveler_3.0_08606E69C773BFC06965007B-0:0-part1
> or is it reported as two lines:
>   /media/user/DebonUSB
>   /usb-Kingston_DataTraveler_3.0_08606E69C773BFC06965007B-0:0-part1
> ?

No it is all connected line and I don't think I can change this.  But so
much for the serial number theory as two different disks have the same
exact address.  That gets really confusing if you don't change the
labels on them.

> The reason why i ask is that i wonder from where xorriso has this
> strange two-line path.  It would be explainable if you had given it to xorriso command "-map".

So are you saying the problem may lie in the name length that xorriso
can't handle?

> (The theory that it might be a symbolic link effect is dead meanwhile.)

the question I have is if this is ....part1 where is the other part/s?

> New approach to get to a mount point of the stick:
> 
> If you do
>   find /media/user/sid | less
> 
> while the stick is plugged in, do you see all the many file paths from
> the stick ?
> (No need to count them all. Just check whether it looks roughly like
> the expected content.)
> 
> If so, then try that address instead of the lengthy one:
> 
>   xorriso \
>   -for_backup \
>   -outdev usb_part1.iso \
>   -map /media/user/sid /

I will report back as I am fried for the day ...

drwxr-xr-x 3 root root 4096 Mar 12 16:23
/mnt/usb-Kingston_DataTraveler_3.0_08606E69C773BFC06965007B-0:0-part1


xorriso 1.4.6 : RockRidge filesystem manipulator, libburnia project.

Drive current: -outdev 'usb_part1.iso'
Media current: stdio file, overwriteable
Media status : is blank
Media summary: 0 sessions, 0 data blocks, 0 data,  451g free
xorriso : UPDATE : 6000 files added in 1 seconds
xorriso : UPDATE : 13800 files added in 2 seconds
xorriso : UPDATE : 24000 files added in 3 seconds
xorriso : UPDATE : 32400 files added in 4 seconds
xorriso : UPDATE : 36400 files added in 5 seconds
xorriso : UPDATE : 36500 files added in 7 seconds
xorriso : UPDATE : 36900 files added in 8 seconds
xorriso : UPDATE : 37700 files added in 12 seconds
xorriso : UPDATE : 38600 files added in 14 seconds
xorriso : UPDATE : 41700 files added in 15 seconds
xorriso : UPDATE : 47300 files added in 18 seconds
xorriso : UPDATE : 49200 files added in 20 seconds
xorriso : UPDATE : 58300 files added in 21 seconds
xorriso : UPDATE : 65300 files added in 22 seconds
xorriso : UPDATE : 71400 files added in 23 seconds
xorriso : UPDATE : 80900 files added in 24 seconds
xorriso : UPDATE : 93500 files added in 25 seconds
xorriso : UPDATE : 105000 files added in 26 seconds
xorriso : UPDATE : 116500 files added in 28 seconds
xorriso : FAILURE : Cannot open as source directory:
'/media/user/sid/lost+found'
xorriso : UPDATE : 127528 files added in 29 seconds
xorriso : aborting : -abort_on 'FAILURE' encountered 'FAILURE'

No file created
But sid is only the mountable partition of the disk where the system is.
 There is the swap partition (empty) and I assume some hidden stuff at
the beggining of the disk, not?

> Have a nice day :)
> Thomas

You have one too!  ;)
kAt

-- 
 "The most violent element in society is ignorance" rEG

[toc] | [prev] | [next] | [standalone]

#178739

On Sun 12 Mar 2017 at 23:50:00 (+0000), GiaThnYgeia wrote:
> Thomas Schmitt:
> > This is not an answer to my question.
> > Is the reported address a single line
> >   /media/user/DebonUSB/usb-Kingston_DataTraveler_3.0_08606E69C773BFC06965007B-0:0-part1
> > or is it reported as two lines:
> >   /media/user/DebonUSB
> >   /usb-Kingston_DataTraveler_3.0_08606E69C773BFC06965007B-0:0-part1
> > ?
> 
> No it is all connected line and I don't think I can change this.  But so
> much for the serial number theory as two different disks have the same
> exact address.  That gets really confusing if you don't change the
> labels on them.

It's always moot what manufacturers actually put in the Serial Number
field on cheap devices like sticks and SD cards. The only instance
where I have several sticks from one source (a Petroleum company at
AAPG) has the ID_SERIAL¹ set to SMI_USB_DISK-0:0 which has no chance
of being unique.

For Kingston's, I was just guessing², based on a sample of two:

S:disk/by-id/usb-Kingston_DataTraveler_2.0_2731524021F1FE28-0:0-part1

E:ID_BUS=usb
E:ID_MODEL=DataTraveler_2.0
E:ID_SERIAL=Kingston_DataTraveler_2.0_2731524021F1FE28-0:0
E:ID_SERIAL_SHORT=2731524021F1FE28
E:ID_VENDOR=Kingston

S:disk/by-id/usb-Kingston_DT_Elite_HS_2.0_06E16840C3009958-0:0-part1

E:ID_BUS=usb
E:ID_MODEL=DT_Elite_HS_2.0
E:ID_SERIAL=Kingston_DT_Elite_HS_2.0_06E16840C3009958-0:0
E:ID_SERIAL_SHORT=06E16840C3009958
E:ID_VENDOR=Kingston

> the question I have is if this is ....part1 where is the other part/s?

"part" does not mean piece/fragment, but is short for partition.
So if you have a single partition (typical for a stick when purchased),
there will only be a part1. Nothing's missing.

¹ from the contents of /run/udev/data/b8:<1+2^N> but also found in
/sys/bus/usb/devices/<foo>/{manufacturer,product,serial,version}.
With hard drives, the Serial Number printed on the carton is as likely
to turn up in ID_SCSI_COMPAT as ID_SERIAL/ID_SERIAL_SHORT.

² Never having depended on these fields, I hadn't given them any
thought. If this is what automounters insist on doing, I shall
keep on mounting them myself by LABEL (ext/FAT) or UUID (FAT).
Thanks for the cautionary tale.

Cheers,
David.

[toc] | [prev] | [next] | [standalone]

#178745

Hi,

i quoted man bzip2:
> >  As  with  compression, supplying no filenames causes decompression from
> >  standard input to standard output."

GiaThnYgeia wrote:
> ...aka screen dump?

If the standard output of bzip2 is not connected to the standard input
of another process or redirected to a file, then i'd call that a
terminal flood. Ugly text salad which may even change display settings of
your terminal window.

Standard input, standard output, and standard error output are three i/o
channels which every process on a Unix-like system has. Connecting them
to other i/o channels or redirecting them to files is a fundamental
gesture of shell programming. (Yes, the shell is a programming language,
although we often only execute single command lines.)


> > In what state is "imagefile" now ? Compressed ? Uncompressed ? Ashes ?

> Now it is at the state of being all thrown to trashcan [...]
> a new project will begin sometime next week ....

Maybe you are giving up too fast.


> I think it was that dreadful Calamares installer that came with this sid
> distro that locks onto the disk and prevents ovewriting.

I deem this unlikely. Your Linux was up and had control. Our problem
is not about overwriting but about finding a directory path under which
we can read the files of the USB stick partition.

> But the usb was so hard locked that gparted would erase its partition,

So you interspersed some other experiments ? That might be entertaining
but also confusing.


> > is it reported as two lines:

> No it is all connected line [...]

> > The reason why i ask is that i wonder from where xorriso has this
> > strange two-line path.  It would be explainable if you had given it to
> > xorriso command "-map".

> So are you saying the problem may lie in the name length that xorriso
> can't handle?

No. The name length should not be an obstacle until it reaches limits
of the X/Open system specification (255 characters per name, 1024 per path).

The problem is rather that xorriso gets to see a file path which does
not lead to an existing file. Either this name stems from the program
arguments of the run (i.e. is given after -map) or it stems from
following a symbolic link, which tells a wrong target path.
In the first case, the operator i(i.e. you) is to blame. In the second
case, some automat installed confusing links.


> the question I have is if this is ....part1 where is the other part/s?

"part" shall mean "partition". I just wanted to keep the name short.


> > New approach to get to a mount point of the stick:

> xorriso : UPDATE : 116500 files added in 28 seconds
> xorriso : FAILURE : Cannot open as source directory: '/media/user/sid/lost+found'
> ...
> xorriso : aborting : -abort_on 'FAILURE' encountered 'FAILURE'

We got some progress now.
The new problem is the fact that xorriso (and possibly other user
processes, too) cannot read the content of
  /media/user/sid/lost+found

So why does this file make trouble ? IIRC, it is a directory which
holds files that were found orphaned during filesystem checks.
Please report the output of

  ls -ld /media/user/sid/lost+found

(I will have to investigate why xorriso tells no further reason for
 being unable to read the directory's file list.)

------------------------------------------------------------------

Whatever, this is a local filesystem problem which we may try to
circumvent by omitting the offending file object.

   xorriso \
   -for_backup \
   -outdev usb_part1.iso \
   -not_paths /media/user/sid/lost+found -- \
   -map /media/user/sid /

The xorriso command -not_paths takes one or more file paths which then
get excluded from the backup. The word "--" marks the end of this path list,
so that the next word "-map" is then interpreted as next xorriso command.

Let's see how far we will get with this try.

If more unreadable "lost+found" directories show up, you may ban the
name from being backed up when found under any path:

   xorriso \
   -for_backup \
   -outdev usb_part1.iso \
   -not_leaf lost+found \
   -map /media/user/sid /

Other than -not_paths, -not_leaf takes exactly one parameter. So no end
makr "--" is necessary.


Have a nice day :)

Thomas

[toc] | [prev] | [next] | [standalone]

#178751

I'll have to learn how to do this trick to (read the fine code of your
email that is that scraps the rest)

Thomas Schmitt:
>  ls -ld /media/user/sid/lost+found

I ommitted some of the usual stuff .... with drwxr-xr-x (the C; is a
joke of course for user@machinename)

What is that + at .. root-directory?  Mounting point?

C;/media/user/sid$  ls -ld /media/user/sid/lost+found
drwx------ 2 root root 16384 Mar 10 03:21 /media/user/sid/lost+found
C;/media/user/sid$ ls -ld /media/user/sid
drwxr-xr-x 24 root root 4096 Mar 10 03:05 /media/user/sid
C;/media/user/sid$ ls -alt -ld /media/user/sid
drwxr-xr-x 24 root root 4096 Mar 10 03:05 /media/user/sid
C;/media/user/sid$ ls -alt /media/user/sid
total 124
drwxr-x---+   5 root root  4096 Mar 13 13:52 ..
drwxr-xr-x  122 root root 12288 Mar 13 04:27 etc
drwx------    9 root root  4096 Mar 12 15:51 root
drwxr-xr-x    3 root root  4096 Mar 11 18:03 boot
drwxr-xr-x    2 root root 12288 Mar 11 15:04 sbin
.......
drwxr-xr-x    3 root root  4096 Mar 11 14:51 media
drwxr-xr-x    2 root root  4096 Mar 10 13:55 bin
drwxrwxrwt    2 root root  4096 Mar 10 04:43 tmp
drwxr-xr-x    3 root root  4096 Mar 10 04:24 home
drwxr-xr-x    2 root root  4096 Mar 10 03:22 mnt
.......
drwxr-xr-x    2 root root  4096 Mar 10 03:22 proc
drwx------    2 root root 16384 Mar 10 03:21 lost+found
drwxr-xr-x   24 root root  4096 Mar 10 03:05 .
-rw-r--r--    1 root root     0 Mar 10 03:05 .xorgconfig-was-here
.....
drwxr-xr-x    2 root root  4096 Mar  5 20:24 lib64
......
drwxr-xr-x   11 root root  4096 Mar  5 20:24 var
C;/media/user/sid$ ls -ld /media/user/sid/lost+found
drwx------ 2 root root 16384 Mar 10 03:21 /media/user/sid/lost+found
C;/media/user/sid$  ls -ld /media/user/sid/lost+found
C;/media/user/sid$  ls -ld /media/user/sid/
drwxr-xr-x 24 root root 4096 Mar 10 03:05 /media/user/sid/
C;/media/user/sid$  ls -ld /media/user/sid/root
drwx------ 9 root root 4096 Mar 12 15:51 /media/user/sid/root
C;/media/user/sid$  ls -ld /media/user/sid/.xorg*
-rw-r--r-- 1 root root 0 Mar 10 03:05 /media/user/sid/.xorgconfig-was-here

-- 
 "The most violent element in society is ignorance" rEG

[toc] | [prev] | [next] | [standalone]

#178754

On Mon, Mar 13, 2017 at 12:15:00PM +0000, GiaThnYgeia wrote:
> What is that + at .. root-directory?  Mounting point?

> C;/media/user/sid$ ls -alt /media/user/sid
> total 124
> drwxr-x---+   5 root root  4096 Mar 13 13:52 ..

It indicates the presence of an ACL (file access control list), as
documented in "info coreutils ls":

     Following the file mode bits is a single character that specifies
     whether an alternate access method such as an access control list
     applies to the file.  When the character following the file mode
     bits is a space, there is no alternate access method.  When it is a
     printing character, then there is such a method.

     GNU `ls' uses a `.' character to indicate a file with a security
     context, but no other alternate access method.

     A file with any other combination of alternate access methods is
     marked with a `+' character.

See "man getfacl" and "man 5 acl" for more information on ACLs.

[toc] | [prev] | [next] | [standalone]

#178758

Hi,

GiaThnYgeia wrote:
> drwx------ 2 root root 16384 Mar 10 03:21 /media/user/sid/lost+found

If you were not superuser or ran xorriso under sudo, then the ownership
and permissions are a valid reason for being unable to read its content.

I do not generally advise to make backups as superuser. But since the
mounted USB stick probably contains files of many users with various
permission sets i now propose to become superuser or run xorriso under
control of program sudo:

   sudo xorriso \
   -for_backup \
   -outdev usb_part1.iso \
   -map /media/user/sid /


> drwxr-x---+   5 root root  4096 Mar 13 13:52 ..

Greg Wooledge wrote:
> It indicates the presence of an ACL (file access control list),

Well, it's the boss directory which we do not want to backup.
Nevertheless, Debian's build of xorriso run with settimg -for_backup
will record ACLs and Extended Attributes. Linux mount will not show them,
but xorriso will be able to restore them.


Have a nice day :)

Thomas

[toc] | [prev] | [next] | [standalone]

#178756

I changed the rights 0755 to this lost+ and it got stuck to an other
folder and contents that had only root/owner  privileges
So I decided to run the whole script as sudo or sudo xorriso and it
seems the problem is solved.

Should I attempt to rebuild it to a test disk to see if it reliable?

Thomas Schmitt:
> Whatever, this is a local filesystem problem which we may try to
> circumvent by omitting the offending file object.
> 
>    xorriso \
>    -for_backup \
>    -outdev usb_part1.iso \
>    -not_paths /media/user/sid/lost+found -- \
>    -map /media/user/sid /
> 

xorriso : UPDATE : Writing:    1925120s   99.8%   fifo 100%  buf  50%
24.2xD
ISO image produced: 1927897 sectors
Written to medium : 1928064 sectors at LBA 32
Writing to 'usb_part2.iso' completed successfully.

ISO image produced: 1927897 sectors
Written to medium : 1928064 sectors at LBA 32
Writing to 'usb_part1.iso' completed successfully.


-- 
 "The most violent element in society is ignorance" rEG

[toc] | [prev] | [next] | [standalone]

#178759

Hi,

fresh mail from GiaThnYgeia:
> So I decided to run the whole script as sudo or sudo xorriso and it
> seems the problem is solved.

Good to know. You are now supposed to have a copy of the files and
directories of the USB stick.


> Should I attempt to rebuild it to a test disk to see if it reliable?

The disk still needs partitioning and possibly boot equipment.
In case of real backup restoration you will need to rebuild these data
by tools like parted and grub.

That's why a (compressed) dd copy of the not mounted USB stick is the
quickest way to get back the whole disk layout.

The ISO provides easy access to the single files. A test for reliablility
would rather be in comparing both file trees:

  sudo mkdir /media/user/iso
  sudo mount -o loop usb_part1.iso /media/user/iso
  sudo diff -q -r /media/user/sid /media/user/iso | less

Program "diff" will list files with differing content.
So no output is a good sign then.

When done, unmount the ISO:

  sudo umount /media/user/iso


You may of course restore the ISO content to some directory in some
filesystem. I will discuss your options in respect to this in another
mail. I have some things to do right now.


Have a nice day :)

Thomas

[toc] | [prev] | [next] | [standalone]

#178766

Hi,

the restore scenario for the xorriso backup would be like this:


- Prepare the storage device to which you want to restore.
  This may be as simple as choosing some directory in a filesystem with
  enough free space, or as complicated as setting up a new operating
  system on a freshly purchased hard disk.


- If the backup has some history of copying or transmission (e.g. by
  being burned to a DVD), then first let xorriso check whether it is
  still undamaged:

    iso=/dev/sr0

    xorriso -for_backup -indev "$iso" -check_media --

  will check the MD5 of superblock and directory tree and then read the
  whole ISO sequentially to look for the MD5 checksum tags. Those MD5s
  were stored by xorriso during the write run with setting -for_backup.

  Example of how a goot verification should look like (with lower speed
  if read from a real DVD):

    xorriso : UPDATE : Found matching MD5 superblock tag: start=32 size=18
    xorriso : UPDATE : Found matching MD5 tree tag: start=32 size=302
    xorriso : UPDATE : Found matching MD5 session tag: start=32 size=81217
    xorriso : UPDATE : 81250 blocks read in 1 seconds = 120.1xD
    Media checks :        lba ,       size , quality
    Media region :          0 ,      81250 , + good
    Media region :      81250 ,        158 , 0 untested
    MD5 checks   :        lba ,       size , result
    MD5 tag range:         32 ,      81217 , + md5_match

  "Media region" with quality "+ good" means that there were no i/o errors.
  "0 untested" means that these blocks are not claimed by the ISO filesystem.

  "MD5 tag range" with result "+ md5_match" means that the MD5 checksum
  of the inspected region matches the recorded MD5.


- If you just want to pick some files by help of your favorite file
  copier tool, then mount the medium

    iso=/dev/sr0
    sudo mount "$iso" /media/user/iso

  or if the ISO is in a data file rather than on a DVD

    iso=usb_part1.iso
    sudo mount -o loop "$iso" /media/user/iso

  Now you may copy files from /media/user/iso to the place where you want
  them to be. My favorite would be cp with option -a, possibly under sudo
  control for the power to assign file ownerships.
  (Caution: This can of course shoot your foot if you do not take care
            when composing the cp command.)


- If you want to copy the whole ISO content from DVD to a directory tree
  on hard disk or if ACLs and Extended Attributes matter, it is advisable
  to let xorriso copy the files out of the ISO:

    target=/my/prepared/restore/directory

    sudo xorriso \
     -osirrox on:sort_lba_on:auto_chmod_on \
     -for_backup \
     -indev "$iso" \
     -extract / "$target"

  Only the superuser or sudo are permitted to assign file ownership to
  other users. Omit "sudo" if all restored files shall belong the user
  who operates xorriso.

  The -osirrox setting "on" enables command -extract. Setting "sort_lba"
  lets xorriso read the files from DVD in the order of their content
  block addresses. This avoids slow and loud laser head movements.
  Because this reading order may cause revisiting of directories which
  were already restored without wx-permission, the setting "auto_chmod_on"
  permits xorriso to temprorarily grant its user those permissions.


Have a nice day :)

Thomas

[toc] | [prev] | [next] | [standalone]

#178772

Very good information but the word sudo comes up everywhere.
If a user does not have sudo rights she/he can back-up files and restore
them as long as s/he has rights to what their backing-up/restoring.  So
if you are in a network public environment you may not even have rights
to even your own disk if some of it is somehow protected.  And in some
places to protect the system and network mounting media is not allowed,
bios may be locked and prevent the machine from booting anything other
than its networked drive.


Thomas Schmitt:
> Hi,
> 
> the restore scenario for the xorriso backup would be like this:
> 
> 
> - Prepare the storage device to which you want to restore.
>   This may be as simple as choosing some directory in a filesystem with
>   enough free space, or as complicated as setting up a new operating
>   system on a freshly purchased hard disk.
> 
> 
> - If the backup has some history of copying or transmission (e.g. by
>   being burned to a DVD), then first let xorriso check whether it is
>   still undamaged:
> 
>     iso=/dev/sr0
> 
>     xorriso -for_backup -indev "$iso" -check_media --
> 
>   will check the MD5 of superblock and directory tree and then read the
>   whole ISO sequentially to look for the MD5 checksum tags. Those MD5s
>   were stored by xorriso during the write run with setting -for_backup.
> 
>   Example of how a goot verification should look like (with lower speed
>   if read from a real DVD):
> 
>     xorriso : UPDATE : Found matching MD5 superblock tag: start=32 size=18
>     xorriso : UPDATE : Found matching MD5 tree tag: start=32 size=302
>     xorriso : UPDATE : Found matching MD5 session tag: start=32 size=81217
>     xorriso : UPDATE : 81250 blocks read in 1 seconds = 120.1xD
>     Media checks :        lba ,       size , quality
>     Media region :          0 ,      81250 , + good
>     Media region :      81250 ,        158 , 0 untested
>     MD5 checks   :        lba ,       size , result
>     MD5 tag range:         32 ,      81217 , + md5_match
> 
>   "Media region" with quality "+ good" means that there were no i/o errors.
>   "0 untested" means that these blocks are not claimed by the ISO filesystem.
> 
>   "MD5 tag range" with result "+ md5_match" means that the MD5 checksum
>   of the inspected region matches the recorded MD5.
> 
> 
> - If you just want to pick some files by help of your favorite file
>   copier tool, then mount the medium
> 
>     iso=/dev/sr0
>     sudo mount "$iso" /media/user/iso
> 
>   or if the ISO is in a data file rather than on a DVD
> 
>     iso=usb_part1.iso
>     sudo mount -o loop "$iso" /media/user/iso
> 
>   Now you may copy files from /media/user/iso to the place where you want
>   them to be. My favorite would be cp with option -a, possibly under sudo
>   control for the power to assign file ownerships.
>   (Caution: This can of course shoot your foot if you do not take care
>             when composing the cp command.)
> 
> 
> - If you want to copy the whole ISO content from DVD to a directory tree
>   on hard disk or if ACLs and Extended Attributes matter, it is advisable
>   to let xorriso copy the files out of the ISO:
> 
>     target=/my/prepared/restore/directory
> 
>     sudo xorriso \
>      -osirrox on:sort_lba_on:auto_chmod_on \
>      -for_backup \
>      -indev "$iso" \
>      -extract / "$target"
> 
>   Only the superuser or sudo are permitted to assign file ownership to
>   other users. Omit "sudo" if all restored files shall belong the user
>   who operates xorriso.
> 
>   The -osirrox setting "on" enables command -extract. Setting "sort_lba"
>   lets xorriso read the files from DVD in the order of their content
>   block addresses. This avoids slow and loud laser head movements.
>   Because this reading order may cause revisiting of directories which
>   were already restored without wx-permission, the setting "auto_chmod_on"
>   permits xorriso to temprorarily grant its user those permissions.
> 
> 
> Have a nice day :)
> 
> Thomas
> 

-- 
 "The most violent element in society is ignorance" rEG

[toc] | [prev] | [next] | [standalone]

#178784

Hi,

GiaThnYgeia wrote:
> Very good information but the word sudo comes up everywhere.

For the purpose of backing up and restoring a whole operating system
with multiple users and partly restrictive permissions: yes.


> If a user does not have sudo rights she/he can back-up files and restore
> them as long as s/he has rights to what their backing-up/restoring.

That's the idea ... until Why-oh-why-oh-why strikes again, of course.


> So
> if you are in a network public environment you may not even have rights
> to even your own disk if some of it is somehow protected.

If you can read directories and files, then you can backup them.
If you can create new directories and write new files, then you can
restore the backup.
If you cannot, then it's hopeless.


Have a nice day :)

Thomas

[toc] | [prev] | [next] | [standalone]

#178732 — Disabling automount, and mounting/ unmounting the "old way"

On 03/11/2017 11:59 PM, Thomas Schmitt wrote:
> I am very happy that i disabled automounting on my system. Life becomes
> so clear and straightforward if one does it the old way.

I use Jesse and Xfce:

2017-03-12 12:42:20 dpchrist@jesse ~
$ cat /etc/debian_version; uname -a
8.7
Linux jesse 3.16.0-4-amd64 #1 SMP Debian 3.16.39-1+deb8u2 (2017-03-07) 
x86_64 GNU/Linux
2017-03-12 12:47:05 dpchrist@jesse ~

$ dpkg-query --list xfce4
Desired=Unknown/Install/Remove/Purge/Hold
| 
Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name           Version      Architecture Description
+++-==============-============-============-=================================
ii  xfce4          4.10.1       all          Meta-package for the Xfce 
Lightwe


I believe Xfce came with automounting disabled OOTB.  I can adjust it 
via Application Menu -> File Manager -> (Thunar) Edit -> Preferences -> 
Advanced -> Volume Management -> Configure -> (Removable Drives and 
Media) Storage -> Removable Storage -> Mount removable drives when 
hot-plugged and Mount removable media when inserted.


When I insert a USB drive or optical disk, an icon appears on the 
desktop and in the side (left) pane of Thunar.  Right-clicking either 
provides a pop-up menu with a "mount" option.  Once mounted, 
right-clicking either provides a pop-up menu with "unmount" and/or 
"eject" options.


I typically use dmesg(1) with USB drives to determine which device file 
(/dev/*) to use with mount(8).  But for optical discs, dmesg(1) doesn't 
provide any clues.


lsblk(8) lists connected USB drives, unmounted or mounted, and lists the 
optical drive (sr0), empty, unmounted, or mounted.  The SIZE, TYPE, 
and/or MOUNTPOINT fields appear to when a file system is mounted.  When 
a USB device is removed, lsblk(8) no longer lists it.  But when an 
optical disc is ejected, lsblk(8) continues to display the unmounted 
information.


blkid(8) against a device file seems to provide information that does 
not change with mounting or unmounting.


Could you please describe how you mount/unmount devices and file systems 
"the old way"?


David

[toc] | [prev] | [next] | [standalone]

#178733 — Re: Disabling automount, and mounting/ unmounting the "old way"

Hi,

David Christensen wrote:
> Could you please describe how you mount/unmount devices and file systems
> "the old way"?

I become superuser and execute commands like
  mount /dev/sr0 /mnt/iso
  mount -o loop /mnt/iso/boot/grub/efi.img /mnt/fat
  mount /dev/sdc2 /mnt/fat2
  umount /mnt/fat
  umount /mnt/iso
  umount /mnt/fat2

If i forgot what i mounted where i run
  mount
(of which the output grows why-oh-why-oh-why always bigger while
 Linux progress moves on.)


Have a nice day :)

Thomas

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.user

csiph-web