Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.user > #157672 > unrolled thread

Extracting directories from an ISO image, command line tool?

Back to article view | Back to linux.debian.user

Contents

  Extracting directories from an ISO image, command line tool? Richard Owlett <rowlett@cloud85.net> - 2015-09-10 15:10 +0200
    Re: Extracting directories from an ISO image, command line tool? Brian <ad44@cityscape.co.uk> - 2015-09-10 15:20 +0200
      Re: Extracting directories from an ISO image, command line tool? Richard Owlett <rowlett@cloud85.net> - 2015-09-10 16:10 +0200
        Re: Extracting directories from an ISO image, command line tool? The Wanderer <wanderer@fastmail.fm> - 2015-09-10 16:30 +0200
          Re: Extracting directories from an ISO image, command line tool? The Wanderer <wanderer@fastmail.fm> - 2015-09-10 16:30 +0200
        Re: Extracting directories from an ISO image, command line tool? Brian <ad44@cityscape.co.uk> - 2015-09-10 17:00 +0200
          Re: Extracting directories from an ISO image, command line tool? Richard Owlett <rowlett@cloud85.net> - 2015-09-10 17:20 +0200
            Re: Extracting directories from an ISO image, command line tool? Brian <ad44@cityscape.co.uk> - 2015-09-10 18:20 +0200
              Re: Extracting directories from an ISO image, command line tool? Hans <hans.ullrich@loop.de> - 2015-09-10 18:30 +0200
                Re: Extracting directories from an ISO image, command line tool? Brian <ad44@cityscape.co.uk> - 2015-09-10 20:50 +0200
                  Re: Extracting directories from an ISO image, command line tool? Hans <hans.ullrich@loop.de> - 2015-09-10 21:10 +0200
                    Re: Extracting directories from an ISO image, command line tool? Brian <ad44@cityscape.co.uk> - 2015-09-10 21:30 +0200
                  Re: Extracting directories from an ISO image, command line tool? Curt <curty@free.fr> - 2015-09-10 22:10 +0200
                    Re: Extracting directories from an ISO image, command line tool? Brian <ad44@cityscape.co.uk> - 2015-09-10 23:00 +0200
                    Re: Extracting directories from an ISO image, command line tool? Lisi Reisz <lisi.reisz@gmail.com> - 2015-09-11 15:40 +0200
                    Re: Extracting directories from an ISO image, command line tool? Chris Bannister <cbannister@slingshot.co.nz> - 2015-09-11 16:50 +0200
            Re: Extracting directories from an ISO image, command line tool? Lisi Reisz <lisi.reisz@gmail.com> - 2015-09-11 15:40 +0200
    Re: Extracting directories from an ISO image, command line tool? The Wanderer <wanderer@fastmail.fm> - 2015-09-10 15:30 +0200
      Re: Extracting directories from an ISO image, command line tool? Richard Owlett <rowlett@cloud85.net> - 2015-09-10 16:20 +0200
        Re: Extracting directories from an ISO image, command line tool? The Wanderer <wanderer@fastmail.fm> - 2015-09-10 16:30 +0200
          Re: Extracting directories from an ISO image, command line tool? Richard Owlett <rowlett@cloud85.net> - 2015-09-10 17:40 +0200
            Re: Extracting directories from an ISO image, command line tool? The Wanderer <wanderer@fastmail.fm> - 2015-09-10 17:50 +0200
        Re: Extracting directories from an ISO image, command line tool? Eduard Bloch <edi@gmx.de> - 2015-09-10 16:30 +0200
    Re: Extracting directories from an ISO image, command line tool? Darac Marjal <mailinglist@darac.org.uk> - 2015-09-10 15:30 +0200
      Re: Extracting directories from an ISO image, command line tool? The Wanderer <wanderer@fastmail.fm> - 2015-09-10 16:30 +0200
      Re: Extracting directories from an ISO image, command line tool? Richard Owlett <rowlett@cloud85.net> - 2015-09-10 16:30 +0200
    Re: Extracting directories from an ISO image, command line tool? "Thomas Schmitt" <scdbackup@gmx.net> - 2015-09-10 16:40 +0200
      Re: Extracting directories from an ISO image, command line tool? Richard Owlett <rowlett@cloud85.net> - 2015-09-10 17:30 +0200
      Re: Extracting directories from an ISO image, command line tool? David Wright <deblis@lionunicorn.co.uk> - 2015-09-10 22:50 +0200
        Re: Extracting directories from an ISO image, command line tool? "Thomas Schmitt" <scdbackup@gmx.net> - 2015-09-11 10:30 +0200
          Re: Extracting directories from an ISO image, command line tool? David Wright <deblis@lionunicorn.co.uk> - 2015-09-11 18:50 +0200
            Re: Extracting directories from an ISO image, command line tool? Curt <curty@free.fr> - 2015-09-12 10:10 +0200
            Re: Extracting directories from an ISO image, command line tool? "Thomas Schmitt" <scdbackup@gmx.net> - 2015-09-12 11:00 +0200
      Re: Extracting directories from an ISO image, command line tool? Richard Owlett <rowlett@cloud85.net> - 2015-09-13 23:20 +0200
        Re: Extracting directories from an ISO image, command line tool? "Thomas Schmitt" <scdbackup@gmx.net> - 2015-09-14 00:40 +0200
          Re: Extracting directories from an ISO image, command line tool? Richard Owlett <rowlett@cloud85.net> - 2015-09-14 03:00 +0200
            Re: Extracting directories from an ISO image, command line tool? "Thomas Schmitt" <scdbackup@gmx.net> - 2015-09-14 10:20 +0200
              Re: Extracting directories from an ISO image, command line tool? Richard Owlett <rowlett@cloud85.net> - 2015-09-14 16:40 +0200
                Narrowing down problem source - was [Re: Extracting directories from  an ISO image, command line tool?] Richard Owlett <rowlett@cloud85.net> - 2015-09-17 17:00 +0200
                  Re: Narrowing down problem source - was [Re: Extracting directories from  an ISO image, command line tool?] Steve McIntyre <steve@einval.com> - 2015-09-17 18:10 +0200
                    Re: Narrowing down problem source - was [Re: Extracting directories  from an ISO image, command line tool?] Richard Owlett <rowlett@cloud85.net> - 2015-09-18 13:20 +0200
                  Re: Narrowing down problem source "Thomas Schmitt" <scdbackup@gmx.net> - 2015-09-17 19:20 +0200
                    Re: Narrowing down problem source Richard Owlett <rowlett@cloud85.net> - 2015-09-18 14:40 +0200
                      Re: Narrowing down problem source "Thomas Schmitt" <scdbackup@gmx.net> - 2015-09-18 15:00 +0200
                        Re: Narrowing down problem source Richard Owlett <rowlett@cloud85.net> - 2015-09-18 15:40 +0200
                          Re: Narrowing down problem source "Thomas Schmitt" <scdbackup@gmx.net> - 2015-09-18 15:50 +0200
                            Re: Narrowing down problem source Richard Owlett <rowlett@cloud85.net> - 2015-09-20 22:30 +0200
                              Re: Narrowing down problem source "Thomas Schmitt" <scdbackup@gmx.net> - 2015-09-21 08:20 +0200
                                Re: Narrowing down problem source Richard Owlett <rowlett@cloud85.net> - 2015-09-21 19:00 +0200
                                  Re: Narrowing down problem source "Thomas Schmitt" <scdbackup@gmx.net> - 2015-09-21 20:30 +0200
    Re: Extracting directories from an ISO image, command line tool? David Wright <deblis@lionunicorn.co.uk> - 2015-09-10 17:10 +0200
      Re: Extracting directories from an ISO image, command line tool? Brian <ad44@cityscape.co.uk> - 2015-09-10 19:40 +0200
    Re: Extracting directories from an ISO image, command line tool? Brian <ad44@cityscape.co.uk> - 2015-09-11 14:10 +0200
      Re: Extracting directories from an ISO image, command line tool? Richard Owlett <rowlett@cloud85.net> - 2015-09-11 17:40 +0200
        Re: Extracting directories from an ISO image, command line tool? Brian <ad44@cityscape.co.uk> - 2015-09-11 18:50 +0200
    Re: Extracting directories from an ISO image, command line tool? Stuart Longland <stuartl@longlandclan.id.au> - 2015-09-17 22:50 +0200
      Re: Extracting directories from an ISO image, command line tool? Richard Owlett <rowlett@cloud85.net> - 2015-09-18 14:00 +0200
        Re: Extracting directories from an ISO image, command line tool? "Thomas Schmitt" <scdbackup@gmx.net> - 2015-09-18 14:30 +0200
          Re: Extracting directories from an ISO image, command line tool? Richard Owlett <rowlett@cloud85.net> - 2015-09-18 15:50 +0200
        Re: Extracting directories from an ISO image, command line tool? Stuart Longland <stuartl@longlandclan.id.au> - 2015-09-23 04:30 +0200
          Re: Extracting directories from an ISO image, command line tool? Richard Owlett <rowlett@cloud85.net> - 2015-09-23 16:00 +0200
            Re: Extracting directories from an ISO image, command line tool? "Thomas Schmitt" <scdbackup@gmx.net> - 2015-09-23 20:10 +0200

Page 3 of 4 — ← Prev page 1 2 [3] 4  Next page →

#158045 — Re: Narrowing down problem source - was [Re: Extracting directories from an ISO image, command line tool?]

Steve McIntyre wrote:
> Richard Owlett wrote:
>>
>> I started over again. I switched to a laptop dedicated to things
>> that may not work as expected. For some of my projects I
>> regularly wipe the disk and do a complete reinstall of Debian.
>> Currently I'm running Debian 8.0 with Mate as DE. No updates.
>>
>> I used
>>    blocks=$(expr $(/sbin/isosize /dev/sr0) / 2048)
>>    echo "Byte count: $(expr $blocks '*' 2048)"
>>    dd if=/dev/sr0 bs=2048 count=$blocks of=dvd8_2.iso
>> as a pattern for each DVD.
>>
>> _Some_ passed your test of
>>   ls -l /media/richard/Lexar/dvd8_2.iso
>>   /sbin/isosize /media/richard/Lexar/dvd8_2.iso
>>
>> Example:
>>   ls -l dvd8_9.iso
>>   /sbin/isosize dvd8_9.iso
>>
>> resulted in
>> richard@debian:~$  ls -l dvd8_9.iso
>> -rw-r--r-- 1 richard richard 3976200192 Sep 16 11:50 dvd8_9.iso
>> richard@debian:~$
>> richard@debian:~$  /sbin/isosize dvd8_9.iso
>> 4460664832
>
> I'll tell you what's most likely wrong - you're storing your extracted
> ISO images on a FAT32 filesystem which doesn't support file sizes
> larger than 4GiB. That's why DVD #1 works, it's explicitly reduced in
> size to fit on a 4GB USB stick too.
>

That was true at the very beginning.
Latest runs were under the home directory of my Debian 8 install.

[toc] | [prev] | [next] | [standalone]

#158028 — Re: Narrowing down problem source

Hi,

Richard Owlett wrote:
> I did a rerun. The only apparent change in test conditions was I used a
> different instance of Mate Terminal.
> ...
> 4460664832 bytes (4.5 GB) copied, 798.488 s, 5.6 MB/s
> ...
> richard@debian:~$ /sbin/isosize rerundvd9.iso
> 4460664832

Did the dd runs with short "ls -l" result tell the "isosize"
result, nevertheless ?


> I've a gut feeling that Mate Terminal isn't redoing
> blocks=$(expr $(/sbin/isosize /dev/sr0) / 2048)
> every time.

Command interpretation is supposed to depend on the
shell interpreter (e.g. bash, dash, ksh) and not on the
terminal emulator (e.g. xterm, gnome-terminal, Konsole).

On the first hand we have to concentrate on the messages
from the dd runs which deliver too few bytes in the .iso
files.
After a dd run turned out to copy not enough bytes, look
into the system log whether any error message about "sr0"
has been freshly recorded.


Steve McIntyre:
> I'll tell you what's most likely wrong - you're storing your extracted
> ISO images on a FAT32 filesystem which doesn't support file sizes
> larger than 4GiB. 

But why did the retry succeed ?

Richard: Did you copy to a different directory ?


> Nagging memories of a similar problem with either Win95 or Win98.

We fiercely deny any similarity with said systems.
No bugs shared.


Have a nice day :)

Thomas

[toc] | [prev] | [next] | [standalone]

#158050 — Re: Narrowing down problem source

Thomas Schmitt wrote:
> Hi,
>
> Richard Owlett wrote:
>> I did a rerun. The only apparent change in test conditions was I used a
>> different instance of Mate Terminal.
>> ...
>> 4460664832 bytes (4.5 GB) copied, 798.488 s, 5.6 MB/s
>> ...
>> richard@debian:~$ /sbin/isosize rerundvd9.iso
>> 4460664832
>
> Did the dd runs with short "ls -l" result tell the "isosize"
> result, nevertheless ?

I think I did.
I'll revise my procedure to use the script command to keep better 
records.

>
>
>> I've a gut feeling that Mate Terminal isn't redoing
>> blocks=$(expr $(/sbin/isosize /dev/sr0) / 2048)
>> every time.
>
> Command interpretation is supposed to depend on the
> shell interpreter (e.g. bash, dash, ksh) and not on the
> terminal emulator (e.g. xterm, gnome-terminal, Konsole).

What should have been identical runs did not have identical 
results -- part of reason I'll make extensive use of script 
command in the next round.

>
> On the first hand we have to concentrate on the messages
> from the dd runs which deliver too few bytes in the .iso
> files.
> After a dd run turned out to copy not enough bytes, look
> into the system log whether any error message about "sr0"
> has been freshly recorded.
>
>
> Steve McIntyre:
>> I'll tell you what's most likely wrong - you're storing your extracted
>> ISO images on a FAT32 filesystem which doesn't support file sizes
>> larger than 4GiB.
>
> But why did the retry succeed ?
>
> Richard: Did you copy to a different directory ?

When reusing a file name I had used filemanager to send file to 
trash and then emptied trash before reusing.

>
>
>> Nagging memories of a similar problem with either Win95 or Win98.
>
> We fiercely deny any similarity with said systems.
> No bugs shared.

Why would creative minds need to "share" ;)

>
>
> Have a nice day :)
>
> Thomas
>
>

[toc] | [prev] | [next] | [standalone]

#158051 — Re: Narrowing down problem source

Hi,

Steve McIntyre wrote:
> > > ... FAT32 filesystem ...

i wrote:
> > Richard: Did you copy to a different directory ?

Richard Owlett wrote:
> When reusing a file name I had used filemanager to send file to trash and
> then emptied trash before reusing.

The question was more about whether you had your
successful dd runs on a filesystem which can take
files larger than 4 GiB, wheras the unsuccessful ones
were on FAT.
Different filesystems would have to be mounted under
different directories.

So staying within the same directory during all tries
would make clear that failure or success does not stick
with a particular filesystem type.


> > No bugs shared.

> Why would creative minds need to "share" ;)

Our bugs look completely different than theirs.
Ours are for free - theirs are proprietary.

Total cost of ownership for an oldfashioned frugal
GNU/Linux bug is far less than for a Windows 10 glitch
which has to last for the whole lifetime of a toddler
born in 2013.


Have a nice day :)

Thomas

[toc] | [prev] | [next] | [standalone]

#158052 — Re: Narrowing down problem source

Thomas Schmitt wrote:
> Hi,
>
> Steve McIntyre wrote:
>>>> ... FAT32 filesystem ...
>
> i wrote:
>>> Richard: Did you copy to a different directory ?
>
> Richard Owlett wrote:
>> When reusing a file name I had used filemanager to send file to trash and
>> then emptied trash before reusing.
>
> The question was more about whether you had your
> successful dd runs on a filesystem which can take
> files larger than 4 GiB, wheras the unsuccessful ones
> were on FAT.
> Different filesystems would have to be mounted under
> different directories.
>
> So staying within the same directory during all tries
> would make clear that failure or success does not stick
> with a particular filesystem type.

My initial xorriso problems were FAT32 related.

My later comments of run-to-run inconsistencies with dd and your 
recommended tests were all using my home directory.

[toc] | [prev] | [next] | [standalone]

#158054 — Re: Narrowing down problem source

Hi,

> My later comments of run-to-run inconsistencies with dd and your recommended
> tests were all using my home directory.

In this case any dd run with short result file should issue
some error message and/or there should be some hardware
error message in the system logs.


Have a nice day :)

Thomas

[toc] | [prev] | [next] | [standalone]

#158099 — Re: Narrowing down problem source

Thomas Schmitt wrote:
> Hi,
>
>> My later comments of run-to-run inconsistencies with dd and your recommended
>> tests were all using my home directory.
>
> In this case any dd run with short result file should issue
> some error message and/or there should be some hardware
> error message in the system logs.
>

I captured a test run with the script command.
NOTE dd copies exactly same number of records for differently 
sized DVDs :{
Result was:

[snip]
]0;richard@debian-8: ~richard@debian-8:~$
]0;richard@debian-8: ~richard@debian-8:~$ N=0
]0;richard@debian-8: ~richard@debian-8:~$
]0;richard@debian-8: ~richard@debian-8:~$ F="JessieDVD"
]0;richard@debian-8: ~richard@debian-8:~$
]0;richard@debian-8: ~richard@debian-8:~$ E='.iso'
]0;richard@debian-8: ~richard@debian-8:~$
]0;richard@debian-8: ~richard@debian-8:~$ while true
>
>   do
>
>     let N=N+1
>
>      echo "Insert medium, wait until blinking ends, press Enter key (or Ctrl+C
  to end)"
>
>     read dummyvar
>
>     blocks=$(expr $(/sbin/isosize /dev/sr0) / 2048)
>
>     echo "Block count: "$blocks
>
>     echo "Byte count: $(expr $blocks '*' 2048)"
>
>     FILENAME=$F$N$E
>
>     echo "The output file will be $FILENAME"
>
>     echo ""
>
>     dd if=/dev/sr0 bs=2048 count=$blocks of=$FILENAME
>
>     echo ""
>
>     echo "next lines are output of ls and isosize commnds"
>
>     ls -l $FILENAME
>
>     /sbin/isosize $FILENAME
>
>     echo "about to repeat loop"
>
>     echo ""
>
>   done
Insert medium, wait until blinking ends, press Enter key (or 
Ctrl+C to end)
Block count: 1941504
Byte count: 3976200192
The output file will be JessieDVD1.iso

1941504+0 records in
1941504+0 records out
3976200192 bytes (4.0 GB) copied, 876.109 s, 4.5 MB/s

next lines are output of ls and isosize commnds
-rw-r--r-- 1 richard richard 3976200192 Sep 20 13:39 JessieDVD1.iso
3976200192
about to repeat loop

Insert medium, wait until blinking ends, press Enter key (or 
Ctrl+C to end)

Block count: 2284052
Byte count: 4677738496
The output file will be JessieDVD2.iso

1941504+0 records in
1941504+0 records out
3976200192 bytes (4.0 GB) copied, 733.727 s, 5.4 MB/s

next lines are output of ls and isosize commnds
-rw-r--r-- 1 richard richard 3976200192 Sep 20 13:52 JessieDVD2.iso
4677738496
about to repeat loop

Insert medium, wait until blinking ends, press Enter key (or 
Ctrl+C to end)
^C
]0;richard@debian-8: ~richard@debian-8:~$ exit

Script done on Sat 20 Sep 2014 01:53:32 PM CDT

[toc] | [prev] | [next] | [standalone]

#158107 — Re: Narrowing down problem source

Hi,

Richard Owlett wrote:
> Block count: 1941504
> ...
> 1941504+0 records in
> 1941504+0 records out

This is ok.

> Block count: 2284052
> ...
> 1941504+0 records in
> 1941504+0 records out

This is not ok.

The variable "block" was computed correctly, but program "dd"
did not retrieve enough blocks.
(I wonder whether it's a bug or a feature that it does not
 report an error. At least this behavior is reproducible.)

It seems that the system does not tell "dd" the block count of
the new DVD but the count of the previous one.

What do you get from this command

  echo "Device block count: $(expr $(cat /sys/class/block/sr0/size) / 4)"

inserted in the script after the line
  echo "Byte count: $(expr $blocks '*' 2048)"

(The /sys filesystem is an inexhaustible source of wonders.)


Have a nice day :)

Thomas

[toc] | [prev] | [next] | [standalone]

#158132 — Re: Narrowing down problem source

Thomas Schmitt wrote:
> Hi,
>
> Richard Owlett wrote:
>> Block count: 1941504
>> ...
>> 1941504+0 records in
>> 1941504+0 records out
>
> This is ok.
>
>> Block count: 2284052
>> ...
>> 1941504+0 records in
>> 1941504+0 records out
>
> This is not ok.
>
> The variable "block" was computed correctly, but program "dd"
> did not retrieve enough blocks.
> (I wonder whether it's a bug or a feature that it does not
>   report an error. At least this behavior is reproducible.)
>
> It seems that the system does not tell "dd" the block count of
> the new DVD but the count of the previous one.
>
> What do you get from this command
>
>    echo "Device block count: $(expr $(cat /sys/class/block/sr0/size) / 4)"
>
> inserted in the script after the line
>    echo "Byte count: $(expr $blocks '*' 2048)"
>
> (The /sys filesystem is an inexhaustible source of wonders.)
>

"Device block count" <> "blocks"
HOWEVER, resulting *.iso for DVD1 and DVD2 are the same size even 
though input sizes were different.

Output of script command below:

Script started on Mon 21 Sep 2015 08:44:20 AM CDT

N=0

F="JessieDVD"

E='.iso'

while true

   do

     let N=N+1

      echo "Insert medium, wait until blinking ends, press Enter 
key (or Ctrl+C to end)"

     read dummyvar

     blocks=$(expr $(/sbin/isosize /dev/sr0) / 2048)

     echo "Block count: "$blocks

     echo "Byte count: $(expr $blocks '*' 2048)"

DEVICEBLKCOUNT=$(expr $(cat /sys/class/block/sr0/size) / 4)

     echo "Device block count: "$DEVICEBLKCOUNT

     FILENAME=$F$N$E

     echo "The output file will be $FILENAME"

     echo ""

     dd if=/dev/sr0 bs=2048 count=$DEVICEBLKCOUNT of=$FILENAME

     echo ""

     echo "next lines are output of ls and isosize commnds"

     ls -l $FILENAME

     /sbin/isosize $FILENAME

     echo "about to repeat loop"

     echo ""

   done

]0;richard@debian-8: ~richard@debian-8:~$
]0;richard@debian-8: ~richard@debian-8:~$ N=0
]0;richard@debian-8: ~richard@debian-8:~$
]0;richard@debian-8: ~richard@debian-8:~$ F="JessieDVD"
]0;richard@debian-8: ~richard@debian-8:~$
]0;richard@debian-8: ~richard@debian-8:~$ E='.iso'
]0;richard@debian-8: ~richard@debian-8:~$
]0;richard@debian-8: ~richard@debian-8:~$ while true
>
>   do
>
>     let N=N+1
>
>      echo "Insert medium, wait until blinking ends, press Enter key (or Ctrl+C
  to end)"
>
>     read dummyvar
>
>     blocks=$(expr $(/sbin/isosize /dev/sr0) / 2048)
>
>     echo "Block count: "$blocks
>
>     echo "Byte count: $(expr $blocks '*' 2048)"
>
> DEVICEBLKCOUNT=$(expr $(cat /sys/class/block/sr0/size) / 4)
>
>     echo "Device block count: "$DEVICEBLKCOUNT
>
>     FILENAME=$F$N$E
>
>     echo "The output file will be $FILENAME"
>
>     echo ""
>
>     dd if=/dev/sr0 bs=2048 count=$DEVICEBLKCOUNT of=$FILENAME
>
>     echo ""
>
>     echo "next lines are output of ls and isosize commnds"
>
>     ls -l $FILENAME
>
>     /sbin/isosize $FILENAME
>
>     echo "about to repeat loop"
>
>     echo ""
>
>   done
Insert medium, wait until blinking ends, press Enter key (or 
Ctrl+C to end)
Block count: 1941504
Byte count: 3976200192
Device block count: 1941504
The output file will be JessieDVD1.iso

1941504+0 records in
1941504+0 records out
3976200192 bytes (4.0 GB) copied, 762.909 s, 5.2 MB/s

next lines are output of ls and isosize commnds
-rw-r--r-- 1 richard richard 3976200192 Sep 21 08:57 JessieDVD1.iso
3976200192
about to repeat loop

Insert medium, wait until blinking ends, press Enter key (or 
Ctrl+C to end)

Block count: 2284052
Byte count: 4677738496
Device block count: 2284064
The output file will be JessieDVD2.iso

1941504+0 records in
1941504+0 records out
3976200192 bytes (4.0 GB) copied, 2655.4 s, 1.5 MB/s

next lines are output of ls and isosize commnds
-rw-r--r-- 1 richard richard 3976200192 Sep 21 09:43 JessieDVD2.iso
4677738496
about to repeat loop

Insert medium, wait until blinking ends, press Enter key (or 
Ctrl+C to end)
^C
]0;richard@debian-8: ~richard@debian-8:~$ exit

Script done on Mon 21 Sep 2015 09:43:50 AM CDT

[toc] | [prev] | [next] | [standalone]

#158142 — Re: Narrowing down problem source

Hi,

Richard Owlett wrote:
> Block count: 2284052
> Device block count: 2284064

This matches. The DVD offers more than the ISO claims.
(Just rounded up to a multiple of 16 blocks, in this case.)


> 1941504+0 records in
> 1941504+0 records out

And this is wrong.
Why does dd not obey but also not report error ?

/bin/dd is from coreutils ... google ...
  http://git.savannah.gnu.org/gitweb/?p=coreutils.git;a=blob;f=src/dd.c;h=e64729477101f471a31adcb6582d03d2274499cf;hb=HEAD

Looks like read(2) gets an End-Of-File and returns 0.
(In line 1075, causing the iread() call in line 2127 to
 get return value 0, which causes end of the copy loop
 in line 2133, hopping to line 2230. From then on it is
 treated as successful.)
read(2) belongs to libc.
The actual work is supposed to happen in the kernel.
I will not dive in there. Not yet.


Well, if the normal i/o is swimming belly up, lets try
whether the drive is willing to hand out the blocks which
it announced to the kernel.
Replace the dd run by

  xorriso -indev /dev/sr0 \
          -check_media data_to="$FILENAME" --

xorriso will find out the size of the ISO on its own.
It will not use system function read(2) but ioctl(SG_IO)
and SCSI command READ10.
In the end it will report something like

  Media checks :        lba ,       size , quality
  Media region :          0 ,      13472 , + good
  Media region :      13472 ,         32 , + slow
  Media region :      13504 ,     958590 , + good
  ...

because it is more for finding bad spots than for retrieving
image files from medium.

Now i am curious how far it gets.
Maybe you see some SCSI error replies. If it does not yield
the expected number of bytes, please copy as much of the
last messages as possible and send them to me in private.


Have a nice day :)

Thomas

[toc] | [prev] | [next] | [standalone]

#157689

Quoting Richard Owlett (rowlett@cloud85.net):
> Environment:
> Using dd I have copied physical [NO INTERNET AVAILABLE ;] Debian
> DVD's to
> /media/distributionA resulting in
> /media/distributionA/DVD1.iso
> /media/distributionA/DVD2.iso
> .
> .
> /media/distributionA/DVDn.iso
> 
> Goal:
> Extract pool directory (and all of its sub-directories) resulting in
> /media/distributionA/poolA/
> /media/distributionA/poolA/contrib
> /media/distributionA/poolA/main
> 
> What commands should I be looking at?

I'm not sure what use you want to make of this archive, ie how
you want it managed, if at all. Personally, I would read
/usr/share/doc/apt-cacher-ng/apt-cacher-ng.pdf carefully to
see how to import iso images, and how it would deal with the
lack of connectivity to import the "dists" part of an archive
which is as necessary as the "pool" part for a smooth operation.
You may have your own way of dealimg with this.

apt-cacher-ng uses a browser interface with a simple user/password
like CUPS to protect write operations, but I don't know whether a text
browser is non-GUI enough to count as a command-line tool for you.

But leaving that suggestion aside, and taking your goal at face value,
I'm assuming that "distributionA" really means "dvd-set-a" and that
you want no mixing of sets even if you come by, say, 8.1 DVDs and
8.2 ones. What is the significance of "poolA"?

As root, I would probably just use the simplest tools available,
mount, cp -Rip and chmod. The -i will log duplicate filenames. If you
don't want it to wait, yes | cp -i... will log them but carry on
copying.

Were I to use mc as suggested, I would do this as a user. I'm not
willing to use such a tool as root. One of the downsides of mc
(there are lots of ups) is that it will only query a duplicate
when it occurs and when you've made a final decision (like nonE
or aLl) you get no further idea of how many files were affected.

Cheers,
David.

[toc] | [prev] | [next] | [standalone]

#157699

On Thu 10 Sep 2015 at 10:07:23 -0500, David Wright wrote:

> Were I to use mc as suggested, I would do this as a user. I'm not
> willing to use such a tool as root. One of the downsides of mc
> (there are lots of ups) is that it will only query a duplicate
> when it occurs and when you've made a final decision (like nonE
> or aLl) you get no further idea of how many files were affected.

Another downside is working out what mc does amd applying it in a
what we might suppose is a scripting situation. The effort is too
much. For a non-scriptable solution it wins hands down.

Anyway, here is Plan B. The Wanderer should like it because there
is no root involvement. Richard should be over the moon because it
is all command line processing and can be scripted if desired.

In $HOME:

  mkdir ISO

Then:

  bsdtar -C ISO -xf /media/distributionA/DVD1.iso

Two commands and nirvana looms on the horizon.

[toc] | [prev] | [next] | [standalone]

#157732

On Thu 10 Sep 2015 at 08:06:36 -0500, Richard Owlett wrote:

> Environment:
> Using dd I have copied physical [NO INTERNET AVAILABLE ;] Debian DVD's to
> /media/distributionA resulting in
> /media/distributionA/DVD1.iso
> /media/distributionA/DVD2.iso
> .
> .
> /media/distributionA/DVDn.iso

Is there a good reason why the DVDs are copied to /media? $HOME is where
they are so that would seem to be the natural place to dd them to. If
the user has no root privileges it is the only place for them. So:

 $HOME/distributionA/DVD1.iso
 $HOME/distributionA/DVD2.iso
 .
 .
 $HOME/distributionA/DVDn.iso

> Goal:
> Extract pool directory (and all of its sub-directories) resulting in
> /media/distributionA/poolA/
> /media/distributionA/poolA/contrib
> /media/distributionA/poolA/main
> 
> What commands should I be looking at?

    bsdtar -C $HOME/distributionA/ -xf $HOME/distributionA/DVD1.iso pool

    chmod -R u=rwX,go=rX $HOME/distributionA/pool

    bsdtar -C $HOME/distributionA/ -xf $HOME/distributionA/DVD2.iso
    .
    .
    bsdtar -C $HOME/distributionA/ -xf $HOME/distributionA/DVDn.iso

    chmod -R u=rX,go=rX $HOME/distributionA/pool

[toc] | [prev] | [next] | [standalone]

#157743

Brian wrote:
> On Thu 10 Sep 2015 at 08:06:36 -0500, Richard Owlett wrote:
>
>> Environment:
>> Using dd I have copied physical [NO INTERNET AVAILABLE ;] Debian DVD's to
>> /media/distributionA resulting in
>> /media/distributionA/DVD1.iso
>> /media/distributionA/DVD2.iso
>> .
>> .
>> /media/distributionA/DVDn.iso
>
> Is there a good reason why the DVDs are copied to /media?

Yes

[toc] | [prev] | [next] | [standalone]

#157747

On Fri 11 Sep 2015 at 10:37:38 -0500, Richard Owlett wrote:

> Brian wrote:
> >On Thu 10 Sep 2015 at 08:06:36 -0500, Richard Owlett wrote:
> >
> >>Environment:
> >>Using dd I have copied physical [NO INTERNET AVAILABLE ;] Debian DVD's to
> >>/media/distributionA resulting in
> >>/media/distributionA/DVD1.iso
> >>/media/distributionA/DVD2.iso
> >>.
> >>.
> >>/media/distributionA/DVDn.iso
> >
> >Is there a good reason why the DVDs are copied to /media?
> 
> Yes

I nearly forgot to thank you for your first post. It prompted me to
return to a project (possibly similar to yours), tidy its code up
and complete it. The command line approach fits better than the one
using mc.

[toc] | [prev] | [next] | [standalone]

#158033

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 10/09/15 23:06, Richard Owlett wrote:
> Environment: Using dd I have copied physical [NO INTERNET AVAILABLE
> ;] Debian DVD's to /media/distributionA resulting in 
> /media/distributionA/DVD1.iso /media/distributionA/DVD2.iso .. .. 
> /media/distributionA/DVDn.iso

Well, I can get you half-way there.

for f in /media/distributionA/*.iso; do
	bn="$( basename "${f}" .iso )"
	dn="$( dirname "${f}" )"
	mount "${f}" /mnt/cdrom -o loop
	rsync -aP /mnt/cdrom "${dn}/${bn}"/
	umount /mnt/cdrom
done

Now you'll have in addition to the above:
/media/distributionA/DVD1/ …
/media/distributionA/DVD2/ …
..
..
/media/distributionA/DVDn/ …

I'm not sure how you'd go about merging the repositories together.
Someone else can help with that.
- -- 
Stuart Longland (aka Redhatter, VK4MSL)

I haven't lost my mind...
  ...it's backed up on a tape somewhere.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQIcBAEBCAAGBQJV+yaxAAoJEE36GRQQveO38CkQALmNl6BUrN4QduBPmREfmRC4
A8GEdua8W9YZ8E4pcKx/9xG6N3pN7stck26/illFewb8MqrgMfsUZfx9UGiqtFSk
HQwtHxWvp0N60d4xlXmtmKSB0EopJ3kgPRtPpvKG33HpiZ24dYIgdoMY/dHIdjM4
LTcdqQdWjo0tJA1CAoVY604tXVYGqR+kFWaLfzM/ysxoj1KxuDRgoARSozCLrXpu
Mr2wIb3qnjnpm9e3fO9le52HurwEY/RVbYOWd0aY9DSekGKaOg7LiK4qlkP73nnO
BjZ8hnqsNmidBZN5647cTEZXCCK0T6Ygxp8CxFNmDDba9TRnDlxQg7Pu6oBseCRW
cJVW3ULmiShgiNArsnszXGPzJjpWIYewtpK025npWEc23LoBTQFLk04tyBhvstVr
8Gkcl2Rzyq8sNf6F/fLqmoaDis/abxdigaj61sHHoVPBIIntabzkP+dnAWi8Z9/a
XmAcdiZXRGiUDo6K0b+/6VKhzyNKNQg9EvFozZWxIumhCFI0FVRPITJCmre8vdyX
rlPDYEL13tyd3UzyTe6X+fhl9xlvCvH3ywkDw7q08k3xZQaHFq91p7idmLT9GI9J
FvFqpu7uFNGTmvS6BoJQLpGh63xBMZy8+aTmCXMkLE6y8RhZQwTUN5QwhAOZK3zw
aFVAGPp3f3KauHkx6abG
=dSnl
-----END PGP SIGNATURE-----

[toc] | [prev] | [next] | [standalone]

#158047

Stuart Longland wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
> On 10/09/15 23:06, Richard Owlett wrote:
>> Environment: Using dd I have copied physical [NO INTERNET AVAILABLE
>> ;] Debian DVD's to /media/distributionA resulting in
>> /media/distributionA/DVD1.iso /media/distributionA/DVD2.iso .. ..
>> /media/distributionA/DVDn.iso
>
> Well, I can get you half-way there.
>
> for f in /media/distributionA/*.iso; do
> 	bn="$( basename "${f}" .iso )"
> 	dn="$( dirname "${f}" )"
> 	mount "${f}" /mnt/cdrom -o loop
> 	rsync -aP /mnt/cdrom "${dn}/${bn}"/
> 	umount /mnt/cdrom
> done
>
> Now you'll have in addition to the above:
> /media/distributionA/DVD1/ …
> /media/distributionA/DVD2/ …
> ..
> ..
> /media/distributionA/DVDn/ …
>
> I'm not sure how you'd go about merging the repositories together.
> Someone else can help with that.
>

I'll your points in reverse order.
The "repository merging" is handled by the -extract option of 
xorriso.
Thomas Schmitt, author of xorriso, has said in this thread to 
copy from DVD using dd.
Is there a way to pause your loop to swap DVDs?

[toc] | [prev] | [next] | [standalone]

#158049

Hi,

(Note to myself: This is the extract thread, not the
 thread about getting .iso files from /dev/sr0.)

Stuart Longland wrote:
> > for f in /media/distributionA/*.iso; do
> >       bn="$( basename "${f}" .iso )"
> >       dn="$( dirname "${f}" )"
> >       mount "${f}" /mnt/cdrom -o loop
> >       rsync -aP /mnt/cdrom "${dn}/${bn}"/
> >       umount /mnt/cdrom
> > done

I wonder why rsync and not "cp -a", and why not only
/mnt/cdrom/pool gets copied.
Maybe one should only do mount and umount as superuser
(via sudo) and leave the loop under control of the normal
user.


Richard Owlett wrote:
> Is there a way to pause your loop to swap DVDs?

  while true
  do
    echo "Insert medium, wait until blinking ends, press Enter key (or Ctrl+C to end)"
    read dummy

    ... mount /dev/sr0, rsync or cp, umount /dev/sr0
        or xorriso ... -indev /dev/sr0 ...

  done


Have a nice day :)

Thomas

[toc] | [prev] | [next] | [standalone]

#158053

Thomas Schmitt wrote:
> Hi,
>
> (Note to myself: This is the extract thread, not the
>   thread about getting .iso files from /dev/sr0.)
>
> Stuart Longland wrote:
>>> for f in /media/distributionA/*.iso; do
>>>        bn="$( basename "${f}" .iso )"
>>>        dn="$( dirname "${f}" )"
>>>        mount "${f}" /mnt/cdrom -o loop
>>>        rsync -aP /mnt/cdrom "${dn}/${bn}"/
>>>        umount /mnt/cdrom
>>> done
>
> [snip]
>
> Richard Owlett wrote:
>> Is there a way to pause your loop to swap DVDs?
>
>    while true
>    do
>      echo "Insert medium, wait until blinking ends, press Enter key (or Ctrl+C to end)"
>      read dummy
>
>      ... mount /dev/sr0, rsync or cp, umount /dev/sr0
>          or xorriso ... -indev /dev/sr0 ...
>
>    done
>

Now have half dozen scripting tutorials up.
I sometimes take the long way 'round [If retirement isn't for 
education, what use is it?]

[toc] | [prev] | [next] | [standalone]

#158177

[Multipart message — attachments visible in raw view] — view raw

On 18/09/15 21:57, Richard Owlett wrote:
> I'll your points in reverse order.
> The "repository merging" is handled by the -extract option of xorriso.
> Thomas Schmitt, author of xorriso, has said in this thread to copy from
> DVD using dd.

Yeah, I basically was just focussing on the mechanics of getting the
data out of the .iso image.

> Is there a way to pause your loop to swap DVDs?

Why would you want to do that, surely the DVD images are separate files?

Or have the DVDs already been burned and you've yet to DD the images of
them?  In which case, you could modify the loop:

dn=/media/distributionA
for disc in $( seq 1 N ); do
    echo "Please insert disc ${disc}"
    read newline
    mount /dev/cdrom /mnt/cdrom
    rsync -aP /mnt/cdrom "${dn}/DVD${disc}"/
    umount /mnt/cdrom
done

That will prompt on each disc.

Someone asked why rsync?  Well, cp would do the job too I guess, rsync
will resume an interrupted copy whereas cp won't.  Piping tar to itself
is another way.
-- 
Stuart Longland (aka Redhatter, VK4MSL)

I haven't lost my mind...
  ...it's backed up on a tape somewhere.

[toc] | [prev] | [next] | [standalone]

Page 3 of 4 — ← Prev page 1 2 [3] 4  Next page →

Back to top | Article view | linux.debian.user

csiph-web