Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.user > #174497 > unrolled thread

Re: dd - proper use or more suitable program

Back to article view | Back to linux.debian.user

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: dd - proper use or more suitable program Andy Smith <andy@strugglers.net> - 2016-11-11 18:00 +0100
    Re: dd - proper use or more suitable program Christian Seiler <christian@iwakd.de> - 2016-11-11 18:40 +0100
      Re: dd - proper use or more suitable program The Wanderer <wanderer@fastmail.fm> - 2016-11-11 19:20 +0100
        Re: dd - proper use or more suitable program Richard Owlett <rowlett@cloud85.net> - 2016-11-11 22:40 +0100
          Re: dd - proper use or more suitable program Andy Smith <andy@strugglers.net> - 2016-11-12 05:50 +0100
            Re: dd - proper use or more suitable program Richard Owlett <rowlett@cloud85.net> - 2016-11-12 09:20 +0100
    Re: dd - proper use or more suitable program Nicolas George <george@nsup.org> - 2016-11-11 19:00 +0100
      Re: dd - proper use or more suitable program Richard Owlett <rowlett@cloud85.net> - 2016-11-11 22:40 +0100
        Re: dd - proper use or more suitable program Christian Seiler <christian@iwakd.de> - 2016-11-12 08:40 +0100
          Re: dd - proper use or more suitable program Richard Owlett <rowlett@cloud85.net> - 2016-11-12 09:30 +0100
            Re: dd - proper use or more suitable program Richard Owlett <rowlett@cloud85.net> - 2016-11-12 18:30 +0100
              Re: dd - proper use or more suitable program Pascal Hambourg <pascal@plouf.fr.eu.org> - 2016-11-12 19:30 +0100
                Re: dd - proper use or more suitable program Richard Owlett <rowlett@cloud85.net> - 2016-11-12 20:30 +0100
                  Re: dd - proper use or more suitable program Ben Caradoc-Davies <ben@transient.nz> - 2016-11-12 21:00 +0100
                    Re: dd - proper use or more suitable program Brian <ad44@cityscape.co.uk> - 2016-11-12 21:10 +0100
                  Re: dd - proper use or more suitable program <tomas@tuxteam.de> - 2016-11-12 22:00 +0100
                    Invoking ddrescue Richard Owlett <rowlett@cloud85.net> - 2016-11-13 04:10 +0100
                      Re: Invoking ddrescue "Thomas Schmitt" <scdbackup@gmx.net> - 2016-11-13 13:00 +0100
                        Re: Invoking ddrescue <tomas@tuxteam.de> - 2016-11-13 13:10 +0100
                      Re: Invoking ddrescue <tomas@tuxteam.de> - 2016-11-13 13:00 +0100
                      Re: Invoking ddrescue "Thomas Schmitt" <scdbackup@gmx.net> - 2016-11-13 13:40 +0100
                        Re: Invoking ddrescue Pascal Hambourg <pascal@plouf.fr.eu.org> - 2016-11-13 13:50 +0100
                          Re: Invoking ddrescue Nicolas George <george@nsup.org> - 2016-11-13 14:00 +0100
                          Re: Invoking ddrescue "Thomas Schmitt" <scdbackup@gmx.net> - 2016-11-13 14:40 +0100
                            Re: Invoking ddrescue Pascal Hambourg <pascal@plouf.fr.eu.org> - 2016-11-13 15:10 +0100
                              Re: Invoking ddrescue "Thomas Schmitt" <scdbackup@gmx.net> - 2016-11-13 16:10 +0100
                                Re: Invoking ddrescue Richard Owlett <rowlett@cloud85.net> - 2016-11-13 17:30 +0100
                      Re: Invoking ddrescue Brian <ad44@cityscape.co.uk> - 2016-11-13 15:20 +0100
                      Progress report Re: Invoking ddrescue Richard Owlett <rowlett@cloud85.net> - 2016-11-14 20:20 +0100
                        Re: Progress report Re: Invoking ddrescue Jonathan Dowland <jmtd@debian.org> - 2016-11-14 21:20 +0100
                          Re: Progress report Re: Invoking ddrescue Richard Owlett <rowlett@cloud85.net> - 2016-11-14 22:00 +0100
                        Re: Progress report Re: Invoking ddrescue <tomas@tuxteam.de> - 2016-11-14 21:30 +0100
                          Re: Progress report Re: Invoking ddrescue Richard Owlett <rowlett@cloud85.net> - 2016-11-14 22:20 +0100
                        Re: Progress report Re: Invoking ddrescue "Thomas Schmitt" <scdbackup@gmx.net> - 2016-11-14 21:30 +0100
                          Re: Progress report Re: Invoking ddrescue Richard Owlett <rowlett@cloud85.net> - 2016-11-14 22:10 +0100
                          Re: Progress report Re: Invoking ddrescue Pascal Hambourg <pascal@plouf.fr.eu.org> - 2016-11-17 20:20 +0100
                        SUCESS!!! - was [Re: Progress report Re: Invoking ddrescue] Richard Owlett <rowlett@cloud85.net> - 2016-11-14 23:40 +0100
                          Re: SUCESS!!! - was [Re: Progress report Re: Invoking ddrescue] Brian <ad44@cityscape.co.uk> - 2016-11-15 00:30 +0100
                            Re: SUCESS!!! - was [Re: Progress report Re: Invoking ddrescue] Richard Owlett <rowlett@cloud85.net> - 2016-11-15 04:00 +0100
                              Re: SUCESS!!! - was [Re: Progress report Re: Invoking ddrescue] Brian <ad44@cityscape.co.uk> - 2016-11-15 21:00 +0100
                                Re: SUCESS!!! - was [Re: Progress report Re: Invoking ddrescue] <tomas@tuxteam.de> - 2016-11-15 22:00 +0100
                          Re: SUCESS!!! - was [Re: Progress report Re: Invoking ddrescue] "Thomas Schmitt" <scdbackup@gmx.net> - 2016-11-15 12:50 +0100
                            Re: SUCESS!!! - was [Re: Progress report Re: Invoking ddrescue] Brian <ad44@cityscape.co.uk> - 2016-11-15 13:30 +0100
                              Re: SUCESS!!! - was [Re: Progress report Re: Invoking ddrescue] "Thomas Schmitt" <scdbackup@gmx.net> - 2016-11-15 13:50 +0100
                            Re: SUCESS!!! - was [Re: Progress report Re: Invoking ddrescue] Richard Owlett <rowlett@cloud85.net> - 2016-11-15 13:50 +0100
                            Re: SUC[C]ESS!!! - was [Re: Progress report Re: Invoking ddrescue] David Wright <deblis@lionunicorn.co.uk> - 2016-11-15 16:30 +0100
                              Re: SUCESS!!! - was [Re: Progress report Re: Invoking ddrescue] "Thomas Schmitt" <scdbackup@gmx.net> - 2016-11-15 17:20 +0100
                                Re: SUCESS!!! - was [Re: Progress report Re: Invoking ddrescue] Greg Wooledge <wooledg@eeg.ccf.org> - 2016-11-15 17:30 +0100

Page 1 of 3  [1] 2 3  Next page →

#174497 — Re: dd - proper use or more suitable program

Hi Richard,

On Fri, Nov 11, 2016 at 10:49:37AM -0600, Richard Owlett wrote:
> I was considering using dd to copy the entire drive to a *SINGLE*
> partition of a 1 TB drive with the intention making a "byte perfect"
> of of the defective drive to a new 300 GB drive at a later time to
> then attempt "data rescue". Partitions other than the first are
> evidently readable.
> 
> Suggestions/comments please.

You are better off using GNU ddrescue for taking images of
possibly-failing devices.

Amongst other issues, dd will either give up or produce zeroes when
it encounters problems whereas ddrescue will keep track of where it
was unable to read and keep trying.

Cheers,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting

[toc] | [next] | [standalone]

#174500

Hi,

Am 11. November 2016 17:57:27 MEZ, schrieb Andy Smith <andy@strugglers.net>:
>Hi Richard,
>
>On Fri, Nov 11, 2016 at 10:49:37AM -0600, Richard Owlett wrote:
>> I was considering using dd to copy the entire drive to a *SINGLE*
>> partition of a 1 TB drive with the intention making a "byte perfect"
>> of of the defective drive to a new 300 GB drive at a later time to
>> then attempt "data rescue". Partitions other than the first are
>> evidently readable.
>> 
>> Suggestions/comments please.
>
>You are better off using GNU ddrescue for taking images of
>possibly-failing devices.

Full ACK: GNU ddrescue has saved my data multiple times in the past, I can really recommend it. (The "log file" is very helpful with resuming at a later point in time if you had to cancel it.)

Just don't confuse it with dd_rescue, which I don't recommend unless you are an expert and have a very special case.

Regards,
Christian

[toc] | [prev] | [next] | [standalone]

#174506

[Multipart message — attachments visible in raw view] — view raw

On 2016-11-11 at 12:37, Christian Seiler wrote:

> Hi,
> 
> Am 11. November 2016 17:57:27 MEZ, schrieb Andy Smith
> <andy@strugglers.net>:
> 
>> Hi Richard,
>> 
>> On Fri, Nov 11, 2016 at 10:49:37AM -0600, Richard Owlett wrote:
>> 
>>> I was considering using dd to copy the entire drive to a
>>> *SINGLE* partition of a 1 TB drive with the intention making a
>>> "byte perfect" of of the defective drive to a new 300 GB drive at
>>> a later time to then attempt "data rescue". Partitions other than
>>> the first are evidently readable.
>>> 
>>> Suggestions/comments please.
>> 
>> You are better off using GNU ddrescue for taking images of
>> possibly-failing devices.
> 
> Full ACK: GNU ddrescue has saved my data multiple times in the past,
> I can really recommend it. (The "log file" is very helpful with
> resuming at a later point in time if you had to cancel it.)
> 
> Just don't confuse it with dd_rescue, which I don't recommend unless
> you are an expert and have a very special case.

There's also myrescue, which is similar in function to both but which
I've found easier and less confusing to use in the past - if perhaps
only because it eliminates the confusion about remembering which of the
other two is the one which is more problematic.

The trouble with all of these is that not only do you need a device with
enough space to store the entire device you're drawing from (ideally in
a file rather than on the device directly), to do it properly and safely
you also need enough extra space - on another device is fine - to store
the log file which gets created during the rescue process.

-- 
   The Wanderer can't figure out why he's now thinking of the game IVAN

The reasonable man adapts himself to the world; the unreasonable one
persists in trying to adapt the world to himself. Therefore all
progress depends on the unreasonable man.         -- George Bernard Shaw

[toc] | [prev] | [next] | [standalone]

#174540

On 11/11/2016 12:13 PM, The Wanderer wrote:
> On 2016-11-11 at 12:37, Christian Seiler wrote:
>
>> Hi,
>>
>> Am 11. November 2016 17:57:27 MEZ, schrieb Andy Smith
>> <andy@strugglers.net>:
>>
>>> Hi Richard,
>>>
>>> On Fri, Nov 11, 2016 at 10:49:37AM -0600, Richard Owlett wrote:
>>>
>>>> I was considering using dd to copy the entire drive to a
>>>> *SINGLE* partition of a 1 TB drive with the intention making a
>>>> "byte perfect" of of the defective drive to a new 300 GB drive at
>>>> a later time to then attempt "data rescue". Partitions other than
>>>> the first are evidently readable.
>>>>
>>>> Suggestions/comments please.
>>>
>>> You are better off using GNU ddrescue for taking images of
>>> possibly-failing devices.
>>
>> Full ACK: GNU ddrescue has saved my data multiple times in the past,
>> I can really recommend it. (The "log file" is very helpful with
>> resuming at a later point in time if you had to cancel it.)
>>
>> Just don't confuse it with dd_rescue, which I don't recommend unless
>> you are an expert and have a very special case.
>
> There's also myrescue, which is similar in function to both but which
> I've found easier and less confusing to use in the past - if perhaps
> only because it eliminates the confusion about remembering which of the
> other two is the one which is more problematic.

http://myrescue.sourceforge.net/ doesn't indicate whether it will 
attempt to do what I want.
https://www.gnu.org/software/ddrescue/manual/ddrescue_manual.html 
is long and not oriented to first time user. But explicitly 
claims to do what I want. I'll have to re-read after a good 
night's sleep.


>
> The trouble with all of these is that not only do you need a device with
> enough space to store the entire device you're drawing from (ideally in
> a file rather than on the device directly), to do it properly and safely
> you also need enough extra space - on another device is fine - to store
> the log file which gets created during the rescue process.
>

How big might the logfile be when trying to recover a known flaky 
300 GB drive. I've lots of space? Some convienient, some not.

[toc] | [prev] | [next] | [standalone]

#174565

Hi Richard,

On Fri, Nov 11, 2016 at 03:31:21PM -0600, Richard Owlett wrote:
> How big might the logfile be when trying to recover a known flaky 300
> GB drive. I've lots of space? Some convienient, some not.

TL;DR: this depends on how many bad sectors you expect to find. If
the number is likely to be low then the map file should be a matter
of kilobytes in size.

I've never looked into this before as it's never been an issue for
me, but looking at:

    https://www.gnu.org/software/ddrescue/manual/ddrescue_manual.html#Mapfile-structure

The header of the map file looks like:

     # Mapfile. Created by GNU ddrescue version 1.21
     # Command line: ddrescue -d -c18 /dev/fd0 fdimage mapfile
     # Start time:   2015-07-21 09:37:44
     # Current time: 2015-07-21 09:38:19
     # Copying non-tried blocks... Pass 1 (forwards)
     # current_pos  current_status
     0x00120000     ?
     #      pos        size  status

…which is 304 bytes.

After that there is one line for each range of blocks depending on
their status (finished, not tried yet, failed etc).

I am thinking that the absolute worst case in terms of maximal
number of lines in this file would be if every other sector were
failed, so you'd have an alternating sequence of:

0x00000000  0x00000001  +
0x00000001  0x00000001  -
0x00000002  0x00000001  +
0x00000003  0x00000001  -

for the entire device. That's 52 bytes for every two blocks.

The default block size is 512 bytes in ddrescue, so two blocks
covers 1024 bytes of your device.

If your device is 300 gigabytes in size—and I'll assume that is SI
power of ten giga- (not binary power of two gibi-) as is common with
drive manufacturers, so 300,000,000,000 bytes—then that's
300,000,000,000 / 1,024 = 292,968,750. That times 54 bytes is
15,820,312,500 bytes. Or 14.73GiB. Plus a ~304 byte header.

As far as I can see that is the absolute worst case and for a more
realistic scenario of a device with only a couple of bad sectors
you'd be looking at mere kilobytes of map file size.

For example, if merely 1% of the sectors were bad (and I would
suggest that even that would represent a catastrophically damaged
device that you will find very difficult to extract any sense out
of) then you'd still only be looking at a map file with 5,859,375
bad blocks in it (5,859,375 bad sectors out of 585,937,500 total
512-byte sectors in a 300,000,000,000 byte device). This would
require 5,859,376 different ranges in the map file, with each range
being 27 bytes, so 27 * 5,859,376 = 158,203,152 bytes = 150.9MiB.

I doubt you will see 5.9 million bad sectors on your 300G drive!

Basically whenever my destination has had noticeably more space than
the source device I haven't spared a thought to this so have never
worked it out before. I think the above is correct but look forward
to a correction from anyone who knows better.

Also do note that should you run out of space when writing the map
file, you still have the map file that has been written to date, so
you can extricate yourself from the situation and rerun ddrescue,
safe in the knowledge that it will pick up from where it got to.

If you are expecting serious numbers of bad sectors then your most
precious resource may actually be time. ddrescue tries REALLY HARD
to read a bad sector with each try potentially taking 2 or more
minutes. So on the hypothetical "1% broken" drive with 5.9 million
bad sectors, a single pass could take upwards of 10 million minutes
(19 years). And sometimes multiple passes are required to read a bad
sector.

Cheers,
Andy

-- 
https://bitfolk.com/ -- No-nonsense VPS hosting

[toc] | [prev] | [next] | [standalone]

#174573

On 11/11/2016 10:45 PM, Andy Smith wrote:
> Hi Richard,
>
> On Fri, Nov 11, 2016 at 03:31:21PM -0600, Richard Owlett wrote:
>> How big might the logfile be when trying to recover a known flaky 300
>> GB drive. I've lots of space? Some convienient, some not.
>
> TL;DR: this depends on how many bad sectors you expect to find. If
> the number is likely to be low then the map file should be a matter
> of kilobytes in size.

Based on your example calculations I should be in good shape. 
Only one partition [the old c: drive] seems to be in bad shape. 
I've found some tutorial material that clears up enough that I'm 
confident of running safely even if not optimized.




>
> I've never looked into this before as it's never been an issue for
> me, but looking at:
>
>      https://www.gnu.org/software/ddrescue/manual/ddrescue_manual.html#Mapfile-structure
>
> The header of the map file looks like:
>
>       # Mapfile. Created by GNU ddrescue version 1.21
>       # Command line: ddrescue -d -c18 /dev/fd0 fdimage mapfile
>       # Start time:   2015-07-21 09:37:44
>       # Current time: 2015-07-21 09:38:19
>       # Copying non-tried blocks... Pass 1 (forwards)
>       # current_pos  current_status
>       0x00120000     ?
>       #      pos        size  status
>
> …which is 304 bytes.
>
> After that there is one line for each range of blocks depending on
> their status (finished, not tried yet, failed etc).
>
> I am thinking that the absolute worst case in terms of maximal
> number of lines in this file would be if every other sector were
> failed, so you'd have an alternating sequence of:
>
> 0x00000000  0x00000001  +
> 0x00000001  0x00000001  -
> 0x00000002  0x00000001  +
> 0x00000003  0x00000001  -
>
> for the entire device. That's 52 bytes for every two blocks.
>
> The default block size is 512 bytes in ddrescue, so two blocks
> covers 1024 bytes of your device.
>
> If your device is 300 gigabytes in size—and I'll assume that is SI
> power of ten giga- (not binary power of two gibi-) as is common with
> drive manufacturers, so 300,000,000,000 bytes—then that's
> 300,000,000,000 / 1,024 = 292,968,750. That times 54 bytes is
> 15,820,312,500 bytes. Or 14.73GiB. Plus a ~304 byte header.
>
> As far as I can see that is the absolute worst case and for a more
> realistic scenario of a device with only a couple of bad sectors
> you'd be looking at mere kilobytes of map file size.
>
> For example, if merely 1% of the sectors were bad (and I would
> suggest that even that would represent a catastrophically damaged
> device that you will find very difficult to extract any sense out
> of) then you'd still only be looking at a map file with 5,859,375
> bad blocks in it (5,859,375 bad sectors out of 585,937,500 total
> 512-byte sectors in a 300,000,000,000 byte device). This would
> require 5,859,376 different ranges in the map file, with each range
> being 27 bytes, so 27 * 5,859,376 = 158,203,152 bytes = 150.9MiB.
>
> I doubt you will see 5.9 million bad sectors on your 300G drive!
>
> Basically whenever my destination has had noticeably more space than
> the source device I haven't spared a thought to this so have never
> worked it out before. I think the above is correct but look forward
> to a correction from anyone who knows better.
>
> Also do note that should you run out of space when writing the map
> file, you still have the map file that has been written to date, so
> you can extricate yourself from the situation and rerun ddrescue,
> safe in the knowledge that it will pick up from where it got to.
>
> If you are expecting serious numbers of bad sectors then your most
> precious resource may actually be time. ddrescue tries REALLY HARD
> to read a bad sector with each try potentially taking 2 or more
> minutes. So on the hypothetical "1% broken" drive with 5.9 million
> bad sectors, a single pass could take upwards of 10 million minutes
> (19 years). And sometimes multiple passes are required to read a bad
> sector.
>
> Cheers,
> Andy
>

[toc] | [prev] | [next] | [standalone]

#174503

[Multipart message — attachments visible in raw view] — view raw

Le primidi 21 brumaire, an CCXXV, Andy Smith a écrit :
> You are better off using GNU ddrescue for taking images of
> possibly-failing devices.

IIRC, there is a catch there: there is another program with a very
similar name that does not work the same way. Be careful when installing
it.

> Amongst other issues, dd will either give up or produce zeroes when
> it encounters problems whereas ddrescue will keep track of where it
> was unable to read and keep trying.

For the record, when using dd to copy data from a damaged medium, always
use conv=noerror,sync. Without noerror, dd would stop at the first
error and without sync it would just skip the block, making the dump
useless.

But of course, as you say, ddrescue keeps a log of the parts that could
not be read, which is even more useful.

Also, I would advise to dump to a file, not a raw partition.

[toc] | [prev] | [next] | [standalone]

#174541

On 11/11/2016 11:49 AM, Nicolas George wrote:
> Le primidi 21 brumaire, an CCXXV, Andy Smith a écrit :
>> You are better off using GNU ddrescue for taking images of
>> possibly-failing devices.
>
> IIRC, there is a catch there: there is another program with a very
> similar name that does not work the same way. Be careful when installing
> it.

Jessie DVD's catch that. What Synaptic fetches is gddrescue, but 
after loading you invoke it as just ddrescue.

>
>> Amongst other issues, dd will either give up or produce zeroes when
>> it encounters problems whereas ddrescue will keep track of where it
>> was unable to read and keep trying.
>
> For the record, when using dd to copy data from a damaged medium, always
> use conv=noerror,sync. Without noerror, dd would stop at the first
> error and without sync it would just skip the block, making the dump
> useless.
>
> But of course, as you say, ddrescue keeps a log of the parts that could
> not be read, which is even more useful.
>
> Also, I would advise to dump to a file, not a raw partition.
>

I was wondering about that.
https://www.gnu.org/software/ddrescue/manual/ddrescue_manual.html 
is not first time user friendly. Will re-read after a good 
night's sleep. Will also look for appropriate tutorials. Suggestions?

[toc] | [prev] | [next] | [standalone]

#174569

On 11/11/2016 10:38 PM, Richard Owlett wrote:
> I was wondering about that. 
> https://www.gnu.org/software/ddrescue/manual/ddrescue_manual.html is
> not first time user friendly. Will re-read after a good night's
> sleep. Will also look for appropriate tutorials. Suggestions?

Well, I would suggest not dumping the result on another partition
but rather into an image file. In that case you'd have the old
drive (not mounted), let's call it /dev/sda, and the new drive
with a partition on it and mounted on /mnt with sufficient free
disk space there.

In the simplest case you'd do:

ddrescue /dev/sda /mnt/defective_drive.img /mnt/defective_drive.log

If the drive is farther gone and has quite a few defective
sectors you may get better results if you use direct disk access
for the phase where you try to read defective sectors. In that case
you'd copy the bulk of the data while disabling the scraping phase
manually,

ddrescue -n /dev/sda /mnt/defective_drive.img /mnt/defective_drive.log

And then use direct access to scrape the rest:

ddrescue -d /dev/sda /mnt/defective_drive.img /mnt/defective_drive.log

Note that ddrescue assumes a default sector size of 512 bytes,
which is likely to be correct for older hard drives smaller than
2 TiB. If you have a newer hard drive, especially if it's rather
large, the physical sector size could be 4096 bytes instead.
(You can check the physical sector size of your disk by running
hdparm -I /dev/sda - that will tell you a lot of things about
your drive, among them the physical sector size in bytes. Note
that the physical sector size is relevant here, not the logical
one.)

In addition, you may also want to specify a number of retries
when trying to read defective sectors. (Default is 0.) You can
do that with the -r flag, e.g. -r3 for three retries per sector.

To recap:

 - Simplest way of calling the program is

   ddrescue /dev/sda output_image_file log_file

 - If the condition of your hard drive is a bit worse, you might
   achieve better results with:

   ddrescue -n /dev/sda output_image_file log_file
   ddrescue -d -r2 /dev/sda output_image_file log_file

   (-r2 is for 2 retries per defective sector, you can change
   that number; you can also call the second ddrescue command
   again in case you want to do more retries.)

 - If you have a new disk with 4096 bytes / sector (not likely)
   please also add a -b4096 to the command line.

 - If your target is not an image file, but a disk drive itself,
   also specify the -f option. (I would really recommend using
   image files though, gives you more flexibility.)

 - I wouldn't really bother with the other options, the default
   behavior is very sensible.

 - Finally, get some coffee or similar, this may take a while.

Regards,
Christian

[toc] | [prev] | [next] | [standalone]

#174574

On 11/12/2016 1:31 AM, Christian Seiler wrote:
> On 11/11/2016 10:38 PM, Richard Owlett wrote:
>> I was wondering about that.
>> https://www.gnu.org/software/ddrescue/manual/ddrescue_manual.html is
>> not first time user friendly. Will re-read after a good night's
>> sleep. Will also look for appropriate tutorials. Suggestions?
>
> Well, I would suggest not dumping the result on another partition
> but rather into an image file. In that case you'd have the old
> drive (not mounted), let's call it /dev/sda, and the new drive
> with a partition on it and mounted on /mnt with sufficient free
> disk space there.
>
> In the simplest case you'd do:
>
> ddrescue /dev/sda /mnt/defective_drive.img /mnt/defective_drive.log
>
> If the drive is farther gone and has quite a few defective
> sectors you may get better results if you use direct disk access
> for the phase where you try to read defective sectors. In that case
> you'd copy the bulk of the data while disabling the scraping phase
> manually,
>
> ddrescue -n /dev/sda /mnt/defective_drive.img /mnt/defective_drive.log
>
> And then use direct access to scrape the rest:
>
> ddrescue -d /dev/sda /mnt/defective_drive.img /mnt/defective_drive.log
>
> Note that ddrescue assumes a default sector size of 512 bytes,
> which is likely to be correct for older hard drives smaller than
> 2 TiB. If you have a newer hard drive, especially if it's rather
> large, the physical sector size could be 4096 bytes instead.
> (You can check the physical sector size of your disk by running
> hdparm -I /dev/sda - that will tell you a lot of things about
> your drive, among them the physical sector size in bytes. Note
> that the physical sector size is relevant here, not the logical
> one.)
>
> In addition, you may also want to specify a number of retries
> when trying to read defective sectors. (Default is 0.) You can
> do that with the -r flag, e.g. -r3 for three retries per sector.
>
> To recap:
>
>   - Simplest way of calling the program is
>
>     ddrescue /dev/sda output_image_file log_file
>
>   - If the condition of your hard drive is a bit worse, you might
>     achieve better results with:
>
>     ddrescue -n /dev/sda output_image_file log_file
>     ddrescue -d -r2 /dev/sda output_image_file log_file
>
>     (-r2 is for 2 retries per defective sector, you can change
>     that number; you can also call the second ddrescue command
>     again in case you want to do more retries.)
>
>   - If you have a new disk with 4096 bytes / sector (not likely)
>     please also add a -b4096 to the command line.
>
>   - If your target is not an image file, but a disk drive itself,
>     also specify the -f option. (I would really recommend using
>     image files though, gives you more flexibility.)
>
>   - I wouldn't really bother with the other options, the default
>     behavior is very sensible.
>
>   - Finally, get some coffee or similar, this may take a while.
>
> Regards,
> Christian

Your timing is perfect. Couldn't sleep so just finished reading a 
couple of tutorials. Your post filled some holes. You also 
confirm my tentative choices of options. Now to go back to bed.

[toc] | [prev] | [next] | [standalone]

#174591

On 11/12/2016 2:29 AM, Richard Owlett wrote:
> On 11/12/2016 1:31 AM, Christian Seiler wrote:
>> On 11/11/2016 10:38 PM, Richard Owlett wrote:
>>> I was wondering about that.
>>> https://www.gnu.org/software/ddrescue/manual/ddrescue_manual.html
>>> is
>>> not first time user friendly. Will re-read after a good night's
>>> sleep. Will also look for appropriate tutorials. Suggestions?
>>
>> Well, I would suggest not dumping the result on another partition
>> but rather into an image file. In that case you'd have the old
>> drive (not mounted), let's call it /dev/sda, and the new drive
>> with a partition on it and mounted on /mnt with sufficient free
>> disk space there.
>>
>> In the simplest case you'd do:
>>
>> ddrescue /dev/sda /mnt/defective_drive.img
>> /mnt/defective_drive.log

I can't get that performing correctly. I have a strong suspicion 
I'm even more confused about mounting than the group suspects :<

Can this be run as a user, or are root permissions required.

May I have a little hand-holding please?
My defective drive is /dev/sdc .
Partition /dev/sdb6 {formatted ext4} is much larger than the 
defective drive.
/dev/sdb is a 1 TB rotating platter drive connected via USB.
How do I mount it and run ddrescue?
I'm tired and likely going in circles. Need to get groceries, 
will be back soon.

Thank you for patience.

[toc] | [prev] | [next] | [standalone]

#174594

Le 12/11/2016 à 18:22, Richard Owlett a écrit :
>
>>> ddrescue /dev/sda /mnt/defective_drive.img /mnt/defective_drive.log
(...)
> Can this be run as a user, or are root permissions required.

Unless the user has read permission on the raw device, it must be run as 
root.

> My defective drive is /dev/sdc .
> Partition /dev/sdb6 {formatted ext4} is much larger than the defective
> drive.
> /dev/sdb is a 1 TB rotating platter drive connected via USB.
> How do I mount it and run ddrescue?

Mount what ?
You don't mount anything from the defective drive.
You mount the filesystem which will receive the disk image file anywhere 
you like with mount. In the above example it was expected to be mounted 
on /mnt, the usual temporary mount point. So :

mount /dev/sdb6 /mnt

The file manager of your desktop environment may have already mounted it 
for you elsewhere in /media/. Check df or mount.

[toc] | [prev] | [next] | [standalone]

#174603

On 11/12/2016 12:26 PM, Pascal Hambourg wrote:
> Le 12/11/2016 à 18:22, Richard Owlett a écrit :
>>
>>>> ddrescue /dev/sda /mnt/defective_drive.img
>>>> /mnt/defective_drive.log
> (...)
>> Can this be run as a user, or are root permissions required.
>
> Unless the user has read permission on the raw device, it must be
> run as root.

I have a STRONG suspicion that by "raw device" you refer to the 
defective device which is enumerated as /dev/sdc . *ALL* 
documentation and tutorials I found make a *MAJOR POINT* of *NOT* 
mounting the defective device. "Permissions" therefor are a murky 
issue. Point of fact, the specific physical defective object 
predates me having more than casual interest in *nix.

There's a reason I asked for "hand holding" and specifically 
asked for tolerance.
In a "user to user" support enviroment ther is an expectation of 
querent doing his due diligence. I tried. I failed. M'aidez s'il 
vous plait.

>
>> My defective drive is /dev/sdc .
>> Partition /dev/sdb6 {formatted ext4} is much larger than the
>> defective
>> drive.
>> /dev/sdb is a 1 TB rotating platter drive connected via USB.
>> How do I mount it and run ddrescue?
>
> Mount what ?

Whatever that needs to be mounted.
There *IS* a reason I explicitly stated that I needed an atypical 
amount of hand holding!

> You don't mount anything from the defective drive.
> You mount the filesystem which will receive the disk image file
> anywhere you like with mount. In the above example it was
> expected to be mounted on /mnt, the usual temporary mount point.
> So :
>
> mount /dev/sdb6 /mnt
>
> The file manager of your desktop environment may have already
> mounted it for you elsewhere in /media/. Check df or mount.
>

I've gone to lenghts to defeat Debian's attempts to out-M$ Wm Gates.
I predate the 8085 when it was acceptable manufacturing practice 
to cut/jumper etchs to "reprogram" 7400 series devices.

Tolerant help please.

[toc] | [prev] | [next] | [standalone]

#174605

On 13/11/16 08:25, Richard Owlett wrote:
> I've gone to lenghts to defeat Debian's attempts to out-M$ Wm Gates.

Redmond is now our ally against the closed garden at Cupertino, with 
cloud image support, open-sourced language platforms, and even Ubuntu 
integration on the desktop.

We've always been at war with Eastasia.

Kind regards,

-- 
Ben Caradoc-Davies <ben@transient.nz>
Director
Transient Software Limited <http://transient.nz/>
New Zealand

[toc] | [prev] | [next] | [standalone]

#174606

On Sun 13 Nov 2016 at 08:54:05 +1300, Ben Caradoc-Davies wrote:

> On 13/11/16 08:25, Richard Owlett wrote:
> >I've gone to lenghts to defeat Debian's attempts to out-M$ Wm Gates.
> 
> Redmond is now our ally against the closed garden at Cupertino, with cloud
> image support, open-sourced language platforms, and even Ubuntu integration
> on the desktop.
> 
> We've always been at war with Eastasia.

I imagine Richard Owlett's comment has attracted this sort of response.
Neither helps with a solution to the issue.

-- 
Brian.

[toc] | [prev] | [next] | [standalone]

#174609

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Sat, Nov 12, 2016 at 01:25:50PM -0600, Richard Owlett wrote:
> On 11/12/2016 12:26 PM, Pascal Hambourg wrote:
> >Le 12/11/2016 à 18:22, Richard Owlett a écrit :
> >>
> >>>>ddrescue /dev/sda /mnt/defective_drive.img
> >>>>/mnt/defective_drive.log
> >(...)
> >>Can this be run as a user, or are root permissions required.
> >
> >Unless the user has read permission on the raw device, it must be
> >run as root.
> 
> I have a STRONG suspicion that by "raw device" you refer to the
> defective device which is enumerated as /dev/sdc . *ALL*
> documentation and tutorials I found make a *MAJOR POINT* of *NOT*
> mounting the defective device. "Permissions" therefor are a murky
> issue.

In this case, the permissions on /dev/sdc were meant (as opposed
to the permissions on the files whithin the file system in /dev/sdc,
which don't count in this case)

> Point of fact, the specific physical defective object
> predates me having more than casual interest in *nix.

Yes: I remember there's a DOS file system in the disk in question,
so no permissions in there anyway. But as stated above, this doesn't
matter, because you're looking at the container: access to that
is ruled by the permissions on the device file, i.e. /dev/sdc.

> There's a reason I asked for "hand holding" and specifically asked
> for tolerance.
> In a "user to user" support enviroment ther is an expectation of
> querent doing his due diligence. I tried. I failed. M'aidez s'il
> vous plait.

In a nutshell: you mount your *new* disk to a directory of your
choice (let's say /mnt). Possibly your OS is set up to do that
for you: it'll typically end then somewhere in /media/blah (for
some suitable value of "blah"). Let's use /mnt as a placeholder.

Then you insert you defective disk, which (let's say) appears
as /dev/sdc. Make sure the OS doesn't mount it automatically,
otherwise unmount yourself.

Then you do

  dd if=/dev/sdc of=/mnt/my-disk-backup bs=4096 count=1000000

(this is over-simplified: you'll probably use ddrescue instead
of dd, or at least use the option 'noerror').

An image of your disk will hopefully appear on 'my-disk-backup'.
You need read access to /dev/sdc and write access to the directory
/mnt (either by doing sudo, I'd do that or by other means).

Those steps are a rough sketch. Many details already flew back
and forth in this thread, so I didn't want to bloat the thing
too much. Feel free to ask where things are unclear.

Regards
- -- tomás
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)

iEYEARECAAYFAlgngikACgkQBcgs9XrR2kYjmACeJyeAvYCCrcfx2UuSglH7Dnvb
eg0An14EEu9c0jwQTcjF5/4bNCm9qHTV
=c2+e
-----END PGP SIGNATURE-----

[toc] | [prev] | [next] | [standalone]

#174624 — Invoking ddrescue

On 11/12/2016 2:57 PM, tomas@tuxteam.de wrote:
> [*SNIP*]
> Those steps are a rough sketch. Many details already flew back
> and forth in this thread, so I didn't want to bloat the thing
> too much. Feel free to ask where things are unclear.
>

I'm tangled up !! I plead for 5 or 6 lines to copy-n-paste.
On my left hand I have a defective hard disk - AKA /dev/sdc .
On my right hand I have a partitioned device waiting for data - 
AKA /dev/sdb6 .
Automount has been disabled explicitly.
I have been EXPLICITLY told that ddrescue is the appropriate tool.
Reading 
https://www.gnu.org/software/ddrescue/manual/ddrescue_manual.html 
and a half dozen "tutorials" convinces me of that. Clear 
explanations given in those documents why dd and other tools 
*NOT* suitable for my circumstances.


How do I prepare to invoke
    ddrescue /dev/sdc /mnt/repaired.img /mnt/repaired.log


Help. Please. Thank you.

[toc] | [prev] | [next] | [standalone]

#174638 — Re: Invoking ddrescue

Hi,

Richard Owlett wrote:
> I'm tangled up !! I plead for 5 or 6 lines to copy-n-paste.

My proposal deviates from some aspects of your original plan.

I would try something like

  ddrescue -p /dev/sdc1 /mnt/my_sdb6/my_sdc1 /mnt/my_sdb6/sdc1_log

Details:

  /dev/sdc1              is a partition of the defective disk.
  /mnt/my_sdb6           is the mount point of the filesystem on your
                         partition sdb6.
  /mnt/my_sdb6/my_sdc1   is the data file name for the copied data.
  /mnt/my_sdb6/sdc1_log  records what ddrescue might need for further read
                         attempts.
  -p creates the data file with full size (i understand), so that you know
     early when your filesystem is full.

Why:

Copying disk to partition is not the best thing to do if you want
to use the copy result directly.
If the original disk contains partitions - even if it is only a single
one -, then each should get into a separate storage container. So you
can mount these containers.

Storage container can be a whole disk device, a partition of a disk
device, or a data file in a large filesystem which can represent
large files (i.e. a normal Linux filesystem in a large partition).

Most normal and harmless to use is a filesystem.
So if you have a partition which is large enough for the whole original
disk, then i advise to equip it with a filesystem (by e.g. mkfs) and
to mount it somewhere (e.g. as /mnt/my_sdb6).


Have a nice day :)

Thomas

[toc] | [prev] | [next] | [standalone]

#174640 — Re: Invoking ddrescue

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Sun, Nov 13, 2016 at 12:53:20PM +0100, Thomas Schmitt wrote:
> Hi,
> 
> Richard Owlett wrote:
> > I'm tangled up !! I plead for 5 or 6 lines to copy-n-paste.
> 
> My proposal deviates from some aspects of your original plan.

Ah, Thomas, you're the fairy :-)

My wish of someone chiming in became true

Yes, option -p makes a lot of sense.

[...]

> Copying disk to partition is not the best thing to do if you want
> to use the copy result directly.
> If the original disk contains partitions - even if it is only a single
> one -, then each should get into a separate storage container. So you
> can mount these containers.
> 
> Storage container can be a whole disk device, a partition of a disk
> device, or a data file in a large filesystem which can represent
> large files (i.e. a normal Linux filesystem in a large partition).
> 
> Most normal and harmless to use is a filesystem.
> So if you have a partition which is large enough for the whole original
> disk, then i advise to equip it with a filesystem (by e.g. mkfs) and
> to mount it somewhere (e.g. as /mnt/my_sdb6).

As far as I understood, Richard is already doing that: his "big" disk
(where the backup is going to) has already a file system and is mounted.
His "first cut" command

     ddrescue /dev/sdc /mnt/repaired.img /mnt/repaired.log

is already copying the raw device /dev/sdc to a file in the target
system.

Thanks
- -- tomás
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)

iEYEARECAAYFAlgoVi0ACgkQBcgs9XrR2kZY9ACfVNmDcQRN1AoyRFqmdJSN+SyW
yrcAn2N6kjwTHawHElYzaV+2jDCWWtUu
=60fJ
-----END PGP SIGNATURE-----

[toc] | [prev] | [next] | [standalone]

#174639 — Re: Invoking ddrescue

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Sat, Nov 12, 2016 at 09:05:17PM -0600, Richard Owlett wrote:
> On 11/12/2016 2:57 PM, tomas@tuxteam.de wrote:
> >[*SNIP*]
> >Those steps are a rough sketch. Many details already flew back
> >and forth in this thread, so I didn't want to bloat the thing
> >too much. Feel free to ask where things are unclear.
> >
> 
> I'm tangled up !! I plead for 5 or 6 lines to copy-n-paste.
> On my left hand I have a defective hard disk - AKA /dev/sdc .
> On my right hand I have a partitioned device waiting for data - AKA
> /dev/sdb6 .
> Automount has been disabled explicitly.
> I have been EXPLICITLY told that ddrescue is the appropriate tool.
> Reading
> https://www.gnu.org/software/ddrescue/manual/ddrescue_manual.html
> and a half dozen "tutorials" convinces me of that. Clear
> explanations given in those documents why dd and other tools *NOT*
> suitable for my circumstances.
> 
> 
> How do I prepare to invoke
>    ddrescue /dev/sdc /mnt/repaired.img /mnt/repaired.log

That sounds about right. Problem is... I haven't used ddrescue (yet),
but only dd -- so better wait for more ddrescue-savvy folks to chime
in for the gory details.

Going by the ddrescue manual [1], your command line looks pretty
sane to me.

Regards
[1] https://www.gnu.org/software/ddrescue/manual/ddrescue_manual.html#Invoking-ddrescue

- -- tomás
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)

iEYEARECAAYFAlgoVOEACgkQBcgs9XrR2kb78wCfQy+xgvHizbvHvAmmt0gLrDaH
OPYAnigypVBMmd+CTV5bsrsb0ZWG7gDI
=Nz9z
-----END PGP SIGNATURE-----

[toc] | [prev] | [next] | [standalone]

Page 1 of 3  [1] 2 3  Next page →

Back to top | Article view | linux.debian.user

csiph-web