Groups > comp.os.linux.misc > #13209 > unrolled thread

How to force copy retries until finally copied all due to a dying old HDD in Linux?

Started by	Ant <ant@zimage.comANT>
First post	2015-01-04 06:55 -0800
Last post	2015-01-05 14:13 +0100
Articles	20 on this page of 55 — 11 participants

Back to article view | Back to comp.os.linux.misc

Page 2 of 3 — ← Prev page 1 [2] 3 Next page →

#13224 — Re: How to force copy retries until finally copied all due to a dying old HDD in Linux?

From	Robert Heller <heller@deepsoft.com>
Date	2015-01-04 12:26 -0600
Subject	Re: How to force copy retries until finally copied all due to a dying old HDD in Linux?
Message-ID	<7sKdnTXDfehKGDTJnZ2dnUU7-fmdnZ2d@giganews.com>
In reply to	#13221

At Sun, 4 Jan 2015 18:12:34 +0000 (UTC) William Unruh <unruh@invalid.ca> wrote:

> 
> On 2015-01-04, Ant <ant@zimage.comANT> wrote:
> > On 1/4/2015 7:39 AM, Lusotec wrote:
> >
> >> https://www.gnu.org/software/ddrescue/manual/ddrescue_manual.html
> >
> > "If you use a device or a partition as destination, any data stored 
> > there will be overwritten." I don't want to erase anything on my 
> > destinated drive (/dev/sdb*) though. I just wanted to copy a few 
> > unreadable files. I do NOT want to copy everything from the very old 
> > dying HDD. This is a lot more complex than I expected as a newbie. So, I 
> > can't copy files directly like cp command? I thought there was a way to 
> > do that to an existing HDD (don't want to mess up my destination drive). :(
> 
> If the data is important to you, go out and buy a new drive and use dd
> from that broken drive to the new one (nothing there to erase). Then
> afterwards reformat/partion the new drive and use it as spare storage
> space (eg backups!).
> 
> When a hard disk fails, it fails. The recording medium is scratched, the
> rust (iron oxide or ferrite) flakes off, etc. It is IMPOSSIBLE to
> recover the data there. dd can be told to ignore errors, on blocks, and
> just go on to the next one. cp cannot. 

Right.  Using 'dd conf=noerrors ...' lets you copy what you can to 'clean' 
media (or an image file).  Then you can mount the clean media (or an image 
file) and then recover what you can.

The *best* option is to *always* make regular backups in the first place. Then
when the drive fails (and it will, sooner or later), all of the data is on the
backup media (and it is best to make multiple backup copies on multiple
instantances of backup media of all of your important data).

> 
>                 

-- 
Robert Heller             -- 978-544-6933
Deepwoods Software        -- Custom Software Services
http://www.deepsoft.com/  -- Linux Administration Services
heller@deepsoft.com       -- Webhosting Services

[toc] | [prev] | [next] | [standalone]

#13237

From	The Natural Philosopher <tnp@invalid.invalid>
Date	2015-01-04 23:23 +0000
Message-ID	<m8ci1o$3q5$2@news.albasani.net>
In reply to	#13215

On 04/01/15 17:25, Ant wrote:
> I just wanted to copy a few unreadable files

Good luck with that one...

-- 
Everything you read in newspapers is absolutely true, except for the 
rare story of which you happen to have first-hand knowledge. – Erwin Knoll

[toc] | [prev] | [next] | [standalone]

#13260

From	Marc Haber <mh+usenetspam1118@zugschl.us>
Date	2015-01-05 14:11 +0100
Message-ID	<m8e2i9$kct$1@news1.tnib.de>
In reply to	#13215

Ant <ant@zimage.comANT> wrote:
>On 1/4/2015 7:39 AM, Lusotec wrote:
>> https://www.gnu.org/software/ddrescue/manual/ddrescue_manual.html
>
>"If you use a device or a partition as destination, any data stored 
>there will be overwritten." I don't want to erase anything on my 
>destinated drive (/dev/sdb*) though.

Then don't use a device or a partition as destination.

>I just wanted to copy a few unreadable files.

If they're unreadable, how do you want to read them?

Greetings
Marc
-- 
-------------------------------------- !! No courtesy copies, please !! -----
Marc Haber         |   " Questions are the         | Mailadresse im Header
Mannheim, Germany  |     Beginning of Wisdom "     | http://www.zugschlus.de/
Nordisch by Nature | Lt. Worf, TNG "Rightful Heir" | Fon: *49 621 72739834

[toc] | [prev] | [next] | [standalone]

#13220

From	William Unruh <unruh@invalid.ca>
Date	2015-01-04 18:07 +0000
Message-ID	<m8bvg9$h4v$3@dont-email.me>
In reply to	#13209

On 2015-01-04, Ant <ant@zimage.comANT> wrote:
> Hi!
>
> I have a very old PATA HDD that is dying due to SMART errors and I am 
> trying to copy my data to another working drive, but it has problems 
> reading some data. Here is an example:
>
> $ cp -R * ~/DebianPATAhddBackups/
> cp: cannot stat `Files/AntsImages': Input/output error
> cp: reading `Mac OS X.dmg': Input/output error
> cp: failed to extend `Mac OS X.dmg': Input/output error
>
> I assume my Debian stable was unable to copy those files. IIRC, there 
> was a way to retry over and over until copied fully. How do I do that? 
> It has been so many years I did this. :(

If the rust has flaked off the drive, there is nothing to copy, no
matter how many times you try. 

>
> Thank you in advance. :)

[toc] | [prev] | [next] | [standalone]

#13228

From	Ant <ant@zimage.comANT>
Date	2015-01-04 13:10 -0800
Message-ID	<8J6dnYfNS9L-MTTJnZ2dnUU7-T2dnZ2d@earthlink.com>
In reply to	#13220

On 1/4/2015 10:07 AM, William Unruh wrote:
>>
>> I have a very old PATA HDD that is dying due to SMART errors and I am
>> trying to copy my data to another working drive, but it has problems
>> reading some data. Here is an example:
>>
>> $ cp -R * ~/DebianPATAhddBackups/
>> cp: cannot stat `Files/AntsImages': Input/output error
>> cp: reading `Mac OS X.dmg': Input/output error
>> cp: failed to extend `Mac OS X.dmg': Input/output error
>>
>> I assume my Debian stable was unable to copy those files. IIRC, there
>> was a way to retry over and over until copied fully. How do I do that?
>> It has been so many years I did this. :(
>
> If the rust has flaked off the drive, there is nothing to copy, no
> matter how many times you try.

Even with ddrescue? I thought it retries until it finally copies.
-- 
"The great companies did not know that the line between hunger and anger 
is a thin line. And money that might have gone to wages went for gas, 
for guns, for agents and spies, for blacklists, for drilling. On the 
highways the people moved like ants and searched for work, for food. And 
the anger began to ferment." --John Steinbeck's The Grapes of Wrath
    /\___/\         Ant(Dude) @ http://antfarm.ma.cx (Personal Web Site)
   / /\ /\ \                Ant's Quality Foraged Links: http://aqfl.net
  | |o   o| |
     \ _ /        If crediting, then use Ant nickname and AQFL URL/link.
      ( )         If e-mailing, then axe ANT from its address if needed.
Ant is currently not listening to any songs on this computer.

[toc] | [prev] | [next] | [standalone]

#13232

From	William Unruh <unruh@invalid.ca>
Date	2015-01-04 22:09 +0000
Message-ID	<m8cdm8$f9s$1@dont-email.me>
In reply to	#13228

On 2015-01-04, Ant <ant@zimage.comANT> wrote:
> On 1/4/2015 10:07 AM, William Unruh wrote:
>>>
>>> I have a very old PATA HDD that is dying due to SMART errors and I am
>>> trying to copy my data to another working drive, but it has problems
>>> reading some data. Here is an example:
>>>
>>> $ cp -R * ~/DebianPATAhddBackups/
>>> cp: cannot stat `Files/AntsImages': Input/output error
>>> cp: reading `Mac OS X.dmg': Input/output error
>>> cp: failed to extend `Mac OS X.dmg': Input/output error
>>>
>>> I assume my Debian stable was unable to copy those files. IIRC, there
>>> was a way to retry over and over until copied fully. How do I do that?
>>> It has been so many years I did this. :(
>>
>> If the rust has flaked off the drive, there is nothing to copy, no
>> matter how many times you try.
>
> Even with ddrescue? I thought it retries until it finally copies.

ddrescue just allows you to conitue if there are errors (it essentially
retries once) . It does NOT
keep retrying until it reads it (well, you can tell it to try, but as
they say, that could damage the disk so that subsequent reads of
formerly good areas could go bad-- rust flakes between the drive head
and the platter do not do good things to the platter).
To try forever  would be stupid, because it can be
impossible to read it. As I said, data that is not there cannot be read,
no matter how hard you try.

[toc] | [prev] | [next] | [standalone]

#13238

From	Ant <ant@zimage.comANT>
Date	2015-01-04 15:25 -0800
Message-ID	<II2dnYwyh_hlVjTJnZ2dnUU7-YGdnZ2d@earthlink.com>
In reply to	#13232

>>>> I have a very old PATA HDD that is dying due to SMART errors and I am
>>>> trying to copy my data to another working drive, but it has problems
>>>> reading some data. Here is an example:
>>>>
>>>> $ cp -R * ~/DebianPATAhddBackups/
>>>> cp: cannot stat `Files/AntsImages': Input/output error
>>>> cp: reading `Mac OS X.dmg': Input/output error
>>>> cp: failed to extend `Mac OS X.dmg': Input/output error
>>>>
>>>> I assume my Debian stable was unable to copy those files. IIRC, there
>>>> was a way to retry over and over until copied fully. How do I do that?
>>>> It has been so many years I did this. :(
>>>
>>> If the rust has flaked off the drive, there is nothing to copy, no
>>> matter how many times you try.
>>
>> Even with ddrescue? I thought it retries until it finally copies.
>
> ddrescue just allows you to conitue if there are errors (it essentially
> retries once) . It does NOT
> keep retrying until it reads it (well, you can tell it to try, but as
> they say, that could damage the disk so that subsequent reads of
> formerly good areas could go bad-- rust flakes between the drive head
> and the platter do not do good things to the platter).
> To try forever  would be stupid, because it can be
> impossible to read it. As I said, data that is not there cannot be read,
> no matter how hard you try.

Ah OK. I was reading 
http://www.kalysto.org/utilities/dd_rhelp/index.en.html , and it sounded 
interesting. I am currently running ddrescue right now from the old 
drive to another old Seagate 80 GB PATA/IDE HDD that I found in one of 
my moving boxes.
-- 
"I am afraid that I am convinced that a community of human beings is a 
far more useful thing than a community of ants, and that if the human 
being is condemned and restricted to perform the same functions over and 
over again, he will not even be a good ant, not to mention a good human 
being." --Norbert Wiener
    /\___/\         Ant(Dude) @ http://antfarm.ma.cx (Personal Web Site)
   / /\ /\ \                Ant's Quality Foraged Links: http://aqfl.net
  | |o   o| |
     \ _ /        If crediting, then use Ant nickname and AQFL URL/link.
      ( )         If e-mailing, then axe ANT from its address if needed.
Ant is currently not listening to any songs on this computer.

[toc] | [prev] | [next] | [standalone]

#13338

From	Chris Davies <chris-usenet@roaima.co.uk>
Date	2015-01-06 20:25 +0000
Message-ID	<pustnbxl1s.ln2@news.roaima.co.uk>
In reply to	#13232

In comp.os.linux.misc William Unruh <unruh@invalid.ca> wrote:
> ddrescue just allows you to conitue if there are errors (it essentially
> retries once) . It does NOT keep retrying until it reads it [...]

You must have a different version of ddrescue to the one that I
use. http://www.gnu.org/software/ddrescue/ddrescue.html

Synopsis: ddrescue /path/to/broken/file /path/to/saved/copy /tmp/log.tmp

Don't omit the third parameter as ddrescue uses the "log file" to record
the current state of play with the copy.

Chris

[toc] | [prev] | [next] | [standalone]

#13233

From	Rich <rich@example.invalid>
Date	2015-01-04 22:12 +0000
Message-ID	<m8cdsa$d4d$1@dont-email.me>
In reply to	#13228

In comp.os.linux.misc Ant <ant@zimage.comant> wrote:
> On 1/4/2015 10:07 AM, William Unruh wrote:
> >>
> >> I have a very old PATA HDD that is dying due to SMART errors and I am
> >> trying to copy my data to another working drive, but it has problems
> >> reading some data. Here is an example:
> >>
> >> $ cp -R * ~/DebianPATAhddBackups/
> >> cp: cannot stat `Files/AntsImages': Input/output error
> >> cp: reading `Mac OS X.dmg': Input/output error
> >> cp: failed to extend `Mac OS X.dmg': Input/output error
> >>
> >> I assume my Debian stable was unable to copy those files. IIRC, there
> >> was a way to retry over and over until copied fully. How do I do that?
> >> It has been so many years I did this. :(
> >
> > If the rust has flaked off the drive, there is nothing to copy, no
> > matter how many times you try.

> Even with ddrescue? I thought it retries until it finally copies.

If the disk sector is corrupted, no amount of retries will make it
"finally copy".  It is simply gone.  Retries are only useful when there
is an intermittent issue that occurs sometimes, but not other times.

This is why having backups on separate media is so very important.

[toc] | [prev] | [next] | [standalone]

#13239

From	Ant <ant@zimage.comANT>
Date	2015-01-04 15:26 -0800
Message-ID	<II2dnY8yh_ixUTTJnZ2dnUU7-YGdnZ2d@earthlink.com>
In reply to	#13233

On 1/4/2015 2:12 PM, Rich wrote:

>> Even with ddrescue? I thought it retries until it finally copies.
>
> If the disk sector is corrupted, no amount of retries will make it
> "finally copy".  It is simply gone.  Retries are only useful when there
> is an intermittent issue that occurs sometimes, but not other times.

Ah, so http://www.kalysto.org/utilities/dd_rhelp/index.en.html is 
useless in this scenario. Interesting. Thanks.
-- 
"The great companies did not know that the line between hunger and anger 
is a thin line. And money that might have gone to wages went for gas, 
for guns, for agents and spies, for blacklists, for drilling. On the 
highways the people moved like ants and searched for work, for food. And 
the anger began to ferment." --John Steinbeck's The Grapes of Wrath
    /\___/\         Ant(Dude) @ http://antfarm.ma.cx (Personal Web Site)
   / /\ /\ \                Ant's Quality Foraged Links: http://aqfl.net
  | |o   o| |
     \ _ /        If crediting, then use Ant nickname and AQFL URL/link.
      ( )         If e-mailing, then axe ANT from its address if needed.
Ant is currently not listening to any songs on this computer.

[toc] | [prev] | [next] | [standalone]

#13244

From	Rich <rich@example.invalid>
Date	2015-01-05 00:21 +0000
Message-ID	<m8clev$81v$1@dont-email.me>
In reply to	#13239

In comp.os.linux.misc Ant <ant@zimage.comant> wrote:
> On 1/4/2015 2:12 PM, Rich wrote:

> >> Even with ddrescue? I thought it retries until it finally copies.
> >
> > If the disk sector is corrupted, no amount of retries will make it
> > "finally copy".  It is simply gone.  Retries are only useful when there
> > is an intermittent issue that occurs sometimes, but not other times.

> Ah, so http://www.kalysto.org/utilities/dd_rhelp/index.en.html is 
> useless in this scenario. Interesting. Thanks.

That simply runs ddrescue on the good parts of the disk first
(therefore you get the maximum amount of readable data copied over in
the shortest time span) before trying to read the bad parts multiple
times.

But nothing will change the physical fact that data which is unreadable
(true unreadable, not intermittent) is just that, unreadable, and no
amount of attempting to read it over and over will magically make it
readable.

[toc] | [prev] | [next] | [standalone]

#13247

From	Ant <ant@zimage.comANT>
Date	2015-01-04 18:32 -0800
Message-ID	<mc2dnemgRPFGajTJnZ2dnUU7-SGdnZ2d@earthlink.com>
In reply to	#13244

>>>> Even with ddrescue? I thought it retries until it finally copies.
>>>
>>> If the disk sector is corrupted, no amount of retries will make it
>>> "finally copy".  It is simply gone.  Retries are only useful when there
>>> is an intermittent issue that occurs sometimes, but not other times.
>
>> Ah, so http://www.kalysto.org/utilities/dd_rhelp/index.en.html is
>> useless in this scenario. Interesting. Thanks.
>
> That simply runs ddrescue on the good parts of the disk first
> (therefore you get the maximum amount of readable data copied over in
> the shortest time span) before trying to read the bad parts multiple
> times.
>
> But nothing will change the physical fact that data which is unreadable
> (true unreadable, not intermittent) is just that, unreadable, and no
> amount of attempting to read it over and over will magically make it
> readable.

Thank you for explaining that. It was technical to read the details. :)

FYI. I was running dd_rescue which was the wrong command from Reddit 
comments and here. So, I aborted it and saw:
...
Bad block: 29983
^C
dd_rescue: (fatal): Caught signal 2 "Interrupt". Exiting!
Summary for /dev/sda5 -> /dev/sdg1:
dd_rescue: (info): ipos:     15004.0k, opos:     15004.0k, xferd: 
15004.0k
                    errs:    336, errxfer:       168.0k, succxfer: 
14836.0k
              +curr.rate:        2kB/s, avg.rate:        2kB/s, 
avg.load:  0.0%


real    166m1.617s
user    0m0.004s
sys     0m0.084s

Ouch, 2 kB/sec! Slower than my crappy 3 kB/sec dial-up Internet 
connections! :( So, I started "ddrescue --force /dev/sda5 /dev/sdg1 
logfile" and recently saw:

# time ddrescue --force /dev/sda5 /dev/sdg1 logfile
GNU ddrescue 1.16
Press Ctrl-C to interrupt
rescued:     2955 MB,  errsize:    109 MB,  current rate:    1703 kB/s
rescued:    41020 MB,  errsize:    109 MB,  current rate:   22675 kB/s
    ipos:    41130 MB,   errors:      47,    average rate:    8898 kB/s
    opos:    41130 MB,     time since last successful read:       0 s
Copying non-tried blocks...

FYI, sdg(1=ext4 storage2 partition) is my old external USB2 HDD. Both 
failing drive and external USB2 HDD are not mounted right now for dd 
rescues.
-- 
"Have I told you how much I like ants, huh? Especially fried in a subtle 
blend of mech fluid and grated gears?" --Rampage to Inferno, 
"Transmutate" in Transformers (Beast Wars)
    /\___/\         Ant(Dude) @ http://antfarm.ma.cx (Personal Web Site)
   / /\ /\ \                Ant's Quality Foraged Links: http://aqfl.net
  | |o   o| |
     \ _ /        If crediting, then use Ant nickname and AQFL URL/link.
      ( )         If e-mailing, then axe ANT from its address if needed.
A song (i/wa)s playing on this computer: Don McLean - American Pie 
(Mastermix Classic Cuts 99 - Party)

[toc] | [prev] | [next] | [standalone]

#13250

From	Ant <ant@zimage.comANT>
Date	2015-01-04 19:36 -0800
Message-ID	<b6Wdnfc8M-01mzfJnZ2dnUU7-I2dnZ2d@earthlink.com>
In reply to	#13244

> FYI, sdg(1=ext4 storage2 partition) is my old external USB2 HDD. Both
> failing drive and external USB2 HDD are not mounted right now for dd
> rescues.

My ddrescue aborted or maybe it is done? I am not sure if it was caused 
by me while disabling Bluetooth (don't have any!) in KDE4's settings or 
not, so I looked around:

# time ddrescue --force /dev/sda5 /dev/sdg1 logfile
GNU ddrescue 1.16
Press Ctrl-C to interrupt
rescued:     2955 MB,  errsize:    109 MB,  current rate:    1703 kB/s
rescued:    79615 MB,  errsize:    109 MB,  current rate:      630 B/s
    ipos:     2147 MB,   errors:      47,    average rate:   11651 kB/s
    opos:     2147 MB,     time since last successful read:       0 s
Trimming failed blocks...
ddrescue: write error: Input/output error

real    113m52.396s
user    0m2.428s
sys     23m27.752s

--

dmesg:
...
[2015-01-04 19:04:59] ata1.00: exception Emask 0x0 SAct 0x0 SErr 0x0 
action 0x0
[2015-01-04 19:04:59] ata1.00: BMDMA stat 0x25
[2015-01-04 19:04:59] ata1.00: failed command: READ DMA
[2015-01-04 19:04:59] ata1.00: cmd c8/00:08:18:ca:4c/00:00:00:00:00/e0 
tag 0 dma 4096 in
[2015-01-04 19:04:59]          res 51/40:00:1c:ca:4c/00:00:00:00:00/e0 
Emask 0x9 (media error)
[2015-01-04 19:04:59] ata1.00: status: { DRDY ERR }
[2015-01-04 19:04:59] ata1.00: error: { UNC }
[2015-01-04 19:04:59] ata1.00: configured for UDMA/100
[2015-01-04 19:04:59] ata1: EH complete
[2015-01-04 19:05:03] ata1.00: exception Emask 0x0 SAct 0x0 SErr 0x0 
action 0x0
[2015-01-04 19:05:03] ata1.00: BMDMA stat 0x25
[2015-01-04 19:05:03] ata1.00: failed command: READ DMA
[2015-01-04 19:05:03] ata1.00: cmd c8/00:08:18:ca:4c/00:00:00:00:00/e0 
tag 0 dma 4096 in
[2015-01-04 19:05:03]          res 51/40:00:1c:ca:4c/00:00:00:00:00/e0 
Emask 0x9 (media error)
[2015-01-04 19:05:03] ata1.00: status: { DRDY ERR }
[2015-01-04 19:05:03] ata1.00: error: { UNC }
[2015-01-04 19:05:04] ata1.00: configured for UDMA/100
[2015-01-04 19:05:04] ata1: EH complete
[2015-01-04 19:05:08] ata1.00: exception Emask 0x0 SAct 0x0 SErr 0x0 
action 0x0
[2015-01-04 19:05:08] ata1.00: BMDMA stat 0x25
[2015-01-04 19:05:08] ata1.00: failed command: READ DMA
[2015-01-04 19:05:08] ata1.00: cmd c8/00:08:18:ca:4c/00:00:00:00:00/e0 
tag 0 dma 4096 in
[2015-01-04 19:05:08]          res 51/40:00:1c:ca:4c/00:00:00:00:00/e0 
Emask 0x9 (media error)
[2015-01-04 19:05:08] ata1.00: status: { DRDY ERR }
[2015-01-04 19:05:08] ata1.00: error: { UNC }
[2015-01-04 19:05:08] ata1.00: configured for UDMA/100
[2015-01-04 19:05:08] ata1: EH complete
[2015-01-04 19:05:13] ata1.00: exception Emask 0x0 SAct 0x0 SErr 0x0 
action 0x0
[2015-01-04 19:05:13] ata1.00: BMDMA stat 0x25
[2015-01-04 19:05:13] ata1.00: failed command: READ DMA
[2015-01-04 19:05:13] ata1.00: cmd c8/00:08:18:ca:4c/00:00:00:00:00/e0 
tag 0 dma 4096 in
[2015-01-04 19:05:13]          res 51/40:00:1c:ca:4c/00:00:00:00:00/e0 
Emask 0x9 (media error)
[2015-01-04 19:05:13] ata1.00: status: { DRDY ERR }
[2015-01-04 19:05:13] ata1.00: error: { UNC }
[2015-01-04 19:05:13] ata1.00: configured for UDMA/100
[2015-01-04 19:05:13] ata1: EH complete
[2015-01-04 19:05:18] ata1.00: exception Emask 0x0 SAct 0x0 SErr 0x0 
action 0x0
[2015-01-04 19:05:18] ata1.00: BMDMA stat 0x25
[2015-01-04 19:05:18] ata1.00: failed command: READ DMA
[2015-01-04 19:05:18] ata1.00: cmd c8/00:08:18:ca:4c/00:00:00:00:00/e0 
tag 0 dma 4096 in
[2015-01-04 19:05:18]          res 51/40:00:1c:ca:4c/00:00:00:00:00/e0 
Emask 0x9 (media error)
[2015-01-04 19:05:18] ata1.00: status: { DRDY ERR }
[2015-01-04 19:05:18] ata1.00: error: { UNC }
[2015-01-04 19:05:18] ata1.00: configured for UDMA/100
[2015-01-04 19:05:18] ata1: EH complete
[2015-01-04 19:05:22] ata1.00: exception Emask 0x0 SAct 0x0 SErr 0x0 
action 0x0
[2015-01-04 19:05:22] ata1.00: BMDMA stat 0x25
[2015-01-04 19:05:22] ata1.00: failed command: READ DMA
[2015-01-04 19:05:22] ata1.00: cmd c8/00:08:18:ca:4c/00:00:00:00:00/e0 
tag 0 dma 4096 in
[2015-01-04 19:05:22]          res 51/40:00:1c:ca:4c/00:00:00:00:00/e0 
Emask 0x9 (media error)
[2015-01-04 19:05:22] ata1.00: status: { DRDY ERR }
[2015-01-04 19:05:22] ata1.00: error: { UNC }
[2015-01-04 19:05:22] ata1.00: configured for UDMA/100
[2015-01-04 19:05:22] sd 0:0:0:0: [sda] Unhandled sense code
[2015-01-04 19:05:22] sd 0:0:0:0: [sda]  Result: hostbyte=DID_OK 
driverbyte=DRIVER_SENSE
[2015-01-04 19:05:22] sd 0:0:0:0: [sda]  Sense Key : Medium Error 
[current] [descriptor]
[2015-01-04 19:05:22] Descriptor sense data with sense descriptors (in hex):
[2015-01-04 19:05:22]         72 03 11 04 00 00 00 0c 00 0a 80 00 00 00 
00 00
[2015-01-04 19:05:22]         00 4c ca 1c
[2015-01-04 19:05:22] sd 0:0:0:0: [sda]  Add. Sense: Unrecovered read 
error - auto reallocate failed
[2015-01-04 19:05:22] sd 0:0:0:0: [sda] CDB: Read(10): 28 00 00 4c ca 18 
00 00 08 00
[2015-01-04 19:05:22] end_request: I/O error, dev sda, sector 5032476
[2015-01-04 19:05:22] Buffer I/O error on device sda5, logical block 556355
[2015-01-04 19:05:22] ata1: EH complete
[2015-01-04 19:05:48] sd 11:0:0:0: [sdg] Unhandled sense code
[2015-01-04 19:05:48] sd 11:0:0:0: [sdg]  Result: hostbyte=DID_ERROR 
driverbyte=DRIVER_SENSE
[2015-01-04 19:05:48] sd 11:0:0:0: [sdg]  Sense Key : Hardware Error 
[current]
[2015-01-04 19:05:48] sd 11:0:0:0: [sdg]  Add. Sense: No additional 
sense information
[2015-01-04 19:05:48] sd 11:0:0:0: [sdg] CDB: Read(10): 28 00 00 40 08 
58 00 00 08 00
[2015-01-04 19:05:48] end_request: I/O error, dev sdg, sector 4196440
...

I reran it, and it failed again and I didn't do anything to cause it:
# time ddrescue --force /dev/sda5 /dev/sdg1 logfile
GNU ddrescue 1.16
Press Ctrl-C to interrupt
Initial status (read from logfile)
rescued:    79615 MB,  errsize:    109 MB,  errors:      47
Current status
rescued:    79615 MB,  errsize:    109 MB,  current rate:        0 B/s
    ipos:     2147 MB,   errors:      47,    average rate:        0 B/s
    opos:     2147 MB,     time since last successful read:      25 s
Trimming failed blocks...
ddrescue: write error: Input/output error

real    0m24.515s
user    0m0.000s
sys     0m0.000s


dmesg:
...
[2015-01-04 19:13:44] kjournald starting.  Commit interval 5 seconds
[2015-01-04 19:13:44] EXT3-fs (sdg1): warning: mounting fs with errors, 
running e2fsck is recommended
[2015-01-04 19:13:44] EXT3-fs (sdg1): using internal journal
[2015-01-04 19:13:44] EXT3-fs (sdg1): mounted filesystem with ordered 
data mode
[2015-01-04 19:15:27] sd 11:0:0:0: [sdg] Unhandled sense code
[2015-01-04 19:15:27] sd 11:0:0:0: [sdg]  Result: hostbyte=DID_ERROR 
driverbyte=DRIVER_SENSE
[2015-01-04 19:15:27] sd 11:0:0:0: [sdg]  Sense Key : Hardware Error 
[current]
[2015-01-04 19:15:27] sd 11:0:0:0: [sdg]  Add. Sense: No additional 
sense information
[2015-01-04 19:15:27] sd 11:0:0:0: [sdg] CDB: Read(10): 28 00 00 40 08 
58 00 00 08 00
[2015-01-04 19:15:27] end_request: I/O error, dev sdg, sector 4196440

I mounted both drives and did a df:
$ df
Filesystem                                             1K-blocks 
Used Available Use% Mounted on
rootfs                                                    960504 
733580    178132  81% /
udev                                                       10240 
0     10240   0% /dev
tmpfs                                                     206056 
11356    194700   6% /run
/dev/disk/by-uuid/242d81f2-8a0d-48e1-a144-22ab4d2b9407    960504 
733580    178132  81% /
tmpfs                                                       5120 
8      5112   1% /run/lock
tmpfs                                                    1193200 
80   1193120   1% /run/shm
/dev/sdb9                                               50013916 
16658680  30814644  36% /home
/dev/sdb5                                                 960504 
36180    875532   4% /tmp
/dev/sdb8                                               49982172 
6441960  41001204  14% /usr
/dev/sdb6                                                4804736 
1738124   2822544  39% /var
/dev/sda5                                               76637568 
31256040  41488692  43% /media/_others
/dev/sdg1                                               76637568 
31256040  41488692  43% /media/_others_

Is it done since they look the same? I briefly looked at the copied 
data, and they seem OK? Also, its logfile:
$ cat logfile
# Rescue Logfile. Created by GNU ddrescue version 1.16
# Command line: ddrescue --force /dev/sda5 /dev/sdg1 logfile
# current_pos  current_status
0x8000C000     *
#      pos        size  status
0x00000000  0x00E18000  +
0x00E18000  0x00008000  *
0x00E20000  0x00000200  -
0x00E20200  0x00010000  *
0x00E30200  0x00000200  -
0x00E30400  0x00020000  *
0x00E50400  0x00002C00  +
0x00E53000  0x00000200  -
0x00E53200  0x0003D400  *
0x00E90600  0x00000A00  +
0x00E91000  0x00000200  -
0x00E91200  0x00079E00  *
0x00F0B000  0x00004000  +
0x00F0F000  0x00000200  -
0x00F0F200  0x000EFC00  *
0x00FFEE00  0x00000200  -
0x00FFF000  0x001DF800  *
0x011DE800  0x00009800  +
0x011E8000  0x00000200  -
0x011E8200  0x003B5800  *
0x0159DA00  0x0001D600  +
0x015BB000  0x00000200  -
0x015BB200  0x0074DA00  *
0x01D08C00  0x00000400  +
0x01D09000  0x00000200  -
0x01D09200  0x00E9B000  *
0x02BA4200  0x002B0E00  +
0x02E55000  0x00000200  -
0x02E55200  0x01A85200  *
0x048DA400  0x005B9C00  +
0x04E94000  0x00000200  -
0x04E94200  0x02F50800  *
0x07DE4A00  0x0BD9B600  +
0x13B80000  0x00010000  *
0x13B90000  0x0B9B1000  +
0x1F541000  0x0000F000  *
0x1F550000  0x04986000  +
0x23ED6000  0x0000A000  *
0x23EE0000  0x03F58000  +
0x27E38000  0x00008000  *
0x27E40000  0x03B95000  +
0x2B9D5000  0x0000B000  *
0x2B9E0000  0x05DF5000  +
0x317D5000  0x0000B000  *
0x317E0000  0x04318000  +
0x35AF8000  0x00008000  *
0x35B00000  0x0A380000  +
0x3FE80000  0x00010000  *
0x3FE90000  0x01D3A000  +
0x41BCA000  0x00006000  *
0x41BD0000  0x0A030000  +
0x4BC00000  0x00010000  *
0x4BC10000  0x000DC000  +
0x4BCEC000  0x00004000  *
0x4BCF0000  0x04310000  +
0x50000000  0x00010000  *
0x50010000  0x07A75000  +
0x57A85000  0x0000B000  *
0x57A90000  0x0852A000  +
0x5FFBA000  0x00006000  *
0x5FFC0000  0x01BE8000  +
0x61BA8000  0x00008000  *
0x61BB0000  0x008D0000  +
0x62480000  0x00010000  *
0x62490000  0x01DC4000  +
0x64254000  0x0000C000  *
0x64260000  0x003B6000  +
0x64616000  0x0000A000  *
0x64620000  0x0145F000  +
0x65A7F000  0x00001000  *
0x65A80000  0x00784000  +
0x66204000  0x0000C000  *
0x66210000  0x0032D000  +
0x6653D000  0x00003000  *
0x66540000  0x01AC0000  +
0x68000000  0x00010000  *
0x68010000  0x027F0000  +
0x6A800000  0x00010000  *
0x6A810000  0x01270000  +
0x6BA80000  0x00010000  *
0x6BA90000  0x08F70000  +
0x74A00000  0x00010000  *
0x74A10000  0x0816C000  +
0x7CB7C000  0x00004000  *
0x7CB80000  0x03482000  +
0x80002000  0x0000A000  *
0x8000C000  0x07D37000  +
0x87D43000  0x00000E00  /
0x87D43E00  0x00000200  -
0x87D44000  0x0043C000  +
0x88180000  0x00000E00  /
0x88180E00  0x00000200  -
0x88181000  0x06954000  +
0x8EAD5000  0x00000E00  /
0x8EAD5E00  0x00000200  -
0x8EAD6000  0x0832A000  +
0x96E00000  0x00000E00  /
0x96E00E00  0x00000200  -
0x96E01000  0x011FF000  +
0x98000000  0x00000E00  /
0x98000E00  0x00000200  -
0x98001000  0x06383000  +
0x9E384000  0x00000E00  /
0x9E384E00  0x00000200  -
0x9E385000  0x066FB000  +
0xA4A80000  0x00000E00  /
0xA4A80E00  0x00000200  -
0xA4A81000  0x0DDF5000  +
0xB2876000  0x00000E00  /
0xB2876E00  0x00000200  -
0xB2877000  0x03DC3000  +
0xB663A000  0x00000E00  /
0xB663AE00  0x00000200  -
0xB663B000  0x006FB000  +
0xB6D36000  0x00000E00  /
0xB6D36E00  0x00000200  -
0xB6D37000  0x021D6000  +
0xB8F0D000  0x00000E00  /
0xB8F0DE00  0x00000200  -
0xB8F0E000  0x11D70F2000  +

What do you think?
-- 
"What, like I had to live with all those ants? Do you know what I did to 
those ants? HoooHooo! No more ants!" --unknown
    /\___/\         Ant(Dude) @ http://antfarm.ma.cx (Personal Web Site)
   / /\ /\ \                Ant's Quality Foraged Links: http://aqfl.net
  | |o   o| |
     \ _ /        If crediting, then use Ant nickname and AQFL URL/link.
      ( )         If e-mailing, then axe ANT from its address if needed.
A song (i/wa)s playing on this computer: Tears For Fears - Mad World 
(World Mix)

[toc] | [prev] | [next] | [standalone]

#13318

From	Robert Nichols <SEE_SIGNATURE@localhost.localdomain.invalid>
Date	2015-01-06 08:58 -0600
Message-ID	<m8gt6q$92t$1@omega-3a.local>
In reply to	#13250

On 01/04/2015 09:36 PM, Ant wrote:
> [2015-01-04 19:05:48] sd 11:0:0:0: [sdg] Unhandled sense code
> [2015-01-04 19:05:48] sd 11:0:0:0: [sdg]  Result: hostbyte=DID_ERROR
> driverbyte=DRIVER_SENSE
> [2015-01-04 19:05:48] sd 11:0:0:0: [sdg]  Sense Key : Hardware Error [current]
> [2015-01-04 19:05:48] sd 11:0:0:0: [sdg]  Add. Sense: No additional sense
> information
> [2015-01-04 19:05:48] sd 11:0:0:0: [sdg] CDB: Read(10): 28 00 00 40 08 58 00 00
> 08 00
> [2015-01-04 19:05:48] end_request: I/O error, dev sdg, sector 4196440
> ...
>
> I reran it, and it failed again and I didn't do anything to cause it:
> # time ddrescue --force /dev/sda5 /dev/sdg1 logfile
> GNU ddrescue 1.16
> Press Ctrl-C to interrupt
> Initial status (read from logfile)
> rescued:    79615 MB,  errsize:    109 MB,  errors:      47
> Current status
> rescued:    79615 MB,  errsize:    109 MB,  current rate:        0 B/s
>     ipos:     2147 MB,   errors:      47,    average rate:        0 B/s
>     opos:     2147 MB,     time since last successful read:      25 s
> Trimming failed blocks...
> ddrescue: write error: Input/output error
>
> real    0m24.515s
> user    0m0.000s
> sys     0m0.000s

That's bad! It says that your destination drive (sdg1) is also having
problems.

-- 
Bob Nichols         AT comcast.net I am "RNichols42"

[toc] | [prev] | [next] | [standalone]

#13392

From	Ant <ant@zimage.comANT>
Date	2015-01-09 18:33 -0800
Message-ID	<D-adnXEAA_EIEi3JnZ2dnUU7-e8AAAAA@earthlink.com>
In reply to	#13244

I resumed this issue since I finally found free time. I decided to do an 
image from scratch since people told me to do that instead, but it looks 
like I run out of disk space on the destination external USB2 HDD. This 
probably why it failed from partition to partition too last weekend. The 
original data used is about 30 GB. Why would it need to fill up the 
whole drive?

# time ddrescue -r -1 /dev/sda5 /media/storage2/CopiedDyingSDA5 ImageFileLog
GNU ddrescue 1.16
Press Ctrl-C to interrupt
rescued:    78579 MB,  errsize:    119 MB,  current rate:   14811 kB/s
    ipos:    78698 MB,   errors:      46,    average rate:   12289 kB/s
    opos:    78698 MB,     time since last successful read:       0 s
Copying non-tried blocks...
ddrescue: write error: No space left on device

real    106m34.617s
user    0m2.536s
sys     23m53.486s


Isn't 70 GB free enough for this? How much disk space do I really need 
for this? This is sure frustrating. :(
-- 
"Hey dad, look. I put honey on my back, and now the ants are carrying me 
home." --Chris from Family Guy
    /\___/\         Ant(Dude) @ http://antfarm.ma.cx (Personal Web Site)
   / /\ /\ \                Ant's Quality Foraged Links: http://aqfl.net
  | |o   o| |
     \ _ /        If crediting, then use Ant nickname and AQFL URL/link.
      ( )         If e-mailing, then axe ANT from its address if needed.
Ant is currently not listening to any songs on this computer.

[toc] | [prev] | [next] | [standalone]

#13394

From	Robert Nichols <SEE_SIGNATURE@localhost.localdomain.invalid>
Date	2015-01-10 08:21 -0600
Message-ID	<m8rcgt$3um$1@omega-3a.local>
In reply to	#13392

On 01/09/2015 08:33 PM, Ant wrote:
> I resumed this issue since I finally found free time. I decided to do an image
> from scratch since people told me to do that instead, but it looks like I run
> out of disk space on the destination external USB2 HDD. This probably why it
> failed from partition to partition too last weekend. The original data used is
> about 30 GB. Why would it need to fill up the whole drive?
>
> # time ddrescue -r -1 /dev/sda5 /media/storage2/CopiedDyingSDA5 ImageFileLog
> GNU ddrescue 1.16
> Press Ctrl-C to interrupt
> rescued:    78579 MB,  errsize:    119 MB,  current rate:   14811 kB/s
>     ipos:    78698 MB,   errors:      46,    average rate:   12289 kB/s
>     opos:    78698 MB,     time since last successful read:       0 s
> Copying non-tried blocks...
> ddrescue: write error: No space left on device
>
> real    106m34.617s
> user    0m2.536s
> sys     23m53.486s
>
>
> Isn't 70 GB free enough for this? How much disk space do I really need for this?
> This is sure frustrating. :(

How large is partition /dev/sda5? That's the whole partition, regardless of how
much space is or is not "used" in the filesystem. That is the amount of space
you need at the destination. /ddrescue/ neither knows nor cares about filesystem
structure. It copies the whole partition.

-- 
Bob Nichols         AT comcast.net I am "RNichols42"

[toc] | [prev] | [next] | [standalone]

#13395

From	Ant <ant@zimage.comANT>
Date	2015-01-10 07:47 -0800
Message-ID	<ZqKdnVdvz4021CzJnZ2dnUU7-QednZ2d@earthlink.com>
In reply to	#13394

On 1/10/2015 6:21 AM, Robert Nichols wrote:

> How large is partition /dev/sda5? That's the whole partition, regardless
> of how
> much space is or is not "used" in the filesystem. That is the amount of
> space
> you need at the destination. /ddrescue/ neither knows nor cares about
> filesystem
> structure. It copies the whole partition.

Oh, that is why! Dang it! So, my destinated drive/partition isn't big 
enough due to imaging it (not just used data) even though both drives 
are the same size. :( As you requested:

$ df -h
Filesystem                                              Size  Used Avail 
Use% Mounted on
rootfs                                                  938M  717M  174M 
  81% /
udev                                                     10M     0   10M 
   0% /dev
tmpfs                                                   202M  6.9M  195M 
   4% /run
/dev/disk/by-uuid/242d81f2-8a0d-48e1-a144-22ab4d2b9407  938M  717M  174M 
  81% /
tmpfs                                                   5.0M  4.0K  5.0M 
   1% /run/lock
tmpfs                                                   1.2G   80K  1.2G 
   1% /run/shm
/dev/sdb9                                                48G   16G   30G 
  36% /home
/dev/sdb5                                               938M   24M  867M 
   3% /tmp
/dev/sdb8                                                48G  6.2G   40G 
  14% /usr
/dev/sdb6                                               4.6G  1.8G  2.7G 
  40% /var
/dev/sdg1                                                74G  180M   70G 
   1% /media/storage2
/dev/sda5                                                74G   30G   40G 
  43% /media/_others

FYI. sda5 (an ext3 partition on the dying drive I care for) is not 
mounted, but /media/storage2 (ext4) is mounted in case that matters.

I do have another external USB2 drive, but it was formatted in FAT32 and 
has data. Can ddrescue use it to make an image file? I don't want to 
lose any data on it though.
-- 
"The ants sought personal revenge for my having sprayed them the day 
before." --Oliver Smith
    /\___/\         Ant(Dude) @ http://antfarm.ma.cx (Personal Web Site)
   / /\ /\ \                Ant's Quality Foraged Links: http://aqfl.net
  | |o   o| |
     \ _ /        If crediting, then use Ant nickname and AQFL URL/link.
      ( )         If e-mailing, then axe ANT from its address if needed.
Ant is currently not listening to any songs on this computer.

[toc] | [prev] | [next] | [standalone]

#13398

From	Gernot Fink <g.fink@gmx.net>
Date	2015-01-10 20:45 +0100
Message-ID	<34c8ob-rp6.ln1@garv.home>
In reply to	#13395

In article <ZqKdnVdvz4021CzJnZ2dnUU7-QednZ2d@earthlink.com>,
	Ant <ant@zimage.comANT> writes:
> /dev/sdg1                                                74G  180M   70G 
>    1% /media/storage2
> /dev/sda5                                                74G   30G   40G 
>   43% /media/_others

Sorce = 74G
freee space on destination is 70G plus some "root reserved" space.
Its to small.

Gernot

[toc] | [prev] | [next] | [standalone]

#13404

From	Robert Nichols <SEE_SIGNATURE@localhost.localdomain.invalid>
Date	2015-01-11 08:04 -0600
Message-ID	<m8tvte$679$1@omega-3a.local>
In reply to	#13395

On 01/10/2015 09:47 AM, Ant wrote:
> On 1/10/2015 6:21 AM, Robert Nichols wrote:
>
>> How large is partition /dev/sda5? That's the whole partition, regardless
>> of how
>> much space is or is not "used" in the filesystem. That is the amount of
>> space
>> you need at the destination. /ddrescue/ neither knows nor cares about
>> filesystem
>> structure. It copies the whole partition.
>
> Oh, that is why! Dang it! So, my destinated drive/partition isn't big enough due
> to imaging it (not just used data) even though both drives are the same size. :(
> As you requested:
>
> $ df -h
> Filesystem                                              Size  Used Avail Use%
> Mounted on
> rootfs                                                  938M  717M  174M  81% /
> udev                                                     10M     0   10M   0% /dev
> tmpfs                                                   202M  6.9M  195M   4% /run
> /dev/disk/by-uuid/242d81f2-8a0d-48e1-a144-22ab4d2b9407  938M  717M  174M  81% /
> tmpfs                                                   5.0M  4.0K  5.0M   1%
> /run/lock
> tmpfs                                                   1.2G   80K  1.2G   1%
> /run/shm
> /dev/sdb9                                                48G   16G   30G  36% /home
> /dev/sdb5                                               938M   24M  867M   3% /tmp
> /dev/sdb8                                                48G  6.2G   40G  14% /usr
> /dev/sdb6                                               4.6G  1.8G  2.7G  40% /var
> /dev/sdg1                                                74G  180M   70G   1%
> /media/storage2
> /dev/sda5                                                74G   30G   40G  43%
> /media/_others
>
> FYI. sda5 (an ext3 partition on the dying drive I care for) is not mounted, but
> /media/storage2 (ext4) is mounted in case that matters.
>
> I do have another external USB2 drive, but it was formatted in FAT32 and has
> data. Can ddrescue use it to make an image file? I don't want to lose any data
> on it though.

FAT32 can't hold a 74GB file. The maximum file size for FAT32 is 4GB
minus 1 byte. /dev/sdg1 *might* be large enough for the image if you
unmounted it and went straight to the raw partition, but that's going to
destroy the current contents of /dev/sdg1. I can't tell from the above
info. Compare the two sizes when you run "grep -E 'sda5|sdg1'
/proc/partitions" or the sizes you see when you run "fdisk -lu /dev/sda
/dev/sdg".

-- 
Bob Nichols         AT comcast.net I am "RNichols42"

[toc] | [prev] | [next] | [standalone]

#13406

From	Ant <ant@zimage.comANT>
Date	2015-01-11 10:14 -0800
Message-ID	<ZrGdnYsSWdAAIC_JnZ2dnUU7-QOdnZ2d@earthlink.com>
In reply to	#13404

On 1/11/2015 6:04 AM, Robert Nichols wrote:
...
> FAT32 can't hold a 74GB file. The maximum file size for FAT32 is 4GB
> minus 1 byte. /dev/sdg1 *might* be large enough for the image if you
> unmounted it and went straight to the raw partition, but that's going to
> destroy the current contents of /dev/sdg1. I can't tell from the above
> info. Compare the two sizes when you run "grep -E 'sda5|sdg1'
> /proc/partitions" or the sizes you see when you run "fdisk -lu /dev/sda
> /dev/sdg".

$ df | grep -E 'sda5|sdg1'
/dev/sdg1                                               76921888 
184068  72830340   1% /media/storage2
/dev/sda5                                               76637568 
31256040  41488692  43% /media/_others


# fdisk -lu /dev/sda /dev/sdg

Disk /dev/sda: 80.0 GB, 80026361856 bytes
255 heads, 63 sectors/track, 9729 cylinders, total 156301488 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x6a3a6846

    Device Boot      Start         End      Blocks   Id  System
/dev/sda2          578340   156296384    77859022+   5  Extended
/dev/sda5          581632   156295167    77856768   83  Linux

Disk /dev/sdg: 80.0 GB, 80026361856 bytes
255 heads, 63 sectors/track, 9729 cylinders, total 156301488 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x1c7391d2

    Device Boot      Start         End      Blocks   Id  System
/dev/sdg1            2048   156301311    78149632   83  Linux


As for ummounted sdg1, I think I did that when I tried to copy sda5 
partition into it (was an ext4, but turned into ext3), but not as an 
image. It failed as shown in 
http://pastie.org/pastes/9813433/text?key=y5rqiou5fsukcgvngnamw ... I'll 
try again as an image file this time and follow-up. :)
-- 
    /\___/\ +1 Year Ant(Dude) @ http://antfarm.ma.cx (Personal Web Site)
   / /\ /\ \                Ant's Quality Foraged Links: http://aqfl.net
  | |o   o| |
     \ _ /        If crediting, then use Ant nickname and AQFL URL/link.
      ( )         If e-mailing, then axe ANT from its address if needed.
Ant is currently not listening to any songs on this computer.

[toc] | [prev] | [next] | [standalone]

Page 2 of 3 — ← Prev page 1 [2] 3 Next page →

csiph-web

How to force copy retries until finally copied all due to a dying old HDD in Linux?

Contents

#13224 — Re: How to force copy retries until finally copied all due to a dying old HDD in Linux?

#13237

#13260

#13220

#13228

#13232

#13238

#13338

#13233

#13239

#13244

#13247

#13250

#13318

#13392

#13394

#13395

#13398

#13404

#13406