Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.user > #223256 > unrolled thread

How long will this take?

Back to article view | Back to linux.debian.user

Contents

  How long will this take? Matthew Campbell <trenix25@pm.me> - 2020-06-08 22:30 +0200
    Re: How long will this take? Dan Ritter <dsr@randomstring.org> - 2020-06-08 22:40 +0200
      Re: How long will this take? deloptes <deloptes@gmail.com> - 2020-06-08 23:00 +0200
        Re: How long will this take? Dan Ritter <dsr@randomstring.org> - 2020-06-09 00:10 +0200
          Re: How long will this take? Michael Stone <mstone@debian.org> - 2020-06-10 16:10 +0200
            Re: How long will this take? Michael Stone <mstone@debian.org> - 2020-06-10 17:00 +0200
            Re: How long will this take? Andrei POPESCU <andreimpopescu@gmail.com> - 2020-06-10 17:00 +0200
            Re: How long will this take? David Christensen <dpchrist@holgerdanske.com> - 2020-06-11 02:40 +0200
      Re: How long will this take? Jude DaShiell <jdashiel@panix.com> - 2020-06-09 02:20 +0200
        Re: How long will this take? Andrei POPESCU <andreimpopescu@gmail.com> - 2020-06-09 09:40 +0200
    Re: How long will this take? Nicolas George <george@nsup.org> - 2020-06-08 22:40 +0200
      Re: How long will this take? Matthew Campbell <trenix25@pm.me> - 2020-06-08 23:10 +0200
        Re: How long will this take? Nicolas George <george@nsup.org> - 2020-06-08 23:20 +0200
    Re: How long will this take? David Wright <deblis@lionunicorn.co.uk> - 2020-06-09 05:10 +0200
      Re: How long will this take? Christopher David Howie <me@chrishowie.com> - 2020-06-09 06:00 +0200
        Re: How long will this take? Nicolas George <george@nsup.org> - 2020-06-09 11:50 +0200
          Fw: How long will this take? Matthew Campbell <trenix25@pm.me> - 2020-06-09 17:40 +0200
          Re: How long will this take? Christopher David Howie <me@chrishowie.com> - 2020-06-10 02:50 +0200
      Re: How long will this take? Michael Stone <mstone@debian.org> - 2020-06-10 16:20 +0200
        Re: How long will this take? David Wright <deblis@lionunicorn.co.uk> - 2020-06-10 19:10 +0200
          Fw: How long will this take? Matthew Campbell <trenix25@pm.me> - 2020-06-10 19:20 +0200
            Re: Fw: How long will this take? David Christensen <dpchrist@holgerdanske.com> - 2020-06-11 03:10 +0200
              Fw: Fw: How long will this take? Matthew Campbell <trenix25@pm.me> - 2020-06-11 20:00 +0200
                Re: Fw: Fw: How long will this take? Dan Ritter <dsr@randomstring.org> - 2020-06-11 20:10 +0200
                  Re: Fw: Fw: How long will this take? Kenneth Parker <sea7kenp@gmail.com> - 2020-06-11 22:20 +0200
                  Re: Fw: Fw: How long will this take? "Rick Thomas" <rick.thomas@pobox.com> - 2020-06-13 11:20 +0200
          Re: How long will this take? Michael Stone <mstone@debian.org> - 2020-06-10 21:00 +0200
            [OT] How to help the OP? (was: How long will this take?) l0f4r0@tuta.io - 2020-06-10 21:30 +0200
              Re: [OT] How to help the OP? (was: How long will this take?) Nicolas George <george@nsup.org> - 2020-06-10 21:30 +0200
                Re: [OT] How to help the OP? (was: How long will this take?) Nicolas George <george@nsup.org> - 2020-06-10 22:00 +0200
                  Re: [OT] How to help the OP? (was: How long will this take?) Michael Stone <mstone@debian.org> - 2020-06-10 22:00 +0200
                    Re: [OT] How to help the OP? (was: How long will this take?) Nicolas George <george@nsup.org> - 2020-06-10 22:30 +0200
                    Re: [OT] How to help the OP? (was: How long will this take?) deloptes <deloptes@gmail.com> - 2020-06-11 07:30 +0200
                      Re: [OT] How to help the OP? (was: How long will this take?) <tomas@tuxteam.de> - 2020-06-11 07:50 +0200
                        Re: [OT] How to help the OP? (was: How long will this take?) deloptes <deloptes@gmail.com> - 2020-06-11 11:00 +0200
                          Re: [OT] How to help the OP? (was: How long will this take?) <tomas@tuxteam.de> - 2020-06-11 11:40 +0200
                          Re: [OT] How to help the OP? (was: How long will this take?) Michael Stone <mstone@debian.org> - 2020-06-11 14:00 +0200
                            Re: [OT] How to help the OP? (was: How long will this take?) Nicolas George <george@nsup.org> - 2020-06-11 14:10 +0200
                Re: [OT] How to help the OP? (was: How long will this take?) Michael Stone <mstone@debian.org> - 2020-06-10 22:00 +0200
              Re: [OT] How to help the OP? (was: How long will this take?) Michael Stone <mstone@debian.org> - 2020-06-10 21:50 +0200
                Re: [OT] How to help the OP? (was: How long will this take?) David Wright <deblis@lionunicorn.co.uk> - 2020-06-12 04:00 +0200
            Re: How long will this take? David Wright <deblis@lionunicorn.co.uk> - 2020-06-12 04:00 +0200
              Re: How long will this take? Michael Stone <mstone@debian.org> - 2020-06-12 14:00 +0200
                Re: How long will this take? David Wright <deblis@lionunicorn.co.uk> - 2020-06-19 04:20 +0200
                  Re: How long will this take? David Christensen <dpchrist@holgerdanske.com> - 2020-06-20 00:00 +0200
                    Re: How long will this take? David Wright <deblis@lionunicorn.co.uk> - 2020-06-26 15:10 +0200
                      Re: How long will this take? David Christensen <dpchrist@holgerdanske.com> - 2020-06-27 00:10 +0200
                        Re: How long will this take? David Wright <deblis@lionunicorn.co.uk> - 2020-06-27 03:30 +0200
                          Re: How long will this take? David Christensen <dpchrist@holgerdanske.com> - 2020-06-27 05:00 +0200
                            Re: How long will this take? David Wright <deblis@lionunicorn.co.uk> - 2020-07-02 03:10 +0200
    Re: How long will this take? David Christensen <dpchrist@holgerdanske.com> - 2020-06-09 09:10 +0200
    Re: How long will this take? l0f4r0@tuta.io - 2020-06-09 20:10 +0200
      Re: How long will this take? Nicolas George <george@nsup.org> - 2020-06-09 20:30 +0200
        Re: How long will this take? Jude DaShiell <jdashiel@panix.com> - 2020-06-09 21:50 +0200
          Re: How long will this take? Nicolas George <george@nsup.org> - 2020-06-09 21:50 +0200
        Re: How long will this take? Anders Andersson <pipatron@gmail.com> - 2020-06-10 12:50 +0200
          Re: How long will this take? Nicolas George <george@nsup.org> - 2020-06-10 13:20 +0200
            Re: How long will this take? Anders Andersson <pipatron@gmail.com> - 2020-06-10 15:30 +0200
              Re: How long will this take? Nicolas George <george@nsup.org> - 2020-06-10 15:40 +0200
                Re: How long will this take? Anders Andersson <pipatron@gmail.com> - 2020-06-10 16:00 +0200
                  Re: How long will this take? Nicolas George <george@nsup.org> - 2020-06-10 16:10 +0200
                    Re: How long will this take? Anders Andersson <pipatron@gmail.com> - 2020-06-10 22:10 +0200
                  Fw: How long will this take? Matthew Campbell <trenix25@pm.me> - 2020-06-10 19:00 +0200
      Re: How long will this take? Jude DaShiell <jdashiel@panix.com> - 2020-06-09 20:30 +0200
    Re: How long will this take? Seeds Notoneofmy <notoneofmyseeds@gmx.de> - 2020-06-27 01:40 +0200
      Re: How long will this take? rhkramer@gmail.com - 2020-06-27 03:30 +0200
        Re: How long will this take? Seeds Notoneofmy <notoneofmyseeds@gmx.de> - 2020-06-27 03:50 +0200
          Re: How long will this take? rhkramer@gmail.com - 2020-06-27 20:10 +0200
      Re: How long will this take? Andrei POPESCU <andreimpopescu@gmail.com> - 2020-06-27 10:40 +0200
      On Subject drift [was: How long will this take?] <tomas@tuxteam.de> - 2020-06-27 11:00 +0200
        Re: On Subject drift [was: How long will this take?] l0f4r0@tuta.io - 2020-06-27 11:40 +0200
        Re: On Subject drift [was: How long will this take?] Seeds Notoneofmy <notoneofmyseeds@gmx.de> - 2020-06-27 16:40 +0200
          Re: On Subject drift [was: How long will this take?] <tomas@tuxteam.de> - 2020-06-27 18:50 +0200

Page 1 of 4  [1] 2 3 4  Next page →

#223256 — How long will this take?

[Multipart message — attachments visible in raw view] — view raw

I bought a new 4 terrabyte hard drive that is connected with a USB cable using USB2. It took about 32 hours to read every sector on the drive to look for bad sectors. I started blanking the sectors using /dev/zero last Friday night. It still isn't done. Is there a way I can find out how much data a particular process has written to the disk? I'm using Debian 10.4.

dd if=/dev/zero of=/dev/sdb ibs=4096 count=976754646

name=Matthew%20Campbell&email=trenix25%40pm.me

[toc] | [next] | [standalone]

#223257

Matthew Campbell wrote: 
> I bought a new 4 terrabyte hard drive that is connected with a USB cable using USB2. It took about 32 hours to read every sector on the drive to look for bad sectors. I started blanking the sectors using /dev/zero last Friday night. It still isn't done. Is there a way I can find out how much data a particular process has written to the disk? I'm using Debian 10.4.
> 
> dd if=/dev/zero of=/dev/sdb ibs=4096 count=976754646

USB2 disks are good for about 25MB/s.

4 seconds gets you 100MB.

40 seconds gets you 1000MB.

4000 * 40 seconds is 160000 seconds, so that's not quite two
days.

Is something wrong? Based on current news reports, I would say
you accidentally purchased an SMR disk. (By accidentally, I mean
that the box didn't say, the ad didn't say, and the manufacturer
might even have lied to you for a while.)

https://www.ixsystems.com/community/resources/list-of-known-smr-drives.141/

Is it one of those?

If so, return it. Tell the store that it's an unlabelled SMR
drive. They'll take it back.

-dsr-

[toc] | [prev] | [next] | [standalone]

#223260

Dan Ritter wrote:

> USB2 disks are good for about 25MB/s.
> 

Where do you have those numbers?

USB 2.0 standard can theoretically transfer data at a very high 480 megabits
per second (mbps), or 60 megabytes per second (MBps) [for example in
wikipedia).

but as you say it is slowing down at some point. It is not working at max
anyway - some people say it will not increase above 20MB/s anyway.

I suggest add  to dd   status=progress

              The LEVEL of information to print to stderr; 'none' suppresses
everything but error messages, 'noxfer' suppresses the final transfer sta-
              tistics, 'progress' shows periodic transfer statistics

And honestly I do not think 4TB disks were meant to be used via USB2 - think
of using eSATA.

[toc] | [prev] | [next] | [standalone]

#223264

deloptes wrote: 
> Dan Ritter wrote:
> 
> > USB2 disks are good for about 25MB/s.
> > 
> 
> Where do you have those numbers?
> 
> USB 2.0 standard can theoretically transfer data at a very high 480 megabits
> per second (mbps), or 60 megabytes per second (MBps) [for example in
> wikipedia).

Yes, that's the theory. In years of running several USB 2.0
attached disks, I found that they were actually good for about
25MB/s long-term. Bursts to 37MB/s were not uncommon.

The USB mass storage protocol forces a queue depth of 1: one
request, one response, nothing else until it's done. 

The OP's 4K writes will be particularly badly performing.

-dsr-

[toc] | [prev] | [next] | [standalone]

#223312

On Mon, Jun 08, 2020 at 08:22:39PM +0000, Matthew Campbell wrote:
>I bought a new 4 terrabyte hard drive that is connected with a USB cable using
>USB2. It took about 32 hours to read every sector on the drive to look for bad
>sectors. I started blanking the sectors using /dev/zero last Friday night. It
>still isn't done. Is there a way I can find out how much data a particular
>process has written to the disk? I'm using Debian 10.4.
>
>dd if=/dev/zero of=/dev/sdb ibs=4096 count=976754646

This command line gets data in 4k chunks from /dev/zero and then writes 
them to the disk in 512 byte chunks. That's pretty much the worst 
possible case for writing to a disk. You want "bs", not "ibs". I'd 
suggest 
dd if=/dev/zero of=/dev/sdb bs=64k
and I wouldn't bother trying to calculate a count if you're trying to 
overwrite the entire disk (any human is likely to screw up the math and 
there's no actual benefit).

IME performance peaks at 16-64k. Beyond that things don't improve, and 
can potentially get worse or cause other issues.

On Mon, Jun 08, 2020 at 04:33:29PM -0400, Dan Ritter wrote:
>Matthew Campbell wrote:
>> I bought a new 4 terrabyte hard drive that is connected with a USB cable using USB2. It took about 32 hours to read every sector on the drive to look for bad sectors. I started blanking the sectors using /dev/zero last Friday night. It still isn't done. Is there a way I can find out how much data a particular process has written to the disk? I'm using Debian 10.4.
>>
>> dd if=/dev/zero of=/dev/sdb ibs=4096 count=976754646
>
>USB2 disks are good for about 25MB/s.

I've been getting sustained USB2 disk writes in the low 40MB/s range for 
more than 15 years. I'd suggest either checking that you're using a 
reasonable block size or getting a better USB2 adapter. 25MB/s is 
definitely low. That said, these days you'd be much better off with
USB3 because any modern disk is going to bottleneck on USB2.

On Mon, Jun 08, 2020 at 06:02:46PM -0400, Dan Ritter wrote:
>deloptes wrote:
>> Dan Ritter wrote:
>>
>> > USB2 disks are good for about 25MB/s.
>> >
>>
>> Where do you have those numbers?
>>
>> USB 2.0 standard can theoretically transfer data at a very high 480 megabits
>> per second (mbps), or 60 megabytes per second (MBps) [for example in
>> wikipedia).
>
>Yes, that's the theory. In years of running several USB 2.0
>attached disks, I found that they were actually good for about
>25MB/s long-term. Bursts to 37MB/s were not uncommon.

# dd if=/dev/zero of=/dev/sdh bs=64k count=10000 conv=fdatasync
10000+0 records in
10000+0 records out
655360000 bytes (655 MB, 625 MiB) copied, 15.1168 s, 43.4 MB/s

>The OP's 4K writes will be particularly badly performing.

OP was doing 512 byte writes.

(same adapter/disk)
# dd if=/dev/zero of=/dev/sdh ibs=4096 count=10000 conv=fdatasync
10000+0 records in
80000+0 records out
40960000 bytes (41 MB, 39 MiB) copied, 3.15622 s, 13.0 MB/s

I didn't have the patience to write the same amount of data. :D

[toc] | [prev] | [next] | [standalone]

#223314

On Wed, Jun 10, 2020 at 05:53:17PM +0300, Andrei POPESCU wrote:
>On Mi, 10 iun 20, 10:00:48, Michael Stone wrote:
>>
>> IME performance peaks at 16-64k. Beyond that things don't improve, and can
>> potentially get worse or cause other issues.
>
>Even so, bs=1M is easy to remember and type ;)

I don't find it all that hard to remember 64k but YMMV.

[toc] | [prev] | [next] | [standalone]

#223315

[Multipart message — attachments visible in raw view] — view raw

On Mi, 10 iun 20, 10:00:48, Michael Stone wrote:
> 
> IME performance peaks at 16-64k. Beyond that things don't improve, and can
> potentially get worse or cause other issues.

Even so, bs=1M is easy to remember and type ;)

Kind regards,
Andrei
-- 
http://wiki.debian.org/FAQsFromDebianUser

[toc] | [prev] | [next] | [standalone]

#223335

On 2020-06-10 07:00, Michael Stone wrote:
> On Mon, Jun 08, 2020 at 08:22:39PM +0000, Matthew Campbell wrote:

>> dd if=/dev/zero of=/dev/sdb ibs=4096 count=976754646
> 
> This command line gets data in 4k chunks from /dev/zero and then writes 
> them to the disk in 512 byte chunks. That's pretty much the worst 
> possible case for writing to a disk. 

 > # dd if=/dev/zero of=/dev/sdh ibs=4096 count=10000 conv=fdatasync
 > 10000+0 records in
 > 80000+0 records out
 > 40960000 bytes (41 MB, 39 MiB) copied, 3.15622 s, 13.0 MB/s

Good catch.


> You want "bs", not "ibs". I'd 
> suggest dd if=/dev/zero of=/dev/sdb bs=64k

+1


(I do not recall having a need for 'ibs' or 'obs'.)


> and I wouldn't bother trying to calculate a count if you're trying to 
> overwrite the entire disk

+1


> IME performance peaks at 16-64k. Beyond that things don't improve, and 
> can potentially get worse or cause other issues.

I've run benchmarks over the years, usually on Linux.  I forget where 
the performance knees are, but do recall that bs=1M has always been in 
between.


> I've been getting sustained USB2 disk writes in the low 40MB/s range for 
> more than 15 years. I'd suggest either checking that you're using a 
> reasonable block size or getting a better USB2 adapter. 25MB/s is 
> definitely low.

> # dd if=/dev/zero of=/dev/sdh bs=64k count=10000 conv=fdatasync
> 10000+0 records in
> 10000+0 records out
> 655360000 bytes (655 MB, 625 MiB) copied, 15.1168 s, 43.4 MB/s

I have Intel desktop and server motherboards, and Dell laptops and one 
server.  I believe they all have Intel USB chips.


Looking at a recent imaging script run of dd(1) with bs=1M over USB 2.0 
to a modern USB 3.0 external enclosure with a vintage SATA I HDD, the 
numbers were better than I was remembering:

13997441024 bytes (14 GB, 13 GiB) copied, 387 s, 36.2 MB/s


David

[toc] | [prev] | [next] | [standalone]

#223265

People are suing Western Digital for sneaking those SMR disks into their
supply chain.  They're supposed to be red in color if what I read in the
news is correct.

On Mon, 8 Jun 2020, Dan Ritter wrote:

> Date: Mon, 8 Jun 2020 16:33:29
> From: Dan Ritter <dsr@randomstring.org>
> To: Matthew Campbell <trenix25@pm.me>
> Cc: Debian User Support <debian-user@lists.debian.org>
> Subject: Re: How long will this take?
> Resent-Date: Mon,  8 Jun 2020 20:33:43 +0000 (UTC)
> Resent-From: debian-user@lists.debian.org
>
> Matthew Campbell wrote:
> > I bought a new 4 terrabyte hard drive that is connected with a USB cable using USB2. It took about 32 hours to read every sector on the drive to look for bad sectors. I started blanking the sectors using /dev/zero last Friday night. It still isn't done. Is there a way I can find out how much data a particular process has written to the disk? I'm using Debian 10.4.
> >
> > dd if=/dev/zero of=/dev/sdb ibs=4096 count=976754646
>
> USB2 disks are good for about 25MB/s.
>
> 4 seconds gets you 100MB.
>
> 40 seconds gets you 1000MB.
>
> 4000 * 40 seconds is 160000 seconds, so that's not quite two
> days.
>
> Is something wrong? Based on current news reports, I would say
> you accidentally purchased an SMR disk. (By accidentally, I mean
> that the box didn't say, the ad didn't say, and the manufacturer
> might even have lied to you for a while.)
>
> https://www.ixsystems.com/community/resources/list-of-known-smr-drives.141/
>
> Is it one of those?
>
> If so, return it. Tell the store that it's an unlabelled SMR
> drive. They'll take it back.
>
> -dsr-
>
>

-- 

[toc] | [prev] | [next] | [standalone]

#223275

[Multipart message — attachments visible in raw view] — view raw

On Lu, 08 iun 20, 20:09:54, Jude DaShiell wrote:
> > From: Dan Ritter <dsr@randomstring.org>
> People are suing Western Digital for sneaking those SMR disks into their
> supply chain.  They're supposed to be red in color if what I read in the
> news is correct.
> >
> > https://www.ixsystems.com/community/resources/list-of-known-smr-drives.141/

According to that list it's only specific model numbers of WD Red and 
Blue.

Kind regards,
Andrei
-- 
http://wiki.debian.org/FAQsFromDebianUser

[toc] | [prev] | [next] | [standalone]

#223258

[Multipart message — attachments visible in raw view] — view raw

Matthew Campbell (12020-06-08):
> I bought a new 4 terrabyte hard drive that is connected with a USB
> cable using USB2. It took about 32 hours to read every sector on the
> drive to look for bad sectors. I started blanking the sectors using
> /dev/zero last Friday night. It still isn't done. Is there a way I can
> find out how much data a particular process has written to the disk?
> I'm using Debian 10.4.

       Sending a USR1 signal to a running 'dd' process makes it print I/O sta‐
       tistics to standard error and then resume copying.

Fron dd(1).

Also, you can go read /proc/$(pidof dd)/fdinfo, it contains the
information too.

Note that it becomes much slower as it nears the center of the disk.

Regards,

-- 
  Nicolas George

[toc] | [prev] | [next] | [standalone]

#223261

[Multipart message — attachments visible in raw view] — view raw

# cat /proc/24283/fdinfo/1
pos: 877106917376
flags: 0100001
mnt_id: 21
#

Is that in bytes?

stdin and stderr both show a position of zero.

name=Matthew%20Campbell&email=trenix25%40pm.me

-------- Original Message --------
On Jun 8, 2020, 1:32 PM, Nicolas George wrote:

> Matthew Campbell (12020-06-08): > I bought a new 4 terrabyte hard drive that is connected with a USB > cable using USB2. It took about 32 hours to read every sector on the > drive to look for bad sectors. I started blanking the sectors using > /dev/zero last Friday night. It still isn't done. Is there a way I can > find out how much data a particular process has written to the disk? > I'm using Debian 10.4. Sending a USR1 signal to a running 'dd' process makes it print I/O sta‐ tistics to standard error and then resume copying. Fron dd(1). Also, you can go read /proc/$(pidof dd)/fdinfo, it contains the information too. Note that it becomes much slower as it nears the center of the disk. Regards, -- Nicolas George

[toc] | [prev] | [next] | [standalone]

#223263

[Multipart message — attachments visible in raw view] — view raw

Matthew Campbell (12020-06-08):
> Is that in bytes?

You can compare with the stats presented by USR1 to be sure.

> stdin and stderr both show a position of zero.

You can look in /proc/$PID/fd to see where the various fd points. I
guess 0 will point to /dev/zero, 1 to the hard drive and 2 to the tty.

Regards,

-- 
  Nicolas George

[toc] | [prev] | [next] | [standalone]

#223268

On Mon 08 Jun 2020 at 20:22:39 (+0000), Matthew Campbell wrote:
> I bought a new 4 terrabyte hard drive that is connected with a USB cable using USB2. It took about 32 hours to read every sector on the drive to look for bad sectors.

I recently ran

# badblocks -c 1024 -s -w -t random -v /dev/sdz

on a 2TB disk with a USB2 connection. The whole process, writing and
checking, took 33⅓ hours. (The disk now holds an encrypted ext4 filesystem.)

> I started blanking the sectors using /dev/zero last Friday night. It still isn't done. Is there a way I can find out how much data a particular process has written to the disk? I'm using Debian 10.4.

I'm not sure why you'd do that. I've only zeroed disks to erase them
before I return them to the owner. (They're inside loaned computers.)

> dd if=/dev/zero of=/dev/sdb ibs=4096 count=976754646

… And I'd be using bs=1M and no count. I, too, determine progress with
# kill -USR1 <pid-of-dd>

Cheers,
David.

[toc] | [prev] | [next] | [standalone]

#223269

On 6/8/2020 11:01 PM, David Wright wrote:
> I, too, determine progress with
> # kill -USR1 <pid-of-dd>

I'd suggest simply adding "status=progress" which gives you a summary 
every second including bytes written, elapsed time, and average transfer 
rate.

-- 
Chris Howie
http://www.chrishowie.com
http://en.wikipedia.org/wiki/User:Crazycomputers

If you correspond with me on a regular basis, please read this document: 
http://www.chrishowie.com/email-preferences/

PGP fingerprint: 2B7A B280 8B12 21CC 260A DF65 6FCE 505A CF83 38F5

------------------------------------------------------------------------
                     IMPORTANT INFORMATION/DISCLAIMER

This document should be read only by those persons to whom it is 
addressed.  If you have received this message it was obviously addressed 
to you and therefore you can read it.

Additionally, by sending an email to ANY of my addresses or to ANY 
mailing lists to which I am subscribed, whether intentionally or 
accidentally, you are agreeing that I am "the intended recipient," and 
that I may do whatever I wish with the contents of any message received 
from you, unless a pre-existing agreement prohibits me from so doing.

This overrides any disclaimer or statement of confidentiality that may 
be included on your message.

[toc] | [prev] | [next] | [standalone]

#223278

[Multipart message — attachments visible in raw view] — view raw

Christopher David Howie (12020-06-08):
> I'd suggest simply adding "status=progress" which gives you a summary every
> second including bytes written, elapsed time, and average transfer rate.

How do you add "status=progress" to a process that has already been
running for three days?

Regards,

-- 
  Nicolas George

[toc] | [prev] | [next] | [standalone]

#223285

[Multipart message — attachments visible in raw view] — view raw

I have started the process over from the beginning. It seems to respond well to obs=4M as the write speed has gone from 3.7 MB/s to 28.2 MB/s, which is about the same as where it was with obs=1M. It should take a couple of days to complete this write process.

The WD drive that I have, which is connected to a separate USB2 port, showed a write speed of 27.5 Mb/s.

I am using:

dd if=/dev/zero of=/dev/sdb obs=4M status=progress

The read speed was faster, of course.

name=Matthew%20Campbell&email=trenix25%40pm.me

-------- Original Message --------
On Jun 9, 2020, 2:39 AM, Nicolas George wrote:

> Christopher David Howie (12020-06-08): > I'd suggest simply adding "status=progress" which gives you a summary every > second including bytes written, elapsed time, and average transfer rate. How do you add "status=progress" to a process that has already been running for three days? Regards, -- Nicolas George

[toc] | [prev] | [next] | [standalone]

#223296

On 6/9/2020 5:39 AM, Nicolas George wrote:
> How do you add "status=progress" to a process that has already been
> running for three days?

You can't, of course.  I was merely suggesting using this in future 
invocations.

-- 
Chris Howie
http://www.chrishowie.com
http://en.wikipedia.org/wiki/User:Crazycomputers

If you correspond with me on a regular basis, please read this document: 
http://www.chrishowie.com/email-preferences/

PGP fingerprint: 2B7A B280 8B12 21CC 260A DF65 6FCE 505A CF83 38F5

------------------------------------------------------------------------
                     IMPORTANT INFORMATION/DISCLAIMER

This document should be read only by those persons to whom it is 
addressed.  If you have received this message it was obviously addressed 
to you and therefore you can read it.

Additionally, by sending an email to ANY of my addresses or to ANY 
mailing lists to which I am subscribed, whether intentionally or 
accidentally, you are agreeing that I am "the intended recipient," and 
that I may do whatever I wish with the contents of any message received 
from you, unless a pre-existing agreement prohibits me from so doing.

This overrides any disclaimer or statement of confidentiality that may 
be included on your message.

[toc] | [prev] | [next] | [standalone]

#223313

On Mon, Jun 08, 2020 at 10:01:13PM -0500, David Wright wrote:
>On Mon 08 Jun 2020 at 20:22:39 (+0000), Matthew Campbell wrote:
>> I bought a new 4 terrabyte hard drive that is connected with a USB 
>> cable using USB2. It took about 32 hours to read every sector on the 
>> drive to look for bad sectors.
>
>I recently ran
>
># badblocks -c 1024 -s -w -t random -v /dev/sdz
>
>on a 2TB disk with a USB2 connection. The whole process, writing and
>checking, took 33⅓ hours. (The disk now holds an encrypted ext4 filesystem.)

Yes, it's a slower process than just writing zeros. A modern drive will 
verify writes as they're made. badblocks is basically a relic of another 
age primarily intended to give a list of bad sectors to avoid when 
making a filesystem. Once upon a time, hard drives actually had a 
handwritten label on the top listing any bad sectors identified at the 
factory so you could avoid them. They don't have that anymore. If any 
modern hard drive has consistent bad (not remappable) sectors it should 
just be thrown away, because that means it is so far gone that it no 
longer has the ability to internally map bad sectors to reserved good 
sectors.

>> I started blanking the sectors using /dev/zero last Friday night. It 
>> still isn't done. Is there a way I can find out how much data a 
>> particular process has written to the disk? I'm using Debian 10.4.

>I'm not sure why you'd do that. I've only zeroed disks to erase them
>before I return them to the owner. (They're inside loaned computers.)

Because it accomplishes what your badblocks run does, in less than half 
the time. :)

[toc] | [prev] | [next] | [standalone]

#223320

On Wed 10 Jun 2020 at 10:14:02 (-0400), Michael Stone wrote:
> On Mon, Jun 08, 2020 at 10:01:13PM -0500, David Wright wrote:
> > On Mon 08 Jun 2020 at 20:22:39 (+0000), Matthew Campbell wrote:
> > > I bought a new 4 terrabyte hard drive that is connected with a
> > > USB cable using USB2. It took about 32 hours to read every
> > > sector on the drive to look for bad sectors.
> > 
> > I recently ran
> > 
> > # badblocks -c 1024 -s -w -t random -v /dev/sdz
> > 
> > on a 2TB disk with a USB2 connection. The whole process, writing and
> > checking, took 33⅓ hours. (The disk now holds an encrypted ext4 filesystem.)
> 
> Yes, it's a slower process than just writing zeros. A modern drive
> will verify writes as they're made. badblocks is basically a relic of
> another age primarily intended to give a list of bad sectors to avoid
> when making a filesystem. Once upon a time, hard drives actually had a
> handwritten label on the top listing any bad sectors identified at the
> factory so you could avoid them. They don't have that anymore. If any
> modern hard drive has consistent bad (not remappable) sectors it
> should just be thrown away, because that means it is so far gone that
> it no longer has the ability to internally map bad sectors to reserved
> good sectors.
> 
> > > I started blanking the sectors using /dev/zero last Friday
> > > night. It still isn't done. Is there a way I can find out how
> > > much data a particular process has written to the disk? I'm
> > > using Debian 10.4.
> 
> > I'm not sure why you'd do that. I've only zeroed disks to erase them
> > before I return them to the owner. (They're inside loaned computers.)
> 
> Because it accomplishes what your badblocks run does, in less than
> half the time. :)

I tried to make clear that my use case differed from that of the OP,
in case you missed that. Just before lockdown (=lockout). I borrowed
an AIO computer and, to make room, returned a 2006 vintage tower that
would no longer pass its POST. I used /dev/zero to erase all the
information from the disk as there was little point in trying to put
Windows XP (licensed to a dead computer) back onto it. Quick, easy,
and quick to check with od. Both l0f4r0 and I have asked why the OP
is zeroing the drive, but no reply yet. Perhaps you can suggest an
answer.

My use case for badblocks was closer to that of the OP, but still
different. Firstly, the disk contained personal data from unencrypted
use in the past. Secondly, I was intending to use it encrypted (as
mentioned) and prefer no high-watermark.  Thirdly, because of its
age (2011), I was interested in seeing how well it performed. I have
no idea whether the disk is "modern" in the sense you used, as I don't
follow the technology like some people on this list evidently do.
Fourthly, I don't make a habit of throwing away 2TB disks.

But, as you know about these things, a few questions:

. How does badblocks do its job in readonly mode, given that it
  doesn't know what any block's content ought to be.

. Why might the OP run badblocks, particularly non-destructively
  (as if to preserve something), and *then* zero the drive.

. What's the easiest way of finding out about "consistent bad
  (not remappable) sectors" on a drive, as I soon will have to
  repeat this result (if not by this exact process) with a 3TB
  disk of 2013 vintage. (The good news: it has a USB3 connection.)

Cheers,
David.

[toc] | [prev] | [next] | [standalone]

Page 1 of 4  [1] 2 3 4  Next page →

Back to top | Article view | linux.debian.user

csiph-web