Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1403330 > unrolled thread

BLKZEROOUT not zeroing md dev on VMDK

Back to article view | Back to linux.kernel

Contents

  BLKZEROOUT not zeroing md dev on VMDK Sitsofe Wheeler <sitsofe@gmail.com> - 2016-05-19 00:40 +0200
    Re: BLKZEROOUT not zeroing md dev on VMDK "Darrick J. Wong" <darrick.wong@oracle.com> - 2016-05-27 06:20 +0200
      Re: BLKZEROOUT not zeroing md dev on VMDK Sitsofe Wheeler <sitsofe@gmail.com> - 2016-05-27 06:50 +0200
        Re: BLKZEROOUT not zeroing md dev on VMDK Tom Yan <tom.ty89@gmail.com> - 2016-05-27 11:40 +0200

#1403330 — BLKZEROOUT not zeroing md dev on VMDK

Hi,

With Ubuntu's 4.4.0-22-generic kernel and a Fedora 23
4.6.0-1.vanilla.knurd.1.fc23.x86_64 kernel I've found that the
BLKZEROOUT syscall can malfunction and not zero data.

When BLKZEROOUT is issued to an MD device atop a PVSCSI controller
supplied VMDK from ESXi 6.0 the call returns immediately and with a zero
return code. Unfortunately, inspecting the data on the MD device shows
that it has not been zeroed and is in fact untouched. The easiest way to
see this behaviour is to boot the VM, create an mdadm device atop
/dev/sd?, scribble some non-zero value on the disk and then use
blkdiscard --zeroout /dev/md??? . If you then inspect the MD disk (e.g.
with hexdump) you will still see the old data and using POSIX_FADV_DONTNEED
on the MD device doesn't change the outcome.

The only clue I've seen is that
/sys/block/sd?/queue/write_same_max_bytes starts out being 33553920 but
after a WRITE SAME is issued it becomes 0. If the MD device is created
after write_same_max_bytes has become 0 on the backing disk then
BLKZEROOUT seems to work correctly.

-- 
Sitsofe | http://sucs.org/~sits/

[toc] | [next] | [standalone]

#1407802

On Wed, May 18, 2016 at 11:39:30PM +0100, Sitsofe Wheeler wrote:
> Hi,
> 
> With Ubuntu's 4.4.0-22-generic kernel and a Fedora 23
> 4.6.0-1.vanilla.knurd.1.fc23.x86_64 kernel I've found that the
> BLKZEROOUT syscall can malfunction and not zero data.
> 
> When BLKZEROOUT is issued to an MD device atop a PVSCSI controller
> supplied VMDK from ESXi 6.0 the call returns immediately and with a zero
> return code. Unfortunately, inspecting the data on the MD device shows
> that it has not been zeroed and is in fact untouched. The easiest way to
> see this behaviour is to boot the VM, create an mdadm device atop
> /dev/sd?, scribble some non-zero value on the disk and then use
> blkdiscard --zeroout /dev/md??? . If you then inspect the MD disk (e.g.
> with hexdump) you will still see the old data and using POSIX_FADV_DONTNEED
> on the MD device doesn't change the outcome.
> 
> The only clue I've seen is that
> /sys/block/sd?/queue/write_same_max_bytes starts out being 33553920 but
> after a WRITE SAME is issued it becomes 0. If the MD device is created
> after write_same_max_bytes has become 0 on the backing disk then
> BLKZEROOUT seems to work correctly.

It's possible that the pvscsi device advertised WRITE SAME, but if the device
sends back ILLEGAL REQUEST then the SCSI disk driver will set
write_same_max_bytes=0.  Subsequent BLKZEROOUT attempts will then issue writes
of zeroes to the drive.

--D

> 
> -- 
> Sitsofe | http://sucs.org/~sits/

[toc] | [prev] | [next] | [standalone]

#1407809

On 27 May 2016 at 05:18, Darrick J. Wong <darrick.wong@oracle.com> wrote:
>
> It's possible that the pvscsi device advertised WRITE SAME, but if the device
> sends back ILLEGAL REQUEST then the SCSI disk driver will set
> write_same_max_bytes=0.  Subsequent BLKZEROOUT attempts will then issue writes
> of zeroes to the drive.

Thanks for following up on this but that's not what happens on the md
device - you can go on to issue as many BLKZEROOUT requests as you
like but the md disk is never zeroed nor is an error returned.

I filed a bug at https://bugzilla.kernel.org/show_bug.cgi?id=118581
(see https://bugzilla.kernel.org/show_bug.cgi?id=118581#c6 for
alternative reproduction steps that use scsi_debug and can be reworked
to impact device mapper) and Shaohua Li noted that
blkdev_issue_write_same could return 0 even when the disk didn't
support write same (see
https://bugzilla.kernel.org/show_bug.cgi?id=118581#c8 ).

Shaohua went on to create a patch for this ("block: correctly fallback
for zeroout" - https://patchwork.kernel.org/patch/9137311/ ) which has
yet to be reviewed.

-- 
Sitsofe | http://sucs.org/~sits/

[toc] | [prev] | [next] | [standalone]

#1407955

There seems to be some sort of race condition between
blkdev_issue_zeroout() and the scsi disk driver (disabling write same
after an illegal request). On my UAS drive, sometimes `blkdiscard -z
/dev/sdX` will return right away, even though if I then check
`write_same_max_bytes` it has turned 0. Sometimes it will just write
zero with SCSI WRITE even if `write_same_max_bytes` is 33553920 before
I issue `blkdiscard -z` (`write_same_max_bytes` also turned 0, as
expected).

Not sure if it is directly related to the case here though.

On 27 May 2016 at 12:45, Sitsofe Wheeler <sitsofe@gmail.com> wrote:
> On 27 May 2016 at 05:18, Darrick J. Wong <darrick.wong@oracle.com> wrote:
>>
>> It's possible that the pvscsi device advertised WRITE SAME, but if the device
>> sends back ILLEGAL REQUEST then the SCSI disk driver will set
>> write_same_max_bytes=0.  Subsequent BLKZEROOUT attempts will then issue writes
>> of zeroes to the drive.
>
> Thanks for following up on this but that's not what happens on the md
> device - you can go on to issue as many BLKZEROOUT requests as you
> like but the md disk is never zeroed nor is an error returned.
>
> I filed a bug at https://bugzilla.kernel.org/show_bug.cgi?id=118581
> (see https://bugzilla.kernel.org/show_bug.cgi?id=118581#c6 for
> alternative reproduction steps that use scsi_debug and can be reworked
> to impact device mapper) and Shaohua Li noted that
> blkdev_issue_write_same could return 0 even when the disk didn't
> support write same (see
> https://bugzilla.kernel.org/show_bug.cgi?id=118581#c8 ).
>
> Shaohua went on to create a patch for this ("block: correctly fallback
> for zeroout" - https://patchwork.kernel.org/patch/9137311/ ) which has
> yet to be reviewed.
>
> --
> Sitsofe | http://sucs.org/~sits/
> --
> To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web