Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1547047 > unrolled thread

LTP rwtest01 blocks on DAX mountpoint

Started byXiong Zhou <xzhou@redhat.com>
First post2016-12-24 12:10 +0100
Last post2017-01-04 17:50 +0100
Articles 11 — 3 participants

Back to article view | Back to linux.kernel

Contents

  LTP rwtest01 blocks on DAX mountpoint Xiong Zhou <xzhou@redhat.com> - 2016-12-24 12:10 +0100
    Re: LTP rwtest01 blocks on DAX mountpoint Xiong Zhou <xzhou@redhat.com> - 2016-12-30 10:40 +0100
      Re: LTP rwtest01 blocks on DAX mountpoint Jan Kara <jack@suse.cz> - 2017-01-02 11:20 +0100
      Re: LTP rwtest01 blocks on DAX mountpoint Jan Kara <jack@suse.cz> - 2017-01-02 18:20 +0100
        Re: LTP rwtest01 blocks on DAX mountpoint Ross Zwisler <ross.zwisler@linux.intel.com> - 2017-01-03 05:00 +0100
          Re: LTP rwtest01 blocks on DAX mountpoint Xiong Zhou <xzhou@redhat.com> - 2017-01-03 07:50 +0100
            Re: LTP rwtest01 blocks on DAX mountpoint Ross Zwisler <ross.zwisler@linux.intel.com> - 2017-01-03 18:00 +0100
              Re: LTP rwtest01 blocks on DAX mountpoint Xiong Zhou <xzhou@redhat.com> - 2017-01-04 02:30 +0100
              Re: LTP rwtest01 blocks on DAX mountpoint Xiong Zhou <xzhou@redhat.com> - 2017-01-04 02:50 +0100
              Re: LTP rwtest01 blocks on DAX mountpoint Xiong Zhou <xzhou@redhat.com> - 2017-01-04 10:50 +0100
                Re: LTP rwtest01 blocks on DAX mountpoint Ross Zwisler <ross.zwisler@linux.intel.com> - 2017-01-04 17:50 +0100

#1547047 — LTP rwtest01 blocks on DAX mountpoint

FromXiong Zhou <xzhou@redhat.com>
Date2016-12-24 12:10 +0100
SubjectLTP rwtest01 blocks on DAX mountpoint
Message-ID<sRSn7-67J-5@gated-at.bofh.it>

[Multipart message — attachments visible in raw view] — view raw

Hi lists,

Since around 20161129 tag, LTP rwtest01 on dax mountpoint blocks
on linux-next tree, now on Linus tree.

In "normal", rwtest01 subcase ends in a few minutes, now it keeps
running for hours on dax mountpoint, both ext4 and xfs. Ctrl + c
can interrupt it.

It is always reproducible, blocking following tests.

It does not happen when mounting without dax option.
It does not happen on v4.9.

Bisect point to:

commit 4b4bb46d00b386e1c972890dc5785a7966eaa9c0
Author: Jan Kara <jack@suse.cz>
Date:   Wed Dec 14 15:07:53 2016 -0800

    dax: clear dirty entry tags on cache flush


Reverting this commit on top of Linus tree "fixes" this issue.

Reproducer:

sh-4.2# cat rwt
rwtest01 export LTPROOT; rwtest -N rwtest01 -c -q -i 60s  -f sync 10%25000:$TMPDIR/rw-sync-$$
sh-4.2# 
mkfs.xfs /dev/pmem0p1
mount -o dax /dev/pmem0p1 /daxmnt && \
/opt/ltp/runltp -q -d /daxmnt -f rwt -p -b /dev/pmem0p2 -B xfs
umount /daxmnt

Bisect log is attached.

Thanks,
Xiong

[toc] | [next] | [standalone]

#1548509

FromXiong Zhou <xzhou@redhat.com>
Date2016-12-30 10:40 +0100
Message-ID<sU1Pj-2xO-1@gated-at.bofh.it>
In reply to#1547047
On Sat, Dec 24, 2016 at 07:07:14PM +0800, Xiong Zhou wrote:
> Hi lists,
> 
> Since around 20161129 tag, LTP rwtest01 on dax mountpoint blocks
> on linux-next tree, now on Linus tree.
> 
> In "normal", rwtest01 subcase ends in a few minutes, now it keeps
> running for hours on dax mountpoint, both ext4 and xfs. Ctrl + c
> can interrupt it.

Test programme is waiting for a memcpy call to return.

From sysrq output, kernel code is not blocking on somewhere,
it just wont return.

> 
> It is always reproducible, blocking following tests.
> 
> It does not happen when mounting without dax option.
> It does not happen on v4.9.
> 
> Bisect point to:
> 
> commit 4b4bb46d00b386e1c972890dc5785a7966eaa9c0
> Author: Jan Kara <jack@suse.cz>
> Date:   Wed Dec 14 15:07:53 2016 -0800
> 
>     dax: clear dirty entry tags on cache flush
> 
> 
> Reverting this commit on top of Linus tree "fixes" this issue.
> 
> Reproducer:
> 
> sh-4.2# cat rwt
> rwtest01 export LTPROOT; rwtest -N rwtest01 -c -q -i 60s  -f sync 10%25000:$TMPDIR/rw-sync-$$
> sh-4.2# 
> mkfs.xfs /dev/pmem0p1
> mount -o dax /dev/pmem0p1 /daxmnt && \
> /opt/ltp/runltp -q -d /daxmnt -f rwt -p -b /dev/pmem0p2 -B xfs
> umount /daxmnt
> 
> Bisect log is attached.
> 
> Thanks,
> Xiong

> git bisect start
> # bad: [50f6584e4c626b8fa39edb66f33fec27bab3996c] Merge tag 'leds_for_4.10_email_update' of git://git.kernel.org/pub/scm/linux/kernel/git/j.anaszewski/linux-leds
> git bisect bad 50f6584e4c626b8fa39edb66f33fec27bab3996c
> # good: [69973b830859bc6529a7a0468ba0d80ee5117826] Linux 4.9
> git bisect good 69973b830859bc6529a7a0468ba0d80ee5117826
> # good: [5266e70335dac35c35b5ca9cea4251c1389d4a68] Merge tag 'tty-4.10-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/tty
> git bisect good 5266e70335dac35c35b5ca9cea4251c1389d4a68
> # bad: [6df8b74b1720db1133ace0861cb6721bfe57819a] Merge tag 'devicetree-for-4.10' of git://git.kernel.org/pub/scm/linux/kernel/git/robh/linux
> git bisect bad 6df8b74b1720db1133ace0861cb6721bfe57819a
> # good: [f4000cd99750065d5177555c0a805c97174d1b9f] Merge tag 'arm64-upstream' of git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux
> git bisect good f4000cd99750065d5177555c0a805c97174d1b9f
> # bad: [e1e14ab8411df344a17687821f8f78f0a1e73cbb] radix tree test suite: delete unused rcupdate.c
> git bisect bad e1e14ab8411df344a17687821f8f78f0a1e73cbb
> # good: [f5b893c947151d424a4ab55ea3a8544b81974b31] scsi: qla4xxx: switch to pci_alloc_irq_vectors
> git bisect good f5b893c947151d424a4ab55ea3a8544b81974b31
> # good: [b9f98bd4034a3196ff068eb0fa376c5f41077480] Merge tag 'mmc-v4.10-2' of git://git.kernel.org/pub/scm/linux/kernel/git/ulfh/mmc
> git bisect good b9f98bd4034a3196ff068eb0fa376c5f41077480
> # good: [4d1f0fb096aedea7bb5489af93498a82e467c480] kernel/watchdog: use nmi registers snapshot in hardlockup handler
> git bisect good 4d1f0fb096aedea7bb5489af93498a82e467c480
> # good: [5b56d49fc31dbb0487e14ead790fc81ca9fb2c99] mm: add locked parameter to get_user_pages_remote()
> git bisect good 5b56d49fc31dbb0487e14ead790fc81ca9fb2c99
> # bad: [cfa40bcfd6fed7010b1633bf127ed8571d3b607e] radix tree test suite: benchmark for iterator
> git bisect bad cfa40bcfd6fed7010b1633bf127ed8571d3b607e
> # good: [a41b70d6dfc28b9e1a17c2a9f3181c2b614bfd54] mm: use vmf->page during WP faults
> git bisect good a41b70d6dfc28b9e1a17c2a9f3181c2b614bfd54
> # bad: [4b4bb46d00b386e1c972890dc5785a7966eaa9c0] dax: clear dirty entry tags on cache flush
> git bisect bad 4b4bb46d00b386e1c972890dc5785a7966eaa9c0
> # good: [a19e25536ed3a20845f642ce531e10c27fb2add5] mm: change return values of finish_mkwrite_fault()
> git bisect good a19e25536ed3a20845f642ce531e10c27fb2add5
> # good: [a6abc2c0e77b16480f4d2c1eb7925e5287ae1526] dax: make cache flushing protected by entry lock
> git bisect good a6abc2c0e77b16480f4d2c1eb7925e5287ae1526
> # good: [2f89dc12a25ddf995b9acd7b6543fe892e3473d6] dax: protect PTE modification on WP fault by radix tree entry lock
> git bisect good 2f89dc12a25ddf995b9acd7b6543fe892e3473d6
> # first bad commit: [4b4bb46d00b386e1c972890dc5785a7966eaa9c0] dax: clear dirty entry tags on cache flush

> _______________________________________________
> Linux-nvdimm mailing list
> Linux-nvdimm@lists.01.org
> https://lists.01.org/mailman/listinfo/linux-nvdimm

[toc] | [prev] | [next] | [standalone]

#1549095

FromJan Kara <jack@suse.cz>
Date2017-01-02 11:20 +0100
Message-ID<sV7SF-5kc-5@gated-at.bofh.it>
In reply to#1548509
On Fri 30-12-16 17:33:53, Xiong Zhou wrote:
> On Sat, Dec 24, 2016 at 07:07:14PM +0800, Xiong Zhou wrote:
> > Hi lists,
> > 
> > Since around 20161129 tag, LTP rwtest01 on dax mountpoint blocks
> > on linux-next tree, now on Linus tree.
> > 
> > In "normal", rwtest01 subcase ends in a few minutes, now it keeps
> > running for hours on dax mountpoint, both ext4 and xfs. Ctrl + c
> > can interrupt it.
> 
> Test programme is waiting for a memcpy call to return.
> 
> From sysrq output, kernel code is not blocking on somewhere,
> it just wont return.

Thanks for report. I'll try to reproduce this. Looks like we are forever
retrying the fault or something like that.

								Honza
-- 
Jan Kara <jack@suse.com>
SUSE Labs, CR

[toc] | [prev] | [next] | [standalone]

#1549309

FromJan Kara <jack@suse.cz>
Date2017-01-02 18:20 +0100
Message-ID<sVer7-1K6-15@gated-at.bofh.it>
In reply to#1548509
On Fri 30-12-16 17:33:53, Xiong Zhou wrote:
> On Sat, Dec 24, 2016 at 07:07:14PM +0800, Xiong Zhou wrote:
> > Hi lists,
> > 
> > Since around 20161129 tag, LTP rwtest01 on dax mountpoint blocks
> > on linux-next tree, now on Linus tree.
> > 
> > In "normal", rwtest01 subcase ends in a few minutes, now it keeps
> > running for hours on dax mountpoint, both ext4 and xfs. Ctrl + c
> > can interrupt it.
> 
> Test programme is waiting for a memcpy call to return.
> 
> From sysrq output, kernel code is not blocking on somewhere,
> it just wont return.

I was trying to reproduce this but for me rwtest01 completes just fine on
dax mountpoint (I've used your reproducer). So can you sample several
kernel stack traces to get a rough idea where the kernel is running?
Thanks!

								Honza
-- 
Jan Kara <jack@suse.com>
SUSE Labs, CR

[toc] | [prev] | [next] | [standalone]

#1549518

FromRoss Zwisler <ross.zwisler@linux.intel.com>
Date2017-01-03 05:00 +0100
Message-ID<sVoqt-cw-5@gated-at.bofh.it>
In reply to#1549309
On Mon, Jan 02, 2017 at 06:16:17PM +0100, Jan Kara wrote:
> On Fri 30-12-16 17:33:53, Xiong Zhou wrote:
> > On Sat, Dec 24, 2016 at 07:07:14PM +0800, Xiong Zhou wrote:
> > > Hi lists,
> > > 
> > > Since around 20161129 tag, LTP rwtest01 on dax mountpoint blocks
> > > on linux-next tree, now on Linus tree.
> > > 
> > > In "normal", rwtest01 subcase ends in a few minutes, now it keeps
> > > running for hours on dax mountpoint, both ext4 and xfs. Ctrl + c
> > > can interrupt it.
> > 
> > Test programme is waiting for a memcpy call to return.
> > 
> > From sysrq output, kernel code is not blocking on somewhere,
> > it just wont return.
> 
> I was trying to reproduce this but for me rwtest01 completes just fine on
> dax mountpoint (I've used your reproducer). So can you sample several
> kernel stack traces to get a rough idea where the kernel is running?
> Thanks!
> 
> 								Honza

I'm also unable to reproduce this issue.  I've tried with both the blamed
commit:

4b4bb46 (HEAD) dax: clear dirty entry tags on cache flush

and with v4.9-rc2.  Both pass the test in my setup.

Perhaps the variable is the size of your PMEM partitions?

# fdisk -l /dev/pmem0
Disk /dev/pmem0: 16 GiB, 17179869184 bytes, 33554432 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: dos
Disk identifier: 0xfe50c900

Device       Boot    Start      End  Sectors Size Id Type
/dev/pmem0p1          4096 25165823 25161728  12G 83 Linux
/dev/pmem0p2      25165824 33550335  8384512   4G 83 Linux

What does your setup look like?

I'm using the current tip of the LTP tree:

8cc4165  waitid02: define _XOPEN_SOURCE 500

Thanks,
- Ross

[toc] | [prev] | [next] | [standalone]

#1549566

FromXiong Zhou <xzhou@redhat.com>
Date2017-01-03 07:50 +0100
Message-ID<sVr4Z-2iY-9@gated-at.bofh.it>
In reply to#1549518
On Mon, Jan 02, 2017 at 02:49:41PM -0700, Ross Zwisler wrote:
> On Mon, Jan 02, 2017 at 06:16:17PM +0100, Jan Kara wrote:
> > On Fri 30-12-16 17:33:53, Xiong Zhou wrote:
> > > On Sat, Dec 24, 2016 at 07:07:14PM +0800, Xiong Zhou wrote:
> > > > Hi lists,
snip
> > I was trying to reproduce this but for me rwtest01 completes just fine on
> > dax mountpoint (I've used your reproducer). So can you sample several
> > kernel stack traces to get a rough idea where the kernel is running?
> > Thanks!
> > 
> > 								Honza
> 
> I'm also unable to reproduce this issue.  I've tried with both the blamed
> commit:
> 4b4bb46 (HEAD) dax: clear dirty entry tags on cache flush
> and with v4.9-rc2.  Both pass the test in my setup.
> Perhaps the variable is the size of your PMEM partitions?
> # fdisk -l /dev/pmem0
> Disk /dev/pmem0: 16 GiB, 17179869184 bytes, 33554432 sectors
> Units: sectors of 1 * 512 = 512 bytes
> Sector size (logical/physical): 512 bytes / 4096 bytes
> I/O size (minimum/optimal): 4096 bytes / 4096 bytes
> Disklabel type: dos
> Disk identifier: 0xfe50c900
> Device       Boot    Start      End  Sectors Size Id Type
> /dev/pmem0p1          4096 25165823 25161728  12G 83 Linux
> /dev/pmem0p2      25165824 33550335  8384512   4G 83 Linux
> 
> What does your setup look like?
> I'm using the current tip of the LTP tree:
> 8cc4165  waitid02: define _XOPEN_SOURCE 500
> Thanks,
> - Ross

Thanks all for looking into it.

Turns out the rc2 relative updates fix this issue, so does
an old issue i reported a while ago:
multi-threads libvmmalloc fork test hang
https://lists.01.org/pipermail/linux-nvdimm/2016-October/007602.html

I'm able to reproduce these issues before rc2, now it
passes on current Linus tree:
c8b4ec8 Merge tag 'fscrypt-for-stable'

Thanks,
Xiong

> --
> To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

[toc] | [prev] | [next] | [standalone]

#1549955

FromRoss Zwisler <ross.zwisler@linux.intel.com>
Date2017-01-03 18:00 +0100
Message-ID<sVABj-Hi-3@gated-at.bofh.it>
In reply to#1549566
On Tue, Jan 03, 2017 at 02:49:22PM +0800, Xiong Zhou wrote:
> On Mon, Jan 02, 2017 at 02:49:41PM -0700, Ross Zwisler wrote:
> > On Mon, Jan 02, 2017 at 06:16:17PM +0100, Jan Kara wrote:
> > > On Fri 30-12-16 17:33:53, Xiong Zhou wrote:
> > > > On Sat, Dec 24, 2016 at 07:07:14PM +0800, Xiong Zhou wrote:
> > > > > Hi lists,
> snip
> > > I was trying to reproduce this but for me rwtest01 completes just fine on
> > > dax mountpoint (I've used your reproducer). So can you sample several
> > > kernel stack traces to get a rough idea where the kernel is running?
> > > Thanks!
> > > 
> > > 								Honza
> > 
> > I'm also unable to reproduce this issue.  I've tried with both the blamed
> > commit:
> > 4b4bb46 (HEAD) dax: clear dirty entry tags on cache flush
> > and with v4.9-rc2.  Both pass the test in my setup.
> > Perhaps the variable is the size of your PMEM partitions?
> > # fdisk -l /dev/pmem0
> > Disk /dev/pmem0: 16 GiB, 17179869184 bytes, 33554432 sectors
> > Units: sectors of 1 * 512 = 512 bytes
> > Sector size (logical/physical): 512 bytes / 4096 bytes
> > I/O size (minimum/optimal): 4096 bytes / 4096 bytes
> > Disklabel type: dos
> > Disk identifier: 0xfe50c900
> > Device       Boot    Start      End  Sectors Size Id Type
> > /dev/pmem0p1          4096 25165823 25161728  12G 83 Linux
> > /dev/pmem0p2      25165824 33550335  8384512   4G 83 Linux
> > 
> > What does your setup look like?
> > I'm using the current tip of the LTP tree:
> > 8cc4165  waitid02: define _XOPEN_SOURCE 500
> > Thanks,
> > - Ross
> 
> Thanks all for looking into it.
> 
> Turns out the rc2 relative updates fix this issue, so does
> an old issue i reported a while ago:
> multi-threads libvmmalloc fork test hang
> https://lists.01.org/pipermail/linux-nvdimm/2016-October/007602.html
> 
> I'm able to reproduce these issues before rc2, now it
> passes on current Linus tree:
> c8b4ec8 Merge tag 'fscrypt-for-stable'

Hmm...I'm able to reproduce the other libvmmalloc issue with both v4.10-rc2
and with "c8b4ec8 Merge tag 'fscrypt-for-stable'".  I'm debugging that issue
today.

It's interesting that both tests started passing for you.  Did you change
something in your test setup?

[toc] | [prev] | [next] | [standalone]

#1550350

FromXiong Zhou <xzhou@redhat.com>
Date2017-01-04 02:30 +0100
Message-ID<sVIyS-668-15@gated-at.bofh.it>
In reply to#1549955
On Tue, Jan 03, 2017 at 09:57:10AM -0700, Ross Zwisler wrote:
> On Tue, Jan 03, 2017 at 02:49:22PM +0800, Xiong Zhou wrote:
> > On Mon, Jan 02, 2017 at 02:49:41PM -0700, Ross Zwisler wrote:
> > > On Mon, Jan 02, 2017 at 06:16:17PM +0100, Jan Kara wrote:
> > > > On Fri 30-12-16 17:33:53, Xiong Zhou wrote:
> > > > > On Sat, Dec 24, 2016 at 07:07:14PM +0800, Xiong Zhou wrote:
> > > > > > Hi lists,
> > snip
> > > > I was trying to reproduce this but for me rwtest01 completes just fine on
> > > > dax mountpoint (I've used your reproducer). So can you sample several
> > > > kernel stack traces to get a rough idea where the kernel is running?
> > > > Thanks!
> > > > 
> > > > 								Honza
> > > 
> > > I'm also unable to reproduce this issue.  I've tried with both the blamed
> > > commit:
> > > 4b4bb46 (HEAD) dax: clear dirty entry tags on cache flush
> > > and with v4.9-rc2.  Both pass the test in my setup.
> > > Perhaps the variable is the size of your PMEM partitions?
> > > # fdisk -l /dev/pmem0
> > > Disk /dev/pmem0: 16 GiB, 17179869184 bytes, 33554432 sectors
> > > Units: sectors of 1 * 512 = 512 bytes
> > > Sector size (logical/physical): 512 bytes / 4096 bytes
> > > I/O size (minimum/optimal): 4096 bytes / 4096 bytes
> > > Disklabel type: dos
> > > Disk identifier: 0xfe50c900
> > > Device       Boot    Start      End  Sectors Size Id Type
> > > /dev/pmem0p1          4096 25165823 25161728  12G 83 Linux
> > > /dev/pmem0p2      25165824 33550335  8384512   4G 83 Linux
> > > 
> > > What does your setup look like?
> > > I'm using the current tip of the LTP tree:
> > > 8cc4165  waitid02: define _XOPEN_SOURCE 500
> > > Thanks,
> > > - Ross
> > 
> > Thanks all for looking into it.
> > 
> > Turns out the rc2 relative updates fix this issue, so does
> > an old issue i reported a while ago:
> > multi-threads libvmmalloc fork test hang
> > https://lists.01.org/pipermail/linux-nvdimm/2016-October/007602.html
> > 
> > I'm able to reproduce these issues before rc2, now it
> > passes on current Linus tree:
> > c8b4ec8 Merge tag 'fscrypt-for-stable'
> 
> Hmm...I'm able to reproduce the other libvmmalloc issue with both v4.10-rc2
> and with "c8b4ec8 Merge tag 'fscrypt-for-stable'".  I'm debugging that issue
> today.
> 
> It's interesting that both tests started passing for you.  Did you change
> something in your test setup?

Nope.

[toc] | [prev] | [next] | [standalone]

#1550362

FromXiong Zhou <xzhou@redhat.com>
Date2017-01-04 02:50 +0100
Message-ID<sVISd-6cB-5@gated-at.bofh.it>
In reply to#1549955
On Tue, Jan 03, 2017 at 09:57:10AM -0700, Ross Zwisler wrote:
> On Tue, Jan 03, 2017 at 02:49:22PM +0800, Xiong Zhou wrote:
> > On Mon, Jan 02, 2017 at 02:49:41PM -0700, Ross Zwisler wrote:
> > > On Mon, Jan 02, 2017 at 06:16:17PM +0100, Jan Kara wrote:
> > > > On Fri 30-12-16 17:33:53, Xiong Zhou wrote:
> > > > > On Sat, Dec 24, 2016 at 07:07:14PM +0800, Xiong Zhou wrote:
> > > > > > Hi lists,
> > snip
> > > > I was trying to reproduce this but for me rwtest01 completes just fine on
> > > > dax mountpoint (I've used your reproducer). So can you sample several
> > > > kernel stack traces to get a rough idea where the kernel is running?
> > > > Thanks!
> > > > 
> > > > 								Honza
> > > 
> > > I'm also unable to reproduce this issue.  I've tried with both the blamed
> > > commit:
> > > 4b4bb46 (HEAD) dax: clear dirty entry tags on cache flush
> > > and with v4.9-rc2.  Both pass the test in my setup.
> > > Perhaps the variable is the size of your PMEM partitions?
> > > # fdisk -l /dev/pmem0
> > > Disk /dev/pmem0: 16 GiB, 17179869184 bytes, 33554432 sectors
> > > Units: sectors of 1 * 512 = 512 bytes
> > > Sector size (logical/physical): 512 bytes / 4096 bytes
> > > I/O size (minimum/optimal): 4096 bytes / 4096 bytes
> > > Disklabel type: dos
> > > Disk identifier: 0xfe50c900
> > > Device       Boot    Start      End  Sectors Size Id Type
> > > /dev/pmem0p1          4096 25165823 25161728  12G 83 Linux
> > > /dev/pmem0p2      25165824 33550335  8384512   4G 83 Linux
> > > 
> > > What does your setup look like?
> > > I'm using the current tip of the LTP tree:
> > > 8cc4165  waitid02: define _XOPEN_SOURCE 500
> > > Thanks,
> > > - Ross
> > 
> > Thanks all for looking into it.
> > 
> > Turns out the rc2 relative updates fix this issue, so does
> > an old issue i reported a while ago:
> > multi-threads libvmmalloc fork test hang
> > https://lists.01.org/pipermail/linux-nvdimm/2016-October/007602.html
> > 
> > I'm able to reproduce these issues before rc2, now it
> > passes on current Linus tree:
> > c8b4ec8 Merge tag 'fscrypt-for-stable'
> 
> Hmm...I'm able to reproduce the other libvmmalloc issue with both v4.10-rc2
> and with "c8b4ec8 Merge tag 'fscrypt-for-stable'".  I'm debugging that issue
> today.
> 
> It's interesting that both tests started passing for you.  Did you change
> something in your test setup?

Er.. After double-checking, I have reduced nvml check test time on
Dec 30,

-make -C src check -j $NR_CPU
+make -C src check -j $NR_CPU TEST_TIME=1m

Other then this, nothing changed but the kernel code, same machine,
same cmdline, same configs.

I'm going to dig this more, and test your patch.

Thanks for looking into this!
> --
> To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

[toc] | [prev] | [next] | [standalone]

#1550584

FromXiong Zhou <xzhou@redhat.com>
Date2017-01-04 10:50 +0100
Message-ID<sVQmJ-2QA-3@gated-at.bofh.it>
In reply to#1549955
On Tue, Jan 03, 2017 at 09:57:10AM -0700, Ross Zwisler wrote:
> On Tue, Jan 03, 2017 at 02:49:22PM +0800, Xiong Zhou wrote:
> > On Mon, Jan 02, 2017 at 02:49:41PM -0700, Ross Zwisler wrote:
> > > On Mon, Jan 02, 2017 at 06:16:17PM +0100, Jan Kara wrote:
> > > > On Fri 30-12-16 17:33:53, Xiong Zhou wrote:
> > > > > On Sat, Dec 24, 2016 at 07:07:14PM +0800, Xiong Zhou wrote:
> > > > > > Hi lists,
> > snip
> > > > I was trying to reproduce this but for me rwtest01 completes just fine on
> > > > dax mountpoint (I've used your reproducer). So can you sample several
> > > > kernel stack traces to get a rough idea where the kernel is running?
> > > > Thanks!
> > > > 
> > > > 								Honza
> > > 
> > > I'm also unable to reproduce this issue.  I've tried with both the blamed
> > > commit:
> > > 4b4bb46 (HEAD) dax: clear dirty entry tags on cache flush
> > > and with v4.9-rc2.  Both pass the test in my setup.
> > > Perhaps the variable is the size of your PMEM partitions?
> > > # fdisk -l /dev/pmem0
> > > Disk /dev/pmem0: 16 GiB, 17179869184 bytes, 33554432 sectors
> > > Units: sectors of 1 * 512 = 512 bytes
> > > Sector size (logical/physical): 512 bytes / 4096 bytes
> > > I/O size (minimum/optimal): 4096 bytes / 4096 bytes
> > > Disklabel type: dos
> > > Disk identifier: 0xfe50c900
> > > Device       Boot    Start      End  Sectors Size Id Type
> > > /dev/pmem0p1          4096 25165823 25161728  12G 83 Linux
> > > /dev/pmem0p2      25165824 33550335  8384512   4G 83 Linux
> > > 
> > > What does your setup look like?
> > > I'm using the current tip of the LTP tree:
> > > 8cc4165  waitid02: define _XOPEN_SOURCE 500
> > > Thanks,
> > > - Ross
> > 
> > Thanks all for looking into it.
> > 
> > Turns out the rc2 relative updates fix this issue, so does
> > an old issue i reported a while ago:
> > multi-threads libvmmalloc fork test hang
> > https://lists.01.org/pipermail/linux-nvdimm/2016-October/007602.html
> > 
> > I'm able to reproduce these issues before rc2, now it
> > passes on current Linus tree:
> > c8b4ec8 Merge tag 'fscrypt-for-stable'
> 
> Hmm...I'm able to reproduce the other libvmmalloc issue with both v4.10-rc2
> and with "c8b4ec8 Merge tag 'fscrypt-for-stable'".  I'm debugging that issue
> today.
> 
> It's interesting that both tests started passing for you.  Did you change
> something in your test setup?

Hi,

Quick update:
  Ross's new patch fixed the vmmaloc_fork issue, not the rc2 update.
  Regression tests is going on, so far so good.

I'm able to reproduce the vmmalloc_fork issue on rc2 kernel
	c8b4ec8 Merge tag 'fscrypt-for-stable'
with nvml commit to
	77c2a5a Merge pull request #1554 from krzycz/win-libvmem_rc

My previous statement about rc2 fixed old vmmalloc_fork issue
was wrong, my mistake. I have changed my test setup.

Now after some tests, Ross's patch
	[PATCH] dax: fix deadlock with DAX 4k holes
on top of Linus tree c8b4ec8 have fixed this vmmalloc_fork issue.
My DAX regression tests is going on, looks good so far. Gonna
update once it have finished.

> --
> To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

[toc] | [prev] | [next] | [standalone]

#1550983

FromRoss Zwisler <ross.zwisler@linux.intel.com>
Date2017-01-04 17:50 +0100
Message-ID<sVWVb-7cW-11@gated-at.bofh.it>
In reply to#1550584
On Wed, Jan 04, 2017 at 05:48:34PM +0800, Xiong Zhou wrote:
> On Tue, Jan 03, 2017 at 09:57:10AM -0700, Ross Zwisler wrote:
> > On Tue, Jan 03, 2017 at 02:49:22PM +0800, Xiong Zhou wrote:
> > > On Mon, Jan 02, 2017 at 02:49:41PM -0700, Ross Zwisler wrote:
> > > > On Mon, Jan 02, 2017 at 06:16:17PM +0100, Jan Kara wrote:
> > > > > On Fri 30-12-16 17:33:53, Xiong Zhou wrote:
> > > > > > On Sat, Dec 24, 2016 at 07:07:14PM +0800, Xiong Zhou wrote:
> > > > > > > Hi lists,
> > > snip
> > > > > I was trying to reproduce this but for me rwtest01 completes just fine on
> > > > > dax mountpoint (I've used your reproducer). So can you sample several
> > > > > kernel stack traces to get a rough idea where the kernel is running?
> > > > > Thanks!
> > > > > 
> > > > > 								Honza
> > > > 
> > > > I'm also unable to reproduce this issue.  I've tried with both the blamed
> > > > commit:
> > > > 4b4bb46 (HEAD) dax: clear dirty entry tags on cache flush
> > > > and with v4.9-rc2.  Both pass the test in my setup.
> > > > Perhaps the variable is the size of your PMEM partitions?
> > > > # fdisk -l /dev/pmem0
> > > > Disk /dev/pmem0: 16 GiB, 17179869184 bytes, 33554432 sectors
> > > > Units: sectors of 1 * 512 = 512 bytes
> > > > Sector size (logical/physical): 512 bytes / 4096 bytes
> > > > I/O size (minimum/optimal): 4096 bytes / 4096 bytes
> > > > Disklabel type: dos
> > > > Disk identifier: 0xfe50c900
> > > > Device       Boot    Start      End  Sectors Size Id Type
> > > > /dev/pmem0p1          4096 25165823 25161728  12G 83 Linux
> > > > /dev/pmem0p2      25165824 33550335  8384512   4G 83 Linux
> > > > 
> > > > What does your setup look like?
> > > > I'm using the current tip of the LTP tree:
> > > > 8cc4165  waitid02: define _XOPEN_SOURCE 500
> > > > Thanks,
> > > > - Ross
> > > 
> > > Thanks all for looking into it.
> > > 
> > > Turns out the rc2 relative updates fix this issue, so does
> > > an old issue i reported a while ago:
> > > multi-threads libvmmalloc fork test hang
> > > https://lists.01.org/pipermail/linux-nvdimm/2016-October/007602.html
> > > 
> > > I'm able to reproduce these issues before rc2, now it
> > > passes on current Linus tree:
> > > c8b4ec8 Merge tag 'fscrypt-for-stable'
> > 
> > Hmm...I'm able to reproduce the other libvmmalloc issue with both v4.10-rc2
> > and with "c8b4ec8 Merge tag 'fscrypt-for-stable'".  I'm debugging that issue
> > today.
> > 
> > It's interesting that both tests started passing for you.  Did you change
> > something in your test setup?
> 
> Hi,
> 
> Quick update:
>   Ross's new patch fixed the vmmaloc_fork issue, not the rc2 update.
>   Regression tests is going on, so far so good.
> 
> I'm able to reproduce the vmmalloc_fork issue on rc2 kernel
> 	c8b4ec8 Merge tag 'fscrypt-for-stable'
> with nvml commit to
> 	77c2a5a Merge pull request #1554 from krzycz/win-libvmem_rc
> 
> My previous statement about rc2 fixed old vmmalloc_fork issue
> was wrong, my mistake. I have changed my test setup.
> 
> Now after some tests, Ross's patch
> 	[PATCH] dax: fix deadlock with DAX 4k holes
> on top of Linus tree c8b4ec8 have fixed this vmmalloc_fork issue.
> My DAX regression tests is going on, looks good so far. Gonna
> update once it have finished.

Cool, thanks for the update.  If you're still able to reproduce this second
issue after my patch we can dig in to the differences between your test setup
and mine so I can reproduce it & debug.

Thanks for the reports!

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web