Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1590943

Re: mm allocation failure and hang when running xfstests generic/269 on xfs

From Xiong Zhou <xzhou@redhat.com>
Newsgroups linux.kernel
Subject Re: mm allocation failure and hang when running xfstests generic/269 on xfs
Date 2017-03-02 10:30 +0100
Message-ID <tgvdE-fw-9@gated-at.bofh.it> (permalink)
References <tg4n7-66C-1@gated-at.bofh.it> <tgmWJ-2z2-3@gated-at.bofh.it> <tgrD3-64c-1@gated-at.bofh.it> <tgsIN-6Nv-7@gated-at.bofh.it> <tguAW-856-11@gated-at.bofh.it>
Organization linux.* mail to news gateway

Show all headers | View raw

On Thu, Mar 02, 2017 at 09:42:23AM +0100, Michal Hocko wrote:
> On Thu 02-03-17 12:17:47, Anshuman Khandual wrote:
> > On 03/02/2017 10:49 AM, Xiong Zhou wrote:
> > > On Wed, Mar 01, 2017 at 04:37:31PM -0800, Christoph Hellwig wrote:
> > >> On Wed, Mar 01, 2017 at 12:46:34PM +0800, Xiong Zhou wrote:
> > >>> Hi,
> > >>>
> > >>> It's reproduciable, not everytime though. Ext4 works fine.
> > >> On ext4 fsstress won't run bulkstat because it doesn't exist.  Either
> > >> way this smells like a MM issue to me as there were not XFS changes
> > >> in that area recently.
> > > Yap.
> > > 
> > > First bad commit:
> > > 
> > > commit 5d17a73a2ebeb8d1c6924b91e53ab2650fe86ffb
> > > Author: Michal Hocko <mhocko@suse.com>
> > > Date:   Fri Feb 24 14:58:53 2017 -0800
> > > 
> > >     vmalloc: back off when the current task is killed
> > > 
> > > Reverting this commit on top of
> > >   e5d56ef Merge tag 'watchdog-for-linus-v4.11'
> > > survives the tests.
> > 
> > Does fsstress test or the system hang ? I am not familiar with this
> > code but If it's the test which is getting hung and its hitting this
> > new check introduced by the above commit that means the requester is
> > currently being killed by OOM killer for some other memory allocation
> > request.
> 
> Well, not exactly. It is sufficient for it to be _killed_ by SIGKILL.
> And for that it just needs to do a group_exit when one thread was still
> in the kernel (see zap_process). While I can change this check to
> actually do the oom specific check I believe a more generic
> fatal_signal_pending is the right thing to do here. I am still not sure
> what is the actual problem here, though. Could you be more specific
> please?

It's blocking the test and system-shutdown. fsstress wont exit.

For anyone interested, a simple ugly reproducer:

cat > fst.sh <<EOFF
#! /bin/bash -x

[ \$# -ne 3 ] && { echo "./single FSTYP TEST XFSTESTS_DIR"; exit 1; }

FST=\$1
BLKSZ=4096

fallocate -l 10G /home/test.img
fallocate -l 15G /home/scratch.img

MNT1=/loopmnt
MNT2=/loopsch

mkdir -p \$MNT1
mkdir -p \$MNT2

DEV1=\$(losetup --find --show /home/test.img)
DEV2=\$(losetup --find --show /home/scratch.img)

cleanup()
{
	umount -d \$MNT1
	umount -d \$MNT2
	umount \$DEV1 \$DEV2
	losetup -D || losetup -a | awk -F: '{print \$1}' | xargs losetup -d
	rm -f /home/{test,scratch}.img
}

trap cleanup 0 1 2

if [[ \$FST =~ ext ]] ; then
	mkfs.\${FST} -Fq -b \${BLKSZ} \$DEV1
elif [[ \$FST =~ xfs ]] ; then
	mkfs.\${FST} -fq -b size=\${BLKSZ} \$DEV1
fi
if test \$? -ne 0 ; then
	echo "mkfs \$DEV1 failed" 
	exit 1
fi

if [[ \$FST =~ ext ]] ; then
	mkfs.\${FST} -Fq -b \${BLKSZ} \$DEV2
elif [[ \$FST =~ xfs ]] ; then
	mkfs.\${FST} -fq -b size=\${BLKSZ} \$DEV2
fi
if test \$? -ne 0 ; then
	echo "mkfs \$DEV2 failed"
	exit 1
fi

mount \$DEV1 \$MNT1
if test \$? -ne 0 ; then
	echo "mount \$DEV1 failed" 
	exit 1
fi
mount \$DEV2 \$MNT2
if test \$? -ne 0 ; then
	echo "mount \$DEV2 failed" 
	exit 1
fi

pushd \$3 || exit 1

cat > local.config <<EOF
TEST_DEV=\$DEV1
TEST_DIR=\$MNT1
SCRATCH_MNT=\$MNT2
SCRATCH_DEV=\$DEV2
EOF

i=0
while [ \$i -lt 50 ] ; do
	./check \$2
	echo \$i
	((i=\$i+1))
done

popd
EOFF

git clone git://git.kernel.org/pub/scm/fs/xfs/xfstests-dev.git
cd xfstests-dev
make -j2 && make install || exit 1
cd -
sh -x ./fst.sh xfs generic/269 xfstests-dev


> -- 
> Michal Hocko
> SUSE Labs

Back to linux.kernel | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Re: mm allocation failure and hang when running xfstests generic/269  on xfs Christoph Hellwig <hch@infradead.org> - 2017-03-02 01:40 +0100
  Re: mm allocation failure and hang when running xfstests generic/269  on xfs Xiong Zhou <xzhou@redhat.com> - 2017-03-02 06:40 +0100
    Re: mm allocation failure and hang when running xfstests generic/269  on xfs Anshuman Khandual <khandual@linux.vnet.ibm.com> - 2017-03-02 07:50 +0100
      Re: mm allocation failure and hang when running xfstests generic/269  on xfs Michal Hocko <mhocko@kernel.org> - 2017-03-02 09:50 +0100
        Re: mm allocation failure and hang when running xfstests generic/269  on xfs Xiong Zhou <xzhou@redhat.com> - 2017-03-02 10:30 +0100
    Re: mm allocation failure and hang when running xfstests generic/269  on xfs Bob Liu <liubo95@huawei.com> - 2017-03-02 07:50 +0100
    Re: mm allocation failure and hang when running xfstests generic/269on xfs Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp> - 2017-03-02 12:50 +0100
    Re: mm allocation failure and hang when running xfstests generic/269  on xfs Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp> - 2017-03-02 13:00 +0100
      Re: mm allocation failure and hang when running xfstests generic/269  on xfs Michal Hocko <mhocko@kernel.org> - 2017-03-02 13:30 +0100
        Re: mm allocation failure and hang when running xfstests generic/269  on xfs Brian Foster <bfoster@redhat.com> - 2017-03-02 13:30 +0100
          Re: mm allocation failure and hang when running xfstests generic/269  on xfs Michal Hocko <mhocko@kernel.org> - 2017-03-02 14:10 +0100
            Re: mm allocation failure and hang when running xfstests generic/269  on xfs Brian Foster <bfoster@redhat.com> - 2017-03-02 15:00 +0100
              Re: mm allocation failure and hang when running xfstests generic/269  on xfs Michal Hocko <mhocko@kernel.org> - 2017-03-02 15:00 +0100
                Re: mm allocation failure and hang when running xfstests generic/269  on xfs Brian Foster <bfoster@redhat.com> - 2017-03-02 16:00 +0100
                Re: mm allocation failure and hang when running xfstests generic/269  on xfs Brian Foster <bfoster@redhat.com> - 2017-03-02 16:40 +0100
                [PATCH 1/2] xfs: allow kmem_zalloc_greedy to fail Michal Hocko <mhocko@kernel.org> - 2017-03-02 16:50 +0100
                [PATCH 2/2] xfs: back off from kmem_zalloc_greedy if the task is killed Michal Hocko <mhocko@kernel.org> - 2017-03-02 16:50 +0100
                Re: [PATCH 2/2] xfs: back off from kmem_zalloc_greedy if the task         is killed Christoph Hellwig <hch@lst.de> - 2017-03-02 17:00 +0100
                Re: [PATCH 2/2] xfs: back off from kmem_zalloc_greedy if the task is  killed Brian Foster <bfoster@redhat.com> - 2017-03-02 17:10 +0100
                Re: [PATCH 1/2] xfs: allow kmem_zalloc_greedy to fail Brian Foster <bfoster@redhat.com> - 2017-03-02 17:10 +0100
                Re: [PATCH 1/2] xfs: allow kmem_zalloc_greedy to fail Christoph Hellwig <hch@lst.de> - 2017-03-02 17:20 +0100
                Re: [PATCH 1/2] xfs: allow kmem_zalloc_greedy to fail Michal Hocko <mhocko@kernel.org> - 2017-03-02 17:40 +0100
                Re: [PATCH 1/2] xfs: allow kmem_zalloc_greedy to fail "Darrick J. Wong" <darrick.wong@oracle.com> - 2017-03-02 17:50 +0100
                Re: [PATCH 1/2] xfs: allow kmem_zalloc_greedy to fail Dave Chinner <david@fromorbit.com> - 2017-03-04 00:00 +0100
                Re: [PATCH 1/2] xfs: allow kmem_zalloc_greedy to fail "Darrick J. Wong" <darrick.wong@oracle.com> - 2017-03-04 00:30 +0100
                Re: [PATCH 1/2] xfs: allow kmem_zalloc_greedy to fail Dave Chinner <david@fromorbit.com> - 2017-03-04 05:50 +0100
                Re: [PATCH 1/2] xfs: allow kmem_zalloc_greedy to fail Michal Hocko <mhocko@kernel.org> - 2017-03-06 14:30 +0100
                Re: mm allocation failure and hang when running xfstests generic/269  on xfs Michal Hocko <mhocko@kernel.org> - 2017-03-02 20:20 +0100
                Re: mm allocation failure and hang when running xfstests generic/269  on xfs Christoph Hellwig <hch@infradead.org> - 2017-03-02 19:00 +0100
                Re: mm allocation failure and hang when running xfstests generic/269  on xfs Michal Hocko <mhocko@kernel.org> - 2017-03-02 20:20 +0100
                Re: mm allocation failure and hang when running xfstests generic/269  on xfs Brian Foster <bfoster@redhat.com> - 2017-03-02 16:00 +0100
                Re: mm allocation failure and hang when running xfstests generic/269  on xfs Michal Hocko <mhocko@kernel.org> - 2017-03-02 18:40 +0100
              Re: mm allocation failure and hang when running xfstests generic/269  on xfs Brian Foster <bfoster@redhat.com> - 2017-03-02 15:00 +0100
              Re: mm allocation failure and hang when running xfstests generic/269  on xfs Michal Hocko <mhocko@kernel.org> - 2017-03-02 15:10 +0100
              Re: mm allocation failure and hang when running xfstests generic/269 on xfs Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp> - 2017-03-02 17:00 +0100

csiph-web