IO hang with kernel above 4.4.21, heavy IO, btrfs, xen

Path	csiph.com!news.mixmin.net!news.unit0.net!news.panservice.it!diesel.cu.mi.it!bofh.it!news.nic.it!robomod
From	Cheyenne Wills <cheyenne.wills@gmail.com>
Newsgroups	linux.kernel
Subject	IO hang with kernel above 4.4.21, heavy IO, btrfs, xen
Date	Wed, 22 Mar 2017 20:50:01 +0100
Message-ID	<tnUqB-7zp-3@gated-at.bofh.it> (permalink)
X-Original-To	linux-kernel@vger.kernel.org
Dkim-Signature	v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=+zgwxcVgYZVTWka1hEg31V+CIomK/wmCF7DC+3pQyFU=; b=rSivEpY4eJkTeaZFky3v5B7fRwaANJxk/zhbB97JDcCNXwtP4CGO7M3iwh3lqzN0dY mJguQMjPcJoNUyq0a++O/Lhn2YfaH664GbRqZy/0hXzWJfDQhW4CjxuS0Ow1axIReTqK LODWpcRcuW5u9Lv5DJhd6MVhvszX4Qf+5m4uaLZVzrDDc7PeyYCWIkFhaPGo48EM94XA HvEQbgSMBl9PJQmQx5gzaSwWgxCKd5UA8evEqzuNRskrg1FajwFO1cZ0aDiAYcOFjYQd l7SAmJ7qdHUFrTpZpA3l99xxbUny+8KWN5tiYdS3t3UYyGWUPFx8/eJAZGN1bzWMSDL8 yJfQ==
X-Google-Dkim-Signature	v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=+zgwxcVgYZVTWka1hEg31V+CIomK/wmCF7DC+3pQyFU=; b=APDknwuRt9n9xhbfgSO2UsA0eSVRdt9uvgm3zhccOYDl4L1fzIwgnUs9jauHhbhtyN IIIOTfRFgaJHYqHQo7CCpQHNxP3PKCIbf/B29KcWeB5w/qSQEaIfzwscusIQMXPAaKY8 ehRcexRGrCemoxqbW8cqenMAMEcBxsmuWApJMzx2JXDotjVT88yBxsjC9mcEyblz/9yM pw1mB+LMvJEj9T52zM3LeQoSbEMFD/lE3BuajVQMDWlo0nlBgfnaVQBxDXdQCmPsKzid ySxArFuYvFO5UZ8tCHgOtf2K175V2zZSifoTpeA12DEntLyb5+d3pBSaA1AcEb7MDEYW 5w+w==
X-Gm-Message-State	AFeK/H2kO/giceOgFuPxIvUmj+4PHty4WSjoAysmwO50GwL1z6ajzMkVbMcYGhpFw660lzbJDQSJJMvddDEl7Q==
X-Received	by 10.202.195.134 with SMTP id t128mr21100977oif.29.1490211755817; Wed, 22 Mar 2017 12:42:35 -0700 (PDT)
MIME-Version	1.0
Content-Type	text/plain; charset=UTF-8
Sender	robomod@news.nic.it
List-ID	<linux-kernel.vger.kernel.org>
X-Mailing-List	linux-kernel@vger.kernel.org
Approved	robomod@news.nic.it
Lines	32
Organization	linux.* mail to news gateway
X-Original-Date	Wed, 22 Mar 2017 13:42:35 -0600
X-Original-Message-ID	<CAHpsFVedoBmbm6SmLL7y9hhzPToSdraPiayMGc8cM9Vpt5HzHg@mail.gmail.com>
X-Original-Sender	linux-kernel-owner@vger.kernel.org
Xref	csiph.com linux.kernel:1606947

Show key headers only | View raw

I have a reproducible system hang that appears to be the result of heavy IO.

I originally posted the problem to the linux-block mailing list back
in January, but haven't heard back.

Not sure if I reported to the correct area.

There is a bugzill.kernel.org issue open that contains the details of
the problem  https://bugzilla.kernel.org/show_bug.cgi?id=193331

The problem occurs if there is a heavy IO load (the system in question
is a FTP server that locked up during a heavy load).  I am able to
reproduce the problem using a btrfs scrub to the file system (the
filesystem spans multiple devices).  I have a "test system" where I
can reproduce the problem at will (it takes only a few minutes to
cause the IO to hang up).

With a 4.0.5 kernel, I'm unable to reproduce the hang (and the system
in question runs just fine).  A kernel above 4.4.21 (maybe earlier?)
results in the hang described in the above bugzilla ticket.

I would appreciate any suggestions on performing further further
diagnostics, or if there is additional information needed to dig
deeper into the problem.

I'm not subscribed to the lkml, so a cc to cheyenne.wills@gmail.com
would be appreciated.

Thanks in advance.

Cheyenne Wills
cheyenne.wills@gmail.com

Back to linux.kernel | Previous | Next | Find similar | Unroll thread

Thread

IO hang with kernel above 4.4.21, heavy IO, btrfs, xen Cheyenne Wills <cheyenne.wills@gmail.com> - 2017-03-22 20:50 +0100

csiph-web