Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.kernel > #63444

Bug#908216: btrfs blocked for more than 120 seconds

From "Russell Mosemann" <rmosemann@futurefoam.com>
Newsgroups linux.debian.bugs.dist, linux.debian.kernel
Subject Bug#908216: btrfs blocked for more than 120 seconds
Date 2019-02-25 19:50 +0100
Message-ID <xvtu9-5wY-1@gated-at.bofh.it> (permalink)
References <wvon0-eY-13@gated-at.bofh.it> <xvctj-3lE-3@gated-at.bofh.it> <xvcWm-3Lg-7@gated-at.bofh.it> <wvon0-eY-13@gated-at.bofh.it> <xvcWm-3Lg-7@gated-at.bofh.it>
Organization linux.* mail to news gateway

Cross-posted to 2 groups.

Show all headers | View raw

[Multipart message — attachments visible in raw view] - view raw


Steps to reproduce

Simply copying a file into the file system can cause things to lock up. In this case, the files will usually be thin-provisioned qcow2 disks for kvm vm's. There is no detailed formula to force the lockup to occur, but it happens regularly, sometimes multiple times in one day.
 
Files are often copied from a master by reference (cp --reflink), one per day to perform a daily backup for up to 45 days. Removing older files is a painfully slow process, even though there are only 45 files in the directory. Doing a scrub is almost a sure way to lock up the system, especially if a copy or delete operation is in progress. On two systems, crashes occur with 4.18 and 4.19 but not 4.17. On the other systems that crash, it does not seem to matter if it is 4.17, 4.18 or 4.19.
 
Unless otherwise indicated
using qgroups:        No

using compression:    Yes, compress-force=zstd
number of snapshots:  Zero
number of subvolumes: top level subvolume only
raid profile:         None
using bcache:         No
layered on md or lvm: No
 
 
vhost002
# grep btrfs /etc/mtab
/dev/sdc1 /usr/local/data/datastore2 btrfs rw,noatime,compress-force=zstd,space_cache,subvolid=5,subvol=/ 0 0
 
# smartctl -l scterc /dev/sdc
smartctl 6.6 2016-05-31 r4324 [x86_64-linux-4.19.0-0.bpo.2-amd64] (local build)
Copyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org

SCT Error Recovery Control:
           Read: Disabled
          Write: Disabled

 
# btrfs dev stats /usr/local/data/datastore2
[/dev/sdc1].write_io_errs   0
[/dev/sdc1].read_io_errs    0
[/dev/sdc1].flush_io_errs   0
[/dev/sdc1].corruption_errs 0
[/dev/sdc1].generation_errs 0

 
 
vhost003
# grep btrfs /etc/mtab
/dev/sdb4 /usr/local/data/datastore2 btrfs rw,relatime,compress-force=zstd,space_cache,subvolid=5,subvol=/ 0 0

 
(RAID controller)
# smartctl -l scterc /dev/sdb
smartctl 6.6 2016-05-31 r4324 [x86_64-linux-4.17.0-0.bpo.1-amd64] (local build)
Copyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org


# btrfs dev stats /usr/local/data/datastore2
[/dev/sdb4].write_io_errs   0
[/dev/sdb4].read_io_errs    0
[/dev/sdb4].flush_io_errs   0
[/dev/sdb4].corruption_errs 0
[/dev/sdb4].generation_errs 0

 
 
vhost004
# grep btrfs /etc/mtab
/dev/sdb4 /usr/local/data/datastore2 btrfs rw,relatime,compress-force=zstd,space_cache,subvolid=5,subvol=/ 0 0

 
(RAID controller)
# smartctl -l scterc /dev/sdb
smartctl 6.6 2016-05-31 r4324 [x86_64-linux-4.17.0-0.bpo.1-amd64] (local build)
Copyright (C) 2002-16, Bruce Allen, Christian Franke, [ www.smartmontools.org ]( http://www.smartmontools.org )

 
# btrfs dev stats /usr/local/data/datastore2
[/dev/sdb4].write_io_errs   0
[/dev/sdb4].read_io_errs    0
[/dev/sdb4].flush_io_errs   0
[/dev/sdb4].corruption_errs 0
[/dev/sdb4].generation_errs 0

 
 
vhost031
# grep btrfs /etc/mtab
/dev/sdc1 /usr/local/data/datastore2 btrfs rw,relatime,compress-force=zstd,space_cache,subvolid=5,subvol=/ 0 0

 
# smartctl -l scterc /dev/sdc
smartctl 6.6 2016-05-31 r4324 [x86_64-linux-4.19.0-0.bpo.2-amd64] (local build)
Copyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org

SCT Error Recovery Control:
           Read: Disabled
          Write: Disabled

 
# btrfs dev stats /usr/local/data/datastore2
[/dev/sdc1].write_io_errs   0
[/dev/sdc1].read_io_errs    0
[/dev/sdc1].flush_io_errs   0
[/dev/sdc1].corruption_errs 0
[/dev/sdc1].generation_errs 0

 
 
vhost032
# grep btrfs /etc/mtab
/dev/sdc1 /usr/local/data/datastore2 btrfs rw,relatime,compress-force=zstd,space_cache,subvolid=5,subvol=/ 0 0

 
# smartctl -l scterc /dev/sdc
smartctl 6.6 2016-05-31 r4324 [x86_64-linux-4.19.0-0.bpo.2-amd64] (local build)
Copyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org

SCT Error Recovery Control:
           Read: Disabled
          Write: Disabled

 
# btrfs dev stats /usr/local/data/datastore2
[/dev/sdc1].write_io_errs   0
[/dev/sdc1].read_io_errs    0
[/dev/sdc1].flush_io_errs   0
[/dev/sdc1].corruption_errs 0
[/dev/sdc1].generation_errs 0

 
 
vhost182
# grep btrfs /etc/mtab
/dev/sdc1 /usr/local/data/datastore2 btrfs rw,noatime,compress-force=zstd,space_cache,subvolid=5,subvol=/ 0 0
 
# smartctl -l scterc /dev/sdc
smartctl 6.6 2016-05-31 r4324 [x86_64-linux-4.19.0-0.bpo.2-amd64] (local build)
Copyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org

SCT Error Recovery Control:
           Read: Disabled
          Write: Disabled
 
# btrfs dev stats /usr/local/data/datastore2
[/dev/sdc1].write_io_errs   0
[/dev/sdc1].read_io_errs    0
[/dev/sdc1].flush_io_errs   0
[/dev/sdc1].corruption_errs 0
[/dev/sdc1].generation_errs 0

 
 
vhost241
# grep btrfs /etc/mtab
/dev/sdc1 /usr/local/data/datastore2 btrfs rw,relatime,compress-force=zstd,space_cache,subvolid=5,subvol=/ 0 0

 
# smartctl -l scterc /dev/sdc
smartctl 6.6 2016-05-31 r4324 [x86_64-linux-4.17.0-0.bpo.1-amd64] (local build)
Copyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org

SCT Error Recovery Control:
           Read: Disabled
          Write: Disabled

 
# btrfs dev stats /usr/local/data/datastore2
[/dev/sdc1].write_io_errs   0
[/dev/sdc1].read_io_errs    0
[/dev/sdc1].flush_io_errs   0
[/dev/sdc1].corruption_errs 0
[/dev/sdc1].generation_errs 0

 
 
lxc008
number of subvolumes: 1416

 
# grep btrfs /etc/mtab
/dev/sdc1 /usr/local/data2 btrfs rw,noatime,compress-force=zstd,space_cache,subvolid=5,subvol=/ 0 0

 
(RAID controller)
# smartctl -d megaraid,0 -l scterc /dev/sdc
smartctl 6.6 2016-05-31 r4324 [x86_64-linux-4.19.0-0.bpo.2-amd64] (local build)
Copyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org

Write SCT (Get) Error Recovery Control Command failed: ATA return descriptor not supported by controller firmware
SCT (Get) Error Recovery Control command failed

 
# btrfs dev stats /usr/local/data2
[/dev/sdc1].write_io_errs   0
[/dev/sdc1].read_io_errs    0
[/dev/sdc1].flush_io_errs   0
[/dev/sdc1].corruption_errs 0
[/dev/sdc1].generation_errs 0

 
 
lxc009
# grep btrfs /etc/mtab
/dev/sda3 / btrfs rw,relatime,space_cache,subvolid=5,subvol=/ 0 0
/dev/sdb1 /usr/local/data2 btrfs rw,noatime,compress-force=zstd,space_cache,subvolid=5,subvol=/ 0 0

 
(RAID controller)
# smartctl -l scterc /dev/sdb
smartctl 6.6 2016-05-31 r4324 [x86_64-linux-4.19.0-0.bpo.2-amd64] (local build)
Copyright (C) 2002-16, Bruce Allen, Christian Franke, [ www.smartmontools.org ]( http://www.smartmontools.org )

 
# btrfs dev stats /usr/local/data2
[/dev/sdb1].write_io_errs   0
[/dev/sdb1].read_io_errs    0
[/dev/sdb1].flush_io_errs   0
[/dev/sdb1].corruption_errs 0
[/dev/sdb1].generation_errs 0

Back to linux.debian.kernel | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Bug#908216: btrfs blocked for more than 120 seconds "Russell Mosemann" <rmosemann@futurefoam.com> - 2019-02-25 01:40 +0100
  Bug#908216: btrfs blocked for more than 120 seconds Nicholas D Steeves <nsteeves@gmail.com> - 2019-02-25 02:10 +0100
    Bug#908216: btrfs blocked for more than 120 seconds "Russell Mosemann" <rmosemann@futurefoam.com> - 2019-02-25 19:50 +0100
      Bug#908216: btrfs blocked for more than 120 seconds Nicholas D Steeves <nsteeves@gmail.com> - 2019-02-26 05:30 +0100
        Bug#908216: btrfs blocked for more than 120 seconds "Russell Mosemann" <rmosemann@futurefoam.com> - 2019-02-27 13:30 +0100
          Bug#908216: btrfs blocked for more than 120 seconds "Russell Mosemann" <rmosemann@futurefoam.com> - 2019-02-27 17:30 +0100
            Bug#908216: btrfs blocked for more than 120 seconds "Russell Mosemann" <rmosemann@futurefoam.com> - 2019-03-02 13:00 +0100
        Bug#908216: btrfs blocked for more than 120 seconds Nicholas D Steeves <nsteeves@gmail.com> - 2019-03-02 23:10 +0100
          Bug#908216: btrfs blocked for more than 120 seconds "Russell Mosemann" <rmosemann@futurefoam.com> - 2019-03-03 00:20 +0100
            Bug#908216: btrfs blocked for more than 120 seconds Nicholas D Steeves <nsteeves@gmail.com> - 2019-03-03 02:10 +0100
          Bug#908216: btrfs blocked for more than 120 seconds "Russell Mosemann" <rmosemann@futurefoam.com> - 2019-03-06 15:00 +0100
            Bug#908216: btrfs blocked for more than 120 seconds Nicholas D Steeves <nsteeves@gmail.com> - 2019-03-09 01:30 +0100

csiph-web