Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > linux.debian.kernel > #73485 > unrolled thread
| Started by | Daniel Black <daniel@mariadb.org> |
|---|---|
| First post | 2021-10-21 11:50 +0200 |
| Last post | 2021-11-15 11:10 +0100 |
| Articles | 10 — 4 participants |
Back to article view | Back to linux.debian.kernel
Bug#996951: linux-image-5.14.0-3-amd64: iouring looses requests Daniel Black <daniel@mariadb.org> - 2021-10-21 11:50 +0200
Bug#996951: linux-image-5.14.0-3-amd64: iouring looses requests Salvatore Bonaccorso <carnil@debian.org> - 2021-10-21 12:50 +0200
Bug#996951: linux-image-5.14.0-3-amd64: iouring looses requests Diederik de Haas <didi.debian@cknow.org> - 2021-10-21 13:20 +0200
Bug#996951: linux-image-5.14.0-3-amd64: iouring looses requests Daniel Black <daniel@mariadb.org> - 2021-10-21 14:10 +0200
Processed: Re: Bug#996951: linux-image-5.14.0-3-amd64: iouring looses requests "Debian Bug Tracking System" <owner@bugs.debian.org> - 2021-10-21 12:50 +0200
Processed: Re: Bug#996951: linux-image-5.14.0-3-amd64: iouring looses requests "Debian Bug Tracking System" <owner@bugs.debian.org> - 2021-10-23 09:40 +0200
Bug#996951: linux-image-5.14.0-3-amd64: iouring looses requests Diederik de Haas <didi.debian@cknow.org> - 2021-10-24 00:40 +0200
Processed: Re: Bug#996951: linux-image-5.14.0-3-amd64: iouring looses requests "Debian Bug Tracking System" <owner@bugs.debian.org> - 2021-10-24 00:40 +0200
Bug#996951: Daniel Black <daniel@mariadb.org> - 2021-11-07 23:50 +0100
Bug#996951: marked as done (linux-image-5.14.0-3-amd64: iouring looses requests) "Debian Bug Tracking System" <owner@bugs.debian.org> - 2021-11-15 11:10 +0100
| From | Daniel Black <daniel@mariadb.org> |
|---|---|
| Date | 2021-10-21 11:50 +0200 |
| Subject | Bug#996951: linux-image-5.14.0-3-amd64: iouring looses requests |
| Message-ID | <DaCRX-gy-1@gated-at.bofh.it> |
Package: src:linux
Version: 5.14.12-1
Severity: grave
Justification: causes non-serious data loss
X-Debbugs-Cc: daniel@mariadb.org
Dear Maintainer,
MariaDB has been investigating a 10.6+ related problem for a while
https://jira.mariadb.org/browse/MDEV-26674
https://jira.mariadb.org/browse/MDEV-26555
The result of this investigation is that between 5.11 and fixed in 5.15
is a uring kernel related fault that results in a write request getting
lost.
The result of this is that MariaDB-10.6 users, and perhaps other
applications using the iouring kernel interface will loose either
availablity or data.
The good news is I've validated that the linux mainline 5.14.14 build
from https://kernel.ubuntu.com/~kernel-ppa/mainline/v5.14.14/ has
actually fixed this problem.
As such this affects bullseye-backports, bookworm and side currently.
This can be validated by installing mariadb-test-10.6 from MariaDB's
repo.
https://mariadb.org/download/#mariadb-repositories
To test run:
cd /usr/share/mysql/mysql-test
./mtr --vardir=/tmp/var --parallel=4 encryption.innochecksum{,,,,,}
./mtr --vardir=/tmp/var --parallel=4 stress.ddl_innodb stress.ddl_innodb stress.ddl_innodb stress.ddl_innodb
A test failure (after a large timeout 10 mins I think) results in the
mariadb error:
2021-10-21 9:08:43 0 [ERROR] [FATAL] InnoDB: innodb_fatal_semaphore_wait_threshold was exceeded for dict_sys.latch. Please refer to https://mariadb.com/kb/en/how-to-produce-a-full-stack-trace-for-mysqld/
Marko (MariaDB innodb maintainer) in https://jira.mariadb.org/browse/MDEV-26674?focusedCommentId=202674&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-202674
already validated the problem on sid.
This is reported on an ubuntu impish machine with the Debian kernel
installed to eliminate any other userspace effects that may have caused
this.
-- Package-specific info:
** Version:
Linux version 5.14.0-3-amd64 (debian-kernel@lists.debian.org) (gcc-10 (Debian 10.3.0-11) 10.3.0, GNU ld (GNU Binutils for Debian) 2.37) #1 SMP Debian 5.14.12-1 (2021-10-14)
** Command line:
BOOT_IMAGE=/vmlinuz-5.14.0-3-amd64 root=/dev/mapper/ubuntu--vg-ubuntu--lv ro
** Not tainted
** Kernel log:
[ 1.866883] ata4: SATA link down (SStatus 0 SControl 300)
[ 1.869434] scsi 0:0:0:0: CD-ROM QEMU QEMU DVD-ROM 2.5+ PQ: 0 ANSI: 5
[ 1.878673] ata6: SATA link down (SStatus 0 SControl 300)
[ 1.880335] ata5: SATA link down (SStatus 0 SControl 300)
[ 1.915393] sr 0:0:0:0: [sr0] scsi3-mmc drive: 4x/4x cd/rw xa/form2 tray
[ 1.916496] cdrom: Uniform CD-ROM driver Revision: 3.20
[ 1.958195] sr 0:0:0:0: Attached scsi CD-ROM sr0
[ 2.217095] input: ImExPS/2 Generic Explorer Mouse as /devices/platform/i8042/serio1/input/input3
[ 2.225457] pcieport 0000:00:02.6: pciehp: Slot(0-6): No device found
[ 2.525250] raid6: avx2x4 gen() 29809 MB/s
[ 2.593249] raid6: avx2x4 xor() 13304 MB/s
[ 2.661249] raid6: avx2x2 gen() 34952 MB/s
[ 2.729248] raid6: avx2x2 xor() 20847 MB/s
[ 2.797248] raid6: avx2x1 gen() 29982 MB/s
[ 2.865248] raid6: avx2x1 xor() 16322 MB/s
[ 2.933259] raid6: sse2x4 gen() 14256 MB/s
[ 3.001247] raid6: sse2x4 xor() 8371 MB/s
[ 3.069248] raid6: sse2x2 gen() 14546 MB/s
[ 3.137248] raid6: sse2x2 xor() 8631 MB/s
[ 3.205249] raid6: sse2x1 gen() 12159 MB/s
[ 3.273248] raid6: sse2x1 xor() 6207 MB/s
[ 3.273649] raid6: using algorithm avx2x2 gen() 34952 MB/s
[ 3.274147] raid6: .... xor() 20847 MB/s, rmw enabled
[ 3.274552] raid6: using avx2x2 recovery algorithm
[ 3.278389] xor: automatically using best checksumming function avx
[ 3.282251] async_tx: api initialized (async)
[ 3.433164] Btrfs loaded, crc32c=crc32c-intel, zoned=yes
[ 3.667862] EXT4-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null). Quota mode: none.
[ 3.759716] Not activating Mandatory Access Control as /sbin/tomoyo-init does not exist.
[ 3.853765] systemd[1]: Inserted module 'autofs4'
[ 3.893289] systemd[1]: systemd 248.3-1ubuntu8 running in system mode. (+PAM +AUDIT +SELINUX +APPARMOR +IMA +SMACK +SECCOMP +GCRYPT +GNUTLS -OPENSSL +ACL +BLKID +CURL +ELFUTILS -FIDO2 +IDN2 -IDN +IPTC +KMOD +LIBCRYPTSETUP -LIBFDISK +PCRE2 -PWQUALITY -P11KIT -QRENCODE +BZIP2 +LZ4 +XZ +ZLIB +ZSTD -XKBCOMMON +UTMP +SYSVINIT default-hierarchy=unified)
[ 3.895610] systemd[1]: Detected virtualization kvm.
[ 3.896183] systemd[1]: Detected architecture x86-64.
[ 3.907380] systemd[1]: Hostname set to <impish>.
[ 4.275407] systemd[1]: Queued start job for default target Graphical Interface.
[ 4.278640] systemd[1]: Created slice system-modprobe.slice.
[ 4.284509] systemd[1]: Created slice system-systemd\x2dfsck.slice.
[ 4.287754] systemd[1]: Created slice User and Session Slice.
[ 4.289457] systemd[1]: Started Forward Password Requests to Wall Directory Watch.
[ 4.291130] systemd[1]: Set up automount Arbitrary Executable File Formats File System Automount Point.
[ 4.293130] systemd[1]: Reached target Slices.
[ 4.295232] systemd[1]: Reached target Local Verity Integrity Protected Volumes.
[ 4.297851] systemd[1]: Listening on Device-mapper event daemon FIFOs.
[ 4.299603] systemd[1]: Listening on LVM2 poll daemon socket.
[ 4.301182] systemd[1]: Listening on multipathd control socket.
[ 4.303065] systemd[1]: Listening on fsck to fsckd communication Socket.
[ 4.304524] systemd[1]: Listening on initctl Compatibility Named Pipe.
[ 4.306292] systemd[1]: Listening on Journal Audit Socket.
[ 4.307941] systemd[1]: Listening on Journal Socket (/dev/log).
[ 4.310054] systemd[1]: Listening on Journal Socket.
[ 4.311913] systemd[1]: Listening on Network Service Netlink Socket.
[ 4.313937] systemd[1]: Listening on udev Control Socket.
[ 4.315747] systemd[1]: Listening on udev Kernel Socket.
[ 4.318290] systemd[1]: Mounting Huge Pages File System...
[ 4.320763] systemd[1]: Mounting POSIX Message Queue File System...
[ 4.323184] systemd[1]: Mounting Kernel Debug File System...
[ 4.328055] systemd[1]: Mounting Kernel Trace File System...
[ 4.332963] systemd[1]: Starting Journal Service...
[ 4.335422] systemd[1]: Starting Set the console keyboard layout...
[ 4.337771] systemd[1]: Starting Create list of static device nodes for the current kernel...
[ 4.340979] systemd[1]: Starting Monitoring of LVM2 mirrors, snapshots etc. using dmeventd or progress polling...
[ 4.346661] systemd[1]: Starting Load Kernel Module configfs...
[ 4.369499] systemd[1]: Starting Load Kernel Module drm...
[ 4.377752] systemd[1]: Starting Load Kernel Module fuse...
[ 4.379529] systemd[1]: Condition check resulted in OpenVSwitch configuration for cleanup being skipped.
[ 4.383612] systemd[1]: Condition check resulted in Set Up Additional Binary Formats being skipped.
[ 4.385537] systemd[1]: Condition check resulted in File System Check on Root Device being skipped.
[ 4.390332] systemd[1]: Starting Load Kernel Modules...
[ 4.395567] systemd[1]: Starting Remount Root and Kernel File Systems...
[ 4.401664] systemd[1]: Starting Coldplug All udev Devices...
[ 4.408033] EXT4-fs (dm-0): re-mounted. Opts: (null). Quota mode: none.
[ 4.413624] fuse: init (API version 7.34)
[ 4.415296] systemd[1]: Started Journal Service.
[ 4.476521] Adding 4019196k swap on /swap.img. Priority:-2 extents:7 across:4846588k FS
[ 4.498562] systemd-journald[375]: Received client request to flush runtime journal.
[ 4.779184] sr 0:0:0:0: Attached scsi generic sg0 type 5
[ 4.905351] RAPL PMU: API unit is 2^-32 Joules, 0 fixed counters, 10737418240 ms ovfl timer
[ 5.043460] snd_hda_codec_generic hdaudioC0D0: autoconfig for Generic: line_outs=1 (0x3/0x0/0x0/0x0/0x0) type:line
[ 5.043465] snd_hda_codec_generic hdaudioC0D0: speaker_outs=0 (0x0/0x0/0x0/0x0/0x0)
[ 5.043467] snd_hda_codec_generic hdaudioC0D0: hp_outs=0 (0x0/0x0/0x0/0x0/0x0)
[ 5.043468] snd_hda_codec_generic hdaudioC0D0: mono: mono_out=0x0
[ 5.043469] snd_hda_codec_generic hdaudioC0D0: inputs:
[ 5.043470] snd_hda_codec_generic hdaudioC0D0: Line=0x5
[ 5.135361] intel_pmc_core intel_pmc_core.0: initialized
[ 5.256674] alua: device handler registered
[ 5.259674] emc: device handler registered
[ 5.262224] rdac: device handler registered
[ 5.364044] EXT4-fs (vda2): mounted filesystem with ordered data mode. Opts: (null). Quota mode: none.
[ 5.450722] audit: type=1400 audit(1634806008.296:2): apparmor="STATUS" operation="profile_load" profile="unconfined" name="lsb_release" pid=581 comm="apparmor_parser"
[ 5.451038] audit: type=1400 audit(1634806008.296:3): apparmor="STATUS" operation="profile_load" profile="unconfined" name="nvidia_modprobe" pid=582 comm="apparmor_parser"
[ 5.451042] audit: type=1400 audit(1634806008.296:4): apparmor="STATUS" operation="profile_load" profile="unconfined" name="nvidia_modprobe//kmod" pid=582 comm="apparmor_parser"
[ 5.468431] audit: type=1400 audit(1634806008.312:5): apparmor="STATUS" operation="profile_load" profile="unconfined" name="/usr/lib/snapd/snap-confine" pid=586 comm="apparmor_parser"
[ 5.468444] audit: type=1400 audit(1634806008.312:6): apparmor="STATUS" operation="profile_load" profile="unconfined" name="/usr/lib/snapd/snap-confine//mount-namespace-capture-helper" pid=586 comm="apparmor_parser"
[ 5.481385] audit: type=1400 audit(1634806008.328:7): apparmor="STATUS" operation="profile_load" profile="unconfined" name="/usr/lib/NetworkManager/nm-dhcp-client.action" pid=585 comm="apparmor_parser"
[ 5.481391] audit: type=1400 audit(1634806008.328:8): apparmor="STATUS" operation="profile_load" profile="unconfined" name="/usr/lib/NetworkManager/nm-dhcp-helper" pid=585 comm="apparmor_parser"
[ 5.481393] audit: type=1400 audit(1634806008.328:9): apparmor="STATUS" operation="profile_load" profile="unconfined" name="/usr/lib/connman/scripts/dhclient-script" pid=585 comm="apparmor_parser"
[ 5.481395] audit: type=1400 audit(1634806008.328:10): apparmor="STATUS" operation="profile_load" profile="unconfined" name="/{,usr/}sbin/dhclient" pid=585 comm="apparmor_parser"
[ 8.437751] loop: module loaded
[ 8.438109] loop0: detected capacity change from 0 to 8
[ 8.458870] squashfs: version 4.0 (2009/01/31) Phillip Lougher
** Model information
sys_vendor: QEMU
product_name: Standard PC (Q35 + ICH9, 2009)
product_version: pc-q35-5.2
chassis_vendor: QEMU
chassis_version: pc-q35-5.2
bios_vendor: SeaBIOS
bios_version: 1.14.0-4.fc34
** Loaded modules:
squashfs
loop
dm_multipath
scsi_dh_rdac
scsi_dh_emc
scsi_dh_alua
intel_rapl_msr
intel_rapl_common
intel_pmc_core_pltdrv
intel_pmc_core
kvm_intel
snd_hda_codec_generic
ledtrig_audio
kvm
snd_hda_intel
irqbypass
snd_intel_dspcfg
snd_intel_sdw_acpi
rapl
snd_hda_codec
snd_hda_core
snd_hwdep
snd_pcm
snd_timer
serio_raw
snd
virtio_balloon
sg
soundcore
virtio_console
joydev
evdev
qemu_fw_cfg
sch_fq_codel
fuse
configfs
virtio_rng
rng_core
ip_tables
x_tables
autofs4
ext4
crc16
mbcache
jbd2
btrfs
blake2b_generic
zstd_compress
raid10
raid456
async_raid6_recov
async_memcpy
async_pq
async_xor
async_tx
xor
raid6_pq
libcrc32c
crc32c_generic
raid1
raid0
multipath
linear
md_mod
sr_mod
cdrom
hid_generic
usbhid
hid
crct10dif_pclmul
crct10dif_common
crc32_pclmul
crc32c_intel
ghash_clmulni_intel
dm_mod
qxl
drm_ttm_helper
ttm
drm_kms_helper
ahci
libahci
virtio_net
virtio_blk
net_failover
failover
cec
rc_core
libata
drm
aesni_intel
crypto_simd
scsi_mod
psmouse
cryptd
xhci_pci
xhci_hcd
usbcore
i2c_i801
i2c_smbus
lpc_ich
usb_common
virtio_pci
virtio_pci_modern_dev
virtio_ring
virtio
button
** PCI devices:
not available
** USB devices:
not available
-- System Information:
Distributor ID: Ubuntu
Description: Ubuntu 21.10
Release: 21.10
Codename: impish
Architecture: amd64 (x86_64)
Kernel: Linux 5.14.0-3-amd64 (SMP w/2 CPU threads)
Locale: LANG=C.UTF-8, LC_CTYPE=C.UTF-8 (charmap=UTF-8), LANGUAGE not set
Shell: /bin/sh linked to /usr/bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled
Versions of packages linux-image-5.14.0-3-amd64 depends on:
ii initramfs-tools [linux-initramfs-tool] 0.140ubuntu6
ii kmod 28-1ubuntu4
ii linux-base 4.5ubuntu9
Versions of packages linux-image-5.14.0-3-amd64 recommends:
ii apparmor 3.0.3-0ubuntu1
pn firmware-linux-free <none>
Versions of packages linux-image-5.14.0-3-amd64 suggests:
pn debian-kernel-handbook <none>
ii grub-pc 2.04-1ubuntu47
pn linux-doc-5.14 <none>
Versions of packages linux-image-5.14.0-3-amd64 is related to:
pn firmware-amd-graphics <none>
pn firmware-atheros <none>
pn firmware-bnx2 <none>
pn firmware-bnx2x <none>
pn firmware-brcm80211 <none>
pn firmware-cavium <none>
pn firmware-intel-sound <none>
pn firmware-intelwimax <none>
pn firmware-ipw2x00 <none>
pn firmware-ivtv <none>
pn firmware-iwlwifi <none>
pn firmware-libertas <none>
pn firmware-linux-nonfree <none>
pn firmware-misc-nonfree <none>
pn firmware-myricom <none>
pn firmware-netxen <none>
pn firmware-qlogic <none>
pn firmware-realtek <none>
pn firmware-samsung <none>
pn firmware-siano <none>
pn firmware-ti-connectivity <none>
pn xen-hypervisor <none>
-- no debconf information
[toc] | [next] | [standalone]
| From | Salvatore Bonaccorso <carnil@debian.org> |
|---|---|
| Date | 2021-10-21 12:50 +0200 |
| Message-ID | <DaDO1-Pc-1@gated-at.bofh.it> |
| In reply to | #73485 |
Control: tags -1 + moreinfo
Hi Daniel,
On Thu, Oct 21, 2021 at 09:38:00AM +0000, Daniel Black wrote:
> Package: src:linux
> Version: 5.14.12-1
> Severity: grave
> Justification: causes non-serious data loss
> X-Debbugs-Cc: daniel@mariadb.org
>
> Dear Maintainer,
>
> MariaDB has been investigating a 10.6+ related problem for a while
> https://jira.mariadb.org/browse/MDEV-26674
> https://jira.mariadb.org/browse/MDEV-26555
>
> The result of this investigation is that between 5.11 and fixed in 5.15
> is a uring kernel related fault that results in a write request getting
> lost.
>
> The result of this is that MariaDB-10.6 users, and perhaps other
> applications using the iouring kernel interface will loose either
> availablity or data.
>
> The good news is I've validated that the linux mainline 5.14.14 build
> from https://kernel.ubuntu.com/~kernel-ppa/mainline/v5.14.14/ has
> actually fixed this problem.
>
> As such this affects bullseye-backports, bookworm and side currently.
>
> This can be validated by installing mariadb-test-10.6 from MariaDB's
> repo.
> https://mariadb.org/download/#mariadb-repositories
>
> To test run:
>
> cd /usr/share/mysql/mysql-test
> ./mtr --vardir=/tmp/var --parallel=4 encryption.innochecksum{,,,,,}
> ./mtr --vardir=/tmp/var --parallel=4 stress.ddl_innodb stress.ddl_innodb stress.ddl_innodb stress.ddl_innodb
>
> A test failure (after a large timeout 10 mins I think) results in the
> mariadb error:
>
> 2021-10-21 9:08:43 0 [ERROR] [FATAL] InnoDB: innodb_fatal_semaphore_wait_threshold was exceeded for dict_sys.latch. Please refer to https://mariadb.com/kb/en/how-to-produce-a-full-stack-trace-for-mysqld/
>
> Marko (MariaDB innodb maintainer) in https://jira.mariadb.org/browse/MDEV-26674?focusedCommentId=202674&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-202674
> already validated the problem on sid.
>
> This is reported on an ubuntu impish machine with the Debian kernel
> installed to eliminate any other userspace effects that may have caused
> this.
Where you able to isolate the upstream change landed upstream which
fixes the issue?
Regards,
Salvatore
[toc] | [prev] | [next] | [standalone]
| From | Diederik de Haas <didi.debian@cknow.org> |
|---|---|
| Date | 2021-10-21 13:20 +0200 |
| Message-ID | <DaEh3-1e9-3@gated-at.bofh.it> |
| In reply to | #73486 |
[Multipart message — attachments visible in raw view] — view raw
On Thursday, 21 October 2021 12:41:34 CEST Salvatore Bonaccorso wrote: > On Thu, Oct 21, 2021 at 09:38:00AM +0000, Daniel Black wrote: > > Package: src:linux > > Version: 5.14.12-1 > > The good news is I've validated that the linux mainline 5.14.14 build > > from https://kernel.ubuntu.com/~kernel-ppa/mainline/v5.14.14/ has > > actually fixed this problem. > > Where you able to isolate the upstream change landed upstream which > fixes the issue? There has been 1 change in fs/io_uring.c that is part of 5.14.14 (and 5.14.13), but not 5.14.12 and that is this commit: e798dcd960a3f0d40302a1bbf7f755879195a163 "io_uring: kill fasync" Possibly the simplest way to verify whether it's indeed that commit that fixes the issue is to apply the patch to Debian's 5.14.12 kernel. Instructions here (also follow 4.2.1): https://kernel-team.pages.debian.net/kernel-handbook/ch-common-tasks.html#s4.2.2 This would narrow it down to this specific commit and would also rule out differences in Ubuntu's kernel config and Debian's. Cheers, Diederik
[toc] | [prev] | [next] | [standalone]
| From | Daniel Black <daniel@mariadb.org> |
|---|---|
| Date | 2021-10-21 14:10 +0200 |
| Message-ID | <DaF3s-1IM-3@gated-at.bofh.it> |
| In reply to | #73488 |
retested 5.14.14. Sorry not fixed there yet. Guess this is going upstream.
few other fixes to try:
https://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux-block.git/log/?h=io_uring-5.15
$ uname -a
Linux impish 5.14.14-051414-generic #202110201037 SMP Wed Oct 20
11:04:11 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
dan@impish:/usr/share/mysql/mysql-test$ ./mtr --vardir=/tmp/var
--parallel=4 stress.ddl_innodb stress.ddl_innodb stress.ddl_innodb
stress.ddl_innodb
Logging: ./mtr --vardir=/tmp/var --parallel=4 stress.ddl_innodb
stress.ddl_innodb stress.ddl_innodb stress.ddl_innodb
vardir: /tmp/var
Removing old var directory...
Creating var directory '/tmp/var'...
Checking supported features...
MariaDB Version 10.6.5-MariaDB-1:10.6.5+maria~impish
- SSL connections supported
- binaries built with wsrep patch
Collecting tests...
Installing system database...
==============================================================================
TEST WORKER RESULT TIME (ms) or COMMENT
--------------------------------------------------------------------------
worker[1] Using MTR_BUILD_THREAD 300, with reserved ports 16000..16019
worker[4] Using MTR_BUILD_THREAD 301, with reserved ports 16020..16039
worker[3] Using MTR_BUILD_THREAD 302, with reserved ports 16040..16059
worker[2] Using MTR_BUILD_THREAD 303, with reserved ports 16060..16079
stress.ddl_innodb 'innodb' w4 [ pass ] 235134
stress.ddl_innodb 'innodb' w1 [ pass ] 245906
stress.ddl_innodb 'innodb' w3 [ pass ] 301078
stress.ddl_innodb 'innodb' w2 [ fail ]
Test ended at 2021-10-21 11:23:30
......
Version: '10.6.5-MariaDB-1:10.6.5+maria~impish-log' socket:
'/tmp/var/tmp/2/mysqld.1.sock' port: 16060 mariadb.org binary
distribution
2021-10-21 11:09:42 0 [Note] InnoDB: Buffer pool(s) load completed at
211021 11:09:42
2021-10-21 11:23:27 0 [ERROR] [FATAL] InnoDB:
innodb_fatal_semaphore_wait_threshold was exceeded for dict_sys.latch.
Please refer to
https://mariadb.com/kb/en/how-to-produce-a-full-stack-trace-for-mysqld/
211021 11:23:27 [ERROR] mysqld got signal 6 ;
On Thu, Oct 21, 2021 at 10:10 PM Diederik de Haas <didi.debian@cknow.org> wrote:
>
> On Thursday, 21 October 2021 12:41:34 CEST Salvatore Bonaccorso wrote:
> > On Thu, Oct 21, 2021 at 09:38:00AM +0000, Daniel Black wrote:
> > > Package: src:linux
> > > Version: 5.14.12-1
> > > The good news is I've validated that the linux mainline 5.14.14 build
> > > from https://kernel.ubuntu.com/~kernel-ppa/mainline/v5.14.14/ has
> > > actually fixed this problem.
> >
> > Where you able to isolate the upstream change landed upstream which
> > fixes the issue?
>
> There has been 1 change in fs/io_uring.c that is part of 5.14.14 (and 5.14.13),
> but not 5.14.12 and that is this commit:
> e798dcd960a3f0d40302a1bbf7f755879195a163 "io_uring: kill fasync"
>
> Possibly the simplest way to verify whether it's indeed that commit that fixes
> the issue is to apply the patch to Debian's 5.14.12 kernel.
> Instructions here (also follow 4.2.1):
> https://kernel-team.pages.debian.net/kernel-handbook/ch-common-tasks.html#s4.2.2
>
> This would narrow it down to this specific commit and would also rule out
> differences in Ubuntu's kernel config and Debian's.
>
> Cheers,
> Diederik
[toc] | [prev] | [next] | [standalone]
| From | "Debian Bug Tracking System" <owner@bugs.debian.org> |
|---|---|
| Date | 2021-10-21 12:50 +0200 |
| Subject | Processed: Re: Bug#996951: linux-image-5.14.0-3-amd64: iouring looses requests |
| Message-ID | <DaDO1-Pc-3@gated-at.bofh.it> |
| In reply to | #73485 |
Processing control commands: > tags -1 + moreinfo Bug #996951 [src:linux] linux-image-5.14.0-3-amd64: iouring looses requests Added tag(s) moreinfo. -- 996951: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=996951 Debian Bug Tracking System Contact owner@bugs.debian.org with problems
[toc] | [prev] | [next] | [standalone]
| From | "Debian Bug Tracking System" <owner@bugs.debian.org> |
|---|---|
| Date | 2021-10-23 09:40 +0200 |
| Subject | Processed: Re: Bug#996951: linux-image-5.14.0-3-amd64: iouring looses requests |
| Message-ID | <DbjNf-1oi-3@gated-at.bofh.it> |
| In reply to | #73485 |
Processing control commands: > tags -1 + moreinfo Bug #996951 [src:linux] linux-image-5.14.0-3-amd64: iouring looses requests Ignoring request to alter tags of bug #996951 to the same tags previously set -- 996951: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=996951 Debian Bug Tracking System Contact owner@bugs.debian.org with problems
[toc] | [prev] | [next] | [standalone]
| From | Diederik de Haas <didi.debian@cknow.org> |
|---|---|
| Date | 2021-10-24 00:40 +0200 |
| Message-ID | <DbxQd-1KJ-1@gated-at.bofh.it> |
| In reply to | #73485 |
[Multipart message — attachments visible in raw view] — view raw
Control: Forwarded -1 https://lore.kernel.org/linux-block/CABVffENnJ8JkP7EtuUTqi+VkJDBFU37w1UXe4Q3cB7-ixxh0VA@mail.gmail.com/ On Saturday, 23 October 2021 23:29:21 CEST Daniel Black wrote: > https://marc.info/?l=linux-block&m=163489378723217&w=2 Looked up your upstream message on lore.k.o and set 'Forwarded' to that.
[toc] | [prev] | [next] | [standalone]
| From | "Debian Bug Tracking System" <owner@bugs.debian.org> |
|---|---|
| Date | 2021-10-24 00:40 +0200 |
| Subject | Processed: Re: Bug#996951: linux-image-5.14.0-3-amd64: iouring looses requests |
| Message-ID | <DbxQd-1KJ-3@gated-at.bofh.it> |
| In reply to | #73485 |
Processing control commands: > Forwarded -1 https://lore.kernel.org/linux-block/CABVffENnJ8JkP7EtuUTqi+VkJDBFU37w1UXe4Q3cB7-ixxh0VA@mail.gmail.com/ Bug #996951 [src:linux] linux-image-5.14.0-3-amd64: iouring looses requests Set Bug forwarded-to-address to 'https://lore.kernel.org/linux-block/CABVffENnJ8JkP7EtuUTqi+VkJDBFU37w1UXe4Q3cB7-ixxh0VA@mail.gmail.com/'. -- 996951: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=996951 Debian Bug Tracking System Contact owner@bugs.debian.org with problems
[toc] | [prev] | [next] | [standalone]
| From | Daniel Black <daniel@mariadb.org> |
|---|---|
| Date | 2021-11-07 23:50 +0100 |
| Subject | Bug#996951: |
| Message-ID | <DgZ97-Xl-3@gated-at.bofh.it> |
| In reply to | #73485 |
Marko tested this is fixed in linux-image-5.14.0-4-amd64 (5.14.16-1). Thanks for the update. https://jira.mariadb.org/browse/MDEV-26674?page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel&focusedCommentId=204907#comment-204907
[toc] | [prev] | [next] | [standalone]
| From | "Debian Bug Tracking System" <owner@bugs.debian.org> |
|---|---|
| Date | 2021-11-15 11:10 +0100 |
| Subject | Bug#996951: marked as done (linux-image-5.14.0-3-amd64: iouring looses requests) |
| Message-ID | <DjH63-2Qa-13@gated-at.bofh.it> |
| In reply to | #73485 |
[Multipart message — attachments visible in raw view] — view raw
Your message dated Mon, 15 Nov 2021 10:00:20 +0000 with message-id <E1mmYmi-000CA4-Lk@fasolo.debian.org> and subject line Bug#996951: fixed in linux 5.15.2-1~exp1 has caused the Debian Bug report #996951, regarding linux-image-5.14.0-3-amd64: iouring looses requests to be marked as done. This means that you claim that the problem has been dealt with. If this is not the case it is now your responsibility to reopen the Bug report if necessary, and/or fix the problem forthwith. (NB: If you are a system administrator and have no idea what this message is talking about, this may indicate a serious mail system misconfiguration somewhere. Please contact owner@bugs.debian.org immediately.) -- 996951: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=996951 Debian Bug Tracking System Contact owner@bugs.debian.org with problems
[toc] | [prev] | [standalone]
Back to top | Article view | linux.debian.kernel
csiph-web