Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > linux.debian.kernel > #94311 > unrolled thread
| Started by | Salvatore Bonaccorso <carnil@debian.org> |
|---|---|
| First post | 2026-10-01 17:30 +0200 |
| Last post | 2026-10-02 13:00 +0200 |
| Articles | 2 — 2 participants |
Back to article view | Back to linux.debian.kernel
This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by
below is the oldest one visible, not the original post.
Bug#1149564: linux-image-6.12.111+deb13-amd64: 6.12.111-1 regression crash bnxt_en: DMAR: DRHD: handling fault status reg 2 Salvatore Bonaccorso <carnil@debian.org> - 2026-10-01 17:30 +0200
Bug#1149564: linux-image-6.12.111+deb13-amd64: 6.12.111-1 regression crash bnxt_en: DMAR: DRHD: handling fault status reg 2 Bernhard Schmidt <berni@debian.org> - 2026-10-02 13:00 +0200
| From | Salvatore Bonaccorso <carnil@debian.org> |
|---|---|
| Date | 2026-10-01 17:30 +0200 |
| Subject | Bug#1149564: linux-image-6.12.111+deb13-amd64: 6.12.111-1 regression crash bnxt_en: DMAR: DRHD: handling fault status reg 2 |
| Message-ID | <NJqTo-2W1U-35@gated-at.bofh.it> |
Hi Bernhard,
On Wed, Sep 30, 2026 at 09:57:56PM +0200, Bernhard Schmidt wrote:
> Package: src:linux
> Version: 6.12.111-1
> Severity: important
>
> After upgrading to todays security kernel a pair of BCM57416 onboard
> ports fail after a couple of minutes uptime with the following error
> message in the log
>
> Sep 30 21:33:27 nm3 kernel: Linux version 6.12.111+deb13-amd64 (debian-kernel@lists.debian.org) (x86_64-linux-gnu-gcc-14 (Debian 14.2.0-19) 14.2.0,
> [...]
> Sep 30 21:33:27 nm3 kernel: bnxt_en 0000:19:00.0 eth0: Broadcom BCM57416 NetXtreme-E 10GBase-T Ethernet found at mem 9dc10000, node addr 00:0a:f7:e7:f0:5c
> Sep 30 21:33:27 nm3 kernel: bnxt_en 0000:19:00.0: 63.008 Gb/s available PCIe bandwidth (8.0 GT/s PCIe x8 link)
> Sep 30 21:33:27 nm3 kernel: bnxt_en 0000:19:00.1 (unnamed net_device) (uninitialized): Device requests max timeout of 100 seconds, may trigger hung task watchdog (kernel default 120s)
> Sep 30 21:33:27 nm3 kernel: bnxt_en 0000:19:00.1 eth1: Broadcom BCM57416 NetXtreme-E 10GBase-T Ethernet found at mem 9dc00000, node addr 00:0a:f7:e7:f0:5d
> Sep 30 21:33:27 nm3 kernel: bnxt_en 0000:19:00.1: 63.008 Gb/s available PCIe bandwidth (8.0 GT/s PCIe x8 link)
> Sep 30 21:33:27 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: renamed from eth0
> Sep 30 21:33:27 nm3 kernel: bnxt_en 0000:19:00.1 eno2np1: renamed from eth1
> [...]
> Sep 30 21:37:42 nm3 kernel: DMAR: DRHD: handling fault status reg 2
> Sep 30 21:37:42 nm3 kernel: DMAR: [DMA Read NO_PASID] Request device [19:00.0] fault addr 0xefffc000 [fault reason 0x06] PTE Read access is not set
> Sep 30 21:37:42 nm3 kernel: bnxt_en 0000:19:00.1 eno2np1: Fatal firmware reset event, data1: 0x201, data2: 0xda27, min wait 1300 ms, max wait 4200 ms
> Sep 30 21:37:42 nm3 systemd-networkd[1269]: eno2np1: Lost carrier
> Sep 30 21:37:42 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: Abandoning msg {0x23 0x39f} len: 0 due to firmware status: 0x2000001
> Sep 30 21:37:42 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: Abandoning msg {0xb4 0x3a0} len: 0 due to firmware status: 0x2000001
> Sep 30 21:37:42 nm3 kernel: bond0: (slave eno2np1): link status definitely down, disabling slave
> Sep 30 21:37:43 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: hwrm req_type 0x23 seq id 0x3a1 error 0xf
> Sep 30 21:37:43 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: hwrm req_type 0xb4 seq id 0x3a2 error 0xf
> Sep 30 21:37:44 nm3 kernel: bnxt_en 0000:19:00.1 eno2np1: Device requests max timeout of 100 seconds, may trigger hung task watchdog (kernel default 120s)
> Sep 30 21:37:50 nm3 kernel: bnxt_en 0000:19:00.1 eno2np1: NIC Link is Down
> Sep 30 21:37:50 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: Abandoning msg {0x23 0x3a3} len: 0 due to firmware status: 0x2000001
> Sep 30 21:37:50 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: Abandoning msg {0xb4 0x3a4} len: 0 due to firmware status: 0x2000001
> Sep 30 21:37:51 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: Abandoning msg {0x23 0x3a5} len: 0 due to firmware status: 0x2000001
> Sep 30 21:37:51 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: Abandoning msg {0xb4 0x3a6} len: 0 due to firmware status: 0x2000001
> Sep 30 21:37:52 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: Abandoning msg {0x23 0x3a7} len: 0 due to firmware status: 0x2000001
> Sep 30 21:37:52 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: Abandoning msg {0xb4 0x3a8} len: 0 due to firmware status: 0x2000001
> Sep 30 21:37:52 nm3 systemd-networkd[1269]: eno2np1: Gained carrier
> Sep 30 21:37:52 nm3 kernel: bnxt_en 0000:19:00.1 eno2np1: NIC Link is Up, 1000 Mbps full duplex, Flow control: none
> Sep 30 21:37:52 nm3 kernel: bnxt_en 0000:19:00.1 eno2np1: EEE is not active
> Sep 30 21:37:52 nm3 kernel: bnxt_en 0000:19:00.1 eno2np1: FEC autoneg off encoding: None
> Sep 30 21:37:52 nm3 kernel: bond0: (slave eno2np1): link status up, enabling it in 60000 ms
> Sep 30 21:37:53 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: Abandoning msg {0xb4 0x3aa} len: 0 due to firmware status: 0x2000001
> Sep 30 21:37:54 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: Abandoning msg {0x23 0x3ab} len: 0 due to firmware status: 0x2000001
> Sep 30 21:37:54 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: Abandoning msg {0xb4 0x3ac} len: 0 due to firmware status: 0x2000001
> Sep 30 21:37:55 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: Abandoning msg {0x23 0x3ad} len: 0 due to firmware status: 0x2000001
> Sep 30 21:37:55 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: Abandoning msg {0xb4 0x3ae} len: 0 due to firmware status: 0x2000001
> [...]
> Sep 30 21:38:26 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: Abandoning msg {0x23 0x3eb} len: 0 due to firmware status: 0x2000001
> Sep 30 21:38:26 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: Abandoning msg {0xb4 0x3ec} len: 0 due to firmware status: 0x2000001
> Sep 30 21:38:27 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: Abandoning msg {0x23 0x3ed} len: 0 due to firmware status: 0x2000001
> Sep 30 21:38:27 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: Abandoning msg {0xb4 0x3ee} len: 0 due to firmware status: 0x2000001
> Sep 30 21:38:28 nm3 systemd[1607]: oxidized.service: Scheduled restart job, restart counter is at 28.
> Sep 30 21:38:28 nm3 systemd[1607]: Started oxidized.service - Oxidized Network Configuration Backup.
> Sep 30 21:38:28 nm3 oxidized[10462]: A server is already running. Check /home/oxidized/.config/oxidized/pid
> Sep 30 21:38:28 nm3 systemd[1607]: oxidized.service: Main process exited, code=exited, status=1/FAILURE
> Sep 30 21:38:28 nm3 systemd[1607]: oxidized.service: Failed with result 'exit-code'.
> Sep 30 21:38:28 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: Abandoning msg {0x23 0x3ef} len: 0 due to firmware status: 0x2000001
> Sep 30 21:38:28 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: Abandoning msg {0xb4 0x3f0} len: 0 due to firmware status: 0x2000001
> Sep 30 21:38:29 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: Abandoning msg {0x23 0x3f1} len: 0 due to firmware status: 0x2000001
> Sep 30 21:38:29 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: Abandoning msg {0xb4 0x3f2} len: 0 due to firmware status: 0x2000001
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: NETDEV WATCHDOG: CPU: 27: transmit queue 13 timed out 5376 ms
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: TX timeout detected, starting reset task!
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: [0.0]: tx{fw_ring: 0 prod: 76f cons: 75d}
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: [0]: rx{fw_ring: 1 prod: 4c29} rx_agg{fw_ring: 15 agg_prod: 80a sw_agg_prod: a}
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: [0]: cp{fw_ring: 0 raw_cons: 96e3}
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: [1.0]: tx{fw_ring: 1 prod: 352 cons: 350}
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: [1]: rx{fw_ring: 2 prod: 2b5} rx_agg{fw_ring: 16 agg_prod: 801 sw_agg_prod: 1}
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: [1]: cp{fw_ring: 16 raw_cons: 294}
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: [2.0]: tx{fw_ring: 2 prod: df cons: df}
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: [2]: rx{fw_ring: 3 prod: 225} rx_agg{fw_ring: 17 agg_prod: 805 sw_agg_prod: 5}
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: [2]: cp{fw_ring: 17 raw_cons: a4}
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: [3.0]: tx{fw_ring: 3 prod: 105 cons: 105}
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: [3]: rx{fw_ring: 4 prod: 252} rx_agg{fw_ring: 18 agg_prod: 805 sw_agg_prod: 5}
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: [3]: cp{fw_ring: 18 raw_cons: 120}
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: [4.0]: tx{fw_ring: 4 prod: 249 cons: 235}
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: [4]: rx{fw_ring: 5 prod: 208} rx_agg{fw_ring: 19 agg_prod: 800 sw_agg_prod: 0}
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: [4]: cp{fw_ring: 19 raw_cons: e6}
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: [5.0]: tx{fw_ring: 5 prod: bb cons: bb}
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: [5]: rx{fw_ring: 6 prod: 231} rx_agg{fw_ring: 20 agg_prod: 870 sw_agg_prod: 70}
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: [5]: cp{fw_ring: 20 raw_cons: 123}
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: [6.0]: tx{fw_ring: 6 prod: a4b cons: a4b}
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: [6]: rx{fw_ring: 7 prod: 211} rx_agg{fw_ring: 21 agg_prod: 801 sw_agg_prod: 1}
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: [6]: cp{fw_ring: 21 raw_cons: 39e}
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: [7.0]: tx{fw_ring: 7 prod: 245 cons: 245}
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: [7]: rx{fw_ring: 8 prod: 203} rx_agg{fw_ring: 22 agg_prod: 7fd sw_agg_prod: 7fd}
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: [7]: cp{fw_ring: 22 raw_cons: d3}
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: [8.0]: tx{fw_ring: 8 prod: 72 cons: 6e}
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: [8]: rx{fw_ring: 9 prod: 20c} rx_agg{fw_ring: 23 agg_prod: 800 sw_agg_prod: 0}
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: [8]: cp{fw_ring: 23 raw_cons: 46}
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: [9.0]: tx{fw_ring: 9 prod: 8d cons: 8d}
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: [9]: rx{fw_ring: 10 prod: 24a} rx_agg{fw_ring: 24 agg_prod: 801 sw_agg_prod: 1}
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: [9]: cp{fw_ring: 24 raw_cons: e1}
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: [10.0]: tx{fw_ring: 10 prod: cf cons: cf}
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: [10]: rx{fw_ring: 11 prod: 207} rx_agg{fw_ring: 25 agg_prod: 7ff sw_agg_prod: 7ff}
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: [10]: cp{fw_ring: 25 raw_cons: 5b}
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: [11.0]: tx{fw_ring: 11 prod: 2b cons: 2b}
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: [11]: rx{fw_ring: 12 prod: 206} rx_agg{fw_ring: 26 agg_prod: 7ff sw_agg_prod: 7ff}
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: [11]: cp{fw_ring: 26 raw_cons: 21}
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: [12.0]: tx{fw_ring: 12 prod: db cons: db}
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: [12]: rx{fw_ring: 13 prod: 2a6} rx_agg{fw_ring: 27 agg_prod: 7fd sw_agg_prod: 7fd}
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: [12]: cp{fw_ring: 27 raw_cons: 19a}
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: [13.0]: tx{fw_ring: 13 prod: eb6 cons: eae}
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: [13]: rx{fw_ring: 14 prod: c67} rx_agg{fw_ring: 28 agg_prod: 99c sw_agg_prod: 19c}
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: [13]: cp{fw_ring: 28 raw_cons: 1b70}
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: hwrm req_type 0x91 seq id 0x3f3 error 0x3
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: Abandoning msg {0x46 0x3f4} len: 0 due to firmware status: 0x2000001
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: Abandoning msg {0x71 0x3f5} len: 0 due to firmware status: 0x2000001
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: Abandoning msg {0x44 0x3f6} len: 0 due to firmware status: 0x2000001
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: hwrm vnic set tpa failure rc for vnic 0: fffffff0
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: Discarding out of seq response: 0x3f6 for msg {0x41 0x3f7}
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: hwrm req_type 0x41 seq id 0x3f7 error 0x3
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: hwrm req_type 0x41 seq id 0x3f8 error 0x3
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: hwrm req_type 0x41 seq id 0x3f9 error 0x3
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: hwrm req_type 0x41 seq id 0x3fa error 0x3
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: hwrm req_type 0x41 seq id 0x3fb error 0x3
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: hwrm req_type 0x41 seq id 0x3fc error 0x3
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: hwrm req_type 0x41 seq id 0x3fd error 0x3
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: hwrm req_type 0x41 seq id 0x3fe error 0x3
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: hwrm req_type 0x41 seq id 0x3ff error 0x3
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: hwrm req_type 0x41 seq id 0x400 error 0x3
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: hwrm req_type 0x41 seq id 0x401 error 0x3
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: hwrm req_type 0x41 seq id 0x402 error 0x3
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: hwrm req_type 0x41 seq id 0x403 error 0x3
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: hwrm req_type 0x41 seq id 0x404 error 0x3
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: hwrm req_type 0x41 seq id 0x405 error 0x3
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: Resp cmpl intr abandoning msg: 0x51 due to firmware status: 0x2000001
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: hwrm_ring_free type 1 failed. rc:fffffff0 err:3
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: Resp cmpl intr abandoning msg: 0x51 due to firmware status: 0x2000001
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: hwrm_ring_free type 1 failed. rc:fffffff0 err:3
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: Resp cmpl intr abandoning msg: 0x51 due to firmware status: 0x2000001
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: hwrm_ring_free type 1 failed. rc:fffffff0 err:3
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: Resp cmpl intr abandoning msg: 0x51 due to firmware status: 0x2000001
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: hwrm_ring_free type 1 failed. rc:fffffff0 err:3
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: Resp cmpl intr abandoning msg: 0x51 due to firmware status: 0x2000001
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: hwrm_ring_free type 1 failed. rc:fffffff0 err:3
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: Resp cmpl intr abandoning msg: 0x51 due to firmware status: 0x2000001
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: hwrm_ring_free type 1 failed. rc:fffffff0 err:3
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: Resp cmpl intr abandoning msg: 0x51 due to firmware status: 0x2000001
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: hwrm_ring_free type 1 failed. rc:fffffff0 err:3
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: Resp cmpl intr abandoning msg: 0x51 due to firmware status: 0x2000001
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: hwrm_ring_free type 1 failed. rc:fffffff0 err:3
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: Resp cmpl intr abandoning msg: 0x51 due to firmware status: 0x2000001
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: hwrm_ring_free type 2 failed. rc:fffffff0 err:3
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: Resp cmpl intr abandoning msg: 0x51 due to firmware status: 0x2000001
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: hwrm_ring_free type 2 failed. rc:fffffff0 err:3
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: hwrm req_type 0x51 seq id 0x430 error 0x3
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: hwrm_ring_free type 0 failed. rc:fffffff3 err:3
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: hwrm req_type 0x51 seq id 0x431 error 0x3
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: hwrm_ring_free type 0 failed. rc:fffffff3 err:3
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: hwrm req_type 0x51 seq id 0x432 error 0x3
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: hwrm_ring_free type 0 failed. rc:fffffff3 err:3
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: hwrm req_type 0x51 seq id 0x433 error 0x3
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: hwrm_ring_free type 0 failed. rc:fffffff3 err:3
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: hwrm req_type 0x51 seq id 0x434 error 0x3
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: hwrm_ring_free type 0 failed. rc:fffffff3 err:3
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: hwrm req_type 0x51 seq id 0x435 error 0x3
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: hwrm_ring_free type 0 failed. rc:fffffff3 err:3
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: hwrm req_type 0x51 seq id 0x436 error 0x3
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: hwrm_ring_free type 0 failed. rc:fffffff3 err:3
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: hwrm req_type 0x51 seq id 0x437 error 0x3
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: hwrm_ring_free type 0 failed. rc:fffffff3 err:3
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: hwrm req_type 0x51 seq id 0x438 error 0x3
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: hwrm_ring_free type 0 failed. rc:fffffff3 err:3
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: hwrm req_type 0x51 seq id 0x439 error 0x3
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: hwrm_ring_free type 0 failed. rc:fffffff3 err:3
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: hwrm req_type 0x51 seq id 0x43a error 0x3
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: hwrm_ring_free type 0 failed. rc:fffffff3 err:3
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: hwrm req_type 0x51 seq id 0x43b error 0x3
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: hwrm_ring_free type 0 failed. rc:fffffff3 err:3
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: hwrm req_type 0x51 seq id 0x43c error 0x3
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: hwrm_ring_free type 0 failed. rc:fffffff3 err:3
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: hwrm req_type 0x51 seq id 0x43d error 0x3
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: hwrm_ring_free type 0 failed. rc:fffffff3 err:3
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: hwrm req_type 0x61 seq id 0x43e error 0x3
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: hwrm req_type 0x61 seq id 0x43f error 0x3
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: hwrm req_type 0x61 seq id 0x440 error 0x3
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: hwrm req_type 0x61 seq id 0x441 error 0x3
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: hwrm req_type 0x61 seq id 0x442 error 0x3
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: hwrm req_type 0x61 seq id 0x443 error 0x3
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: hwrm req_type 0x61 seq id 0x444 error 0x3
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: hwrm req_type 0x61 seq id 0x445 error 0x3
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: hwrm req_type 0x61 seq id 0x446 error 0x3
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: hwrm req_type 0xb1 seq id 0x456 error 0x3
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: hwrm req_type 0xb1 seq id 0x457 error 0x3
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: hwrm req_type 0xb1 seq id 0x458 error 0x3
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: hwrm req_type 0xb1 seq id 0x459 error 0x3
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: hwrm req_type 0xb0 seq id 0x45a error 0x3
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: hwrm stat ctx alloc failure rc: fffffff3
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: bnxt_init_nic err: fffffff3
> Sep 30 21:38:30 nm3 systemd-networkd[1269]: eno1np0: Link DOWN
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: nic open fail (rc: fffffff3)
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: Abandoning msg {0x20 0x45b} len: 0 due to firmware status: 0x2000001
> Sep 30 21:38:30 nm3 kernel: bnxt_en 0000:19:00.0 eno1np0: Abandoning msg {0xd 0x45c} len: 0 due to firmware status: 0x2000001
> Sep 30 21:38:30 nm3 systemd-networkd[1269]: eno1np0: Lost carrier
> Sep 30 21:38:30 nm3 kernel: bond0: (slave eno1np0): link status definitely down, disabling slave
> Sep 30 21:38:30 nm3 kernel: bond0: (slave eno2np1): making interface the new active one 37200 ms earlier
>
> It's an active/backup bond, then the second port is down it stays down
> until the box has been rebooted. Downgrading to 6.12.107 fixes the
> problem.
Do you have non-production machine with the same setup which you
couls spare to bisect the changes? If you can that would be the most
helpful step to isolate the breaking change.
Or if you can spare the machine for a couple of cycles then bisecting
the issue would be most efficient forward in the following way:
git clone --single-branch -b linux-6.12.y https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git
cd linux-stable
git checkout v6.12.107
cp /boot/config-$(uname -r) .config
yes '' | make localmodconfig
make savedefconfig
mv defconfig arch/x86/configs/my_defconfig
# test 6.12.107 to ensure this is "good"
make my_defconfig
make -j $(nproc) bindeb-pkg
... install the resulting .deb package and confirm problem does not exist
# test 6.12.111 to ensure this is "bad"
git checkout v6.12.111
make my_defconfig
make -j $(nproc) bindeb-pkg
... install the resulting .deb package and confirm problem exists
With that confirmed, the bisection can start:
git bisect start
git bisect good v6.12.107
git bisect bad v6.12.111
In each bisection step git checks out a state between the oldest
known-bad and the newest known-good commit. In each step test using:
make my_defconfig
make -j $(nproc) bindeb-pkg
... install, if problem exists
and if the problem is hit run:
git bisect bad
and if the problem doesn't trigger run:
git bisect good
. Please pay attention to always select the just built kernel for
booting, it won't always be the default kernel picked up by grub.
Iterate until git announces to have identified the first bad commit.
Then provide the output of
git bisect log
In the course of the bisection you might have to uninstall previous
kernels again to not exhaust the disk space in /boot. Also in the end
uninstall all self-built kernels again.
Regards,
Salvatore
[toc] | [next] | [standalone]
| From | Bernhard Schmidt <berni@debian.org> |
|---|---|
| Date | 2026-10-02 13:00 +0200 |
| Message-ID | <NJJ9D-35O0-1@gated-at.bofh.it> |
| In reply to | #94311 |
On 01/10/26 05:26 PM, Salvatore Bonaccorso wrote:
Hi,
> Do you have non-production machine with the same setup which you
> couls spare to bisect the changes? If you can that would be the most
> helpful step to isolate the breaking change.
>
> Or if you can spare the machine for a couple of cycles then bisecting
> the issue would be most efficient forward in the following way:
The machine is semi-productive so we could do it, but so far we've
failed.
> git clone --single-branch -b linux-6.12.y https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git
> cd linux-stable
> git checkout v6.12.107
> cp /boot/config-$(uname -r) .config
> yes '' | make localmodconfig
> make savedefconfig
> mv defconfig arch/x86/configs/my_defconfig
>
> # test 6.12.107 to ensure this is "good"
> make my_defconfig
> make -j $(nproc) bindeb-pkg
> ... install the resulting .deb package and confirm problem does not exist
>
> # test 6.12.111 to ensure this is "bad"
> git checkout v6.12.111
> make my_defconfig
> make -j $(nproc) bindeb-pkg
> ... install the resulting .deb package and confirm problem exists
That 6.12.111 is running fine for the last 4,5h, where it usually failed
within a few minutes (up to 30). So right now we don't have a known bad.
Also interesting, while the compiled kernel works fine the binary sizes
are way different from the ones in the Debian packages
-rw-r--r-- 1 root root 171116 Oct 2 01:00 config-6.12.107
-rw-r--r-- 1 root root 283337 Aug 29 06:19 config-6.12.107+deb13-amd64
-rw-r--r-- 1 root root 171310 Oct 2 01:05 config-6.12.111
-rw-r--r-- 1 root root 283542 Sep 28 14:18 config-6.12.111+deb13-amd64
-rw-r--r-- 1 root root 283542 Sep 28 14:18 config-6.12.111+deb13r1-amd64
-rw-r--r-- 1 root root 29809382 Oct 2 01:05 initrd.img-6.12.107
-rw-r--r-- 1 root root 53517010 Sep 12 12:11 initrd.img-6.12.107+deb13-amd64
-rw-r--r-- 1 root root 29814131 Oct 2 01:09 initrd.img-6.12.111
-rw-r--r-- 1 root root 53535601 Sep 30 03:00 initrd.img-6.12.111+deb13-amd64
-rw-r--r-- 1 root root 53543460 Oct 2 08:42 initrd.img-6.12.111+deb13r1-amd64
-rw-r--r-- 1 root root 11944448 Oct 2 01:00 vmlinuz-6.12.107
-rw-r--r-- 1 root root 12142528 Aug 29 06:19 vmlinuz-6.12.107+deb13-amd64
-rw-r--r-- 1 root root 11964928 Oct 2 01:05 vmlinuz-6.12.111
-rw-r--r-- 1 root root 12154816 Sep 28 14:18 vmlinuz-6.12.111+deb13-amd64
-rw-r--r-- 1 root root 12153344 Sep 28 14:18 vmlinuz-6.12.111+deb13r1-amd64
.107 and .111 are ones created by bindeb-pkg from the upstream source,
+deb13 are the ones from the official debian packages and deb13r1 is one
that was created using the manual at https://kernel-team.pages.debian.net/kernel-handbook/ch-common-tasks.html#s-common-official
We did do a half-hearted attempt to revert a couple of bnxt_en patches
or pull in additional patches from the 6.12-stable patch queue, but so
far it went nowhere.
However, we do have some more information.
As I said before (but forgot a digit in the kernel version) a friend saw
these errors in Proxmox upgrading from 6.17.13-21-pve to 7.0.14-12-pve.
They used the iommu=pt workaround and have not followed up.
Our own proxmox clusters were throwing the same errors after upgrading
from 7.0.14-19-pve to 7.0.14-20-pve. iommu=pt fixes it. In contrary to
the Dell R740 we have been looking at these are Lenovo AMD systems, so a
different IOMMU implementation?
This is their changelog
---
proxmox-kernel-signed-7.0 (7.0.14+20) trixie; urgency=medium
* update submodules and patches to Ubuntu-7.0.0-39.39
- upstream stable changes 6.18.50-6.18.51 and 7.2.2-7.2.5
* cherry-picks for the following CVEs from stable 7.2 and mainline 7.1:
- CVE-2026-90063: virtio-net: Ensure that TCP packets don't overflow
gso_segs
- CVE-2026-90105: vxlan: fix reading neigh ha
- CVE-2026-90106: net: bridge: arp/nd proxy: fix reading neigh ha
- CVE-2026-90232: amt: Don't support cross-netns setup
- CVE-2026-90061: netfilter: nf_tables: skip double clone set expressions
on element insert
- CVE-2026-90071: net/sched: sch_teql: restore skb->dev on the slave
failure path
- CVE-2026-90073: net/sched: hhf: clamp quantum before hhf_change() to
avoid overflow
- CVE-2026-90074: net/sched: fq_pie: clamp default quantum to avoid signed
overflow
- CVE-2026-90075: net/sched: fq_codel: clamp default quantum and mtu
- CVE-2026-90076: net/sched: fq: add overflow bounds to quantum and initial
quantum
- CVE-2026-90050: net/sched: fq: clamp quantum and initial_quantum in
change path
- CVE-2026-90078: net/sched: act_skbmod: fix length calculations and avoid
invalid header warnings
- CVE-2026-90109: net: sched: fix 32-bit backlog wrap in gred, bfifo and
plug enqueue
- CVE-2026-90248: net/sched: cls_api: fix teardown of an adopted proto on
insert-race loss
- CVE-2026-90111: ip6mr: do not clone dst in ip6mr_cache_report()
- CVE-2026-90110: inetpeer: randomize RB-tree node comparison using SipHash
- CVE-2026-90201: net: page_pool: fix UAF in __page_pool_release_netmem_dma
on xa_cmpxchg race
- CVE-2026-64190: net: team: fix NULL pointer dereference in team_xmit
during mode change
- CVE-2026-89789: gtp: add synchronize_net() in gtp_newlink() error path to
prevent use-after-free
- CVE-2026-90067: libceph: validate banner payload length
- CVE-2026-90227: nvme/ioctl: check SUBMIT_IO with nvme_cmd_allowed()
- CVE-2026-90148: NFSv4: Fix incorrect argument passed to
nfs4_delete_lease() in nfs4_add_lease()
- CVE-2026-90234: NFS: Return a delegation the client fails to record
- CVE-2026-90400: md: recheck spare changes before starting sync
- CVE-2026-90325: blk-cgroup: skip dying blkg in blkcg_activate_policy()
- CVE-2026-90326: blk-cgroup: fix race between policy activation and blkg
destruction
- CVE-2026-90240: iommu/vt-d: Flush context cache with correct SID when
tearing down aliases
- CVE-2026-90435: RDMA/mlx5: Fix integer overflow of user QP buffer size
- CVE-2026-92495: RDMA/bnxt_re: Clear VM_MAYWRITE on DBR/toggle page mmap
- CVE-2026-90101: bnxt_en: Fix call to hardware monitoring event handler
- CVE-2026-93182: sched/fair: Fix overflow in update_tg_cfs_runnable()
- CVE-2026-90246: apparmor: fix integer overflow in verify_tags() bounds
check
- CVE-2026-93146: time/namespace: Validate nanosecond field in
proc_timens_set_offset()
- CVE-2026-93066: x86/mm/pat: Take cpa_lock around large-page collapse
* cherry-pick fixes from stable 7.2 that have no CVE assigned yet:
- af_packet: Don't cast tpacket_hdr.tp_len to int in tpacket_parse_header()
- af_unix: Update last skb marker in manage_oob()
- af_unix: Return immediately when manage_oob() returns NULL for 0-length
buffer
- ipv6: fix fib6 walker UAF on seq stop
- ipv6: mcast: fix RCU list diversion in ip6_mc_del1_src()
- ipv6: mcast: use copy-on-write RCU updates in ip6_mc_source()
- ipv6: mcast: use rcu_assign_pointer() for __rcu list updates
- ipv6: null-check fib6_node before accessing in __ip6_del_rt_siblings()
- ipv6: flowlabel: cap duplicate leases per socket
- ipv4: fib: bound automatic table ID allocation
- inet: frags: invalidate queues before flushing them
- net: iptunnel: fix stale transport header during tunnel decapsulation
- net: cap tx_queue_len at S16_MAX to prevent oversized ring allocations
- net/sched: defer qdisc freeing after failed creation
- net/sched: cls_route: free emptied bucket on filter move
- net/sched: cls_api: Don't replay RTM_GETCHAIN in tc_ctl_chain()
- net/sched: cls_u32: fix duplicate handle when node ID pool is exhausted
- net/sched: drr: clamp quantum in change class
- net/sched: ets: clamp quantum in parse and fallback paths
- net/sched: fq_pie: clamp quantum in change path
- net/sched: hhf: clamp quantum in change and init paths
- net/sched: sfq: clamp quantum in change path
- vxlan: mdb: Fix use-after-free in vxlan_mdb_flush()
- vxlan: mdb: Fix use-after-free in vxlan_mdb_remote_src_del()
- vxlan: initialize _md in vxlan_xmit_one()
- net: bridge: mcast: properly convert mglist to rcu
- netfilter: nfnetlink_log: cope with concurrent instance destruction
- bonding: alb: fix uninitialized transport header access in
alb_determine_nd()
- scsi: mpt3sas: Avoid out-of-bounds cpumask_of_node() call in
_base_assign_reply_queues()
- x86/mm/pat: Acquire init_mm write lock on collapse to avoid UAF
* fix user-space data loss with MADV_FREE on transparent huge pages, where
reclaim under memory pressure could discard data written after the
MADV_FREE call if the page protection got changed in between
-- Proxmox Support Team <support@proxmox.com> Thu, 24 Sep 2026 12:43:24 +0200
---
There are only two bnxt_en related commits, as far as I can see those
were also included to 6.12.108 and .110. Those would be the prime
suspect (but no, I don't have an explaination why it would fail for my
friend on 7.0.14-12 which did not have those patches).
There are also a few reports of Proxmox breakage being broken with the
recent updates with other network cards, i.e.
https://forum.proxmox.com/threads/broadcom-bcm57504-100g-bnxt_en-tx-timeout-and-nic-reset-on-proxmox-8-1-5-%E2%80%94-while-bcm57414-25g-works-fine-on-same-host.179837/#post-871482
for RTL8125
so maybe we're completely barking up the wrong tree and the issues lies
somewhere else.
Bernhard
[toc] | [prev] | [standalone]
Back to top | Article view | linux.debian.kernel
csiph-web