Bug#1107521: ath12k_pci errors and loss of connectivity in 6.12.y branch

From	Matt Mower <mowerm@gmail.com>
Newsgroups	linux.debian.bugs.dist, linux.debian.kernel
Subject	Bug#1107521: ath12k_pci errors and loss of connectivity in 6.12.y branch
Date	2025-07-02 07:30 +0200
Message-ID	<L3YcF-eDSj-3@gated-at.bofh.it> (permalink)
References	(1 earlier) <L2eEV-dwDG-11@gated-at.bofh.it> <L2eEV-dwDG-9@gated-at.bofh.it> <L3YcF-eDSj-5@gated-at.bofh.it> <KVpON-995M-5@gated-at.bofh.it> <L3YcF-eDSj-5@gated-at.bofh.it>
Organization	linux.* mail to news gateway

Cross-posted to 2 groups.

Show all headers | View raw

> A couple more things I'd try on the ath12k side: firstly, boot with
> "iommu.strict=1" and see if that makes the faults any more
> frequent/reproducible;

The issue is easy enough to reproduce in 6.12.27 onward and I may be
mistaken about the rarity in 6.12.22; I reproduced it relatively
quickly in .22 today, so if this was the primary purpose for setting
iommu.strict=1, then testing with or without strict works. FWIW, I did
test iommu.strict=1 with 6.15.3 and still have not reproduced this
issue there.

> if a fault is fairly easily reproducible, then
> use the DMA API and/or IOMMU API tracepoints to compare the fault
> address to prior DMA mapping activity - that can usually reveal the
> nature of the bug enough to then know what to go looking for.

This is unfamiliar territory for me, so I hope the following is at
least close to what you requested. If not, happy to provide more test
results based on a set of instructions. Here's what I did:

1. Set CONFIG_DMA_API_DEBUG=y
2. Set kernel command line to: iommu.strict=1 log_buf_len=100M
dma_debug_driver=ath12k_pci trace_event=dma:*,iommu:*
3. Booted and waited for page fault, then cat'd
/sys/kernel/tracing/trace to a file.

Additionally, though I'm pretty sure this is irrelevant now, I added
logging after each dma_map_single() in the ath12k driver to print the
function name and resultant address to the kernel log.

Comparing the addresses of several io_page_fault lines in the trace
and in the kernel log, they line up. So, I'm hopeful this is on the
right track.

DMA/IOMMU trace: https://cmphys.com/ath12k/iommu_dma_trace-20250701.log
Kernel log with additional logging:
https://cmphys.com/ath12k/dmesg-6.12.35-20250701.log
Diff showing extra logging added to v6.12.35:
https://cmphys.com/ath12k/ath12k-extra-logging-6.12.35-20250701.diff

Thanks,
Matt

Thread

Bug#1107521: linux-image-6.12.27+bpo-amd64: ath12k_pci errors and eventual kernel panic starting with linux-image-6.12.27+bpo-amd64 Matt Mower <mowerm@gmail.com> - 2025-06-08 17:10 +0200
  Processed: Re: Bug#1107521: linux-image-6.12.27+bpo-amd64:  ath12k_pci errors and eventual kernel panic starting with linux-image-6.12.27+bpo-amd64 "Debian Bug Tracking System" <owner@bugs.debian.org> - 2025-06-11 23:00 +0200
  Bug#1107521: linux-image-6.12.27+bpo-amd64: ath12k_pci errors and eventual kernel panic starting with linux-image-6.12.27+bpo-amd64 Uwe Kleine-König <u.kleine-koenig@baylibre.com> - 2025-06-11 23:00 +0200
  Bug#1107521: linux-image-6.12.27+bpo-amd64: ath12k_pci errors and eventual kernel panic starting with linux-image-6.12.27+bpo-amd64 Matt Mower <mowerm@gmail.com> - 2025-06-12 22:50 +0200
    Bug#1107521: linux-image-6.12.27+bpo-amd64: ath12k_pci errors and eventual kernel panic starting with linux-image-6.12.27+bpo-amd64 Salvatore Bonaccorso <carnil@debian.org> - 2025-06-14 21:40 +0200
  Processed: Re: Bug#1107521: linux-image-6.12.27+bpo-amd64:  ath12k_pci errors and eventual kernel panic starting with linux-image-6.12.27+bpo-amd64 "Debian Bug Tracking System" <owner@bugs.debian.org> - 2025-06-16 08:30 +0200
  Processed: Re: Bug#1107521: linux-image-6.12.27+bpo-amd64:  ath12k_pci errors and eventual kernel panic starting with linux-image-6.12.27+bpo-amd64 "Debian Bug Tracking System" <owner@bugs.debian.org> - 2025-06-18 11:20 +0200
  Bug#1107521: linux-image-6.12.27+bpo-amd64: ath12k_pci errors and eventual kernel panic starting with linux-image-6.12.27+bpo-amd64 Matt Mower <mowerm@gmail.com> - 2025-06-25 07:20 +0200
  Bug#1107521: ath12k_pci errors and loss of connectivity in 6.12.y branch Robin Murphy <robin.murphy@arm.com> - 2025-06-27 12:40 +0200
  Bug#1107521: ath12k_pci errors and loss of connectivity in 6.12.y branch Matt Mower <mowerm@gmail.com> - 2025-07-02 07:30 +0200
  Bug#1107521: ath12k_pci errors and loss of connectivity in 6.12.y branch Matt Mower <mowerm@gmail.com> - 2025-07-02 17:00 +0200
  Bug#1107521: Info received (ath12k_pci errors and loss of connectivity in 6.12.y branch) Matt Mower <mowerm@gmail.com> - 2025-08-17 18:40 +0200

csiph-web