Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]


Groups > linux.debian.kernel > #72795 > unrolled thread

Bug#993612: bugs.debian.org: Socionext SynQuacer fails to mount rootfs after upgrade to Bullseye

Started bySteve McIntyre <steve@einval.com>
First post2021-09-04 20:00 +0200
Last post2022-03-29 20:10 +0200
Articles 4 — 2 participants

Back to article view | Back to linux.debian.kernel

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.


Contents

  Bug#993612: bugs.debian.org: Socionext SynQuacer fails to mount rootfs after upgrade to Bullseye Steve McIntyre <steve@einval.com> - 2021-09-04 20:00 +0200
    Bug#993612: bugs.debian.org: Socionext SynQuacer fails to mount rootfs after upgrade to Bullseye Steve McIntyre <steve@einval.com> - 2021-09-05 18:10 +0200
      Bug#993612: bugs.debian.org: Socionext SynQuacer fails to mount rootfs after upgrade to Bullseye Mark Brown <broonie@debian.org> - 2022-03-29 02:10 +0200
        Bug#993612: bugs.debian.org: Socionext SynQuacer fails to mount rootfs after upgrade to Bullseye Mark Brown <broonie@debian.org> - 2022-03-29 20:10 +0200

#72795 — Bug#993612: bugs.debian.org: Socionext SynQuacer fails to mount rootfs after upgrade to Bullseye

FromSteve McIntyre <steve@einval.com>
Date2021-09-04 20:00 +0200
SubjectBug#993612: bugs.debian.org: Socionext SynQuacer fails to mount rootfs after upgrade to Bullseye
Message-ID<CTI7n-6gR-7@gated-at.bofh.it>
Hey Luca!

On Fri, Sep 03, 2021 at 03:50:52PM -0700, Don Armstrong wrote:
>Control: reassign -1 linux-signed-arm64
>Control: found -1 5.10.46+4, 5.10.46+4~bpo10+1
>Control: tag -1 moreinfo
>Control: severity -1 important
>
>
>On Fri, 03 Sep 2021, Luca Di Stefano wrote:
>> A few days ago I tried to upgrade one of the six Socionext SynQuacers
>> that we have to the latest Debian release.
>> 
>> It was running fine on Buster using the 4.19 kernel and had no previous issues.
>
>[...]
>
>> The next boot sequence would start and get to the point where it would
>> look for the rootfs without finding it and going into initramfs
>> 
>> I've then proceeded to reinstall buster on that machine and it just
>> worked fine, then also tried installing the kernel from backports
>> linux- image-5.10.0-0.bpo.8-arm64 and after reboot it caused the same
>> problem not finding the rootfs and going into initramfs.
>
>I'm not familiar with the hardware of this particular device, but I
>suspect that some necessary driver was (likely inadvertently) excluded
>from the configuration of the 5.10 kernel, but included in 4.19.
>
>Looking at the output from the boot of both kernels should give you an
>idea of what module/device is broken, and providing that to this bug
>will give one of the maintainers of the arm64 kernel a chance of helping
>fix the issue.

I have a synquacer here still and I'll take a look. I noticed on
bullseye release day that USB stuff didn't seem to work in the
installer on the synquacer either. Maybe there's been a regression in
config. :-/

I'll take a look...

-- 
Steve McIntyre, Cambridge, UK.                                steve@einval.com
"Further comment on how I feel about IBM will appear once I've worked out
 whether they're being malicious or incompetent. Capital letters are forecast."
 Matthew Garrett, http://www.livejournal.com/users/mjg59/30675.html

[toc] | [next] | [standalone]


#72808

FromSteve McIntyre <steve@einval.com>
Date2021-09-05 18:10 +0200
Message-ID<CU2Su-2XZ-5@gated-at.bofh.it>
In reply to#72795

[Multipart message — attachments visible in raw view] — view raw

On Sat, Sep 04, 2021 at 06:50:16PM +0100, Steve McIntyre wrote:
>
>I have a synquacer here still and I'll take a look. I noticed on
>bullseye release day that USB stuff didn't seem to work in the
>installer on the synquacer either. Maybe there's been a regression in
>config. :-/
>
>I'll take a look...

Hmmm, booting 5.10.0-0.bpo.8-arm64 here does not work at all. I'm
seeing a lot of errors around DMA setup, e.g.:

[    0.142797] OF: translation of DMA address(0) to CPU address failed node()
[    0.142824] sram 45200000.sram: Invalid size 0x1 for dma-range(s)
[    0.142991] OF: translation of DMA address(0) to CPU address failed node()
[    0.143008] uart-pl011 2a400000.uart: Invalid size 0x1 for dma-range(s)
...
[    3.843964] OF: translation of DMA address(0) to CPU address failed node()
[    3.850875] armv8-pmu pmu: Invalid size 0x1 for dma-range(s)
[    3.857518] hw perfevents: enabled with armv8_pmuv3 PMU driver, 7 counters available
...
[    3.954986] OF: translation of DMA address(0) to CPU address failed node()
...
[    4.391564] OF: translation of DMA address(0) to CPU address failed node()
[    4.398449] pcieport 0000:00:00.0: Invalid size 0x1 for dma-range(s)
[    4.405253] OF: translation of DMA address(0) to CPU address failed node()
[    4.412161] pcieport 0000:01:01.0: Invalid size 0x1 for dma-range(s)
[    4.418885] OF: translation of DMA address(0) to CPU address failed node()
[    4.425778] pcieport 0000:01:03.0: Invalid size 0x1 for dma-range(s)
[    4.432487] OF: translation of DMA address(0) to CPU address failed node()
[    4.439385] pcieport 0000:01:05.0: Invalid size 0x1 for dma-range(s)
[    4.446095] OF: translation of DMA address(0) to CPU address failed node()
[    4.452988] pcieport 0000:01:07.0: Invalid size 0x1 for dma-range(s)
[    4.459693] pci 0000:04:00.0: enabling device (0000 -> 0002)
[    4.465488] OF: translation of DMA address(0) to CPU address failed node()
...
[    4.472385] pci-host-generic 70000000.pcie: Invalid size 0x1 for dma-range(s)

Shortly after this, I see looping errors in the sdhci driver:
[    5.728589] xhci_hcd: probe of 0000:04:00.0 failed with error -12
[    5.734347] sp : ffff8000142bbb10
[    5.734351] x29: ffff8000142bbb10 x28: 0000000000000000 
[    5.734358] x27: ffff000801ebf848 x26: ffffffffffffffff 
[    5.768201] x25: 0000000000000080 x24: 0000000000010000 
[    5.773514] x23: ffff00080176ac10 x22: 0000000889b60080 
[    5.778826] x21: 0000000000000000 x20: 0000000000000000 
[    5.784139] x19: ffff8000117c9788 x18: 0000000000000020 
[    5.789451] x17: ffff00080833da00 x16: 0000000000000080 
[    5.794763] x15: ffffffffffffffff x14: 2820776f6c667265 
[    5.800076] x13: 766f203633353536 x12: 2b66666666666666 
[    5.805389] x11: 6666666666666666 x10: 6678302072646461 
[    5.810701] x9 : ffff80001011159c x8 : 62202c6666666666 
[    5.816013] x7 : 666666206b73616d x6 : ffff00087f4647c0 
[    5.821325] x5 : ffff8000142bb828 x4 : 0000000000000000 
[    5.826637] x3 : 0000000000000000 x2 : 00000000ffffffff 
[    5.831949] x1 : 3082d45f0379e900 x0 : 0000000000000000 
[    5.837262] Call trace:
[    5.839708]  swiotlb_map+0x294/0x2a8
[    5.843281]  dma_map_page_attrs+0xec/0x228
[    5.847382]  sdhci_setup_host+0xa8c/0xee8 [sdhci]
[    5.852086]  sdhci_add_host+0x20/0x58 [sdhci]
[    5.856444]  sdhci_f_sdh30_probe+0x2a8/0x380 [sdhci_f_sdh30]
[    5.862104]  platform_drv_probe+0x5c/0xb0
[    5.866113]  really_probe+0xf0/0x4d8
[    5.869686]  driver_probe_device+0xfc/0x168
[    5.873868]  __driver_attach_async_helper+0xbc/0xc8
[    5.878743]  async_run_entry_fn+0x50/0x218
[    5.882837]  process_one_work+0x1bc/0x480
[    5.886843]  worker_thread+0x158/0x4e0
[    5.890590]  kthread+0x12c/0x130
[    5.893817]  ret_from_fork+0x10/0x30
[    5.897389] ---[ end trace 986b9b1a11578595 ]---

Longer log attached. I'm wondering if there's a DTB mismatch or
something - my system here is still using the built-in DTB from the
EFI firmware, and I don't see a newer DTB available in the debian
package nor in upstream git.

@Arnd any clues please?

-- 
Steve McIntyre, Cambridge, UK.                                steve@einval.com
“Changing random stuff until your program works is bad coding
 practice, but if you do it fast enough it’s Machine Learning.”
   -- https://twitter.com/manisha72617183

[toc] | [prev] | [next] | [standalone]


#74896

FromMark Brown <broonie@debian.org>
Date2022-03-29 02:10 +0200
Message-ID<E67AR-4SSe-1@gated-at.bofh.it>
In reply to#72808

[Multipart message — attachments visible in raw view] — view raw

On Sun, Sep 05, 2021 at 05:01:18PM +0100, Steve McIntyre wrote:
> On Sat, Sep 04, 2021 at 06:50:16PM +0100, Steve McIntyre wrote:
> >
> >I have a synquacer here still and I'll take a look. I noticed on
> >bullseye release day that USB stuff didn't seem to work in the
> >installer on the synquacer either. Maybe there's been a regression in
> >config. :-/
> >
> >I'll take a look...
> 
> Hmmm, booting 5.10.0-0.bpo.8-arm64 here does not work at all. I'm
> seeing a lot of errors around DMA setup, e.g.:

I bisected this to

7a8b64d17e35810dc3176fe61208b45c15d25402 is the first bad commit
commit 7a8b64d17e35810dc3176fe61208b45c15d25402
Author: Rob Herring <robh@kernel.org>
Date:   Thu Feb 6 14:02:30 2020 +0000

    of/address: use range parser for of_dma_get_range

on what appears to have been a DT system with a built in DT from
the firmware, using upstream's defconfig.  I dimly recall seeing
some discussion of issues here in the past, though I don't have
one of these boxes myself so wasn't paying huge attention.  I'd
guess there's some bug in the DT which given that this is in the
firmware the kernel ought to fix up during early init, if someone
with better access to one of these systems could supply a DT for
inspection that might help people figure things out.

I suspect the machines might work fine when booted with ACPI
firmware.

The bisect log:

git bisect start
# bad: [3d77e6a8804abcc0504c904bd6e5cdf3a5cf8162] Linux 5.7
git bisect bad 3d77e6a8804abcc0504c904bd6e5cdf3a5cf8162
# good: [7111951b8d4973bda27ff663f2cf18b663d15b48] Linux 5.6
git bisect good 0ad2c0e5fc7bd5c5a60f88be1174271410254e32
# skip: [50a5de895dbe5df947b3a695777db5b2c313e065] Merge tag 'for-linus-hmm' of git://git.kernel.org/pub/scm/linux/kernel/git/rdma/rdma
git bisect skip 50a5de895dbe5df947b3a695777db5b2c313e065
# good: [422c032afcf57d5e8109a54912e22ffc53d99068] netfilter: flowtable: Use rw sem as flow block lock
git bisect good 422c032afcf57d5e8109a54912e22ffc53d99068
# skip: [8c1b724ddb218f221612d4c649bc9c7819d8d7a6] Merge tag 'for-linus' of git://git.kernel.org/pub/scm/virt/kvm/kvm
git bisect skip 8c1b724ddb218f221612d4c649bc9c7819d8d7a6
# bad: [88d7fcfa3b1fe670f0412b95be785aafca63352b] net: inet_csk: Fix so_reuseport bind-address cache in tb->fast*
git bisect bad 88d7fcfa3b1fe670f0412b95be785aafca63352b
# skip: [6cad420cc695867b4ca710bac21fde21a4102e4b] Merge branch 'akpm' (patches from Andrew)
git bisect skip 6cad420cc695867b4ca710bac21fde21a4102e4b
# good: [77a36a3ab4ff17fad23831192e3694a3c5a1750d] HID: Add driver fixing Glorious PC Gaming Race mouse report descriptor
git bisect good 77a36a3ab4ff17fad23831192e3694a3c5a1750d
# bad: [8ec91c0fce1500306a858d0c35d1383fd9eb6ba6] Merge tag 'gpio-v5.7-2' of git://git.kernel.org/pub/scm/linux/kernel/git/linusw/linux-gpio
git bisect bad 8ec91c0fce1500306a858d0c35d1383fd9eb6ba6
# skip: [7be97138e7276c71cc9ad1752dcb502d28f4400d] Merge tag 'xfs-5.7-merge-8' of git://git.kernel.org/pub/scm/fs/xfs/xfs-linux
git bisect skip 7be97138e7276c71cc9ad1752dcb502d28f4400d
# bad: [f5637d3b42ab0465ef71d5fb8461bce97fba95e8] mm/memory_hotplug: rename mhp_restrictions to mhp_params
git bisect bad f5637d3b42ab0465ef71d5fb8461bce97fba95e8
# skip: [f365ab31efacb70bed1e821f7435626e0b2528a6] Merge tag 'drm-next-2020-04-01' of git://anongit.freedesktop.org/drm/drm
git bisect skip f365ab31efacb70bed1e821f7435626e0b2528a6
# skip: [c101e9bbce4ae2947b35a660f17d617fc3827595] Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/hid/hid
git bisect skip c101e9bbce4ae2947b35a660f17d617fc3827595
# good: [dc8c609bd31d2b410fd47a82a7b259f68056b244] drm/rcar-du: Plug atomic state hooks to the default implementation
git bisect good dc8c609bd31d2b410fd47a82a7b259f68056b244
# skip: [ccfc569347f870830e7c7cf854679a06cf9c45b5] mlxsw: spectrum_flower: Do not stop at FLOW_ACTION_VLAN_MANGLE
git bisect skip ccfc569347f870830e7c7cf854679a06cf9c45b5
# skip: [e964f1e04a1ce562f0d748b29326244d3cb35ba4] Merge tag 'dmaengine-5.7-rc1' of git://git.infradead.org/users/vkoul/slave-dma
git bisect skip e964f1e04a1ce562f0d748b29326244d3cb35ba4
# good: [1fd34184aab0fe04c5d50af01a37fe1bb8bd6595] drm/meson: dw-hdmi: stop enforcing input_bus_format
git bisect good 1fd34184aab0fe04c5d50af01a37fe1bb8bd6595
# skip: [ff2ae607c6f329d11a3b0528801ea7474be8c3e9] Merge tag 'spdx-5.7-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/spdx
git bisect skip ff2ae607c6f329d11a3b0528801ea7474be8c3e9
# good: [cc674ef252f4750bdcea1560ff491081bb960954] KVM: s390: introduce module parameter kvm.use_gisa
git bisect good cc674ef252f4750bdcea1560ff491081bb960954
# good: [b17350e4037257d25f1ca9772ba5daced9f1bf07] soundwire: cadence: commit changes in the exit_reset() sequence
git bisect good b17350e4037257d25f1ca9772ba5daced9f1bf07
# bad: [aa1a8ce533324d12696a9f4b71dbc5eb561a2e04] Merge tag 'trace-v5.7' of git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-trace
git bisect bad aa1a8ce533324d12696a9f4b71dbc5eb561a2e04
# skip: [848960e576dafc8ed54c691b2f70b92e1fdea9ba] Merge tag 'sound-5.7-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/tiwai/sound
git bisect skip 848960e576dafc8ed54c691b2f70b92e1fdea9ba
# good: [0897f301bc285409ab6453839fa6a43d9ceb26e0] RDMA/mlx5: Remove duplicate definitions of SW_ICM macros
git bisect good 0897f301bc285409ab6453839fa6a43d9ceb26e0
# skip: [397a97946798890b9bdaa6122fcfad7147690670] Merge tag 'linux-kselftest-5.7-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/shuah/linux-kselftest
git bisect skip 397a97946798890b9bdaa6122fcfad7147690670
# bad: [9354371ded2445f674625397b5aaf9ceb881b110] dt-bindings: i2c: i2c-rk3x: add description for rk3308
git bisect bad 9354371ded2445f674625397b5aaf9ceb881b110
# good: [6bb984a3cea02b76d54f2b4bcad7ef0cdca52bc6] dt-bindings: interrupt-controller: Convert UniPhier AIDET to json-schema
git bisect good 6bb984a3cea02b76d54f2b4bcad7ef0cdca52bc6
# bad: [776ab6a3c2460404530345c270b90c51480c9473] dt-bindings: reset: rcar-rst: Convert to json-schema
git bisect bad 776ab6a3c2460404530345c270b90c51480c9473
# bad: [9d55bebd9816903b821a403a69a94190442ac043] of/address: Support multiple 'dma-ranges' entries
git bisect bad 9d55bebd9816903b821a403a69a94190442ac043
# good: [6a9166b5beb82bbfcf86b9215785c6b74a4608ff] powerpc: Drop using struct of_pci_range.pci_space field
git bisect good 6a9166b5beb82bbfcf86b9215785c6b74a4608ff
# good: [bc5e522ec47174770a75df0a76d90f9ebb20132e] of/address: Rework of_pci_range parsing for non-PCI buses
git bisect good bc5e522ec47174770a75df0a76d90f9ebb20132e
# bad: [7a8b64d17e35810dc3176fe61208b45c15d25402] of/address: use range parser for of_dma_get_range
git bisect bad 7a8b64d17e35810dc3176fe61208b45c15d25402
# first bad commit: [7a8b64d17e35810dc3176fe61208b45c15d25402] of/address: use range parser for of_dma_get_range

[toc] | [prev] | [next] | [standalone]


#74902

FromMark Brown <broonie@debian.org>
Date2022-03-29 20:10 +0200
Message-ID<E6os1-53oM-3@gated-at.bofh.it>
In reply to#74896

[Multipart message — attachments visible in raw view] — view raw

On Tue, Mar 29, 2022 at 01:02:34AM +0100, Mark Brown wrote:
> On Sun, Sep 05, 2021 at 05:01:18PM +0100, Steve McIntyre wrote:

> I bisected this to
> 
> commit 7a8b64d17e35810dc3176fe61208b45c15d25402
> 
>     of/address: use range parser for of_dma_get_range
> 
> on what appears to have been a DT system with a built in DT from
> the firmware, using upstream's defconfig.  I dimly recall seeing

Oh, and probably worth pointing out that current mainline handles
the failures with ATA a lot more gracefully, still failing to
bring up ATA but just printing:

[    2.652674] ahci 0000:02:00.0: SSS flag set, parallel bus scan disabled
[    2.659364] ahci 0000:02:00.0: AHCI 0001.0200 32 slots 2 ports 6 Gbps 0x3 impl SATA mode
[    2.667474] ahci 0000:02:00.0: flags: 64bit ncq sntf stag led clo pmp pio slum part ccc sxs 
[    2.676825] ahci 0000:02:00.0: failed to start port 0 (errno=-12)
[    2.682955] ahci: probe of 0000:02:00.0 failed with error -12

and MMC still also fails but this time with an identified oops
rather than just timeouts:

[    2.987953] ------------[ cut here ]------------
[    2.994834] usbcore: registered new interface driver usbhid
[    2.998370] f_sdh30 52300000.sdhci: swiotlb addr 0xffffffffffffffff+65536 overflow (mask ffffffff, bus limit 0).
[    2.998395] WARNING: CPU: 2 PID: 8 at kernel/dma/swiotlb.c:740 swiotlb_map+0x1e4/0x1f8
[    3.003981] usbhid: USB HID core driver
[    3.014149] Modules linked in:
[    3.014159] CPU: 2 PID: 8 Comm: kworker/u48:0 Not tainted 5.17.0-11137-gb953c6822375 #1
[    3.026681] NET: Registered PF_PACKET protocol family
[    3.028964] Hardware name: Socionext SynQuacer E-series DeveloperBox, BIOS build #54 Mar  6 2019
[    3.028973] Workqueue: events_unbound async_run_entry_fn
[    3.028987] pstate: 60000005 (nZCv daif -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[    3.037169] 9pnet: Installing 9P2000 support
[    3.042056] pc : swiotlb_map+0x1e4/0x1f8
[    3.042068] lr : swiotlb_map+0x1e4/0x1f8
[    3.042078] sp : ffff80000818bb80
[    3.050930] Key type dns_resolver registered
[    3.056180] x29: ffff80000818bb80 x28: 0000000000000000 x27: 0000000000000000
[    3.063504] registered taskstats version 1
[    3.067423] 
[    3.067427] x26: 0000000000010000 x25: 0000000000000080 x24: ffff000801787010
[    3.067439] x23: 0000000000000000 x22: 0000000000000000
[    3.071388] Loading compiled-in X.509 certificates
[    3.075292]  x21: 0000000000010000
[    3.075299] x20: ffff8000098c3998 x19: ffff000801787010
[    3.079127] pstore: Using crash dump compression: deflate
[    3.082890]  x18: 0000000000000020
[    3.082897] x17: 0000000000000000 x16: 206f74206b636162 x15: ffffffffffffffff
[    3.082909] x14: 0000000000000001
[    3.097298] OF: translation of DMA address(0) to CPU address failed node()
[    3.102762]  x13: ffff000f7bf76556 x12: ffff000f7bf76551
[    3.102773] x11: 0720072007200720 x10: 000000000000000a x9 : ffff80000818bb80
[    3.102785] x8 : ffff80000a331198 x7 : ffff80000818b980
[    3.108037] gpio-keys gpio-keys: Invalid size 0x1 for dma-range(s)
[    3.112816]  x6 : ffff000f7bf3ef80
[    3.112823] x5 : 0000000000000000 x4 : 0000000000000000
[    3.116564] input: gpio-keys as /devices/platform/gpio-keys/input/input0
[    3.121457]  x3 : 00000000ffffffff
[    3.121464] x2 : ffff80000a0125a0 x1 : fbd95b9fae668f00 x0 : 0000000000000000
[    3.197275] Call trace:
[    3.199721]  swiotlb_map+0x1e4/0x1f8
[    3.203305]  dma_map_page_attrs+0xec/0x228
[    3.207408]  sdhci_setup_host+0x8f0/0xd30
[    3.211430]  sdhci_add_host+0x18/0x50
[    3.215098]  sdhci_f_sdh30_probe+0x2a0/0x378
[    3.219373]  platform_probe+0x68/0xd8
[    3.223043]  really_probe+0xb8/0x300
[    3.226622]  __driver_probe_device+0x78/0xe0
[    3.230896]  driver_probe_device+0x3c/0xe8
[    3.234997]  __driver_attach_async_helper+0x2c/0x50
[    3.239880]  async_run_entry_fn+0x34/0xd0
[    3.243896]  process_one_work+0x1ec/0x330
[    3.247912]  worker_thread+0x44/0x420
[    3.251579]  kthread+0x110/0x120
[    3.254815]  ret_from_fork+0x10/0x20
[    3.258397] ---[ end trace 0000000000000000 ]---

followed by more in sdhci_send_command().

[   23.661522] WARNING: CPU: 0 PID: 0 at drivers/mmc/host/sdhci.c:1151 sdhci_send_command+0x954/0xde8

I'm guessing these are going to be the same underlying issue with the
firmware but manifesting a bit differently, the network device
does still complain about the 1 byte dma-range.  The system gets
to a ramdisk but has no useful I/O.

[toc] | [prev] | [standalone]


Back to top | Article view | linux.debian.kernel


csiph-web