Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.kernel > #55714 > unrolled thread

Bug#843448: linux-image-4.8.0-1-armmp-lpae: fails to boot on Odroid-Xu4 with rootfs on USB

Back to article view | Back to linux.debian.kernel

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Bug#843448: linux-image-4.8.0-1-armmp-lpae: fails to boot on Odroid-Xu4 with rootfs on USB Ben Hutchings <ben@decadent.org.uk> - 2016-11-13 05:30 +0100
    Bug#843448: linux-image-4.8.0-1-armmp-lpae: fails to boot on Odroid-Xu4 with rootfs on USB Vagrant Cascadian <vagrant@debian.org> - 2017-04-22 18:40 +0200
      Bug#843448: linux-image-4.8.0-1-armmp-lpae: fails to boot on Odroid-Xu4 with rootfs on USB Vagrant Cascadian <vagrant@debian.org> - 2017-04-22 19:20 +0200
    Bug#843448: linux-image-4.8.0-1-armmp-lpae: fails to boot on Odroid-Xu4 with rootfs on USB "Steinar H. Gunderson" <sgunderson@bigfoot.com> - 2017-05-14 16:20 +0200
      Bug#843448: linux-image-4.8.0-1-armmp-lpae: fails to boot on Odroid-Xu4 with rootfs on USB "Steinar H. Gunderson" <sgunderson@bigfoot.com> - 2017-05-24 00:40 +0200
        Bug#843448: linux-image-4.8.0-1-armmp-lpae: fails to boot on Odroid-Xu4 with rootfs on USB Vagrant Cascadian <vagrant@debian.org> - 2017-05-24 23:20 +0200
          Bug#843448: linux-image-4.8.0-1-armmp-lpae: fails to boot on Odroid-Xu4 with rootfs on USB "Steinar H. Gunderson" <sgunderson@bigfoot.com> - 2017-05-24 23:40 +0200
            Bug#843448: linux-image-4.8.0-1-armmp-lpae: fails to boot on Odroid-Xu4 with rootfs on USB "Steinar H. Gunderson" <sgunderson@bigfoot.com> - 2017-05-27 11:30 +0200
              Bug#843448: linux-image-4.8.0-1-armmp-lpae: fails to boot on Odroid-Xu4 with rootfs on USB Vagrant Cascadian <vagrant@debian.org> - 2017-05-27 17:50 +0200
                Bug#843448: linux-image-4.8.0-1-armmp-lpae: fails to boot on Odroid-Xu4 with rootfs on USB "Steinar H. Gunderson" <sgunderson@bigfoot.com> - 2017-05-27 18:10 +0200
                Bug#843448: linux-image-4.8.0-1-armmp-lpae: fails to boot on Odroid-Xu4 with rootfs on USB Vagrant Cascadian <vagrant@debian.org> - 2017-05-27 18:10 +0200
        Bug#843448: linux-image-4.8.0-1-armmp-lpae: fails to boot on Odroid-Xu4 with rootfs on USB "Steinar H. Gunderson" <sgunderson@bigfoot.com> - 2017-05-27 18:50 +0200
        Bug#843448: linux-image-4.8.0-1-armmp-lpae: fails to boot on Odroid-Xu4 with rootfs on USB gustavo panizzo <gfa@zumbi.com.ar> - 2017-05-27 18:50 +0200

#55714 — Bug#843448: linux-image-4.8.0-1-armmp-lpae: fails to boot on Odroid-Xu4 with rootfs on USB

[Multipart message — attachments visible in raw view] — view raw

On Sun, 2016-11-06 at 10:00 -0800, Vagrant Cascadian wrote:
> Package: src:linux
> Version: 4.8.5-1
> Severity: important
> 
> When upgrading to linux 4.8.x on an Odroid-XU4 system, it no longer
> detects the USB sata adapter where rootfs is located.
> 
> Downgrading to the 4.7.x kernel from jessie-backports works fine.
> 
> I didn't see anything obviously missing when diffing /boot/config-*.
> 
> Boot log attached.
> 
> If there's nothing obvious fromt the boot log, I can try taking it
> offline and doing an install with stretch or sid to do more
> debugging.

This message (from the core hub driver) looks important:

2016-11-06_17:33:29.19077 [    7.510995] usb usb3-port1: connect-debounce failed

The device tree source for Exynos 5422 was changed between 4.7 and 4.8
so some of the USB definitions could be shared with the 5410 (commit
cb0896562228703209644e92926ed445150cc594).  This obviously shouldn't
have a functional difference to the 5422 but maybe it did?

Ben.

-- 
Ben Hutchings
Nothing is ever a complete failure; it can always serve as a bad
example.

[toc] | [next] | [standalone]

#57639

[Multipart message — attachments visible in raw view] — view raw

On 2016-11-12, Ben Hutchings wrote:
> On Sun, 2016-11-06 at 10:00 -0800, Vagrant Cascadian wrote:
>> When upgrading to linux 4.8.x on an Odroid-XU4 system, it no longer
>> detects the USB sata adapter where rootfs is located.
>> 
>> Downgrading to the 4.7.x kernel from jessie-backports works fine.
...
> This message (from the core hub driver) looks important:
>
> 2016-11-06_17:33:29.19077 [    7.510995] usb usb3-port1: connect-debounce failed
>
> The device tree source for Exynos 5422 was changed between 4.7 and 4.8
> so some of the USB definitions could be shared with the 5410 (commit
> cb0896562228703209644e92926ed445150cc594).  This obviously shouldn't
> have a functional difference to the 5422 but maybe it did?

I tried booting 4.9 from jessie-backports just now using the dts from
4.7, and it had the same behavior. So I'm guessing that's not the issue.

I also tried with an initramfs that had all of the available driver
modules included, with no luck.

Will try with stretch's d-i next...


live well,
  vagrant

[toc] | [prev] | [next] | [standalone]

#57640

[Multipart message — attachments visible in raw view] — view raw

On 2017-04-22, Vagrant Cascadian wrote:
> On 2016-11-12, Ben Hutchings wrote:
>> On Sun, 2016-11-06 at 10:00 -0800, Vagrant Cascadian wrote:
>>> When upgrading to linux 4.8.x on an Odroid-XU4 system, it no longer
>>> detects the USB sata adapter where rootfs is located.
>>> 
>>> Downgrading to the 4.7.x kernel from jessie-backports works fine.
> ...
>> This message (from the core hub driver) looks important:
>>
>> 2016-11-06_17:33:29.19077 [    7.510995] usb usb3-port1: connect-debounce failed
...
> Will try with stretch's d-i next...

With stretch's d-i, I don't see the debounce messages, but USB fails to
work, and ethernet is implemented over USB, so this fails on the d-i
netboot image.

Attached is the full dmesg output from within d-i.

~ # uname -a
Linux (none) 4.9.0-2-armmp #1 SMP Debian 4.9.18-1 (2017-03-30) armv7l
GNU/Linux
~ #
~ # lsmod
Module                  Size  Used by
dwc3_exynos             3827  0
ohci_exynos             4178  0
ohci_hcd               37898  1 ohci_exynos
ehci_exynos             4269  0
ehci_hcd               64996  1 ehci_exynos
dw_mmc_exynos           5942  0
usbcore               195888  4
ehci_exynos,ehci_hcd,ohci_hcd,ohci_exynos
dw_mmc_pltfm            2485  1 dw_mmc_exynos
dw_mmc                 25740  2 dw_mmc_pltfm,dw_mmc_exynos
phy_exynos_usb2         9238  2
phy_exynos5_usbdrd      7053  0
usb_common              3659  1 usbcore


live well,
  vagrant

[toc] | [prev] | [next] | [standalone]

#57781

On Sun, Nov 13, 2016 at 04:21:30AM +0000, Ben Hutchings wrote:
> This message (from the core hub driver) looks important:
> 
> 2016-11-06_17:33:29.19077 [    7.510995] usb usb3-port1: connect-debounce failed
> 
> The device tree source for Exynos 5422 was changed between 4.7 and 4.8
> so some of the USB definitions could be shared with the 5410 (commit
> cb0896562228703209644e92926ed445150cc594).  This obviously shouldn't
> have a functional difference to the 5422 but maybe it did?

There was some discussion upstream, but with no definitive conclusion:

  https://lkml.org/lkml/2016/8/28/183

I've verified that this fails on my XU4 in both Debian 4.9 and 4.11,
as well as upstream 4.12-rc1 (haven't tried 4.10, probably doesn't work).
The discussion says this broke in 4.8. The bisect pointed to a patch, which
has a partial revert here:

  https://lkml.org/lkml/2016/10/16/63

They seem too hackish to go into upstream, though.

This bug unfortunately makes the XU4 pretty useless, since the NIC is hooked
up over USB. 4.7 works, but has no security support anymore (final release
was 4.7.10, in October 2016).

/* Steinar */
-- 
Homepage: https://www.sesse.net/

[toc] | [prev] | [next] | [standalone]

#57831

On Sun, May 14, 2017 at 04:08:56PM +0200, Steinar H. Gunderson wrote:
> I've verified that this fails on my XU4 in both Debian 4.9 and 4.11,
> as well as upstream 4.12-rc1 (haven't tried 4.10, probably doesn't work).
> The discussion says this broke in 4.8. The bisect pointed to a patch, which
> has a partial revert here:
> 
>   https://lkml.org/lkml/2016/10/16/63
> 
> They seem too hackish to go into upstream, though.

I checked out ODROID's own 4.9 tree, and it works fine (tested for a few
days). So I diffed drivers/usb/dwc3/, and these are the only two patches that
actually are missing that I can see:

  https://github.com/hardkernel/linux/commit/74b9605e5587b30912d6b6093e9d7fb06d043c33
  https://github.com/hardkernel/linux/commit/2166ffd004e04a61887eb2a39f8639dc12140c58

The second doesn't apply cleanly, and I don't understand the commit message
anyway (does it only matter if dwc3 is _not_ built as a module?), but the
former seems safe enough to apply. Given that it's been in their tree since
2016, it also would seem to have received a fair amount of user testing.

I've applied the former patch (the revert) on top of Debian's 4.9.25 kernel
(linux-image-4.9.0-3-armmp-lpae), and it boots with working USB and thus
networking on my XU4. I saw this at some point after boot, though:

  [  550.087389] xhci-hcd xhci-hcd.5.auto: Cannot set link state.
  [  550.087426] usb usb6-port1: cannot disable (err = -32)

In any case, it essentially takes the system from “totally broken” to “might
have some problems”. Would you consider taking it in for stretch, so that the
XU4 has networking? I'll keep running with this kernel to get more
experience.

/* Steinar */
-- 
Homepage: https://www.sesse.net/

[toc] | [prev] | [next] | [standalone]

#57833

[Multipart message — attachments visible in raw view] — view raw

On 2017-05-23, Steinar H. Gunderson wrote:
>   https://github.com/hardkernel/linux/commit/74b9605e5587b30912d6b6093e9d7fb06d043c33
...
> I've applied the former patch (the revert) on top of Debian's 4.9.25 kernel
> (linux-image-4.9.0-3-armmp-lpae), and it boots with working USB and thus
> networking on my XU4. I saw this at some point after boot, though:
>
>   [  550.087389] xhci-hcd xhci-hcd.5.auto: Cannot set link state.
>   [  550.087426] usb usb6-port1: cannot disable (err = -32)

I've also applied the patch and was able to boot an odroid-xu4 with
it. I've got three of them used for the reproducible builds farm that
have been stuck on the 4.7.x kernel, so many thanks for identifying a
workaround! Will start using 4.9.x with this patch...


> In any case, it essentially takes the system from “totally broken” to “might
> have some problems”. Would you consider taking it in for stretch, so that the
> XU4 has networking? I'll keep running with this kernel to get more
> experience.

Unfortunately, there are numerous other boards using dwc3, so I'm not
sure this is a good idea...


live well,
  vagrant

[toc] | [prev] | [next] | [standalone]

#57834

On Wed, May 24, 2017 at 02:15:46PM -0700, Vagrant Cascadian wrote:
>> In any case, it essentially takes the system from “totally broken” to “might
>> have some problems”. Would you consider taking it in for stretch, so that the
>> XU4 has networking? I'll keep running with this kernel to get more
>> experience.
> Unfortunately, there are numerous other boards using dwc3, so I'm not
> sure this is a good idea...

Well, it's a revert of something that's supposed to only be a
cleanup/refactor (but isn't, presumably due to phy bugs). So it _should_ be
safe, at least in theory.

FWIW, after a bit over two hours, this happened:

  [ 7912.391226] usb 4-1: new SuperSpeed USB device number 2 using xhci-hcd
  [ 7912.413744] usb 4-1: New USB device found, idVendor=05e3, idProduct=0616
  [ 7912.413752] usb 4-1: New USB device strings: Mfr=1, Product=2, SerialNumber=0
  [ 7912.413758] usb 4-1: Product: USB3.0 Hub
  [ 7912.413764] usb 4-1: Manufacturer: GenesysLogic
  [ 7912.417680] hub 4-1:1.0: USB hub found
  [ 7912.417989] hub 4-1:1.0: 2 ports detected

This might be the issue of warm USB3 boot the other patch talked about?
(I don't have any devices connected to the external ports, so I don't really
know whether they work. I just want the NIC :-) )

/* Steinar */
-- 
Homepage: https://www.sesse.net/

[toc] | [prev] | [next] | [standalone]

#57845

On Wed, May 24, 2017 at 11:37:36PM +0200, Steinar H. Gunderson wrote:
> Well, it's a revert of something that's supposed to only be a
> cleanup/refactor (but isn't, presumably due to phy bugs). So it _should_ be
> safe, at least in theory.
> 
> FWIW, after a bit over two hours, this happened:
> 
>   [ 7912.391226] usb 4-1: new SuperSpeed USB device number 2 using xhci-hcd
>   [ 7912.413744] usb 4-1: New USB device found, idVendor=05e3, idProduct=0616
>   [ 7912.413752] usb 4-1: New USB device strings: Mfr=1, Product=2, SerialNumber=0
>   [ 7912.413758] usb 4-1: Product: USB3.0 Hub
>   [ 7912.413764] usb 4-1: Manufacturer: GenesysLogic
>   [ 7912.417680] hub 4-1:1.0: USB hub found
>   [ 7912.417989] hub 4-1:1.0: 2 ports detected

72 hours uptime now, with no more ill effects. NIC works fine.

/* Steinar */
-- 
Homepage: https://www.sesse.net/

[toc] | [prev] | [next] | [standalone]

#57847

[Multipart message — attachments visible in raw view] — view raw

On 2017-05-27, Steinar H. Gunderson wrote:
> On Wed, May 24, 2017 at 11:37:36PM +0200, Steinar H. Gunderson wrote:
>> Well, it's a revert of something that's supposed to only be a
>> cleanup/refactor (but isn't, presumably due to phy bugs). So it _should_ be
>> safe, at least in theory.
>> 
>> FWIW, after a bit over two hours, this happened:
>> 
>>   [ 7912.391226] usb 4-1: new SuperSpeed USB device number 2 using xhci-hcd
>>   [ 7912.413744] usb 4-1: New USB device found, idVendor=05e3, idProduct=0616
>>   [ 7912.413752] usb 4-1: New USB device strings: Mfr=1, Product=2, SerialNumber=0
>>   [ 7912.413758] usb 4-1: Product: USB3.0 Hub
>>   [ 7912.413764] usb 4-1: Manufacturer: GenesysLogic
>>   [ 7912.417680] hub 4-1:1.0: USB hub found
>>   [ 7912.417989] hub 4-1:1.0: 2 ports detected
>
> 72 hours uptime now, with no more ill effects. NIC works fine.

The three odroid-xu4 reproducible builds nodes, with rootfs and a build
partition on USB, have mostly been working under load with ~2.5 days
uptime for 2/3 of the machines:

  https://jenkins.debian.net/munin/debian.net/odxu4-armhf-rb.debian.net/index.html
  https://jenkins.debian.net/munin/debian.net/odxu4b-armhf-rb.debian.net/index.html
  https://jenkins.debian.net/munin/debian.net/odxu4c-armhf-rb.debian.net/index.html

I've also got a BeagleBoard-X15 that works reasonably well without the
patch; will test the patched kernel to see if it appears to cause any
problems.


live well,
  vagrant

[toc] | [prev] | [next] | [standalone]

#57848

On Sat, May 27, 2017 at 08:57:00AM -0700, Vagrant Cascadian wrote:
> Of course, the BeagleBoard-X15 isn't using USB for anything, so this
> isn't much of a test, but it's the only other board I have available
> that uses dwc3.

I suppose you could plug a device into it, even if there's nothing onboard
connected to it? I don't know the board, but from pictures and specs, it
seems to have multiple USB 3.0 ports.

/* Steinar */
-- 
Homepage: https://www.sesse.net/

[toc] | [prev] | [next] | [standalone]

#57849

[Multipart message — attachments visible in raw view] — view raw

On 2017-05-27, Vagrant Cascadian wrote:
> I've also got a BeagleBoard-X15 that works reasonably well without the
> patch; will test the patched kernel to see if it appears to cause any
> problems.

Of course, the BeagleBoard-X15 isn't using USB for anything, so this
isn't much of a test, but it's the only other board I have available
that uses dwc3.


live well,
  vagrant

[toc] | [prev] | [next] | [standalone]

#57850

On Sun, May 28, 2017 at 12:38:12AM +0800, gustavo panizzo wrote:
> NIC works right away, I have to cold boot to see the USB3 hard
> drives.
> The only kernel that worked reliably after reboot was 4.5.5-1

Have you tried to apply the other patch, too? It's supposedly helping with
this very problem. (I haven't tried the USB3 external ports, really.)

/* Steinar */
-- 
Homepage: https://www.sesse.net/

[toc] | [prev] | [next] | [standalone]

#57851

[Multipart message — attachments visible in raw view] — view raw

On Wed, May 24, 2017 at 12:37:38AM +0200, Steinar H. Gunderson wrote:
>On Sun, May 14, 2017 at 04:08:56PM +0200, Steinar H. Gunderson wrote:
>> I've verified that this fails on my XU4 in both Debian 4.9 and 4.11,
>> as well as upstream 4.12-rc1 (haven't tried 4.10, probably doesn't work).
>> The discussion says this broke in 4.8. The bisect pointed to a patch, which
>> has a partial revert here:
>>
>>   https://lkml.org/lkml/2016/10/16/63
>>
>> They seem too hackish to go into upstream, though.
>
>I checked out ODROID's own 4.9 tree, and it works fine (tested for a few
>days). So I diffed drivers/usb/dwc3/, and these are the only two patches that
>actually are missing that I can see:
>
>  https://github.com/hardkernel/linux/commit/74b9605e5587b30912d6b6093e9d7fb06d043c33

I've applied this patch and booted my XU4

NIC works right away, I have to cold boot to see the USB3 hard
drives.
The only kernel that worked reliably after reboot was 4.5.5-1 

My UX4 is a fileserver, it sees a lot of disk and nic activity, I'll
report if I face any problems with this kernel

[...]

>networking on my XU4. I saw this at some point after boot, though:
>
>  [  550.087389] xhci-hcd xhci-hcd.5.auto: Cannot set link state.
>  [  550.087426] usb usb6-port1: cannot disable (err = -32)

I don't have those errors on my dmesg, at least yet


-- 
IRC: gfa
GPG: 0X44BB1BA79F6C6333

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.kernel

csiph-web