Re: [PATCH 0/2] PCI: Workaround for bus reset on Cavium cn8xxx root ports

Path	csiph.com!news.mixmin.net!aioe.org!bofh.it!news.nic.it!robomod
From	Bjorn Helgaas <helgaas@kernel.org>
Newsgroups	linux.kernel
Subject	Re: [PATCH 0/2] PCI: Workaround for bus reset on Cavium cn8xxx root ports
Date	Tue, 23 May 2017 23:30:01 +0200
Message-ID	<tKpxn-AI-9@gated-at.bofh.it> (permalink)
References	<tHR6N-1Me-7@gated-at.bofh.it> <tKoUG-8uq-17@gated-at.bofh.it> <tKpe3-rD-57@gated-at.bofh.it>
X-Original-To	Alex Williamson <alex.williamson@redhat.com>
Dmarc-Filter	OpenDMARC Filter v1.3.2 mail.kernel.org 6854923961
Authentication-Results	mail.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.org
Authentication-Results	mail.kernel.org; spf=none smtp.mailfrom=helgaas@kernel.org
MIME-Version	1.0
Content-Type	text/plain; charset=us-ascii
Content-Disposition	inline
User-Agent	Mutt/1.5.21 (2010-09-15)
Sender	robomod@news.nic.it
List-ID	<linux-kernel.vger.kernel.org>
X-Mailing-List	linux-kernel@vger.kernel.org
Approved	robomod@news.nic.it
Lines	69
Organization	linux.* mail to news gateway
X-Original-Cc	David Daney <david.daney@cavium.com>, Bjorn Helgaas <bhelgaas@google.com>, linux-pci@vger.kernel.org, Jon Masters <jcm@redhat.com>, Robert Richter <robert.richter@cavium.com>, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org
X-Original-Date	Tue, 23 May 2017 16:20:00 -0500
X-Original-Message-ID	<20170523212000.GA20928@bhelgaas-glaptop.roam.corp.google.com>
X-Original-References	<20170516001736.4001-1-david.daney@cavium.com> <20170523204750.GC15055@bhelgaas-glaptop.roam.corp.google.com> <20170523150404.221b01c7@w520.home>
X-Original-Sender	linux-kernel-owner@vger.kernel.org
Xref	csiph.com linux.kernel:1648760

Show key headers only | View raw

On Tue, May 23, 2017 at 03:04:04PM -0600, Alex Williamson wrote:
> On Tue, 23 May 2017 15:47:50 -0500
> Bjorn Helgaas <helgaas@kernel.org> wrote:
> 
> > On Mon, May 15, 2017 at 05:17:34PM -0700, David Daney wrote:
> > > With the recent improvements in arm64 and vfio-pci, we are seeing
> > > failures like this (on cn8890 based systems):
> > > 
> > > [  235.622361] Unhandled fault: synchronous external abort (0x96000210) at 0xfffffc00c1000100
> > > [  235.630625] Internal error: : 96000210 [#1] PREEMPT SMP
> > > .
> > > .
> > > .
> > > [  236.208820] [<fffffc0008411250>] pci_generic_config_read+0x38/0x9c
> > > [  236.214992] [<fffffc0008435ed4>] thunder_pem_config_read+0x54/0x1e8
> > > [  236.221250] [<fffffc0008411620>] pci_bus_read_config_dword+0x74/0xa0
> > > [  236.227596] [<fffffc000841853c>] pci_find_next_ext_capability.part.15+0x40/0xb8
> > > [  236.234896] [<fffffc0008419428>] pci_find_ext_capability+0x20/0x30
> > > [  236.241068] [<fffffc0008423e2c>] pci_restore_vc_state+0x34/0x88
> > > [  236.246979] [<fffffc000841af3c>] pci_restore_state.part.37+0x2c/0x1fc
> > > [  236.253410] [<fffffc000841b174>] pci_dev_restore+0x4c/0x50
> > > [  236.258887] [<fffffc000841b19c>] pci_bus_restore+0x24/0x4c
> > > [  236.264362] [<fffffc000841c2dc>] pci_try_reset_bus+0x7c/0xa0
> > > [  236.270021] [<fffffc00060a1ab0>] vfio_pci_ioctl+0xc34/0xc3c [vfio_pci]
> > > [  236.276547] [<fffffc0005eb0410>] vfio_device_fops_unl_ioctl+0x20/0x30 [vfio]
> > > [  236.283587] [<fffffc000824b314>] do_vfs_ioctl+0xac/0x744
> > > [  236.288890] [<fffffc000824ba30>] SyS_ioctl+0x84/0x98
> > > [  236.293846] [<fffffc0008082ca0>] __sys_trace_return+0x0/0x4
> > > 
> > > These are caused by the inability of the PCIe root port and Intel
> > > e1000e to sucessfully do a bus reset.
> > > 
> > > The proposed fix is to not do a bus reset on these systems.
> > > 
> > > David Daney (2):
> > >   PCI: Allow PCI_DEV_FLAGS_NO_BUS_RESET to be used on bus device.
> > >   PCI: Avoid bus reset for Cavium cn8xxx root ports.
> > > 
> > >  drivers/pci/pci.c    | 4 ++++
> > >  drivers/pci/quirks.c | 8 ++++++++
> > >  2 files changed, 12 insertions(+)  
> > 
> > Applied with Eric's reviewed-by and typo fixes to pci/virtualization for
> > v4.13, thanks!
> 
> Hmm, well let me again express my concerns that I'm really not sure how
> to support this since it removes our last opportunity to reset devices
> that may otherwise have no reset mechanism.  Certain classes of devices
> are entirely unsupportable for the code path indicated above without a
> bus reset.  If we have an endpoint device that goes bonkers at a bus
> reset, at least we know it's going to behave just as poorly no matter
> what the host platform.  This series allows endpoints that work
> perfectly well on one host to be handled differently on another.  It
> certainly suggests something non-spec compliant about the root port
> implementation and I wish there was more analysis about exactly what
> that problem is since this is coming from the hardware vendor.
> 
> https://lkml.org/lkml/2017/5/16/662

I almost poked you about this on IRC; guess I should have :)

Is it better to leave it as-is, and just take the aborts David
reported?

I agree, it would be nice to know what's really going on.  I assume
Cavium is interested in that as well to make sure future parts don't
have the issue.

Bjorn

Thread

Re: [PATCH 0/2] PCI: Workaround for bus reset on Cavium cn8xxx root  ports Bjorn Helgaas <helgaas@kernel.org> - 2017-05-23 22:50 +0200
  Re: [PATCH 0/2] PCI: Workaround for bus reset on Cavium cn8xxx root  ports Alex Williamson <alex.williamson@redhat.com> - 2017-05-23 23:10 +0200
    Re: [PATCH 0/2] PCI: Workaround for bus reset on Cavium cn8xxx root  ports Bjorn Helgaas <helgaas@kernel.org> - 2017-05-23 23:30 +0200
    Re: [PATCH 0/2] PCI: Workaround for bus reset on Cavium cn8xxx root  ports Alex Williamson <alex.williamson@redhat.com> - 2017-05-24 00:20 +0200

csiph-web