Re: [PATCH 1/4] net: thunderx: Fix IOMMU translation faults

From	Sunil Kovvuri <sunil.kovvuri@gmail.com>
Newsgroups	linux.kernel
Subject	Re: [PATCH 1/4] net: thunderx: Fix IOMMU translation faults
Date	2017-03-07 13:50 +0100
Message-ID	<timIX-7Bl-41@gated-at.bofh.it> (permalink)
References	(2 earlier) <tgZEK-4Br-13@gated-at.bofh.it> <thaTv-4bC-1@gated-at.bofh.it> <ti0fn-8f0-17@gated-at.bofh.it> <ti0yK-9I-9@gated-at.bofh.it> <ti3PY-2mY-13@gated-at.bofh.it>
Organization	linux.* mail to news gateway

Show all headers | View raw

On Mon, Mar 6, 2017 at 10:02 PM, Robin Murphy <robin.murphy@arm.com> wrote:
> On 06/03/17 12:57, Sunil Kovvuri wrote:
>>>>
>>>> We are seeing a 0.75Mpps drop with IP forwarding rate due to that.
>>>> Hence I have restricted calling DMA interfaces to only when IOMMU is enabled.
>>>
>>> What's 0.07Mpps as a percentage of baseline? On a correctly configured
>>> coherent arm64 system, in the absence of an IOMMU, dma_map_*() is
>>> essentially just virt_to_phys() behind a function call or two, so I'd be
>>> interested to know where any non-trivial overhead might be coming from.
>>
>> It's a 5% drop and yes device is configured as coherent.
>> And the drop is due to additional function calls.
>
> OK, interesting - sounds like there's potential for some optimisation
> there as well. AFAICS the callchain goes:
>
> dma_map_single_attrs (inline)
> - ops->map_page (__swiotlb_map_page)
>   - swiotlb_map_page
>     - phys_to_dma (inline)
>     - dma_capable (inline)
>
> Do you happen to have a breakdown of where the time goes? If it's mostly
> just in the indirect branch our options are limited (I'm guessing
> ThunderX doesn't have a particularly fancy branch predictor, if it's not
> even got a data prefetcher), but if it's in the SWIOTLB code then
> there's certainly room for improvement (which will hopefully tie in with
> some DMA ops work I'm planning to do soon anyway).

It's the branching which is costing the performance, as you said nothing
much can be done in the common code for this. Anyway I have submitted
new patch without conditional calling of DMA APIs, will look into reducing
performance impact (if possible implement recycling) a bit later.

Thanks,
Sunil.

Back to linux.kernel | Previous | Next — Previous in thread | Find similar | Unroll thread

Thread

[PATCH 1/4] net: thunderx: Fix IOMMU translation faults sunil.kovvuri@gmail.com - 2017-03-03 12:00 +0100
  Re: [PATCH 1/4] net: thunderx: Fix IOMMU translation faults David Miller <davem@davemloft.net> - 2017-03-03 19:00 +0100
    Re: [PATCH 1/4] net: thunderx: Fix IOMMU translation faults Sunil Kovvuri <sunil.kovvuri@gmail.com> - 2017-03-04 07:00 +0100
      Re: [PATCH 1/4] net: thunderx: Fix IOMMU translation faults Robin Murphy <robin.murphy@arm.com> - 2017-03-06 13:50 +0100
        Re: [PATCH 1/4] net: thunderx: Fix IOMMU translation faults Sunil Kovvuri <sunil.kovvuri@gmail.com> - 2017-03-06 14:10 +0100
          Re: [PATCH 1/4] net: thunderx: Fix IOMMU translation faults Robin Murphy <robin.murphy@arm.com> - 2017-03-06 17:40 +0100
            Re: [PATCH 1/4] net: thunderx: Fix IOMMU translation faults Sunil Kovvuri <sunil.kovvuri@gmail.com> - 2017-03-07 13:50 +0100

csiph-web