Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > linux.kernel > #1507530 > unrolled thread
| Started by | Alexander Duyck <alexander.h.duyck@intel.com> |
|---|---|
| First post | 2016-10-24 20:10 +0200 |
| Last post | 2016-10-24 20:20 +0200 |
| Articles | 13 — 2 participants |
Back to article view | Back to linux.kernel
[net-next PATCH RFC 00/26] Add support for DMA writable pages being writable by the network stack Alexander Duyck <alexander.h.duyck@intel.com> - 2016-10-24 20:10 +0200
[net-next PATCH RFC 26/26] igb: Revert "igb: Revert support for build_skb in igb" Alexander Duyck <alexander.h.duyck@intel.com> - 2016-10-24 20:10 +0200
[net-next PATCH RFC 07/26] arch/c6x: Add option to skip sync on DMA map and unmap Alexander Duyck <alexander.h.duyck@intel.com> - 2016-10-24 20:10 +0200
[net-next PATCH RFC 10/26] arch/m68k: Add option to skip DMA sync as a part of mapping Alexander Duyck <alexander.h.duyck@intel.com> - 2016-10-24 20:10 +0200
[net-next PATCH RFC 13/26] arch/mips: Add option to skip DMA sync as a part of map and unmap Alexander Duyck <alexander.h.duyck@intel.com> - 2016-10-24 20:10 +0200
[net-next PATCH RFC 12/26] arch/microblaze: Add option to skip DMA sync as a part of map and unmap Alexander Duyck <alexander.h.duyck@intel.com> - 2016-10-24 20:10 +0200
Re: [net-next PATCH RFC 01/26] swiotlb: Drop unused function swiotlb_map_sg Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> - 2016-10-24 20:20 +0200
[net-next PATCH RFC 02/26] swiotlb: Add support for DMA_ATTR_SKIP_CPU_SYNC Alexander Duyck <alexander.h.duyck@intel.com> - 2016-10-24 20:20 +0200
[net-next PATCH RFC 01/26] swiotlb: Drop unused function swiotlb_map_sg Alexander Duyck <alexander.h.duyck@intel.com> - 2016-10-24 20:20 +0200
[net-next PATCH RFC 09/26] arch/hexagon: Add option to skip DMA sync as a part of mapping Alexander Duyck <alexander.h.duyck@intel.com> - 2016-10-24 20:20 +0200
[net-next PATCH RFC 08/26] arch/frv: Add option to skip sync on DMA map Alexander Duyck <alexander.h.duyck@intel.com> - 2016-10-24 20:20 +0200
[net-next PATCH RFC 15/26] arch/openrisc: Add option to skip DMA sync as a part of mapping Alexander Duyck <alexander.h.duyck@intel.com> - 2016-10-24 20:20 +0200
[net-next PATCH RFC 03/26] arch/arc: Add option to skip sync on DMA mapping Alexander Duyck <alexander.h.duyck@intel.com> - 2016-10-24 20:20 +0200
| From | Alexander Duyck <alexander.h.duyck@intel.com> |
|---|---|
| Date | 2016-10-24 20:10 +0200 |
| Subject | [net-next PATCH RFC 00/26] Add support for DMA writable pages being writable by the network stack |
| Message-ID | <svRR7-2Kt-9@gated-at.bofh.it> |
The first 21 patches in the set add support for the DMA attribute
DMA_ATTR_SKIP_CPU_SYNC on multiple platforms/architectures. This is needed
so that we can flag the calls to dma_map/unmap_page so that we do not
invalidate cache lines that do not currently belong to the device. Instead
we have to take care of this in the driver via a call to
sync_single_range_for_cpu prior to freeing the Rx page.
Patch 22 adds support for dma_map_page_attrs and dma_unmap_page_attrs so
that we can unmap and map a page using the DMA_ATTR_SKIP_CPU_SYNC
attribute.
Patch 23 adds support for freeing a page that has multiple references being
held by a single caller. This way we can free page fragments that were
allocated by a given driver.
The last 3 patches use these updates in the igb driver to allow for us to
reimplement the use of build_skb which hands a writable page off to the
stack.
My hope is to get the series accepted into the net-next tree as I have a
number of other Intel drivers I could then begin updating once these
patches are accepted.
Any feedback is welcome. Specifically if there is something I overlooked
design-wise or an architecture I missed please let me know and I will add
it to this patch set. If needed I can look into breaking this into a
smaller set of patches but this set is all that should be needed to then
start looking at putting together a DMA page pool per device which I know
is something Jesper has been working on.
---
Alexander Duyck (26):
swiotlb: Drop unused function swiotlb_map_sg
swiotlb: Add support for DMA_ATTR_SKIP_CPU_SYNC
arch/arc: Add option to skip sync on DMA mapping
arch/arm: Add option to skip sync on DMA map and unmap
arch/avr32: Add option to skip sync on DMA map
arch/blackfin: Add option to skip sync on DMA map
arch/c6x: Add option to skip sync on DMA map and unmap
arch/frv: Add option to skip sync on DMA map
arch/hexagon: Add option to skip DMA sync as a part of mapping
arch/m68k: Add option to skip DMA sync as a part of mapping
arch/metag: Add option to skip DMA sync as a part of map and unmap
arch/microblaze: Add option to skip DMA sync as a part of map and unmap
arch/mips: Add option to skip DMA sync as a part of map and unmap
arch/nios2: Add option to skip DMA sync as a part of map and unmap
arch/openrisc: Add option to skip DMA sync as a part of mapping
arch/parisc: Add option to skip DMA sync as a part of map and unmap
arch/powerpc: Add option to skip DMA sync as a part of mapping
arch/sh: Add option to skip DMA sync as a part of mapping
arch/sparc: Add option to skip DMA sync as a part of map and unmap
arch/tile: Add option to skip DMA sync as a part of map and unmap
arch/xtensa: Add option to skip DMA sync as a part of mapping
dma: Add calls for dma_map_page_attrs and dma_unmap_page_attrs
mm: Add support for releasing multiple instances of a page
igb: Update driver to make use of DMA_ATTR_SKIP_CPU_SYNC
igb: Update code to better handle incrementing page count
igb: Revert "igb: Revert support for build_skb in igb"
arch/arc/mm/dma.c | 3
arch/arm/common/dmabounce.c | 16 +-
arch/avr32/mm/dma-coherent.c | 7 +
arch/blackfin/kernel/dma-mapping.c | 7 +
arch/c6x/kernel/dma.c | 16 ++
arch/frv/mb93090-mb00/pci-dma-nommu.c | 16 ++
arch/frv/mb93090-mb00/pci-dma.c | 7 +
arch/hexagon/kernel/dma.c | 6 +
arch/m68k/kernel/dma.c | 8 +
arch/metag/kernel/dma.c | 16 ++
arch/microblaze/kernel/dma.c | 10 +
arch/mips/loongson64/common/dma-swiotlb.c | 2
arch/mips/mm/dma-default.c | 8 +
arch/nios2/mm/dma-mapping.c | 14 ++
arch/openrisc/kernel/dma.c | 3
arch/parisc/kernel/pci-dma.c | 20 ++-
arch/powerpc/kernel/dma.c | 9 +
arch/sh/kernel/dma-nommu.c | 7 +
arch/sparc/kernel/iommu.c | 4 -
arch/sparc/kernel/ioport.c | 4 -
arch/tile/kernel/pci-dma.c | 12 +-
arch/xtensa/kernel/pci-dma.c | 7 +
drivers/net/ethernet/intel/igb/igb.h | 36 ++++-
drivers/net/ethernet/intel/igb/igb_main.c | 207 +++++++++++++++++++++++------
drivers/xen/swiotlb-xen.c | 40 +++---
include/linux/dma-mapping.h | 20 ++-
include/linux/gfp.h | 2
include/linux/swiotlb.h | 10 +
lib/swiotlb.c | 56 ++++----
mm/page_alloc.c | 14 ++
30 files changed, 435 insertions(+), 152 deletions(-)
--
Signature
[toc] | [next] | [standalone]
| From | Alexander Duyck <alexander.h.duyck@intel.com> |
|---|---|
| Date | 2016-10-24 20:10 +0200 |
| Subject | [net-next PATCH RFC 26/26] igb: Revert "igb: Revert support for build_skb in igb" |
| Message-ID | <svRR9-2Kt-85@gated-at.bofh.it> |
| In reply to | #1507530 |
This reverts commit f9d40f6a9921 ("igb: Revert support for build_skb in
igb") and adds a few changes to update it to work with the latest version
of igb. We are now able to revert the removal of this due to the fact
that with the recent changes to the page count and the use of
DMA_ATTR_SKIP_CPU_SYNC we can make the pages writable so we should not be
invalidating the additional data added when we call build_skb.
The biggest risk with this change is that we are now not able to support
full jumbo frames when using build_skb. Instead we can only support up to
2K minus the skb overhead and padding offset.
Signed-off-by: Alexander Duyck <alexander.h.duyck@intel.com>
---
drivers/net/ethernet/intel/igb/igb.h | 29 ++++++
drivers/net/ethernet/intel/igb/igb_main.c | 130 ++++++++++++++++++++++++++---
2 files changed, 142 insertions(+), 17 deletions(-)
diff --git a/drivers/net/ethernet/intel/igb/igb.h b/drivers/net/ethernet/intel/igb/igb.h
index acbc3ab..c3420f3 100644
--- a/drivers/net/ethernet/intel/igb/igb.h
+++ b/drivers/net/ethernet/intel/igb/igb.h
@@ -145,6 +145,10 @@ struct vf_data_storage {
#define IGB_RX_HDR_LEN IGB_RXBUFFER_256
#define IGB_RX_BUFSZ IGB_RXBUFFER_2048
+#define IGB_SKB_PAD (NET_SKB_PAD + NET_IP_ALIGN)
+#define IGB_MAX_BUILD_SKB_SIZE \
+ (SKB_WITH_OVERHEAD(IGB_RX_BUFSZ) - (IGB_SKB_PAD + IGB_TS_HDR_LEN))
+
/* How many Rx Buffers do we bundle into one write to the hardware ? */
#define IGB_RX_BUFFER_WRITE 16 /* Must be power of 2 */
@@ -301,12 +305,29 @@ struct igb_q_vector {
};
enum e1000_ring_flags_t {
- IGB_RING_FLAG_RX_SCTP_CSUM,
- IGB_RING_FLAG_RX_LB_VLAN_BSWAP,
- IGB_RING_FLAG_TX_CTX_IDX,
- IGB_RING_FLAG_TX_DETECT_HANG
+ IGB_RING_FLAG_RX_SCTP_CSUM = 0,
+#if (NET_IP_ALIGN != 0)
+ IGB_RING_FLAG_RX_BUILD_SKB_ENABLED = 1,
+#endif
+ IGB_RING_FLAG_RX_LB_VLAN_BSWAP = 2,
+ IGB_RING_FLAG_TX_CTX_IDX = 3,
+ IGB_RING_FLAG_TX_DETECT_HANG = 4,
+#if (NET_IP_ALIGN == 0)
+#if (L1_CACHE_SHIFT < 5)
+ IGB_RING_FLAG_RX_BUILD_SKB_ENABLED = 5,
+#else
+ IGB_RING_FLAG_RX_BUILD_SKB_ENABLED = L1_CACHE_SHIFT,
+#endif
+#endif
};
+#define ring_uses_build_skb(ring) \
+ test_bit(IGB_RING_FLAG_RX_BUILD_SKB_ENABLED, &(ring)->flags)
+#define set_ring_build_skb_enabled(ring) \
+ set_bit(IGB_RING_FLAG_RX_BUILD_SKB_ENABLED, &(ring)->flags)
+#define clear_ring_build_skb_enabled(ring) \
+ clear_bit(IGB_RING_FLAG_RX_BUILD_SKB_ENABLED, &(ring)->flags)
+
#define IGB_TXD_DCMD (E1000_ADVTXD_DCMD_EOP | E1000_ADVTXD_DCMD_RS)
#define IGB_RX_DESC(R, i) \
diff --git a/drivers/net/ethernet/intel/igb/igb_main.c b/drivers/net/ethernet/intel/igb/igb_main.c
index 83fdef6..7674a50 100644
--- a/drivers/net/ethernet/intel/igb/igb_main.c
+++ b/drivers/net/ethernet/intel/igb/igb_main.c
@@ -3761,6 +3761,16 @@ void igb_configure_rx_ring(struct igb_adapter *adapter,
wr32(E1000_RXDCTL(reg_idx), rxdctl);
}
+static void igb_set_rx_buffer_len(struct igb_adapter *adapter,
+ struct igb_ring *rx_ring)
+{
+ /* set build_skb flag */
+ if (adapter->max_frame_size <= IGB_MAX_BUILD_SKB_SIZE)
+ set_ring_build_skb_enabled(rx_ring);
+ else
+ clear_ring_build_skb_enabled(rx_ring);
+}
+
/**
* igb_configure_rx - Configure receive Unit after Reset
* @adapter: board private structure
@@ -3778,8 +3788,12 @@ static void igb_configure_rx(struct igb_adapter *adapter)
/* Setup the HW Rx Head and Tail Descriptor Pointers and
* the Base and Length of the Rx Descriptor Ring
*/
- for (i = 0; i < adapter->num_rx_queues; i++)
- igb_configure_rx_ring(adapter, adapter->rx_ring[i]);
+ for (i = 0; i < adapter->num_rx_queues; i++) {
+ struct igb_ring *rx_ring = adapter->rx_ring[i];
+
+ igb_set_rx_buffer_len(adapter, rx_ring);
+ igb_configure_rx_ring(adapter, rx_ring);
+ }
}
/**
@@ -4238,7 +4252,7 @@ static void igb_set_rx_mode(struct net_device *netdev)
struct igb_adapter *adapter = netdev_priv(netdev);
struct e1000_hw *hw = &adapter->hw;
unsigned int vfn = adapter->vfs_allocated_count;
- u32 rctl = 0, vmolr = 0;
+ u32 rctl = 0, vmolr = 0, rlpml = MAX_JUMBO_FRAME_SIZE;
int count;
/* Check for Promiscuous and All Multicast modes */
@@ -4310,12 +4324,18 @@ static void igb_set_rx_mode(struct net_device *netdev)
vmolr |= rd32(E1000_VMOLR(vfn)) &
~(E1000_VMOLR_ROPE | E1000_VMOLR_MPME | E1000_VMOLR_ROMPE);
- /* enable Rx jumbo frames, no need for restriction */
+ /* enable Rx jumbo frames, restrict as needed to support build_skb */
vmolr &= ~E1000_VMOLR_RLPML_MASK;
- vmolr |= MAX_JUMBO_FRAME_SIZE | E1000_VMOLR_LPE;
+ vmolr |= E1000_VMOLR_LPE;
+ vmolr |= (adapter->max_frame_size <= IGB_MAX_BUILD_SKB_SIZE) ?
+ IGB_MAX_BUILD_SKB_SIZE : MAX_JUMBO_FRAME_SIZE;
+
+ if (!adapter->vfs_allocated_count &&
+ (adapter->max_frame_size <= IGB_MAX_BUILD_SKB_SIZE))
+ rlpml = IGB_MAX_BUILD_SKB_SIZE;
wr32(E1000_VMOLR(vfn), vmolr);
- wr32(E1000_RLPML, MAX_JUMBO_FRAME_SIZE);
+ wr32(E1000_RLPML, rlpml);
igb_restore_vf_multicasts(adapter);
}
@@ -5046,9 +5066,9 @@ static void igb_tx_csum(struct igb_ring *tx_ring, struct igb_tx_buffer *first)
}
#define IGB_SET_FLAG(_input, _flag, _result) \
- ((_flag <= _result) ? \
- ((u32)(_input & _flag) * (_result / _flag)) : \
- ((u32)(_input & _flag) / (_flag / _result)))
+ (((_flag) <= (_result)) ? \
+ ((u32)(_input & (_flag)) * ((_result) / (_flag))) : \
+ ((u32)(_input & (_flag)) / ((_flag) / (_result))))
static u32 igb_tx_cmd_type(struct sk_buff *skb, u32 tx_flags)
{
@@ -6829,7 +6849,7 @@ static inline bool igb_page_is_reserved(struct page *page)
static bool igb_can_reuse_rx_page(struct igb_rx_buffer *rx_buffer,
struct page *page,
- unsigned int truesize)
+ const unsigned int truesize)
{
unsigned int pagecnt_bias = rx_buffer->pagecnt_bias--;
@@ -6888,7 +6908,7 @@ static bool igb_add_rx_frag(struct igb_ring *rx_ring,
struct page *page = rx_buffer->page;
unsigned char *va = page_address(page) + rx_buffer->page_offset;
#if (PAGE_SIZE < 8192)
- unsigned int truesize = IGB_RX_BUFSZ;
+ const unsigned int truesize = IGB_RX_BUFSZ;
#else
unsigned int truesize = SKB_DATA_ALIGN(size);
#endif
@@ -6933,6 +6953,78 @@ static bool igb_add_rx_frag(struct igb_ring *rx_ring,
return igb_can_reuse_rx_page(rx_buffer, page, truesize);
}
+static struct sk_buff *igb_build_rx_buffer(struct igb_ring *rx_ring,
+ union e1000_adv_rx_desc *rx_desc)
+{
+ unsigned int size = le16_to_cpu(rx_desc->wb.upper.length);
+ struct igb_rx_buffer *rx_buffer;
+ struct sk_buff *skb;
+ struct page *page;
+ void *va;
+#if (PAGE_SIZE < 8192)
+ const unsigned int truesize = IGB_RX_BUFSZ;
+#else
+ unsigned int truesize = SKB_DATA_ALIGN(sizeof(struct skb_shared_info)) +
+ SKB_DATA_ALIGN(NET_SKB_PAD +
+ NET_IP_ALIGN +
+ size);
+#endif
+
+ rx_buffer = &rx_ring->rx_buffer_info[rx_ring->next_to_clean];
+ page = rx_buffer->page;
+ prefetchw(page);
+
+ /* we are reusing so sync this buffer for CPU use */
+ dma_sync_single_range_for_cpu(rx_ring->dev,
+ rx_buffer->dma,
+ rx_buffer->page_offset + IGB_SKB_PAD,
+ size,
+ DMA_FROM_DEVICE);
+
+ va = page_address(page) + rx_buffer->page_offset;
+
+ /* prefetch first cache line of first page */
+ prefetch(va + IGB_SKB_PAD);
+#if L1_CACHE_BYTES < 128
+ prefetch(va + L1_CACHE_BYTES + IGB_SKB_PAD);
+#endif
+
+ /* build an skb to around the page buffer */
+ skb = build_skb(va, truesize);
+ if (unlikely(!skb)) {
+ rx_ring->rx_stats.alloc_failed++;
+ return NULL;
+ }
+
+ /* update pointers within the skb to store the data */
+ skb_reserve(skb, IGB_SKB_PAD);
+ __skb_put(skb, size);
+
+ /* pull timestamp out of packet data */
+ if (igb_test_staterr(rx_desc, E1000_RXDADV_STAT_TSIP)) {
+ igb_ptp_rx_pktstamp(rx_ring->q_vector, skb->data, skb);
+ __skb_pull(skb, IGB_TS_HDR_LEN);
+ }
+
+ if (igb_can_reuse_rx_page(rx_buffer, page, truesize)) {
+ /* hand second half of page back to the ring */
+ igb_reuse_rx_page(rx_ring, rx_buffer);
+ } else {
+ /* We are not reusing the buffer so unmap it and free
+ * any references we are holding to it
+ */
+ dma_unmap_page_attrs(rx_ring->dev, rx_buffer->dma,
+ PAGE_SIZE, DMA_FROM_DEVICE,
+ DMA_ATTR_SKIP_CPU_SYNC);
+ __page_frag_drain(page, 0, rx_buffer->pagecnt_bias);
+ }
+
+ /* clear contents of rx_buffer */
+ rx_buffer->page = NULL;
+
+ return skb;
+}
+
static struct sk_buff *igb_fetch_rx_buffer(struct igb_ring *rx_ring,
union e1000_adv_rx_desc *rx_desc,
struct sk_buff *skb)
@@ -7178,7 +7270,10 @@ static int igb_clean_rx_irq(struct igb_q_vector *q_vector, const int budget)
dma_rmb();
/* retrieve a buffer from the ring */
- skb = igb_fetch_rx_buffer(rx_ring, rx_desc, skb);
+ if (ring_uses_build_skb(rx_ring))
+ skb = igb_build_rx_buffer(rx_ring, rx_desc);
+ else
+ skb = igb_fetch_rx_buffer(rx_ring, rx_desc, skb);
/* exit if we failed to retrieve a buffer */
if (!skb)
@@ -7266,6 +7361,13 @@ static bool igb_alloc_mapped_page(struct igb_ring *rx_ring,
return true;
}
+static inline unsigned int igb_rx_offset(struct igb_ring *rx_ring)
+{
+ return IGB_SET_FLAG(rx_ring->flags,
+ 1 << IGB_RING_FLAG_RX_BUILD_SKB_ENABLED,
+ IGB_SKB_PAD);
+}
+
/**
* igb_alloc_rx_buffers - Replace used receive buffers; packet split
* @adapter: address of board private structure
@@ -7297,7 +7399,9 @@ void igb_alloc_rx_buffers(struct igb_ring *rx_ring, u16 cleaned_count)
/* Refresh the desc even if buffer_addrs didn't change
* because each write-back erases this info.
*/
- rx_desc->read.pkt_addr = cpu_to_le64(bi->dma + bi->page_offset);
+ rx_desc->read.pkt_addr = cpu_to_le64(bi->dma +
+ bi->page_offset +
+ igb_rx_offset(rx_ring));
rx_desc++;
bi++;
[toc] | [prev] | [next] | [standalone]
| From | Alexander Duyck <alexander.h.duyck@intel.com> |
|---|---|
| Date | 2016-10-24 20:10 +0200 |
| Subject | [net-next PATCH RFC 07/26] arch/c6x: Add option to skip sync on DMA map and unmap |
| Message-ID | <svRRa-2Kt-107@gated-at.bofh.it> |
| In reply to | #1507530 |
This change allows us to pass DMA_ATTR_SKIP_CPU_SYNC which allows us to
avoid invoking cache line invalidation if the driver will just handle it
later via a sync_for_cpu or sync_for_device call.
Cc: Mark Salter <msalter@redhat.com>
Cc: Aurelien Jacquiot <a-jacquiot@ti.com>
Cc: linux-c6x-dev@linux-c6x.org
Signed-off-by: Alexander Duyck <alexander.h.duyck@intel.com>
---
arch/c6x/kernel/dma.c | 16 +++++++++++-----
1 file changed, 11 insertions(+), 5 deletions(-)
diff --git a/arch/c6x/kernel/dma.c b/arch/c6x/kernel/dma.c
index db4a6a3..d28df74 100644
--- a/arch/c6x/kernel/dma.c
+++ b/arch/c6x/kernel/dma.c
@@ -42,14 +42,17 @@ static dma_addr_t c6x_dma_map_page(struct device *dev, struct page *page,
{
dma_addr_t handle = virt_to_phys(page_address(page) + offset);
- c6x_dma_sync(handle, size, dir);
+ if (!(attrs & DMA_ATTR_SKIP_CPU_SYNC))
+ c6x_dma_sync(handle, size, dir);
+
return handle;
}
static void c6x_dma_unmap_page(struct device *dev, dma_addr_t handle,
size_t size, enum dma_data_direction dir, unsigned long attrs)
{
- c6x_dma_sync(handle, size, dir);
+ if (!(attrs & DMA_ATTR_SKIP_CPU_SYNC))
+ c6x_dma_sync(handle, size, dir);
}
static int c6x_dma_map_sg(struct device *dev, struct scatterlist *sglist,
@@ -60,7 +63,8 @@ static int c6x_dma_map_sg(struct device *dev, struct scatterlist *sglist,
for_each_sg(sglist, sg, nents, i) {
sg->dma_address = sg_phys(sg);
- c6x_dma_sync(sg->dma_address, sg->length, dir);
+ if (!(attrs & DMA_ATTR_SKIP_CPU_SYNC))
+ c6x_dma_sync(sg->dma_address, sg->length, dir);
}
return nents;
@@ -72,8 +76,10 @@ static void c6x_dma_unmap_sg(struct device *dev, struct scatterlist *sglist,
struct scatterlist *sg;
int i;
- for_each_sg(sglist, sg, nents, i)
- c6x_dma_sync(sg_dma_address(sg), sg->length, dir);
+ for_each_sg(sglist, sg, nents, i) {
+ if (!(attrs & DMA_ATTR_SKIP_CPU_SYNC))
+ c6x_dma_sync(sg_dma_address(sg), sg->length, dir);
+ }
}
[toc] | [prev] | [next] | [standalone]
| From | Alexander Duyck <alexander.h.duyck@intel.com> |
|---|---|
| Date | 2016-10-24 20:10 +0200 |
| Subject | [net-next PATCH RFC 10/26] arch/m68k: Add option to skip DMA sync as a part of mapping |
| Message-ID | <svRRa-2Kt-111@gated-at.bofh.it> |
| In reply to | #1507530 |
This change allows us to pass DMA_ATTR_SKIP_CPU_SYNC which allows us to
avoid invoking cache line invalidation if the driver will just handle it
later via a sync_for_cpu or sync_for_device call.
Cc: Geert Uytterhoeven <geert@linux-m68k.org>
Cc: linux-m68k@lists.linux-m68k.org
Signed-off-by: Alexander Duyck <alexander.h.duyck@intel.com>
---
arch/m68k/kernel/dma.c | 8 +++++++-
1 file changed, 7 insertions(+), 1 deletion(-)
diff --git a/arch/m68k/kernel/dma.c b/arch/m68k/kernel/dma.c
index 8cf97cb..0707006 100644
--- a/arch/m68k/kernel/dma.c
+++ b/arch/m68k/kernel/dma.c
@@ -134,7 +134,9 @@ static dma_addr_t m68k_dma_map_page(struct device *dev, struct page *page,
{
dma_addr_t handle = page_to_phys(page) + offset;
- dma_sync_single_for_device(dev, handle, size, dir);
+ if (!(attrs & DMA_ATTR_SKIP_CPU_SYNC))
+ dma_sync_single_for_device(dev, handle, size, dir);
+
return handle;
}
@@ -146,6 +148,10 @@ static int m68k_dma_map_sg(struct device *dev, struct scatterlist *sglist,
for_each_sg(sglist, sg, nents, i) {
sg->dma_address = sg_phys(sg);
+
+ if (attrs & DMA_ATTR_SKIP_CPU_SYNC)
+ continue;
+
dma_sync_single_for_device(dev, sg->dma_address, sg->length,
dir);
}
[toc] | [prev] | [next] | [standalone]
| From | Alexander Duyck <alexander.h.duyck@intel.com> |
|---|---|
| Date | 2016-10-24 20:10 +0200 |
| Subject | [net-next PATCH RFC 13/26] arch/mips: Add option to skip DMA sync as a part of map and unmap |
| Message-ID | <svRR9-2Kt-103@gated-at.bofh.it> |
| In reply to | #1507530 |
This change allows us to pass DMA_ATTR_SKIP_CPU_SYNC which allows us to
avoid invoking cache line invalidation if the driver will just handle it
via a sync_for_cpu or sync_for_device call.
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: Keguang Zhang <keguang.zhang@gmail.com>
Cc: linux-mips@linux-mips.org
Signed-off-by: Alexander Duyck <alexander.h.duyck@intel.com>
---
arch/mips/loongson64/common/dma-swiotlb.c | 2 +-
arch/mips/mm/dma-default.c | 8 +++++---
2 files changed, 6 insertions(+), 4 deletions(-)
diff --git a/arch/mips/loongson64/common/dma-swiotlb.c b/arch/mips/loongson64/common/dma-swiotlb.c
index 1a80b6f..aab4fd6 100644
--- a/arch/mips/loongson64/common/dma-swiotlb.c
+++ b/arch/mips/loongson64/common/dma-swiotlb.c
@@ -61,7 +61,7 @@ static int loongson_dma_map_sg(struct device *dev, struct scatterlist *sg,
int nents, enum dma_data_direction dir,
unsigned long attrs)
{
- int r = swiotlb_map_sg_attrs(dev, sg, nents, dir, 0);
+ int r = swiotlb_map_sg_attrs(dev, sg, nents, dir, attrs);
mb();
return r;
diff --git a/arch/mips/mm/dma-default.c b/arch/mips/mm/dma-default.c
index b2eadd6..dd998d7 100644
--- a/arch/mips/mm/dma-default.c
+++ b/arch/mips/mm/dma-default.c
@@ -293,7 +293,7 @@ static inline void __dma_sync(struct page *page,
static void mips_dma_unmap_page(struct device *dev, dma_addr_t dma_addr,
size_t size, enum dma_data_direction direction, unsigned long attrs)
{
- if (cpu_needs_post_dma_flush(dev))
+ if (cpu_needs_post_dma_flush(dev) && !(attrs & DMA_ATTR_SKIP_CPU_SYNC))
__dma_sync(dma_addr_to_page(dev, dma_addr),
dma_addr & ~PAGE_MASK, size, direction);
plat_post_dma_flush(dev);
@@ -307,7 +307,8 @@ static int mips_dma_map_sg(struct device *dev, struct scatterlist *sglist,
struct scatterlist *sg;
for_each_sg(sglist, sg, nents, i) {
- if (!plat_device_is_coherent(dev))
+ if (!plat_device_is_coherent(dev) &&
+ !(attrs & DMA_ATTR_SKIP_CPU_SYNC))
__dma_sync(sg_page(sg), sg->offset, sg->length,
direction);
#ifdef CONFIG_NEED_SG_DMA_LENGTH
@@ -324,7 +325,7 @@ static dma_addr_t mips_dma_map_page(struct device *dev, struct page *page,
unsigned long offset, size_t size, enum dma_data_direction direction,
unsigned long attrs)
{
- if (!plat_device_is_coherent(dev))
+ if (!plat_device_is_coherent(dev) && !(attrs & DMA_ATTR_SKIP_CPU_SYNC))
__dma_sync(page, offset, size, direction);
return plat_map_dma_mem_page(dev, page) + offset;
@@ -339,6 +340,7 @@ static void mips_dma_unmap_sg(struct device *dev, struct scatterlist *sglist,
for_each_sg(sglist, sg, nhwentries, i) {
if (!plat_device_is_coherent(dev) &&
+ !(attrs & DMA_ATTR_SKIP_CPU_SYNC) &&
direction != DMA_TO_DEVICE)
__dma_sync(sg_page(sg), sg->offset, sg->length,
direction);
[toc] | [prev] | [next] | [standalone]
| From | Alexander Duyck <alexander.h.duyck@intel.com> |
|---|---|
| Date | 2016-10-24 20:10 +0200 |
| Subject | [net-next PATCH RFC 12/26] arch/microblaze: Add option to skip DMA sync as a part of map and unmap |
| Message-ID | <svRRa-2Kt-119@gated-at.bofh.it> |
| In reply to | #1507530 |
This change allows us to pass DMA_ATTR_SKIP_CPU_SYNC which allows us to
avoid invoking cache line invalidation if the driver will just handle it
via a sync_for_cpu or sync_for_device call.
Cc: Michal Simek <monstr@monstr.eu>
Signed-off-by: Alexander Duyck <alexander.h.duyck@intel.com>
---
arch/microblaze/kernel/dma.c | 10 ++++++++--
1 file changed, 8 insertions(+), 2 deletions(-)
diff --git a/arch/microblaze/kernel/dma.c b/arch/microblaze/kernel/dma.c
index ec04dc1..818daf2 100644
--- a/arch/microblaze/kernel/dma.c
+++ b/arch/microblaze/kernel/dma.c
@@ -61,6 +61,10 @@ static int dma_direct_map_sg(struct device *dev, struct scatterlist *sgl,
/* FIXME this part of code is untested */
for_each_sg(sgl, sg, nents, i) {
sg->dma_address = sg_phys(sg);
+
+ if (attrs & DMA_ATTR_SKIP_CPU_SYNC)
+ continue;
+
__dma_sync(page_to_phys(sg_page(sg)) + sg->offset,
sg->length, direction);
}
@@ -80,7 +84,8 @@ static inline dma_addr_t dma_direct_map_page(struct device *dev,
enum dma_data_direction direction,
unsigned long attrs)
{
- __dma_sync(page_to_phys(page) + offset, size, direction);
+ if (!(attrs & DMA_ATTR_SKIP_CPU_SYNC))
+ __dma_sync(page_to_phys(page) + offset, size, direction);
return page_to_phys(page) + offset;
}
@@ -95,7 +100,8 @@ static inline void dma_direct_unmap_page(struct device *dev,
* phys_to_virt is here because in __dma_sync_page is __virt_to_phys and
* dma_address is physical address
*/
- __dma_sync(dma_address, size, direction);
+ if (!(attrs & DMA_ATTR_SKIP_CPU_SYNC))
+ __dma_sync(dma_address, size, direction);
}
static inline void
[toc] | [prev] | [next] | [standalone]
| From | Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> |
|---|---|
| Date | 2016-10-24 20:20 +0200 |
| Subject | Re: [net-next PATCH RFC 01/26] swiotlb: Drop unused function swiotlb_map_sg |
| Message-ID | <svS0N-2QU-1@gated-at.bofh.it> |
| In reply to | #1507530 |
On Mon, Oct 24, 2016 at 08:04:31AM -0400, Alexander Duyck wrote:
> There are no users for swiotlb_map_sg so we might as well just drop it.
>
> Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Acked-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Thought I swear I saw a familiar patch by Christopher Hellwig at some point..
but maybe that patchset had been dropped.
> Signed-off-by: Alexander Duyck <alexander.h.duyck@intel.com>
> ---
> include/linux/swiotlb.h | 4 ----
> lib/swiotlb.c | 8 --------
> 2 files changed, 12 deletions(-)
>
> diff --git a/include/linux/swiotlb.h b/include/linux/swiotlb.h
> index 5f81f8a..e237b6f 100644
> --- a/include/linux/swiotlb.h
> +++ b/include/linux/swiotlb.h
> @@ -72,10 +72,6 @@ extern void swiotlb_unmap_page(struct device *hwdev, dma_addr_t dev_addr,
> size_t size, enum dma_data_direction dir,
> unsigned long attrs);
>
> -extern int
> -swiotlb_map_sg(struct device *hwdev, struct scatterlist *sg, int nents,
> - enum dma_data_direction dir);
> -
> extern void
> swiotlb_unmap_sg(struct device *hwdev, struct scatterlist *sg, int nents,
> enum dma_data_direction dir);
> diff --git a/lib/swiotlb.c b/lib/swiotlb.c
> index 22e13a0..47aad37 100644
> --- a/lib/swiotlb.c
> +++ b/lib/swiotlb.c
> @@ -910,14 +910,6 @@ void swiotlb_unmap_page(struct device *hwdev, dma_addr_t dev_addr,
> }
> EXPORT_SYMBOL(swiotlb_map_sg_attrs);
>
> -int
> -swiotlb_map_sg(struct device *hwdev, struct scatterlist *sgl, int nelems,
> - enum dma_data_direction dir)
> -{
> - return swiotlb_map_sg_attrs(hwdev, sgl, nelems, dir, 0);
> -}
> -EXPORT_SYMBOL(swiotlb_map_sg);
> -
> /*
> * Unmap a set of streaming mode DMA translations. Again, cpu read rules
> * concerning calls here are the same as for swiotlb_unmap_page() above.
>
[toc] | [prev] | [next] | [standalone]
| From | Alexander Duyck <alexander.h.duyck@intel.com> |
|---|---|
| Date | 2016-10-24 20:20 +0200 |
| Subject | [net-next PATCH RFC 02/26] swiotlb: Add support for DMA_ATTR_SKIP_CPU_SYNC |
| Message-ID | <svRR9-2Kt-83@gated-at.bofh.it> |
| In reply to | #1507530 |
As a first step to making DMA_ATTR_SKIP_CPU_SYNC apply to architectures
beyond just ARM I need to make it so that the swiotlb will respect the
flag. In order to do that I also need to update the swiotlb-xen since it
heavily makes use of the functionality.
Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Signed-off-by: Alexander Duyck <alexander.h.duyck@intel.com>
---
drivers/xen/swiotlb-xen.c | 40 ++++++++++++++++++++++----------------
include/linux/swiotlb.h | 6 ++++--
lib/swiotlb.c | 48 +++++++++++++++++++++++++++------------------
3 files changed, 56 insertions(+), 38 deletions(-)
diff --git a/drivers/xen/swiotlb-xen.c b/drivers/xen/swiotlb-xen.c
index 87e6035..cf047d8 100644
--- a/drivers/xen/swiotlb-xen.c
+++ b/drivers/xen/swiotlb-xen.c
@@ -405,7 +405,8 @@ dma_addr_t xen_swiotlb_map_page(struct device *dev, struct page *page,
*/
trace_swiotlb_bounced(dev, dev_addr, size, swiotlb_force);
- map = swiotlb_tbl_map_single(dev, start_dma_addr, phys, size, dir);
+ map = swiotlb_tbl_map_single(dev, start_dma_addr, phys, size, dir,
+ attrs);
if (map == SWIOTLB_MAP_ERROR)
return DMA_ERROR_CODE;
@@ -416,11 +417,13 @@ dma_addr_t xen_swiotlb_map_page(struct device *dev, struct page *page,
/*
* Ensure that the address returned is DMA'ble
*/
- if (!dma_capable(dev, dev_addr, size)) {
- swiotlb_tbl_unmap_single(dev, map, size, dir);
- dev_addr = 0;
- }
- return dev_addr;
+ if (dma_capable(dev, dev_addr, size))
+ return dev_addr;
+
+ swiotlb_tbl_unmap_single(dev, map, size, dir,
+ attrs | DMA_ATTR_SKIP_CPU_SYNC);
+
+ return DMA_ERROR_CODE;
}
EXPORT_SYMBOL_GPL(xen_swiotlb_map_page);
@@ -444,7 +447,7 @@ static void xen_unmap_single(struct device *hwdev, dma_addr_t dev_addr,
/* NOTE: We use dev_addr here, not paddr! */
if (is_xen_swiotlb_buffer(dev_addr)) {
- swiotlb_tbl_unmap_single(hwdev, paddr, size, dir);
+ swiotlb_tbl_unmap_single(hwdev, paddr, size, dir, attrs);
return;
}
@@ -557,16 +560,9 @@ void xen_swiotlb_unmap_page(struct device *hwdev, dma_addr_t dev_addr,
start_dma_addr,
sg_phys(sg),
sg->length,
- dir);
- if (map == SWIOTLB_MAP_ERROR) {
- dev_warn(hwdev, "swiotlb buffer is full\n");
- /* Don't panic here, we expect map_sg users
- to do proper error handling. */
- xen_swiotlb_unmap_sg_attrs(hwdev, sgl, i, dir,
- attrs);
- sg_dma_len(sgl) = 0;
- return 0;
- }
+ dir, attrs);
+ if (map == SWIOTLB_MAP_ERROR)
+ goto map_error;
xen_dma_map_page(hwdev, pfn_to_page(map >> PAGE_SHIFT),
dev_addr,
map & ~PAGE_MASK,
@@ -589,6 +585,16 @@ void xen_swiotlb_unmap_page(struct device *hwdev, dma_addr_t dev_addr,
sg_dma_len(sg) = sg->length;
}
return nelems;
+map_error:
+ dev_warn(hwdev, "swiotlb buffer is full\n");
+ /*
+ * Don't panic here, we expect map_sg users
+ * to do proper error handling.
+ */
+ xen_swiotlb_unmap_sg_attrs(hwdev, sgl, i, dir,
+ attrs | DMA_ATTR_SKIP_CPU_SYNC);
+ sg_dma_len(sgl) = 0;
+ return 0;
}
EXPORT_SYMBOL_GPL(xen_swiotlb_map_sg_attrs);
diff --git a/include/linux/swiotlb.h b/include/linux/swiotlb.h
index e237b6f..4517be9 100644
--- a/include/linux/swiotlb.h
+++ b/include/linux/swiotlb.h
@@ -44,11 +44,13 @@ enum dma_sync_target {
extern phys_addr_t swiotlb_tbl_map_single(struct device *hwdev,
dma_addr_t tbl_dma_addr,
phys_addr_t phys, size_t size,
- enum dma_data_direction dir);
+ enum dma_data_direction dir,
+ unsigned long attrs);
extern void swiotlb_tbl_unmap_single(struct device *hwdev,
phys_addr_t tlb_addr,
- size_t size, enum dma_data_direction dir);
+ size_t size, enum dma_data_direction dir,
+ unsigned long attrs);
extern void swiotlb_tbl_sync_single(struct device *hwdev,
phys_addr_t tlb_addr,
diff --git a/lib/swiotlb.c b/lib/swiotlb.c
index 47aad37..b538d39 100644
--- a/lib/swiotlb.c
+++ b/lib/swiotlb.c
@@ -425,7 +425,8 @@ static void swiotlb_bounce(phys_addr_t orig_addr, phys_addr_t tlb_addr,
phys_addr_t swiotlb_tbl_map_single(struct device *hwdev,
dma_addr_t tbl_dma_addr,
phys_addr_t orig_addr, size_t size,
- enum dma_data_direction dir)
+ enum dma_data_direction dir,
+ unsigned long attrs)
{
unsigned long flags;
phys_addr_t tlb_addr;
@@ -526,7 +527,8 @@ phys_addr_t swiotlb_tbl_map_single(struct device *hwdev,
*/
for (i = 0; i < nslots; i++)
io_tlb_orig_addr[index+i] = orig_addr + (i << IO_TLB_SHIFT);
- if (dir == DMA_TO_DEVICE || dir == DMA_BIDIRECTIONAL)
+ if (!(attrs & DMA_ATTR_SKIP_CPU_SYNC) &&
+ (dir == DMA_TO_DEVICE || dir == DMA_BIDIRECTIONAL))
swiotlb_bounce(orig_addr, tlb_addr, size, DMA_TO_DEVICE);
return tlb_addr;
@@ -539,18 +541,20 @@ phys_addr_t swiotlb_tbl_map_single(struct device *hwdev,
static phys_addr_t
map_single(struct device *hwdev, phys_addr_t phys, size_t size,
- enum dma_data_direction dir)
+ enum dma_data_direction dir, unsigned long attrs)
{
dma_addr_t start_dma_addr = phys_to_dma(hwdev, io_tlb_start);
- return swiotlb_tbl_map_single(hwdev, start_dma_addr, phys, size, dir);
+ return swiotlb_tbl_map_single(hwdev, start_dma_addr, phys, size,
+ dir, attrs);
}
/*
* dma_addr is the kernel virtual address of the bounce buffer to unmap.
*/
void swiotlb_tbl_unmap_single(struct device *hwdev, phys_addr_t tlb_addr,
- size_t size, enum dma_data_direction dir)
+ size_t size, enum dma_data_direction dir,
+ unsigned long attrs)
{
unsigned long flags;
int i, count, nslots = ALIGN(size, 1 << IO_TLB_SHIFT) >> IO_TLB_SHIFT;
@@ -561,6 +565,7 @@ void swiotlb_tbl_unmap_single(struct device *hwdev, phys_addr_t tlb_addr,
* First, sync the memory before unmapping the entry
*/
if (orig_addr != INVALID_PHYS_ADDR &&
+ !(attrs & DMA_ATTR_SKIP_CPU_SYNC) &&
((dir == DMA_FROM_DEVICE) || (dir == DMA_BIDIRECTIONAL)))
swiotlb_bounce(orig_addr, tlb_addr, size, DMA_FROM_DEVICE);
@@ -654,7 +659,8 @@ void swiotlb_tbl_sync_single(struct device *hwdev, phys_addr_t tlb_addr,
* GFP_DMA memory; fall back on map_single(), which
* will grab memory from the lowest available address range.
*/
- phys_addr_t paddr = map_single(hwdev, 0, size, DMA_FROM_DEVICE);
+ phys_addr_t paddr = map_single(hwdev, 0, size,
+ DMA_FROM_DEVICE, 0);
if (paddr == SWIOTLB_MAP_ERROR)
goto err_warn;
@@ -669,7 +675,8 @@ void swiotlb_tbl_sync_single(struct device *hwdev, phys_addr_t tlb_addr,
/* DMA_TO_DEVICE to avoid memcpy in unmap_single */
swiotlb_tbl_unmap_single(hwdev, paddr,
- size, DMA_TO_DEVICE);
+ size, DMA_TO_DEVICE,
+ DMA_ATTR_SKIP_CPU_SYNC);
goto err_warn;
}
}
@@ -699,7 +706,7 @@ void swiotlb_tbl_sync_single(struct device *hwdev, phys_addr_t tlb_addr,
free_pages((unsigned long)vaddr, get_order(size));
else
/* DMA_TO_DEVICE to avoid memcpy in swiotlb_tbl_unmap_single */
- swiotlb_tbl_unmap_single(hwdev, paddr, size, DMA_TO_DEVICE);
+ swiotlb_tbl_unmap_single(hwdev, paddr, size, DMA_TO_DEVICE, 0);
}
EXPORT_SYMBOL(swiotlb_free_coherent);
@@ -755,7 +762,7 @@ dma_addr_t swiotlb_map_page(struct device *dev, struct page *page,
trace_swiotlb_bounced(dev, dev_addr, size, swiotlb_force);
/* Oh well, have to allocate and map a bounce buffer. */
- map = map_single(dev, phys, size, dir);
+ map = map_single(dev, phys, size, dir, attrs);
if (map == SWIOTLB_MAP_ERROR) {
swiotlb_full(dev, size, dir, 1);
return phys_to_dma(dev, io_tlb_overflow_buffer);
@@ -764,12 +771,13 @@ dma_addr_t swiotlb_map_page(struct device *dev, struct page *page,
dev_addr = phys_to_dma(dev, map);
/* Ensure that the address returned is DMA'ble */
- if (!dma_capable(dev, dev_addr, size)) {
- swiotlb_tbl_unmap_single(dev, map, size, dir);
- return phys_to_dma(dev, io_tlb_overflow_buffer);
- }
+ if (dma_capable(dev, dev_addr, size))
+ return dev_addr;
+
+ swiotlb_tbl_unmap_single(dev, map, size, dir,
+ attrs | DMA_ATTR_SKIP_CPU_SYNC);
- return dev_addr;
+ return phys_to_dma(dev, io_tlb_overflow_buffer);
}
EXPORT_SYMBOL_GPL(swiotlb_map_page);
@@ -782,14 +790,15 @@ dma_addr_t swiotlb_map_page(struct device *dev, struct page *page,
* whatever the device wrote there.
*/
static void unmap_single(struct device *hwdev, dma_addr_t dev_addr,
- size_t size, enum dma_data_direction dir)
+ size_t size, enum dma_data_direction dir,
+ unsigned long attrs)
{
phys_addr_t paddr = dma_to_phys(hwdev, dev_addr);
BUG_ON(dir == DMA_NONE);
if (is_swiotlb_buffer(paddr)) {
- swiotlb_tbl_unmap_single(hwdev, paddr, size, dir);
+ swiotlb_tbl_unmap_single(hwdev, paddr, size, dir, attrs);
return;
}
@@ -809,7 +818,7 @@ void swiotlb_unmap_page(struct device *hwdev, dma_addr_t dev_addr,
size_t size, enum dma_data_direction dir,
unsigned long attrs)
{
- unmap_single(hwdev, dev_addr, size, dir);
+ unmap_single(hwdev, dev_addr, size, dir, attrs);
}
EXPORT_SYMBOL_GPL(swiotlb_unmap_page);
@@ -891,7 +900,7 @@ void swiotlb_unmap_page(struct device *hwdev, dma_addr_t dev_addr,
if (swiotlb_force ||
!dma_capable(hwdev, dev_addr, sg->length)) {
phys_addr_t map = map_single(hwdev, sg_phys(sg),
- sg->length, dir);
+ sg->length, dir, attrs);
if (map == SWIOTLB_MAP_ERROR) {
/* Don't panic here, we expect map_sg users
to do proper error handling. */
@@ -925,7 +934,8 @@ void swiotlb_unmap_page(struct device *hwdev, dma_addr_t dev_addr,
BUG_ON(dir == DMA_NONE);
for_each_sg(sgl, sg, nelems, i)
- unmap_single(hwdev, sg->dma_address, sg_dma_len(sg), dir);
+ unmap_single(hwdev, sg->dma_address, sg_dma_len(sg), dir,
+ attrs);
}
EXPORT_SYMBOL(swiotlb_unmap_sg_attrs);
[toc] | [prev] | [next] | [standalone]
| From | Alexander Duyck <alexander.h.duyck@intel.com> |
|---|---|
| Date | 2016-10-24 20:20 +0200 |
| Subject | [net-next PATCH RFC 01/26] swiotlb: Drop unused function swiotlb_map_sg |
| Message-ID | <svS0N-2QU-3@gated-at.bofh.it> |
| In reply to | #1507530 |
There are no users for swiotlb_map_sg so we might as well just drop it.
Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Signed-off-by: Alexander Duyck <alexander.h.duyck@intel.com>
---
include/linux/swiotlb.h | 4 ----
lib/swiotlb.c | 8 --------
2 files changed, 12 deletions(-)
diff --git a/include/linux/swiotlb.h b/include/linux/swiotlb.h
index 5f81f8a..e237b6f 100644
--- a/include/linux/swiotlb.h
+++ b/include/linux/swiotlb.h
@@ -72,10 +72,6 @@ extern void swiotlb_unmap_page(struct device *hwdev, dma_addr_t dev_addr,
size_t size, enum dma_data_direction dir,
unsigned long attrs);
-extern int
-swiotlb_map_sg(struct device *hwdev, struct scatterlist *sg, int nents,
- enum dma_data_direction dir);
-
extern void
swiotlb_unmap_sg(struct device *hwdev, struct scatterlist *sg, int nents,
enum dma_data_direction dir);
diff --git a/lib/swiotlb.c b/lib/swiotlb.c
index 22e13a0..47aad37 100644
--- a/lib/swiotlb.c
+++ b/lib/swiotlb.c
@@ -910,14 +910,6 @@ void swiotlb_unmap_page(struct device *hwdev, dma_addr_t dev_addr,
}
EXPORT_SYMBOL(swiotlb_map_sg_attrs);
-int
-swiotlb_map_sg(struct device *hwdev, struct scatterlist *sgl, int nelems,
- enum dma_data_direction dir)
-{
- return swiotlb_map_sg_attrs(hwdev, sgl, nelems, dir, 0);
-}
-EXPORT_SYMBOL(swiotlb_map_sg);
-
/*
* Unmap a set of streaming mode DMA translations. Again, cpu read rules
* concerning calls here are the same as for swiotlb_unmap_page() above.
[toc] | [prev] | [next] | [standalone]
| From | Alexander Duyck <alexander.h.duyck@intel.com> |
|---|---|
| Date | 2016-10-24 20:20 +0200 |
| Subject | [net-next PATCH RFC 09/26] arch/hexagon: Add option to skip DMA sync as a part of mapping |
| Message-ID | <svS0N-2QU-21@gated-at.bofh.it> |
| In reply to | #1507530 |
This change allows us to pass DMA_ATTR_SKIP_CPU_SYNC which allows us to
avoid invoking cache line invalidation if the driver will just handle it
later via a sync_for_cpu or sync_for_device call.
Cc: Richard Kuo <rkuo@codeaurora.org>
Cc: linux-hexagon@vger.kernel.org
Signed-off-by: Alexander Duyck <alexander.h.duyck@intel.com>
---
arch/hexagon/kernel/dma.c | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/arch/hexagon/kernel/dma.c b/arch/hexagon/kernel/dma.c
index b901778..dbc4f10 100644
--- a/arch/hexagon/kernel/dma.c
+++ b/arch/hexagon/kernel/dma.c
@@ -119,6 +119,9 @@ static int hexagon_map_sg(struct device *hwdev, struct scatterlist *sg,
s->dma_length = s->length;
+ if (attrs & DMA_ATTR_SKIP_CPU_SYNC)
+ continue;
+
flush_dcache_range(dma_addr_to_virt(s->dma_address),
dma_addr_to_virt(s->dma_address + s->length));
}
@@ -180,7 +183,8 @@ static dma_addr_t hexagon_map_page(struct device *dev, struct page *page,
if (!check_addr("map_single", dev, bus, size))
return bad_dma_address;
- dma_sync(dma_addr_to_virt(bus), size, dir);
+ if (!(attrs & DMA_ATTR_SKIP_CPU_SYNC))
+ dma_sync(dma_addr_to_virt(bus), size, dir);
return bus;
}
[toc] | [prev] | [next] | [standalone]
| From | Alexander Duyck <alexander.h.duyck@intel.com> |
|---|---|
| Date | 2016-10-24 20:20 +0200 |
| Subject | [net-next PATCH RFC 08/26] arch/frv: Add option to skip sync on DMA map |
| Message-ID | <svS0N-2QU-23@gated-at.bofh.it> |
| In reply to | #1507530 |
The use of DMA_ATTR_SKIP_CPU_SYNC was not consistent across all of the DMA
APIs in the arch/arm folder. This change is meant to correct that so that
we get consistent behavior.
Signed-off-by: Alexander Duyck <alexander.h.duyck@intel.com>
---
arch/frv/mb93090-mb00/pci-dma-nommu.c | 16 +++++++++++-----
arch/frv/mb93090-mb00/pci-dma.c | 7 ++++++-
2 files changed, 17 insertions(+), 6 deletions(-)
diff --git a/arch/frv/mb93090-mb00/pci-dma-nommu.c b/arch/frv/mb93090-mb00/pci-dma-nommu.c
index 90f2e4c..ff606d1 100644
--- a/arch/frv/mb93090-mb00/pci-dma-nommu.c
+++ b/arch/frv/mb93090-mb00/pci-dma-nommu.c
@@ -109,16 +109,19 @@ static int frv_dma_map_sg(struct device *dev, struct scatterlist *sglist,
int nents, enum dma_data_direction direction,
unsigned long attrs)
{
- int i;
struct scatterlist *sg;
+ int i;
+
+ WARN_ON(direction == DMA_NONE);
+
+ if (attrs & DMA_ATTR_SKIP_CPU_SYNC)
+ return nents;
for_each_sg(sglist, sg, nents, i) {
frv_cache_wback_inv(sg_dma_address(sg),
sg_dma_address(sg) + sg_dma_len(sg));
}
- BUG_ON(direction == DMA_NONE);
-
return nents;
}
@@ -126,8 +129,11 @@ static dma_addr_t frv_dma_map_page(struct device *dev, struct page *page,
unsigned long offset, size_t size,
enum dma_data_direction direction, unsigned long attrs)
{
- BUG_ON(direction == DMA_NONE);
- flush_dcache_page(page);
+ WARN_ON(direction == DMA_NONE);
+
+ if (!(attrs & DMA_ATTR_SKIP_CPU_SYNC))
+ flush_dcache_page(page);
+
return (dma_addr_t) page_to_phys(page) + offset;
}
diff --git a/arch/frv/mb93090-mb00/pci-dma.c b/arch/frv/mb93090-mb00/pci-dma.c
index f585745..ee5dadf 100644
--- a/arch/frv/mb93090-mb00/pci-dma.c
+++ b/arch/frv/mb93090-mb00/pci-dma.c
@@ -52,6 +52,9 @@ static int frv_dma_map_sg(struct device *dev, struct scatterlist *sglist,
for_each_sg(sglist, sg, nents, i) {
vaddr = kmap_atomic_primary(sg_page(sg));
+ if (attrs & DMA_ATTR_SKIP_CPU_SYNC)
+ continue;
+
frv_dcache_writeback((unsigned long) vaddr,
(unsigned long) vaddr + PAGE_SIZE);
@@ -70,7 +73,9 @@ static dma_addr_t frv_dma_map_page(struct device *dev, struct page *page,
unsigned long offset, size_t size,
enum dma_data_direction direction, unsigned long attrs)
{
- flush_dcache_page(page);
+ if (!(attr & DMA_ATTR_SKIP_CPU_SYNC))
+ flush_dcache_page(page);
+
return (dma_addr_t) page_to_phys(page) + offset;
}
[toc] | [prev] | [next] | [standalone]
| From | Alexander Duyck <alexander.h.duyck@intel.com> |
|---|---|
| Date | 2016-10-24 20:20 +0200 |
| Subject | [net-next PATCH RFC 15/26] arch/openrisc: Add option to skip DMA sync as a part of mapping |
| Message-ID | <svS0O-2QU-37@gated-at.bofh.it> |
| In reply to | #1507530 |
This change allows us to pass DMA_ATTR_SKIP_CPU_SYNC which allows us to
avoid invoking cache line invalidation if the driver will just handle it
via a sync_for_cpu or sync_for_device call.
Cc: Jonas Bonn <jonas@southpole.se>
Signed-off-by: Alexander Duyck <alexander.h.duyck@intel.com>
---
arch/openrisc/kernel/dma.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/arch/openrisc/kernel/dma.c b/arch/openrisc/kernel/dma.c
index 140c991..906998b 100644
--- a/arch/openrisc/kernel/dma.c
+++ b/arch/openrisc/kernel/dma.c
@@ -141,6 +141,9 @@
unsigned long cl;
dma_addr_t addr = page_to_phys(page) + offset;
+ if (attrs & DMA_ATTR_SKIP_CPU_SYNC)
+ return addr;
+
switch (dir) {
case DMA_TO_DEVICE:
/* Flush the dcache for the requested range */
[toc] | [prev] | [next] | [standalone]
| From | Alexander Duyck <alexander.h.duyck@intel.com> |
|---|---|
| Date | 2016-10-24 20:20 +0200 |
| Subject | [net-next PATCH RFC 03/26] arch/arc: Add option to skip sync on DMA mapping |
| Message-ID | <svS0O-2QU-41@gated-at.bofh.it> |
| In reply to | #1507530 |
This change allows us to pass DMA_ATTR_SKIP_CPU_SYNC which allows us to
avoid invoking cache line invalidation if the driver will just handle it
later via a sync_for_cpu or sync_for_device call.
Cc: Vineet Gupta <vgupta@synopsys.com>
Cc: linux-snps-arc@lists.infradead.org
Signed-off-by: Alexander Duyck <alexander.h.duyck@intel.com>
---
arch/arc/mm/dma.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/arch/arc/mm/dma.c b/arch/arc/mm/dma.c
index 20afc65..d0c4b28 100644
--- a/arch/arc/mm/dma.c
+++ b/arch/arc/mm/dma.c
@@ -133,7 +133,8 @@ static dma_addr_t arc_dma_map_page(struct device *dev, struct page *page,
unsigned long attrs)
{
phys_addr_t paddr = page_to_phys(page) + offset;
- _dma_cache_sync(paddr, size, dir);
+ if (!(attrs & DMA_ATTR_SKIP_CPU_SYNC))
+ _dma_cache_sync(paddr, size, dir);
return plat_phys_to_dma(dev, paddr);
}
[toc] | [prev] | [standalone]
Back to top | Article view | linux.kernel
csiph-web