Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > linux.kernel > #1722168 > unrolled thread
| Started by | Sukadev Bhattiprolu <sukadev@linux.vnet.ibm.com> |
|---|---|
| First post | 2017-08-29 08:30 +0200 |
| Last post | 2017-09-07 09:10 +0200 |
| Articles | 3 — 2 participants |
Back to article view | Back to linux.kernel
[PATCH v8 00/10] Enable VAS Sukadev Bhattiprolu <sukadev@linux.vnet.ibm.com> - 2017-08-29 08:30 +0200
[PATCH v8 05/10] powerpc/vas: Define helpers to init window context Sukadev Bhattiprolu <sukadev@linux.vnet.ibm.com> - 2017-08-29 08:30 +0200
Re: [PATCH v8 00/10] Enable VAS Michael Neuling <mikey@neuling.org> - 2017-09-07 09:10 +0200
| From | Sukadev Bhattiprolu <sukadev@linux.vnet.ibm.com> |
|---|---|
| Date | 2017-08-29 08:30 +0200 |
| Subject | [PATCH v8 00/10] Enable VAS |
| Message-ID | <ujIc9-7vX-3@gated-at.bofh.it> |
Power9 introduces a hardware subsystem referred to as the Virtual
Accelerator Switchboard (VAS). VAS allows kernel subsystems and user
space processes to directly access the Nest Accelerator (NX) engines
which implement compression and encryption algorithms in the hardware.
NX has been in Power processors since Power7+, but access to the NX
engines was through the 'icswx' instruction which is only available
to the kernel/hypervisor. Starting with Power9, access to the NX
engines is provided to both kernel and user space processes through
VAS.
The switchboard (i.e VAS) multiplexes accesses between "receivers" and
"senders", where the "receivers" are typically the NX engines and
"senders" are the kernel subsystems and user processors that wish to
access the receivers (NX engines). Once a sender is "connected" to
a receiver through the switchboard, the senders can submit compression/
encryption requests to the hardware using the new (PowerISA 3.0)
"copy" and "paste" instructions.
In the initial OPAL and PowerNV kernel patchsets, the "senders" can
only be kernel subsystems (eg NX-842 driver) and receivers can only
be the NX-842 engine. Follow-on patch sets will allow senders/receivers
to be user-space processes and receivers to be NX-GZIP engines.
Provides:
This kernel patch set configures the VAS subsystems and provides
kernel interfaces to drivers like NX-842 to open receive and send
windows in VAS and to submit compression requests to the NX engine.
Requires:
This patch set needs corresponding VAS/NX skiboot patches which
were merged into skiboot tree. i.e skiboot must include:
commit b503dcf ("vas: Set mmio enable bits in DD2")
Tests:
In-kernel compression requests were tested on DD1 and DD2 POWER9
hardware using compression self-test module and the following
NX-842 patch set from Haren Myneni:
https://lists.ozlabs.org/pipermail/linuxppc-dev/2017-July/160620.html
and by dropping the last parameters to both vas_copy_crb() and
vas_paste_crb() calls in drivers/crypto/nx/nx-842-powernv.c.
See also PATCH 10/10.
Git Tree:
https://github.com/sukadev/linux/
Branch: vas-kern-v8
Thanks to input from Ben Herrenschmidt, Michael Neuling, Michael Ellerman
and Haren Myneni.
Changelog[v8]:
- [Michael Ellerman] Use kernel int types (u64, u32 etc); make VAS
a built-in rather than a module; drop unnecessary fields from
struct vas_instance; Update ISA references; use 0 or 1 with
SET_FIELD macros instead of bool; skip writing to SPARE registers;
minor cleanup of debug/error messages; retry if ida_get_new()
fails with EAGAIN; fix couple of leaks in ids in error handling;
drop vas_initialized() check; drop vas_win_id() and vas_paste_addr()
interfaces as they are not yet used; Set task_state() and fix
parameter to schedule_timeout(); Reuse existing copy/paste macros
drop unnecessary parameters and add cr0 to clobbers list
Changelog[v7]:
- Drop support for user space send/receive FTW windows (will be
posted separately) Simplifies the rx-win-open interface a bit.
- [Michael Ellerman] Move GET_FIELD/SET_FIELD macros from
uapi/asm/vas.h to asm/vas.h.
Changelog[v6]
- Add support for user space send/receive FTW windows
- Add a new, NX-FTW driver which provides the FTW user interface
Changelog[v5]
- [Ben Herrenschmidt] Make VAS a platform device in the device tree
and use the core platform functions to parse the VAS properties.
Map the VAS MMIO regions as non-cachable and paste regions as
cachable. Use CONFIG_PPC_VAS rather than CONFIG_VAS; Don't assume
VAS ids are sequential.
- Copy the FIFO address as is into LFIFO_BAR (don't shift it).
Changelog[v4]
Comments from Michael Neuling:
- Move VAS code from drivers/misc/vas to arch/powerpc/platforms/powernv
since VAS only provides interfaces to other drivers like NX-842.
- Drop vas-internal.h and use vas.h in separate dirs for VAS
internal, kernel API and user API
- Rather than create 6 separate device tree properties windows
and window context, combine them into 6 "reg" properties.
- Drop vas_window_reset() since windows are reset/cleared before
being assigned to kernel/users.
- Use ilog2() and radix_enabled() helpers
Changelog[v3]
- Rebase to v4.11-rc1
- Add interfaces to initialize send/receive window attributes to
defaults that drivers can use (see arch/powerpc/include/asm/vas.h)
- Modify interface vas_paste() to return 0 or error code
- Fix a bug in setting Translation Control Mode (0b11 not 0x11)
- Enable send-window-credit checking
- Reorg code in vas_win_close()
- Minor reorgs and tweaks to register field settings to make it
easier to add support for user space windows.
- Skip writing to read-only registers
- Start window indexing from 0 rather than 1
Changelog[v2]
- Use vas-id, HVWC, UWC and paste address, entries from device tree
rather than defining/computing them in kernel and reorg code.
Sukadev Bhattiprolu (10):
powerpc/vas: Define macros, register fields and structures
Move GET_FIELD/SET_FIELD to vas.h
powerpc/vas: Define vas_init() and vas_exit()
powerpc/vas: Define helpers to access MMIO regions
powerpc/vas: Define helpers to init window context
powerpc/vas: Define helpers to alloc/free windows
powerpc/vas: Define vas_rx_win_open() interface
powerpc/vas: Define vas_win_close() interface
powerpc/vas: Define vas_tx_win_open()
powerpc/vas: Define copy/paste interfaces
.../devicetree/bindings/powerpc/ibm,vas.txt | 23 +
MAINTAINERS | 9 +
arch/powerpc/include/asm/ppc-opcode.h | 2 +
arch/powerpc/include/asm/vas.h | 160 +++
arch/powerpc/platforms/powernv/Kconfig | 14 +
arch/powerpc/platforms/powernv/Makefile | 1 +
arch/powerpc/platforms/powernv/copy-paste.h | 46 +
arch/powerpc/platforms/powernv/vas-window.c | 1134 ++++++++++++++++++++
arch/powerpc/platforms/powernv/vas.c | 151 +++
arch/powerpc/platforms/powernv/vas.h | 467 ++++++++
drivers/crypto/nx/nx-842-powernv.c | 7 +-
drivers/crypto/nx/nx-842.h | 5 -
12 files changed, 2011 insertions(+), 8 deletions(-)
create mode 100644 Documentation/devicetree/bindings/powerpc/ibm,vas.txt
create mode 100644 arch/powerpc/include/asm/vas.h
create mode 100644 arch/powerpc/platforms/powernv/copy-paste.h
create mode 100644 arch/powerpc/platforms/powernv/vas-window.c
create mode 100644 arch/powerpc/platforms/powernv/vas.c
create mode 100644 arch/powerpc/platforms/powernv/vas.h
--
2.7.4
[toc] | [next] | [standalone]
| From | Sukadev Bhattiprolu <sukadev@linux.vnet.ibm.com> |
|---|---|
| Date | 2017-08-29 08:30 +0200 |
| Subject | [PATCH v8 05/10] powerpc/vas: Define helpers to init window context |
| Message-ID | <ujIca-7vX-29@gated-at.bofh.it> |
| In reply to | #1722168 |
Define helpers to initialize window context registers of the VAS
hardware. These will be used in follow-on patches when opening/closing
VAS windows.
Signed-off-by: Sukadev Bhattiprolu <sukadev@linux.vnet.ibm.com>
---
Changelog[v8]:
- Update comments (ISA references and some cleanup)
- Use 0 or 1 when setting boolean fields with SET_FIELD()
- Don't write to spare/unused registers.
- Use kernel integer types (u64/u32/s32)
Changelog[v6]
- Add support for FTW windows and drop the fault window id
code since it is not needed for FTW/kernel windows.
Changelog[v5]
- Fix: Copy the FIFO address into LFIFO_BAR register as is (don't
shift address into bits 8:53).
Changelog[v4]
- Michael Neuling] Use ilog2(), radix_enabled() helpers;
drop warning when 32-bit app uses VAS (a follow-on patch
will check and return error). Set MSR_PR state to 0 for
kernel (rather than reading from MSR).
Changelog[v3]
- Have caller, rather than init_xlate_regs() reset window regs
so we don't reset any settings caller may already have set.
- Translation mode should be 0x3 (0b11) not 0x11.
- Skip initilaizing read-only registers NX_UTIL and NX_UTIL_SE
- Skip initializing adder registers from UWC - they are already
initialized from the HVWC.
- Check winctx->user_win when setting translation registers
---
arch/powerpc/platforms/powernv/vas-window.c | 299 ++++++++++++++++++++++++++++
arch/powerpc/platforms/powernv/vas.h | 55 +++++
2 files changed, 354 insertions(+)
diff --git a/arch/powerpc/platforms/powernv/vas-window.c b/arch/powerpc/platforms/powernv/vas-window.c
index 642814a2..68dfe53 100644
--- a/arch/powerpc/platforms/powernv/vas-window.c
+++ b/arch/powerpc/platforms/powernv/vas-window.c
@@ -13,6 +13,7 @@
#include <linux/mutex.h>
#include <linux/slab.h>
#include <linux/io.h>
+#include <linux/log2.h>
#include "vas.h"
@@ -186,6 +187,304 @@ int map_winctx_mmio_bars(struct vas_window *window)
return 0;
}
+/*
+ * Reset all valid registers in the HV and OS/User Window Contexts for
+ * the window identified by @window.
+ *
+ * NOTE: We cannot really use a for loop to reset window context. Not all
+ * offsets in a window context are valid registers and the valid
+ * registers are not sequential. And, we can only write to offsets
+ * with valid registers.
+ */
+void reset_window_regs(struct vas_window *window)
+{
+ write_hvwc_reg(window, VREG(LPID), 0ULL);
+ write_hvwc_reg(window, VREG(PID), 0ULL);
+ write_hvwc_reg(window, VREG(XLATE_MSR), 0ULL);
+ write_hvwc_reg(window, VREG(XLATE_LPCR), 0ULL);
+ write_hvwc_reg(window, VREG(XLATE_CTL), 0ULL);
+ write_hvwc_reg(window, VREG(AMR), 0ULL);
+ write_hvwc_reg(window, VREG(SEIDR), 0ULL);
+ write_hvwc_reg(window, VREG(FAULT_TX_WIN), 0ULL);
+ write_hvwc_reg(window, VREG(OSU_INTR_SRC_RA), 0ULL);
+ write_hvwc_reg(window, VREG(HV_INTR_SRC_RA), 0ULL);
+ write_hvwc_reg(window, VREG(PSWID), 0ULL);
+ write_hvwc_reg(window, VREG(LFIFO_BAR), 0ULL);
+ write_hvwc_reg(window, VREG(LDATA_STAMP_CTL), 0ULL);
+ write_hvwc_reg(window, VREG(LDMA_CACHE_CTL), 0ULL);
+ write_hvwc_reg(window, VREG(LRFIFO_PUSH), 0ULL);
+ write_hvwc_reg(window, VREG(CURR_MSG_COUNT), 0ULL);
+ write_hvwc_reg(window, VREG(LNOTIFY_AFTER_COUNT), 0ULL);
+ write_hvwc_reg(window, VREG(LRX_WCRED), 0ULL);
+ write_hvwc_reg(window, VREG(LRX_WCRED_ADDER), 0ULL);
+ write_hvwc_reg(window, VREG(TX_WCRED), 0ULL);
+ write_hvwc_reg(window, VREG(TX_WCRED_ADDER), 0ULL);
+ write_hvwc_reg(window, VREG(LFIFO_SIZE), 0ULL);
+ write_hvwc_reg(window, VREG(WINCTL), 0ULL);
+ write_hvwc_reg(window, VREG(WIN_STATUS), 0ULL);
+ write_hvwc_reg(window, VREG(WIN_CTX_CACHING_CTL), 0ULL);
+ write_hvwc_reg(window, VREG(TX_RSVD_BUF_COUNT), 0ULL);
+ write_hvwc_reg(window, VREG(LRFIFO_WIN_PTR), 0ULL);
+ write_hvwc_reg(window, VREG(LNOTIFY_CTL), 0ULL);
+ write_hvwc_reg(window, VREG(LNOTIFY_PID), 0ULL);
+ write_hvwc_reg(window, VREG(LNOTIFY_LPID), 0ULL);
+ write_hvwc_reg(window, VREG(LNOTIFY_TID), 0ULL);
+ write_hvwc_reg(window, VREG(LNOTIFY_SCOPE), 0ULL);
+ write_hvwc_reg(window, VREG(NX_UTIL_ADDER), 0ULL);
+
+ /* Skip read-only registers: NX_UTIL and NX_UTIL_SE */
+
+ /*
+ * The send and receive window credit adder registers are also
+ * accessible from HVWC and have been initialized above. We don't
+ * need to initialize from the OS/User Window Context, so skip
+ * following calls:
+ *
+ * write_uwc_reg(window, VREG(TX_WCRED_ADDER), 0ULL);
+ * write_uwc_reg(window, VREG(LRX_WCRED_ADDER), 0ULL);
+ */
+}
+
+/*
+ * Initialize window context registers related to Address Translation.
+ * These registers are common to send/receive windows although they
+ * differ for user/kernel windows. As we resolve the TODOs we may
+ * want to add fields to vas_winctx and move the initialization to
+ * init_vas_winctx_regs().
+ */
+static void init_xlate_regs(struct vas_window *window, bool user_win)
+{
+ u64 lpcr, val;
+
+ /*
+ * MSR_TA, MSR_US are false for both kernel and user.
+ * MSR_DR and MSR_PR are false for kernel.
+ */
+ val = 0ULL;
+ val = SET_FIELD(VAS_XLATE_MSR_HV, val, 1);
+ val = SET_FIELD(VAS_XLATE_MSR_SF, val, 1);
+ if (user_win) {
+ val = SET_FIELD(VAS_XLATE_MSR_DR, val, 1);
+ val = SET_FIELD(VAS_XLATE_MSR_PR, val, 1);
+ }
+ write_hvwc_reg(window, VREG(XLATE_MSR), val);
+
+ lpcr = mfspr(SPRN_LPCR);
+ val = 0ULL;
+ /*
+ * NOTE: From Section 5.7.8.1 Segment Lookaside Buffer of the
+ * Power ISA, v3.0B, Page size encoding is 0 = 4KB, 5 = 64KB.
+ *
+ * NOTE: From Section 1.3.1, Address Translation Context of the
+ * Nest MMU Workbook, LPCR_SC should be 0 for Power9.
+ */
+ val = SET_FIELD(VAS_XLATE_LPCR_PAGE_SIZE, val, 5);
+ val = SET_FIELD(VAS_XLATE_LPCR_ISL, val, lpcr & LPCR_ISL);
+ val = SET_FIELD(VAS_XLATE_LPCR_TC, val, lpcr & LPCR_TC);
+ val = SET_FIELD(VAS_XLATE_LPCR_SC, val, 0);
+ write_hvwc_reg(window, VREG(XLATE_LPCR), val);
+
+ /*
+ * Section 1.3.1 (Address translation Context) of NMMU workbook.
+ * 0b00 Hashed Page Table mode
+ * 0b01 Reserved
+ * 0b10 Radix on HPT
+ * 0b11 Radix on Radix
+ */
+ val = 0ULL;
+ val = SET_FIELD(VAS_XLATE_MODE, val, radix_enabled() ? 3 : 2);
+ write_hvwc_reg(window, VREG(XLATE_CTL), val);
+
+ /*
+ * TODO: Can we mfspr(AMR) even for user windows?
+ */
+ val = 0ULL;
+ val = SET_FIELD(VAS_AMR, val, mfspr(SPRN_AMR));
+ write_hvwc_reg(window, VREG(AMR), val);
+
+ val = 0ULL;
+ val = SET_FIELD(VAS_SEIDR, val, 0);
+ write_hvwc_reg(window, VREG(SEIDR), val);
+}
+
+/*
+ * Initialize Reserved Send Buffer Count for the send window. It involves
+ * writing to the register, reading it back to confirm that the hardware
+ * has enough buffers to reserve. See section 1.3.1.2.1 of VAS workbook.
+ *
+ * Since we can only make a best-effort attempt to fulfill the request,
+ * we don't return any errors if we cannot.
+ *
+ * TODO: Reserved (aka dedicated) send buffers are not supported yet.
+ */
+static void init_rsvd_tx_buf_count(struct vas_window *txwin,
+ struct vas_winctx *winctx)
+{
+ write_hvwc_reg(txwin, VREG(TX_RSVD_BUF_COUNT), 0ULL);
+}
+
+/*
+ * init_winctx_regs()
+ * Initialize window context registers for a receive window.
+ * Except for caching control and marking window open, the registers
+ * are initialized in the order listed in Section 3.1.4 (Window Context
+ * Cache Register Details) of the VAS workbook although they don't need
+ * to be.
+ *
+ * Design note: For NX receive windows, NX allocates the FIFO buffer in OPAL
+ * (so that it can get a large contiguous area) and passes that buffer
+ * to kernel via device tree. We now write that buffer address to the
+ * FIFO BAR. Would it make sense to do this all in OPAL? i.e have OPAL
+ * write the per-chip RX FIFO addresses to the windows during boot-up
+ * as a one-time task? That could work for NX but what about other
+ * receivers? Let the receivers tell us the rx-fifo buffers for now.
+ */
+int init_winctx_regs(struct vas_window *window, struct vas_winctx *winctx)
+{
+ u64 val;
+ int fifo_size;
+
+ reset_window_regs(window);
+
+ val = 0ULL;
+ val = SET_FIELD(VAS_LPID, val, winctx->lpid);
+ write_hvwc_reg(window, VREG(LPID), val);
+
+ val = 0ULL;
+ val = SET_FIELD(VAS_PID_ID, val, winctx->pidr);
+ write_hvwc_reg(window, VREG(PID), val);
+
+ init_xlate_regs(window, winctx->user_win);
+
+ val = 0ULL;
+ val = SET_FIELD(VAS_FAULT_TX_WIN, val, 0);
+ write_hvwc_reg(window, VREG(FAULT_TX_WIN), val);
+
+ /* In PowerNV, interrupts go to HV. */
+ write_hvwc_reg(window, VREG(OSU_INTR_SRC_RA), 0ULL);
+
+ val = 0ULL;
+ val = SET_FIELD(VAS_HV_INTR_SRC_RA, val, winctx->irq_port);
+ write_hvwc_reg(window, VREG(HV_INTR_SRC_RA), val);
+
+ val = 0ULL;
+ val = SET_FIELD(VAS_PSWID_EA_HANDLE, val, winctx->pswid);
+ write_hvwc_reg(window, VREG(PSWID), val);
+
+ write_hvwc_reg(window, VREG(SPARE1), 0ULL);
+ write_hvwc_reg(window, VREG(SPARE2), 0ULL);
+ write_hvwc_reg(window, VREG(SPARE3), 0ULL);
+
+ /*
+ * NOTE: VAS expects the FIFO address to be copied into the LFIFO_BAR
+ * register as is - do NOT shift the address into VAS_LFIFO_BAR
+ * bit fields! Ok to set the page migration select fields -
+ * VAS ignores the lower 10+ bits in the address anyway, because
+ * the minimum FIFO size is 1K?
+ *
+ * See also: Design note in function header.
+ */
+ val = __pa(winctx->rx_fifo);
+ val = SET_FIELD(VAS_PAGE_MIGRATION_SELECT, val, 0);
+ write_hvwc_reg(window, VREG(LFIFO_BAR), val);
+
+ val = 0ULL;
+ val = SET_FIELD(VAS_LDATA_STAMP, val, winctx->data_stamp);
+ write_hvwc_reg(window, VREG(LDATA_STAMP_CTL), val);
+
+ val = 0ULL;
+ val = SET_FIELD(VAS_LDMA_TYPE, val, winctx->dma_type);
+ val = SET_FIELD(VAS_LDMA_FIFO_DISABLE, val, winctx->fifo_disable);
+ write_hvwc_reg(window, VREG(LDMA_CACHE_CTL), val);
+
+ write_hvwc_reg(window, VREG(LRFIFO_PUSH), 0ULL);
+ write_hvwc_reg(window, VREG(CURR_MSG_COUNT), 0ULL);
+ write_hvwc_reg(window, VREG(LNOTIFY_AFTER_COUNT), 0ULL);
+
+ val = 0ULL;
+ val = SET_FIELD(VAS_LRX_WCRED, val, winctx->wcreds_max);
+ write_hvwc_reg(window, VREG(LRX_WCRED), val);
+
+ val = 0ULL;
+ val = SET_FIELD(VAS_TX_WCRED, val, winctx->wcreds_max);
+ write_hvwc_reg(window, VREG(TX_WCRED), val);
+
+ write_hvwc_reg(window, VREG(LRX_WCRED_ADDER), 0ULL);
+ write_hvwc_reg(window, VREG(TX_WCRED_ADDER), 0ULL);
+
+ fifo_size = winctx->rx_fifo_size / 1024;
+
+ val = 0ULL;
+ val = SET_FIELD(VAS_LFIFO_SIZE, val, ilog2(fifo_size));
+ write_hvwc_reg(window, VREG(LFIFO_SIZE), val);
+
+ /* Update window control and caching control registers last so
+ * we mark the window open only after fully initializing it and
+ * pushing context to cache.
+ */
+
+ write_hvwc_reg(window, VREG(WIN_STATUS), 0ULL);
+
+ init_rsvd_tx_buf_count(window, winctx);
+
+ /* for a send window, point to the matching receive window */
+ val = 0ULL;
+ val = SET_FIELD(VAS_LRX_WIN_ID, val, winctx->rx_win_id);
+ write_hvwc_reg(window, VREG(LRFIFO_WIN_PTR), val);
+
+ write_hvwc_reg(window, VREG(SPARE4), 0ULL);
+
+ val = 0ULL;
+ val = SET_FIELD(VAS_NOTIFY_DISABLE, val, winctx->notify_disable);
+ val = SET_FIELD(VAS_INTR_DISABLE, val, winctx->intr_disable);
+ val = SET_FIELD(VAS_NOTIFY_EARLY, val, winctx->notify_early);
+ val = SET_FIELD(VAS_NOTIFY_OSU_INTR, val, winctx->notify_os_intr_reg);
+ write_hvwc_reg(window, VREG(LNOTIFY_CTL), val);
+
+ val = 0ULL;
+ val = SET_FIELD(VAS_LNOTIFY_PID, val, winctx->lnotify_pid);
+ write_hvwc_reg(window, VREG(LNOTIFY_PID), val);
+
+ val = 0ULL;
+ val = SET_FIELD(VAS_LNOTIFY_LPID, val, winctx->lnotify_lpid);
+ write_hvwc_reg(window, VREG(LNOTIFY_LPID), val);
+
+ val = 0ULL;
+ val = SET_FIELD(VAS_LNOTIFY_TID, val, winctx->lnotify_tid);
+ write_hvwc_reg(window, VREG(LNOTIFY_TID), val);
+
+ val = 0ULL;
+ val = SET_FIELD(VAS_LNOTIFY_MIN_SCOPE, val, winctx->min_scope);
+ val = SET_FIELD(VAS_LNOTIFY_MAX_SCOPE, val, winctx->max_scope);
+ write_hvwc_reg(window, VREG(LNOTIFY_SCOPE), val);
+
+ /* Skip read-only registers NX_UTIL and NX_UTIL_SE */
+
+ write_hvwc_reg(window, VREG(SPARE5), 0ULL);
+ write_hvwc_reg(window, VREG(NX_UTIL_ADDER), 0ULL);
+ write_hvwc_reg(window, VREG(SPARE6), 0ULL);
+
+ /* Finally, push window context to memory and... */
+ val = 0ULL;
+ val = SET_FIELD(VAS_PUSH_TO_MEM, val, 1);
+ write_hvwc_reg(window, VREG(WIN_CTX_CACHING_CTL), val);
+
+ /* ... mark the window open for business */
+ val = 0ULL;
+ val = SET_FIELD(VAS_WINCTL_REJ_NO_CREDIT, val, winctx->rej_no_credit);
+ val = SET_FIELD(VAS_WINCTL_PIN, val, winctx->pin_win);
+ val = SET_FIELD(VAS_WINCTL_TX_WCRED_MODE, val, winctx->tx_wcred_mode);
+ val = SET_FIELD(VAS_WINCTL_RX_WCRED_MODE, val, winctx->rx_wcred_mode);
+ val = SET_FIELD(VAS_WINCTL_TX_WORD_MODE, val, winctx->tx_word_mode);
+ val = SET_FIELD(VAS_WINCTL_RX_WORD_MODE, val, winctx->rx_word_mode);
+ val = SET_FIELD(VAS_WINCTL_FAULT_WIN, val, winctx->fault_win);
+ val = SET_FIELD(VAS_WINCTL_NX_WIN, val, winctx->nx_win);
+ val = SET_FIELD(VAS_WINCTL_OPEN, val, 1);
+ write_hvwc_reg(window, VREG(WINCTL), val);
+
+ return 0;
+}
+
/* stub for now */
int vas_win_close(struct vas_window *window)
{
diff --git a/arch/powerpc/platforms/powernv/vas.h b/arch/powerpc/platforms/powernv/vas.h
index 650805d..60a3c3c 100644
--- a/arch/powerpc/platforms/powernv/vas.h
+++ b/arch/powerpc/platforms/powernv/vas.h
@@ -12,6 +12,7 @@
#include <linux/atomic.h>
#include <linux/idr.h>
#include <asm/vas.h>
+#include <linux/io.h>
/*
* Overview of Virtual Accelerator Switchboard (VAS).
@@ -381,4 +382,58 @@ struct vas_winctx {
extern struct vas_instance *find_vas_instance(int vasid);
+/*
+ * VREG(x):
+ * Expand a register's short name (eg: LPID) into two parameters:
+ * - the register's short name in string form ("LPID"), and
+ * - the name of the macro (eg: VAS_LPID_OFFSET), defining the
+ * register's offset in the window context
+ */
+#define VREG_SFX(n, s) __stringify(n), VAS_##n##s
+#define VREG(r) VREG_SFX(r, _OFFSET)
+
+#ifdef vas_debug
+static inline void vas_log_write(struct vas_window *win, char *name,
+ void *regptr, u64 val)
+{
+ if (val)
+ pr_err("%swin #%d: %s reg %p, val 0x%016llx\n",
+ win->tx_win ? "Tx" : "Rx", win->winid, name,
+ regptr, val);
+}
+
+#else /* vas_debug */
+
+#define vas_log_write(win, name, reg, val)
+
+#endif /* vas_debug */
+
+static inline void write_uwc_reg(struct vas_window *win, char *name,
+ s32 reg, u64 val)
+{
+ void *regptr;
+
+ regptr = win->uwc_map + reg;
+ vas_log_write(win, name, regptr, val);
+
+ out_be64(regptr, val);
+}
+
+static inline void write_hvwc_reg(struct vas_window *win, char *name,
+ s32 reg, u64 val)
+{
+ void *regptr;
+
+ regptr = win->hvwc_map + reg;
+ vas_log_write(win, name, regptr, val);
+
+ out_be64(regptr, val);
+}
+
+static inline u64 read_hvwc_reg(struct vas_window *win,
+ char *name __maybe_unused, s32 reg)
+{
+ return in_be64(win->hvwc_map+reg);
+}
+
#endif /* _VAS_H */
--
2.7.4
[toc] | [prev] | [next] | [standalone]
| From | Michael Neuling <mikey@neuling.org> |
|---|---|
| Date | 2017-09-07 09:10 +0200 |
| Message-ID | <umZ6O-56x-21@gated-at.bofh.it> |
| In reply to | #1722168 |
So this is upstream now but it will cause a crash on boot with older skiboots
with:
powernv-cpufreq: cpufreq pstate min 101 nominal 50 max 0
powernv-cpufreq: Workload Optimized Frequency is enabled in the platform
Disabling lock debugging due to kernel taint
Severe Machine check interrupt [Not recovered]
NIP [c000000000098530]: reset_window_regs+0x20/0x220
Initiator: CPU
Error type: Unknown
opal: Machine check interrupt unrecoverable: MSR(RI=0)
opal: Hardware platform error: Unrecoverable Machine Check exception
CPU: 1 PID: 1 Comm: swapper/0 Tainted: G M 4.13.0-rc7-00708-g8b680911e774-dirty #10
task: c000000f22680000 task.stack: c000000f22700000
NIP: c000000000098530 LR: c000000000098758 CTR: 0000000000000000
REGS: c00000003ffebd80 TRAP: 0200 Tainted: G M (4.13.0-rc7-00708-g8b680911e774-dirty)
MSR: 9000000008349031 <SF,HV,EE,ME,IR,DR,LE> CR: 24000224 XER: 00000000
CFAR: c000000000098754 DAR: 00000000100bef30 DSISR: 40000000 SOFTE: 0
GPR00: c000000000098f44 c000000f22703a00 c000000000eff200 c000000f1cf861e0
GPR04: c000000f22703a50 0000000000000001 0000000000000fff 0000000000000003
GPR08: c00c0000842f0000 0000000000000001 0000000000000000 0000000000000000
GPR12: 0000000000000000 c00000000fd40580 c000000000c03590 c000000000c1f428
GPR16: c000000000c4a640 c000000000c31360 c000000000c92738 c000000000c92690
GPR20: c000000000c926a0 c000000000c926f0 c000000f1d672940 0000000000000000
GPR24: c000000000c92638 0000000000000000 c000000000e888b0 c000000000dce428
GPR28: 0000000000000002 c000000f0e800000 c000000f22703a50 c000000f1cf861e0
NIP [c000000000098530] reset_window_regs+0x20/0x220
LR [c000000000098758] init_winctx_regs+0x28/0x6c0
Call Trace:
[c000000f22703a00] [0000000000000002] 0x2 (unreliable)
[c000000f22703a30] [c000000000098f44] vas_rx_win_open.part.11+0x154/0x210
[c000000f22703ae0] [c000000000d668e8] nx842_powernv_init+0x6b4/0x824
[c000000f22703c[ 38.412765557,0] OPAL: Reboot requested due to Platform error.
[ 38.412828287,3] OPAL: Reboot requested due to Platform error.40] [c00000000000ca60] do_one_initcall+0x60/0x1c0
If you see this you need a new skiboot with at least these two patches:
b503dcf16d vas: Set mmio enable bits in DD2
a5c124072f vas: Set FIRs according to workbook
This is a community announcement brought to you by OzLabs.
OzLabs: Making Linux better since 1999
Mikey
On Mon, 2017-08-28 at 23:23 -0700, Sukadev Bhattiprolu wrote:
> Power9 introduces a hardware subsystem referred to as the Virtual
> Accelerator Switchboard (VAS). VAS allows kernel subsystems and user
> space processes to directly access the Nest Accelerator (NX) engines
> which implement compression and encryption algorithms in the hardware.
>
> NX has been in Power processors since Power7+, but access to the NX
> engines was through the 'icswx' instruction which is only available
> to the kernel/hypervisor. Starting with Power9, access to the NX
> engines is provided to both kernel and user space processes through
> VAS.
>
> The switchboard (i.e VAS) multiplexes accesses between "receivers" and
> "senders", where the "receivers" are typically the NX engines and
> "senders" are the kernel subsystems and user processors that wish to
> access the receivers (NX engines). Once a sender is "connected" to
> a receiver through the switchboard, the senders can submit compression/
> encryption requests to the hardware using the new (PowerISA 3.0)
> "copy" and "paste" instructions.
>
> In the initial OPAL and PowerNV kernel patchsets, the "senders" can
> only be kernel subsystems (eg NX-842 driver) and receivers can only
> be the NX-842 engine. Follow-on patch sets will allow senders/receivers
> to be user-space processes and receivers to be NX-GZIP engines.
>
> Provides:
>
> This kernel patch set configures the VAS subsystems and provides
> kernel interfaces to drivers like NX-842 to open receive and send
> windows in VAS and to submit compression requests to the NX engine.
>
> Requires:
>
> This patch set needs corresponding VAS/NX skiboot patches which
> were merged into skiboot tree. i.e skiboot must include:
> commit b503dcf ("vas: Set mmio enable bits in DD2")
>
> Tests:
> In-kernel compression requests were tested on DD1 and DD2 POWER9
> hardware using compression self-test module and the following
> NX-842 patch set from Haren Myneni:
>
> https://lists.ozlabs.org/pipermail/linuxppc-dev/2017-July/160620.html
>
> and by dropping the last parameters to both vas_copy_crb() and
> vas_paste_crb() calls in drivers/crypto/nx/nx-842-powernv.c.
> See also PATCH 10/10.
>
> Git Tree:
>
> https://github.com/sukadev/linux/
> Branch: vas-kern-v8
>
> Thanks to input from Ben Herrenschmidt, Michael Neuling, Michael Ellerman
> and Haren Myneni.
>
> Changelog[v8]:
> - [Michael Ellerman] Use kernel int types (u64, u32 etc); make VAS
> a built-in rather than a module; drop unnecessary fields from
> struct vas_instance; Update ISA references; use 0 or 1 with
> SET_FIELD macros instead of bool; skip writing to SPARE registers;
> minor cleanup of debug/error messages; retry if ida_get_new()
> fails with EAGAIN; fix couple of leaks in ids in error handling;
> drop vas_initialized() check; drop vas_win_id() and vas_paste_addr()
> interfaces as they are not yet used; Set task_state() and fix
> parameter to schedule_timeout(); Reuse existing copy/paste macros
> drop unnecessary parameters and add cr0 to clobbers list
>
> Changelog[v7]:
> - Drop support for user space send/receive FTW windows (will be
> posted separately) Simplifies the rx-win-open interface a bit.
> - [Michael Ellerman] Move GET_FIELD/SET_FIELD macros from
> uapi/asm/vas.h to asm/vas.h.
>
> Changelog[v6]
> - Add support for user space send/receive FTW windows
> - Add a new, NX-FTW driver which provides the FTW user interface
>
> Changelog[v5]
> - [Ben Herrenschmidt] Make VAS a platform device in the device tree
> and use the core platform functions to parse the VAS properties.
> Map the VAS MMIO regions as non-cachable and paste regions as
> cachable. Use CONFIG_PPC_VAS rather than CONFIG_VAS; Don't assume
> VAS ids are sequential.
> - Copy the FIFO address as is into LFIFO_BAR (don't shift it).
>
> Changelog[v4]
> Comments from Michael Neuling:
> - Move VAS code from drivers/misc/vas to arch/powerpc/platforms/powernv
> since VAS only provides interfaces to other drivers like NX-842.
> - Drop vas-internal.h and use vas.h in separate dirs for VAS
> internal, kernel API and user API
> - Rather than create 6 separate device tree properties windows
> and window context, combine them into 6 "reg" properties.
> - Drop vas_window_reset() since windows are reset/cleared before
> being assigned to kernel/users.
> - Use ilog2() and radix_enabled() helpers
>
> Changelog[v3]
> - Rebase to v4.11-rc1
> - Add interfaces to initialize send/receive window attributes to
> defaults that drivers can use (see arch/powerpc/include/asm/vas.h)
> - Modify interface vas_paste() to return 0 or error code
> - Fix a bug in setting Translation Control Mode (0b11 not 0x11)
> - Enable send-window-credit checking
> - Reorg code in vas_win_close()
> - Minor reorgs and tweaks to register field settings to make it
> easier to add support for user space windows.
> - Skip writing to read-only registers
> - Start window indexing from 0 rather than 1
>
> Changelog[v2]
> - Use vas-id, HVWC, UWC and paste address, entries from device tree
> rather than defining/computing them in kernel and reorg code.
>
>
> Sukadev Bhattiprolu (10):
> powerpc/vas: Define macros, register fields and structures
> Move GET_FIELD/SET_FIELD to vas.h
> powerpc/vas: Define vas_init() and vas_exit()
> powerpc/vas: Define helpers to access MMIO regions
> powerpc/vas: Define helpers to init window context
> powerpc/vas: Define helpers to alloc/free windows
> powerpc/vas: Define vas_rx_win_open() interface
> powerpc/vas: Define vas_win_close() interface
> powerpc/vas: Define vas_tx_win_open()
> powerpc/vas: Define copy/paste interfaces
>
> .../devicetree/bindings/powerpc/ibm,vas.txt | 23 +
> MAINTAINERS | 9 +
> arch/powerpc/include/asm/ppc-opcode.h | 2 +
> arch/powerpc/include/asm/vas.h | 160 +++
> arch/powerpc/platforms/powernv/Kconfig | 14 +
> arch/powerpc/platforms/powernv/Makefile | 1 +
> arch/powerpc/platforms/powernv/copy-paste.h | 46 +
> arch/powerpc/platforms/powernv/vas-window.c | 1134
> ++++++++++++++++++++
> arch/powerpc/platforms/powernv/vas.c | 151 +++
> arch/powerpc/platforms/powernv/vas.h | 467 ++++++++
> drivers/crypto/nx/nx-842-powernv.c | 7 +-
> drivers/crypto/nx/nx-842.h | 5 -
> 12 files changed, 2011 insertions(+), 8 deletions(-)
> create mode 100644 Documentation/devicetree/bindings/powerpc/ibm,vas.txt
> create mode 100644 arch/powerpc/include/asm/vas.h
> create mode 100644 arch/powerpc/platforms/powernv/copy-paste.h
> create mode 100644 arch/powerpc/platforms/powernv/vas-window.c
> create mode 100644 arch/powerpc/platforms/powernv/vas.c
> create mode 100644 arch/powerpc/platforms/powernv/vas.h
>
[toc] | [prev] | [standalone]
Back to top | Article view | linux.kernel
csiph-web