Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1550202

Re: [RFC] memcpy_nocache() and memcpy_writethrough()

Show all headers | View raw

On Sun, Jan 1, 2017 at 9:09 PM, Al Viro <viro@zeniv.linux.org.uk> wrote:
> On Mon, Jan 02, 2017 at 02:35:36AM +0000, Elliott, Robert (Persistent Memory) wrote:
>> > -----Original Message-----
>> > From: linux-kernel-owner@vger.kernel.org [mailto:linux-kernel-
>> > owner@vger.kernel.org] On Behalf Of Al Viro
>> > Sent: Friday, December 30, 2016 8:26 PM
>> > Subject: [RFC] memcpy_nocache() and memcpy_writethrough()
>> >
>> ...
>> > Why does pmem need writethrough warranties, anyway?
>>
>> Using either
>> * nontemporal store instructions; or
>> * following regular store instructions with a sequence of cache flush
>> and store fence instructions (e.g., clflushopt or clwb + sfence)
>>
>> ensures that write data has reached an "ADR-safe zone" that the system
>> promises will be persistent even if there is a surprise power loss or
>> a CPU suffers from an error that isn't totally catastrophic (e.g., the
>> CPU getting disconnected from the SDRAM will always lose data on an
>> NVDIMM-N).
>
> Wait a sec...  In which places do you need sfence in all that?  movnt*
> itself can be reordered, right?  So using that for copying and storing
> the pointer afterwards would still need sfence inbetween, unless I'm
> seriously misunderstanding the situation...

Robert was describing the overall flow / mechanics, but I think it is
easier to visualize the sfence as a flush command sent to a disk
device with a volatile cache. In fact, that's how we implemented it in
the pmem block device driver. The pmem block device registers itself
as requiring REQ_FLUSH to be sent to persist writes. The driver issues
sfence on the assumption that all writes to pmem have either bypassed
the cache with movnt, or are scheduled for write-back via one of the
flush instructions (clflush, clwb, or clflushopt).

>> Newly written data becomes globally visible before it becomes ADR-safe.
>> This means software could act on the new data before a power loss, then
>> see the old data reappear after the power loss - not good.  Software
>> needs to understand that any data in the process of being written is
>> indeterminate until the persistence guarantee is met.  The BTT shows
>> one way that software can avoid that problem.
>
> Joy.  What happens in terms of latency?  I.e. how much of a stall does
> clwb inflict?

Unlike clflush, clwb is unordered, so it has lower overhead. It
schedules writeback, but does not wait for it to complete. The
clflushopt instruction is also unordered, but in addition to writeback
it also invalidates the line.

Back to linux.kernel | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Re: [PATCH v2 1/3] introduce memcpy_nocache() Al Viro <viro@ZenIV.linux.org.uk> - 2016-12-29 00:50 +0100
  Re: [PATCH v2 1/3] introduce memcpy_nocache() Dan Williams <dan.j.williams@intel.com> - 2016-12-29 19:30 +0100
    Re: [PATCH v2 1/3] introduce memcpy_nocache() Al Viro <viro@ZenIV.linux.org.uk> - 2016-12-30 05:00 +0100
      Re: [PATCH v2 1/3] introduce memcpy_nocache() Dan Williams <dan.j.williams@intel.com> - 2016-12-30 06:00 +0100
        [RFC] memcpy_nocache() and memcpy_writethrough() Al Viro <viro@ZenIV.linux.org.uk> - 2016-12-31 03:30 +0100
          RE: [RFC] memcpy_nocache() and memcpy_writethrough() "Elliott, Robert (Persistent Memory)" <elliott@hpe.com> - 2017-01-02 03:40 +0100
            Re: [RFC] memcpy_nocache() and memcpy_writethrough() Al Viro <viro@ZenIV.linux.org.uk> - 2017-01-02 06:10 +0100
              Re: [RFC] memcpy_nocache() and memcpy_writethrough() Dan Williams <dan.j.williams@intel.com> - 2017-01-03 22:20 +0100
                Re: [RFC] memcpy_nocache() and memcpy_writethrough() Al Viro <viro@ZenIV.linux.org.uk> - 2017-01-04 00:30 +0100
                Re: [RFC] memcpy_nocache() and memcpy_writethrough() Linus Torvalds <torvalds@linux-foundation.org> - 2017-01-04 00:50 +0100
                Re: [RFC] memcpy_nocache() and memcpy_writethrough() Dan Williams <dan.j.williams@intel.com> - 2017-01-04 02:00 +0100
                Re: [RFC] memcpy_nocache() and memcpy_writethrough() Dan Williams <dan.j.williams@intel.com> - 2017-01-04 02:40 +0100
                Re: [RFC] memcpy_nocache() and memcpy_writethrough() Al Viro <viro@ZenIV.linux.org.uk> - 2017-01-04 03:00 +0100
                Re: [RFC] memcpy_nocache() and memcpy_writethrough() Dan Williams <dan.j.williams@intel.com> - 2017-01-04 03:20 +0100

csiph-web