Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1217725

Re: [PATCH] dax, pmem: add support for msync

Show all headers | View raw

On 09/02/2015 06:39 PM, Dave Hansen wrote:
> On 09/02/2015 08:18 AM, Boaz Harrosh wrote:
>> On 09/02/2015 05:23 PM, Dave Hansen wrote:
>>>> I'd be curious what the cost is in practice.  Do you have any actual
>>>> numbers of the cost of doing it this way?
>>>>
>>>> Even if the instruction is a "noop", I'd really expect the overhead to
>>>> really add up for a tens-of-gigabytes mapping, no matter how much the
>>>> CPU optimizes it.
>> What tens-of-gigabytes mapping? I have yet to encounter an application
>> that does that. Our tests show that usually the mmaps are small.
> 
> We are going to have 2-socket systems with 6TB of persistent memory in
> them.  I think it's important to design this mechanism so that it scales
> to memory sizes like that and supports large mmap()s.
> 
> I'm not sure the application you've seen thus far are very
> representative of what we want to design for.
> 

We have a patch pending to introduce a new mmap flag that pmem aware
applications can set to eliminate any kind of flushing. MMAP_PMEM_AWARE.

This is good for the like of libnvdimm that does one large mmap of the
all 6T and does not want the clflush penalty on unmap.

>> I can send you a micro benchmark results of an mmap vs direct-io random
>> write. Our code will jump over holes in the file BTW, but I'll ask to also
>> run it with falloc that will make all blocks allocated.
> 
> I'm really just more curious about actual clflush performance on large
> ranges.  I'm curious how good the CPU is at optimizing it.
> 

Again our test does not do this, because it will only flush written-extents
of the file. the most we have in one machine is 64G of pmem, so even on a
very large mmap the most that can be is 64G of data, and the actual modify
of 64G of data will be much slower then the added clflush to each cache_line.

Thanks
Boaz

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Back to linux.kernel | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

[PATCH] dax, pmem: add support for msync Ross Zwisler <ross.zwisler@linux.intel.com> - 2015-08-31 21:10 +0200
  Re: [PATCH] dax, pmem: add support for msync Christoph Hellwig <hch@lst.de> - 2015-08-31 21:10 +0200
    Re: [PATCH] dax, pmem: add support for msync Ross Zwisler <ross.zwisler@linux.intel.com> - 2015-08-31 21:30 +0200
      Re: [PATCH] dax, pmem: add support for msync Christoph Hellwig <hch@lst.de> - 2015-08-31 21:40 +0200
  Re: [PATCH] dax, pmem: add support for msync Dave Chinner <david@fromorbit.com> - 2015-09-01 01:40 +0200
    Re: [PATCH] dax, pmem: add support for msync Christoph Hellwig <hch@lst.de> - 2015-09-01 09:10 +0200
      Re: [PATCH] dax, pmem: add support for msync Boaz Harrosh <boaz@plexistor.com> - 2015-09-01 14:20 +0200
        Re: [PATCH] dax, pmem: add support for msync Ross Zwisler <ross.zwisler@linux.intel.com> - 2015-09-02 21:10 +0200
          Re: [PATCH] dax, pmem: add support for msync "Kirill A. Shutemov" <kirill@shutemov.name> - 2015-09-02 22:20 +0200
          Re: [PATCH] dax, pmem: add support for msync Boaz Harrosh <boaz@plexistor.com> - 2015-09-03 08:40 +0200
            Re: [PATCH] dax, pmem: add support for msync Ross Zwisler <ross.zwisler@linux.intel.com> - 2015-09-03 18:50 +0200
      Re: [PATCH] dax, pmem: add support for msync Dave Chinner <david@fromorbit.com> - 2015-09-02 00:30 +0200
        Re: [PATCH] dax, pmem: add support for msync Ross Zwisler <ross.zwisler@linux.intel.com> - 2015-09-02 05:20 +0200
          Re: [PATCH] dax, pmem: add support for msync Dave Chinner <david@fromorbit.com> - 2015-09-02 07:20 +0200
            Re: [PATCH] dax, pmem: add support for msync Boaz Harrosh <boaz@plexistor.com> - 2015-09-02 12:30 +0200
              Re: [PATCH] dax, pmem: add support for msync Dave Hansen <dave.hansen@linux.intel.com> - 2015-09-02 16:30 +0200
                Re: [PATCH] dax, pmem: add support for msync Boaz Harrosh <boaz@plexistor.com> - 2015-09-02 17:20 +0200
                Re: [PATCH] dax, pmem: add support for msync Dave Hansen <dave.hansen@linux.intel.com> - 2015-09-02 17:50 +0200
                Re: [PATCH] dax, pmem: add support for msync Boaz Harrosh <boaz@plexistor.com> - 2015-09-02 18:10 +0200
                Re: [PATCH] dax, pmem: add support for msync Dave Hansen <dave.hansen@linux.intel.com> - 2015-09-02 18:20 +0200
                Re: [PATCH] dax, pmem: add support for msync Boaz Harrosh <boaz@plexistor.com> - 2015-09-03 08:50 +0200
          Re: [PATCH] dax, pmem: add support for msync Boaz Harrosh <boaz@plexistor.com> - 2015-09-02 12:10 +0200
    Re: [PATCH] dax, pmem: add support for msync "Kirill A. Shutemov" <kirill@shutemov.name> - 2015-09-01 12:10 +0200
      Re: [PATCH] dax, pmem: add support for msync Boaz Harrosh <boaz@plexistor.com> - 2015-09-01 13:30 +0200
      Re: [PATCH] dax, pmem: add support for msync Dave Chinner <david@fromorbit.com> - 2015-09-02 00:50 +0200
        Re: [PATCH] dax, pmem: add support for msync "Kirill A. Shutemov" <kirill@shutemov.name> - 2015-09-02 11:20 +0200
          Re: [PATCH] dax, pmem: add support for msync Boaz Harrosh <boaz@plexistor.com> - 2015-09-02 11:40 +0200
            Re: [PATCH] dax, pmem: add support for msync Boaz Harrosh <boaz@plexistor.com> - 2015-09-02 11:50 +0200
              Re: [PATCH] dax, pmem: add support for msync "Kirill A. Shutemov" <kirill@shutemov.name> - 2015-09-02 11:50 +0200
                Re: [PATCH] dax, pmem: add support for msync Boaz Harrosh <boaz@plexistor.com> - 2015-09-02 12:30 +0200
          Re: [PATCH] dax, pmem: add support for msync Dave Chinner <david@fromorbit.com> - 2015-09-03 03:00 +0200
  Re: [PATCH] dax, pmem: add support for msync Boaz Harrosh <boaz@plexistor.com> - 2015-09-01 15:20 +0200
    Re: [PATCH] dax, pmem: add support for msync Ross Zwisler <ross.zwisler@linux.intel.com> - 2015-09-02 19:50 +0200

csiph-web