Re: [v3 0/9] parallelized "struct page" zeroing

From	Pasha Tatashin <pasha.tatashin@oracle.com>
Newsgroups	linux.kernel
Subject	Re: [v3 0/9] parallelized "struct page" zeroing
Date	2017-05-11 23:10 +0200
Message-ID	<tG3vu-37w-65@gated-at.bofh.it> (permalink)
References	(3 earlier) <tFuem-6dd-7@gated-at.bofh.it> <tFAa6-1mz-19@gated-at.bofh.it> <tFBfQ-20m-9@gated-at.bofh.it> <tFBpw-2iN-21@gated-at.bofh.it> <tG3c5-2La-1@gated-at.bofh.it>
Organization	linux.* mail to news gateway

Show all headers | View raw

We should either keep memset() only for deferred struct pages as what I 
have in my patches.

Another option is to add a new function struct_page_clear() which would 
default to memset() and to something else on platforms that decide to 
optimize it.

On SPARC it would call STBIs, and we would do one membar call after all 
"struct pages" are initialized.

I think what I sent out already is cleaner and better solution, because 
I am not sure what kind of performance we would see on other chips.

On 05/11/2017 04:47 PM, Pasha Tatashin wrote:
>>>
>>> Have you measured that? I do not think it would be super hard to
>>> measure. I would be quite surprised if this added much if anything at
>>> all as the whole struct page should be in the cache line already. We do
>>> set reference count and other struct members. Almost nobody should be
>>> looking at our page at this time and stealing the cache line. On the
>>> other hand a large memcpy will basically wipe everything away from the
>>> cpu cache. Or am I missing something?
>>>
> 
> Here is data for single thread (deferred struct page init is disabled):
> 
> Intel CPU E7-8895 v3 @ 2.60GHz  1T memory
> -----------------------------------------
> time to memset "struct pages in memblock: 11.28s
> time to init "struct pag"es:               4.90s
> 
> Moving memset into __init_single_page()
> time to init and memset "struct page"es:   8.39s
> 
> SPARC M6 @ 3600 MHz  1T memory
> -----------------------------------------
> time to memset "struct pages in memblock:  1.60s
> time to init "struct pag"es:               3.37s
> 
> Moving memset into __init_single_page()
> time to init and memset "struct page"es:  12.99s
> 
> 
> So, moving memset() into __init_single_page() benefits Intel. I am 
> actually surprised why memset() is so slow on intel when it is called 
> from memblock. But, hurts SPARC, I guess these membars at the end of 
> memset() kills the performance.
> 
> Also, when looking at these values, remeber that Intel has twice as many 
> "struct page" for the same amount of memory.
> 
> Pasha
> -- 
> To unsubscribe from this list: send the line "unsubscribe sparclinux" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Thread

[v3 0/9] parallelized "struct page" zeroing Pavel Tatashin <pasha.tatashin@oracle.com> - 2017-05-05 19:10 +0200
  [v3 8/9] powerpc: teach platforms not to zero struct pages memory Pavel Tatashin <pasha.tatashin@oracle.com> - 2017-05-05 19:10 +0200
  [v3 9/9] s390: teach platforms not to zero struct pages memory Pavel Tatashin <pasha.tatashin@oracle.com> - 2017-05-05 19:10 +0200
    Re: [v3 9/9] s390: teach platforms not to zero struct pages memory Heiko Carstens <heiko.carstens@de.ibm.com> - 2017-05-08 13:40 +0200
      Re: [v3 9/9] s390: teach platforms not to zero struct pages memory Pasha Tatashin <pasha.tatashin@oracle.com> - 2017-05-15 20:30 +0200
        Re: [v3 9/9] s390: teach platforms not to zero struct pages memory Heiko Carstens <heiko.carstens@de.ibm.com> - 2017-05-16 01:20 +0200
          Re: [v3 9/9] s390: teach platforms not to zero struct pages memory Pasha Tatashin <pasha.tatashin@oracle.com> - 2017-05-16 02:40 +0200
  [v3 1/9] sparc64: simplify vmemmap_populate Pavel Tatashin <pasha.tatashin@oracle.com> - 2017-05-05 19:10 +0200
  Re: [v3 0/9] parallelized "struct page" zeroing Michal Hocko <mhocko@kernel.org> - 2017-05-09 20:20 +0200
    Re: [v3 0/9] parallelized "struct page" zeroing Pasha Tatashin <pasha.tatashin@oracle.com> - 2017-05-09 21:00 +0200
      Re: [v3 0/9] parallelized "struct page" zeroing Michal Hocko <mhocko@kernel.org> - 2017-05-10 09:30 +0200
        Re: [v3 0/9] parallelized "struct page" zeroing Pasha Tatashin <pasha.tatashin@oracle.com> - 2017-05-10 15:50 +0200
          Re: [v3 0/9] parallelized "struct page" zeroing Michal Hocko <mhocko@kernel.org> - 2017-05-10 17:00 +0200
            Re: [v3 0/9] parallelized "struct page" zeroing Pasha Tatashin <pasha.tatashin@oracle.com> - 2017-05-10 17:10 +0200
              Re: [v3 0/9] parallelized "struct page" zeroing David Miller <davem@davemloft.net> - 2017-05-10 17:30 +0200
              Re: [v3 0/9] parallelized "struct page" zeroing Pasha Tatashin <pasha.tatashin@oracle.com> - 2017-05-11 22:50 +0200
                Re: [v3 0/9] parallelized "struct page" zeroing Pasha Tatashin <pasha.tatashin@oracle.com> - 2017-05-11 23:10 +0200
                Re: [v3 0/9] parallelized "struct page" zeroing David Miller <davem@davemloft.net> - 2017-05-12 19:00 +0200
                Re: [v3 0/9] parallelized "struct page" zeroing Pasha Tatashin <pasha.tatashin@oracle.com> - 2017-05-12 19:30 +0200
                Re: [v3 0/9] parallelized "struct page" zeroing David Miller <davem@davemloft.net> - 2017-05-12 19:40 +0200
                Re: [v3 0/9] parallelized "struct page" zeroing Benjamin Herrenschmidt <benh@au1.ibm.com> - 2017-05-17 02:00 +0200
                Re: [v3 0/9] parallelized "struct page" zeroing David Miller <davem@davemloft.net> - 2017-05-12 19:00 +0200
            Re: [v3 0/9] parallelized "struct page" zeroing David Miller <davem@davemloft.net> - 2017-05-10 17:20 +0200
              Re: [v3 0/9] parallelized "struct page" zeroing Matthew Wilcox <willy@infradead.org> - 2017-05-10 19:20 +0200
                Re: [v3 0/9] parallelized "struct page" zeroing David Miller <davem@davemloft.net> - 2017-05-10 20:10 +0200
                Re: [v3 0/9] parallelized "struct page" zeroing Matthew Wilcox <willy@infradead.org> - 2017-05-10 23:20 +0200
              Re: [v3 0/9] parallelized "struct page" zeroing Michal Hocko <mhocko@kernel.org> - 2017-05-11 10:10 +0200
                Re: [v3 0/9] parallelized "struct page" zeroing David Miller <davem@davemloft.net> - 2017-05-11 16:40 +0200
    Re: [v3 0/9] parallelized "struct page" zeroing Pasha Tatashin <pasha.tatashin@oracle.com> - 2017-05-15 20:20 +0200
      Re: [v3 0/9] parallelized "struct page" zeroing Michal Hocko <mhocko@kernel.org> - 2017-05-15 21:40 +0200
        Re: [v3 0/9] parallelized "struct page" zeroing Pasha Tatashin <pasha.tatashin@oracle.com> - 2017-05-15 22:50 +0200
          Re: [v3 0/9] parallelized "struct page" zeroing Michal Hocko <mhocko@kernel.org> - 2017-05-16 10:40 +0200

csiph-web