Re: [PATCH v4 1/5] getcpu_cache system call: cache CPU number of running thread

From	Rasmus Villemoes <linux@rasmusvillemoes.dk>
Newsgroups	linux.kernel
Subject	Re: [PATCH v4 1/5] getcpu_cache system call: cache CPU number of running thread
Date	2016-02-26 00:40 +0100
Message-ID	<r6dFM-2OI-9@gated-at.bofh.it> (permalink)
References	<r5uyZ-4cb-5@gated-at.bofh.it> <r5uz0-4cb-19@gated-at.bofh.it> <r5FE7-3ID-43@gated-at.bofh.it>
Organization	D03

Show all headers | View raw

On Wed, Feb 24 2016, Mathieu Desnoyers <mathieu.desnoyers@efficios.com> wrote:

>
>        Typically, a library or application will keep the cpu  number
>        cache  in  a  thread-local  storage variable, or other memory
>        areas belonging to each thread. It is recommended to  perform
>        a  volatile  read of the cpu number cache to prevent the com‐
>        piler from doing load tearing. An alternative approach is  to
>        read  the  cpu  number cache from inline assembly in a single
>        instruction.
>
>        Each thread is responsible for registering its own cpu number
>        cache.   Only  one  cpu  cache  address can be registered per
>        thread.
>
>        The symbol  __getcpu_cache_tls  is  recommended  to  be  used
>        across  libraries  and  applications  wishing  to  register a
>        thread-local getcpu_cache. The  attribute  "weak"  is  recom‐
>        mended  when  declaring this variable in libraries.  Applica‐
>        tions can choose to define their own version of  this  symbol
>        without the weak attribute as a performance improvement.
>
>        In  a  typical usage scenario, the thread registering the cpu
>        number cache will be performing reads from that cache. It  is
>        however  also allowed to read the cpu number cache from other
>        threads. The cpu number cache updates performed by the kernel
>        provide single-copy atomicity semantics, which guarantee that
>        other threads performing single-copy atomic reads of the  cpu
>        number cache will always observe a consistent value.
>
>        Memory registered as cpu number cache should never be deallo‐
>        cated before the thread which registered it  exits:  specifi‐
>        cally, it should not be freed, and the library containing the
>        registered thread-local storage should not be dlclose'd.

Maybe spell out the consequence if this is violated - since the SIGSEGV
only happens on migration, it may take a while to strike.

Random thoughts: The current implementation ensures that getcpu_cache is
"idempotent" from within a single thread - once set, it can never get
unset nor set to some other pointer. I think that can be useful, since
it means a library can reliably use the TLS variable itself (initialized
with some negative number) as an indicator of whether
getcpu_cache(GETCPU_CACHE_SET) has been called. So if a single test on a
fast path where the library would need to load __getcpu_cache_tls anyway
is acceptable, it can avoid requiring some library init function to be
called in each thread - which can sometimes be hard to arrange. Is this
something we want to guarantee - that is, will we never implement
GETCPU_CACHE_UNSET or a "force" flag to _SET? Either way, I think we
should spend a few words on it to avoid the current behaviour becoming
accidental ABI.

In another thread:

> However, there are other use-cases for having a fast mechanism for
> reading the current CPU number, besides restartable sequences.  For
> instance, it can be used by glibc to implement a faster sched_getcpu.

Will glibc do that? It may be a little contentious for glibc to claim a
unique resource such as task_struct::cpu_cache for itself, even if
everybody is supposed to use the same symbol. Hm, maybe one could say
that if an application does define the symbol __getcpu_cache_tls (which
is techically in the implementation namespace), that gives glibc (and
any other library) license to do getcpu_cache(SET, &&__getcpu_cache_tls)
(pseudo-code, of course). If a library initializes its own weak version
with -2 it can check whether the application defined
__getcpu_cache_tls. Ok, I'm probably overthinking this...

Rasmus

Thread

[PATCH v4 1/5] getcpu_cache system call: cache CPU number of running thread Mathieu Desnoyers <mathieu.desnoyers@efficios.com> - 2016-02-24 00:30 +0100
  Re: [PATCH v4 1/5] getcpu_cache system call: cache CPU number of  running thread Thomas Gleixner <tglx@linutronix.de> - 2016-02-24 12:20 +0100
    Re: [PATCH v4 1/5] getcpu_cache system call: cache CPU number of  running thread Mathieu Desnoyers <mathieu.desnoyers@efficios.com> - 2016-02-24 18:20 +0100
    Re: [PATCH v4 1/5] getcpu_cache system call: cache CPU number of running thread Rasmus Villemoes <linux@rasmusvillemoes.dk> - 2016-02-26 00:40 +0100
      Re: [PATCH v4 1/5] getcpu_cache system call: cache CPU number of  running thread Mathieu Desnoyers <mathieu.desnoyers@efficios.com> - 2016-02-26 18:50 +0100
  Re: [PATCH v4 1/5] getcpu_cache system call: cache CPU number of  running thread Peter Zijlstra <peterz@infradead.org> - 2016-02-25 11:00 +0100
    Re: [PATCH v4 1/5] getcpu_cache system call: cache CPU number of  running thread Mathieu Desnoyers <mathieu.desnoyers@efficios.com> - 2016-02-25 18:00 +0100
      Re: [PATCH v4 1/5] getcpu_cache system call: cache CPU number of  running thread Peter Zijlstra <peterz@infradead.org> - 2016-02-25 18:10 +0100
        Re: [PATCH v4 1/5] getcpu_cache system call: cache CPU number of  running thread Mathieu Desnoyers <mathieu.desnoyers@efficios.com> - 2016-02-25 18:20 +0100
          Re: [PATCH v4 1/5] getcpu_cache system call: cache CPU number of  running thread Peter Zijlstra <peterz@infradead.org> - 2016-02-26 12:40 +0100
            Re: [PATCH v4 1/5] getcpu_cache system call: cache CPU number of  running thread Thomas Gleixner <tglx@linutronix.de> - 2016-02-26 17:40 +0100
              Re: [PATCH v4 1/5] getcpu_cache system call: cache CPU number of  running thread Mathieu Desnoyers <mathieu.desnoyers@efficios.com> - 2016-02-26 18:30 +0100
                Re: [PATCH v4 1/5] getcpu_cache system call: cache CPU number of  running thread Thomas Gleixner <tglx@linutronix.de> - 2016-02-26 19:10 +0100
                Re: [PATCH v4 1/5] getcpu_cache system call: cache CPU number of  running thread Mathieu Desnoyers <mathieu.desnoyers@efficios.com> - 2016-02-26 21:30 +0100
                Re: [PATCH v4 1/5] getcpu_cache system call: cache CPU number of running thread "H. Peter Anvin" <hpa@zytor.com> - 2016-02-27 00:10 +0100
                Re: [PATCH v4 1/5] getcpu_cache system call: cache CPU number of  running thread Mathieu Desnoyers <mathieu.desnoyers@efficios.com> - 2016-02-27 01:50 +0100
                Re: [PATCH v4 1/5] getcpu_cache system call: cache CPU number of  running thread "H. Peter Anvin" <hpa@zytor.com> - 2016-02-27 07:30 +0100
                Re: [PATCH v4 1/5] getcpu_cache system call: cache CPU number of  running thread Mathieu Desnoyers <mathieu.desnoyers@efficios.com> - 2016-02-27 15:20 +0100
                Re: [PATCH v4 1/5] getcpu_cache system call: cache CPU number of  running thread Peter Zijlstra <peterz@infradead.org> - 2016-02-27 16:00 +0100
                Re: [PATCH v4 1/5] getcpu_cache system call: cache CPU number of  running thread Linus Torvalds <torvalds@linux-foundation.org> - 2016-02-27 19:40 +0100
                Re: [PATCH v4 1/5] getcpu_cache system call: cache CPU number of running thread "H. Peter Anvin" <hpa@zytor.com> - 2016-02-27 20:10 +0100
                Re: [PATCH v4 1/5] getcpu_cache system call: cache CPU number of  running thread Mathieu Desnoyers <mathieu.desnoyers@efficios.com> - 2016-02-28 01:00 +0100
                Re: [PATCH v4 1/5] getcpu_cache system call: cache CPU number of  running thread Linus Torvalds <torvalds@linux-foundation.org> - 2016-02-28 02:00 +0100
                Re: [PATCH v4 1/5] getcpu_cache system call: cache CPU number of  running thread Mathieu Desnoyers <mathieu.desnoyers@efficios.com> - 2016-02-28 15:40 +0100
                Re: [PATCH v4 1/5] getcpu_cache system call: cache CPU number of  running thread Peter Zijlstra <peterz@infradead.org> - 2016-02-29 11:40 +0100
                Re: [PATCH v4 1/5] getcpu_cache system call: cache CPU number of  running thread Mathieu Desnoyers <mathieu.desnoyers@efficios.com> - 2016-03-01 21:30 +0100
                Re: [PATCH v4 1/5] getcpu_cache system call: cache CPU number of  running thread Peter Zijlstra <peterz@infradead.org> - 2016-03-01 22:40 +0100
                Re: [PATCH v4 1/5] getcpu_cache system call: cache CPU number of  running thread Peter Zijlstra <peterz@infradead.org> - 2016-03-01 22:40 +0100
                Re: [PATCH v4 1/5] getcpu_cache system call: cache CPU number of  running thread "H. Peter Anvin" <hpa@zytor.com> - 2016-03-01 23:00 +0100
                Re: [PATCH v4 1/5] getcpu_cache system call: cache CPU number of  running thread Peter Zijlstra <peterz@infradead.org> - 2016-03-02 11:40 +0100
                Re: [PATCH v4 1/5] getcpu_cache system call: cache CPU number of  running thread Peter Zijlstra <peterz@infradead.org> - 2016-02-29 11:40 +0100
                Re: [PATCH v4 1/5] getcpu_cache system call: cache CPU number of running thread Arnd Bergmann <arnd@arndb.de> - 2016-02-29 11:50 +0100
                Re: [PATCH v4 1/5] getcpu_cache system call: cache CPU number of  running thread Mathieu Desnoyers <mathieu.desnoyers@efficios.com> - 2016-02-29 13:50 +0100
                Re: [PATCH v4 1/5] getcpu_cache system call: cache CPU number of running thread Arnd Bergmann <arnd@arndb.de> - 2016-02-29 14:20 +0100
                Re: [PATCH v4 1/5] getcpu_cache system call: cache CPU number of running thread "H. Peter Anvin" <hpa@zytor.com> - 2016-02-29 19:30 +0100
                Re: [PATCH v4 1/5] getcpu_cache system call: cache CPU number of  running thread Geert Uytterhoeven <geert@linux-m68k.org> - 2016-03-02 11:50 +0100
                Re: [PATCH v4 1/5] getcpu_cache system call: cache CPU number of  running thread "H. Peter Anvin" <hpa@zytor.com> - 2016-03-01 19:30 +0100
                Re: [PATCH v4 1/5] getcpu_cache system call: cache CPU number of  running thread Mathieu Desnoyers <mathieu.desnoyers@efficios.com> - 2016-03-01 19:50 +0100
                Re: [PATCH v4 1/5] getcpu_cache system call: cache CPU number of  running thread Geert Uytterhoeven <geert@linux-m68k.org> - 2016-02-28 14:10 +0100
                Re: [PATCH v4 1/5] getcpu_cache system call: cache CPU number of  running thread Linus Torvalds <torvalds@linux-foundation.org> - 2016-02-28 17:30 +0100
                Re: [PATCH v4 1/5] getcpu_cache system call: cache CPU number of  running thread Peter Zijlstra <peterz@infradead.org> - 2016-02-29 11:10 +0100
                Re: [PATCH v4 1/5] getcpu_cache system call: cache CPU number of running thread "H. Peter Anvin" <hpa@zytor.com> - 2016-02-27 16:10 +0100

csiph-web