Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1698340

Re: [PATCH v2] cpuset: fix a deadlock due to incomplete patching of cpusets_enabled()

Show all headers | View raw

On Thu, Jul 27, 2017 at 12:48 PM, Andrew Morton
<akpm@linux-foundation.org> wrote:
> On Thu, 27 Jul 2017 09:46:08 -0700 Dima Zavin <dmitriyz@waymo.com> wrote:
>
>> In codepaths that use the begin/retry interface for reading
>> mems_allowed_seq with irqs disabled, there exists a race condition that
>> stalls the patch process after only modifying a subset of the
>> static_branch call sites.
>>
>> This problem manifested itself as a dead lock in the slub
>> allocator, inside get_any_partial. The loop reads
>> mems_allowed_seq value (via read_mems_allowed_begin),
>> performs the defrag operation, and then verifies the consistency
>> of mem_allowed via the read_mems_allowed_retry and the cookie
>> returned by xxx_begin. The issue here is that both begin and retry
>> first check if cpusets are enabled via cpusets_enabled() static branch.
>> This branch can be rewritted dynamically (via cpuset_inc) if a new
>> cpuset is created. The x86 jump label code fully synchronizes across
>> all CPUs for every entry it rewrites. If it rewrites only one of the
>> callsites (specifically the one in read_mems_allowed_retry) and then
>> waits for the smp_call_function(do_sync_core) to complete while a CPU is
>> inside the begin/retry section with IRQs off and the mems_allowed value
>> is changed, we can hang. This is because begin() will always return 0
>> (since it wasn't patched yet) while retry() will test the 0 against
>> the actual value of the seq counter.
>>
>> The fix is to cache the value that's returned by cpusets_enabled() at the
>> top of the loop, and only operate on the seqcount (both begin and retry) if
>> it was true.
>
> Tricky.  Hence we should have a nice code comment somewhere describing
> all of this.
>
>> --- a/include/linux/cpuset.h
>> +++ b/include/linux/cpuset.h
>> @@ -16,6 +16,11 @@
>>  #include <linux/mm.h>
>>  #include <linux/jump_label.h>
>>
>> +struct cpuset_mems_cookie {
>> +     unsigned int seq;
>> +     bool was_enabled;
>> +};
>
> At cpuset_mems_cookie would be a good site - why it exists, what it
> does, when it is used and how.

Will do. I actually had a comment here but removed it in lieu of
commit message :) Will put it back.

Back to linux.kernel | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

[RFC PATCH] mm/slub: fix a deadlock due to incomplete patching of cpusets_enabled() Dima Zavin <dmitriyz@waymo.com> - 2017-07-26 19:00 +0200
  Re: [RFC PATCH] mm/slub: fix a deadlock due to incomplete patching  of cpusets_enabled() Christopher Lameter <cl@linux.com> - 2017-07-26 19:10 +0200
    Re: [RFC PATCH] mm/slub: fix a deadlock due to incomplete patching of cpusets_enabled() Dima Zavin <dmitriyz@waymo.com> - 2017-07-26 22:00 +0200
    [PATCH v2] cpuset: fix a deadlock due to incomplete patching of cpusets_enabled() Dima Zavin <dmitriyz@waymo.com> - 2017-07-27 18:50 +0200
      Re: [PATCH v2] cpuset: fix a deadlock due to incomplete patching of  cpusets_enabled() Andrew Morton <akpm@linux-foundation.org> - 2017-07-27 21:50 +0200
        Re: [PATCH v2] cpuset: fix a deadlock due to incomplete patching of cpusets_enabled() Dima Zavin <dmitriyz@waymo.com> - 2017-07-27 23:50 +0200
      Re: [PATCH v2] cpuset: fix a deadlock due to incomplete patching of  cpusets_enabled() Andrew Morton <akpm@linux-foundation.org> - 2017-07-27 22:00 +0200
        Re: [PATCH v2] cpuset: fix a deadlock due to incomplete patching of cpusets_enabled() Dima Zavin <dmitriyz@waymo.com> - 2017-07-27 23:50 +0200
      Re: [PATCH v2] cpuset: fix a deadlock due to incomplete patching of  cpusets_enabled() Vlastimil Babka <vbabka@suse.cz> - 2017-07-28 09:50 +0200
        Re: [PATCH v2] cpuset: fix a deadlock due to incomplete patching of cpusets_enabled() Dima Zavin <dmitriyz@waymo.com> - 2017-07-28 10:50 +0200
        Re: [PATCH v2] cpuset: fix a deadlock due to incomplete patching of  cpusets_enabled() Peter Zijlstra <peterz@infradead.org> - 2017-07-28 11:40 +0200
          Re: [PATCH v2] cpuset: fix a deadlock due to incomplete patching of  cpusets_enabled() Vlastimil Babka <vbabka@suse.cz> - 2017-07-28 16:10 +0200
            Re: [PATCH v2] cpuset: fix a deadlock due to incomplete patching of cpusets_enabled() Dima Zavin <dmitriyz@waymo.com> - 2017-07-28 19:00 +0200
            [PATCH v3] cpuset: fix a deadlock due to incomplete patching of cpusets_enabled() Dima Zavin <dmitriyz@waymo.com> - 2017-07-31 06:10 +0200
              Re: [PATCH v3] cpuset: fix a deadlock due to incomplete patching of cpusets_enabled() Dima Zavin <dmitriyz@waymo.com> - 2017-07-31 06:10 +0200
              Re: [PATCH v3] cpuset: fix a deadlock due to incomplete patching of  cpusets_enabled() Vlastimil Babka <vbabka@suse.cz> - 2017-07-31 10:10 +0200
                Re: [PATCH v3] cpuset: fix a deadlock due to incomplete patching of cpusets_enabled() Dima Zavin <dmitriyz@waymo.com> - 2017-07-31 11:10 +0200

csiph-web