Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]


Groups > linux.kernel > #1534760

Re: INFO: rcu_sched detected stalls on CPUs/tasks with `kswapd` and `mem_cgroup_shrink_node`

From Donald Buczek <buczek@molgen.mpg.de>
Newsgroups linux.kernel
Subject Re: INFO: rcu_sched detected stalls on CPUs/tasks with `kswapd` and `mem_cgroup_shrink_node`
Date 2016-12-02 10:20 +0100
Message-ID <sJSaB-1Wx-19@gated-at.bofh.it> (permalink)
References (7 earlier) <sIrYT-1kN-65@gated-at.bofh.it> <sIteh-22H-15@gated-at.bofh.it> <sJajf-4SL-21@gated-at.bofh.it> <sJb5D-5oB-7@gated-at.bofh.it> <sJbyF-5yf-9@gated-at.bofh.it>
Organization linux.* mail to news gateway

Show all headers | View raw


On 11/30/16 12:43, Donald Buczek wrote:
> On 11/30/16 12:09, Michal Hocko wrote:
>> [CCing Paul]
>>
>> On Wed 30-11-16 11:28:34, Donald Buczek wrote:
>> [...]
>>> shrink_active_list gets and releases the spinlock and calls 
>>> cond_resched().
>>> This should give other tasks a chance to run. Just as an experiment, 
>>> I'm
>>> trying
>>>
>>> --- a/mm/vmscan.c
>>> +++ b/mm/vmscan.c
>>> @@ -1921,7 +1921,7 @@ static void shrink_active_list(unsigned long
>>> nr_to_scan,
>>>          spin_unlock_irq(&pgdat->lru_lock);
>>>
>>>          while (!list_empty(&l_hold)) {
>>> -               cond_resched();
>>> +               cond_resched_rcu_qs();
>>>                  page = lru_to_page(&l_hold);
>>>                  list_del(&page->lru);
>>>
>>> and didn't hit a rcu_sched warning for >21 hours uptime now. We'll see.
>> This is really interesting! Is it possible that the RCU stall detector
>> is somehow confused?
>
> Wait... 21 hours is not yet a test result.

For the records: We didn't have any stall warnings after 2 days and 20 
hours now and so I'm quite confident, that my above patch fixed the 
problem for v4.8.0. On previous boots the rcu warnings started after 
37,0.2,1,2,0.8 hours uptime.

Now I've applied this patch to stable latest (v4.8.11) on another backup 
machine which suffered even more rcu stalls.

Donald

> [...]

-- 
Donald Buczek
buczek@molgen.mpg.de
Tel: +49 30 8413 1433

Back to linux.kernel | Previous | NextPrevious in thread | Next in thread | Find similar | Unroll thread


Thread

Re: INFO: rcu_sched detected stalls on CPUs/tasks with `kswapd` and  `mem_cgroup_shrink_node` Donald Buczek <buczek@molgen.mpg.de> - 2016-11-27 10:20 +0100
  Re: INFO: rcu_sched detected stalls on CPUs/tasks with `kswapd` and  `mem_cgroup_shrink_node` Michal Hocko <mhocko@kernel.org> - 2016-11-28 12:10 +0100
    Re: INFO: rcu_sched detected stalls on CPUs/tasks with `kswapd` and  `mem_cgroup_shrink_node` Paul Menzel <pmenzel@molgen.mpg.de> - 2016-11-28 13:30 +0100
      Re: INFO: rcu_sched detected stalls on CPUs/tasks with `kswapd` and  `mem_cgroup_shrink_node` Donald Buczek <buczek@molgen.mpg.de> - 2016-11-30 11:30 +0100
        Re: INFO: rcu_sched detected stalls on CPUs/tasks with `kswapd` and  `mem_cgroup_shrink_node` Michal Hocko <mhocko@kernel.org> - 2016-11-30 12:20 +0100
          Re: INFO: rcu_sched detected stalls on CPUs/tasks with `kswapd` and  `mem_cgroup_shrink_node` Donald Buczek <buczek@molgen.mpg.de> - 2016-11-30 12:50 +0100
            Re: INFO: rcu_sched detected stalls on CPUs/tasks with `kswapd` and  `mem_cgroup_shrink_node` Donald Buczek <buczek@molgen.mpg.de> - 2016-12-02 10:20 +0100
              Re: INFO: rcu_sched detected stalls on CPUs/tasks with `kswapd` and  `mem_cgroup_shrink_node` Donald Buczek <buczek@molgen.mpg.de> - 2016-12-06 09:40 +0100
          Re: INFO: rcu_sched detected stalls on CPUs/tasks with `kswapd` and  `mem_cgroup_shrink_node` "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2016-11-30 13:00 +0100
            Re: INFO: rcu_sched detected stalls on CPUs/tasks with `kswapd` and  `mem_cgroup_shrink_node` Paul Menzel <pmenzel@molgen.mpg.de> - 2016-11-30 13:40 +0100
              Re: INFO: rcu_sched detected stalls on CPUs/tasks with `kswapd` and  `mem_cgroup_shrink_node` "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2016-11-30 15:40 +0100
          Re: INFO: rcu_sched detected stalls on CPUs/tasks with `kswapd` and  `mem_cgroup_shrink_node` "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2016-11-30 13:00 +0100
            Re: INFO: rcu_sched detected stalls on CPUs/tasks with `kswapd` and  `mem_cgroup_shrink_node` Michal Hocko <mhocko@kernel.org> - 2016-11-30 14:20 +0100
              Re: INFO: rcu_sched detected stalls on CPUs/tasks with `kswapd` and  `mem_cgroup_shrink_node` "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2016-11-30 15:40 +0100
                Re: INFO: rcu_sched detected stalls on CPUs/tasks with `kswapd` and  `mem_cgroup_shrink_node` Peter Zijlstra <peterz@infradead.org> - 2016-11-30 17:40 +0100
                Re: INFO: rcu_sched detected stalls on CPUs/tasks with `kswapd` and  `mem_cgroup_shrink_node` Michal Hocko <mhocko@kernel.org> - 2016-11-30 18:10 +0100
                Re: INFO: rcu_sched detected stalls on CPUs/tasks with `kswapd` and  `mem_cgroup_shrink_node` "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2016-11-30 18:30 +0100
                Re: INFO: rcu_sched detected stalls on CPUs/tasks with `kswapd` and  `mem_cgroup_shrink_node` Michal Hocko <mhocko@kernel.org> - 2016-11-30 18:40 +0100
                Re: INFO: rcu_sched detected stalls on CPUs/tasks with `kswapd` and  `mem_cgroup_shrink_node` Peter Zijlstra <peterz@infradead.org> - 2016-11-30 19:00 +0100
                Re: INFO: rcu_sched detected stalls on CPUs/tasks with `kswapd` and  `mem_cgroup_shrink_node` "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2016-11-30 22:40 +0100
                Re: INFO: rcu_sched detected stalls on CPUs/tasks with `kswapd` and  `mem_cgroup_shrink_node` Peter Zijlstra <peterz@infradead.org> - 2016-12-01 06:40 +0100
                Re: INFO: rcu_sched detected stalls on CPUs/tasks with `kswapd` and  `mem_cgroup_shrink_node` "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2016-12-01 13:50 +0100
                Re: INFO: rcu_sched detected stalls on CPUs/tasks with `kswapd` and  `mem_cgroup_shrink_node` Peter Zijlstra <peterz@infradead.org> - 2016-12-01 17:40 +0100
                Re: INFO: rcu_sched detected stalls on CPUs/tasks with `kswapd` and  `mem_cgroup_shrink_node` "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2016-12-01 18:00 +0100
                Re: INFO: rcu_sched detected stalls on CPUs/tasks with `kswapd` and  `mem_cgroup_shrink_node` Peter Zijlstra <peterz@infradead.org> - 2016-12-01 19:20 +0100
                Re: INFO: rcu_sched detected stalls on CPUs/tasks with `kswapd` and  `mem_cgroup_shrink_node` "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2016-12-01 19:50 +0100
                Re: INFO: rcu_sched detected stalls on CPUs/tasks with `kswapd` and  `mem_cgroup_shrink_node` Peter Zijlstra <peterz@infradead.org> - 2016-12-01 20:00 +0100
                Re: INFO: rcu_sched detected stalls on CPUs/tasks with `kswapd` and  `mem_cgroup_shrink_node` "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2016-11-30 18:10 +0100

csiph-web