Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]


Groups > linux.kernel > #1252780 > unrolled thread

[PATCH] mm,vmscan: Use accurate values for zone_reclaimable() checks

Started byTetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp>
First post2015-10-21 14:30 +0200
Last post2015-10-22 17:40 +0200
Articles 20 on this page of 52 — 5 participants

Back to article view | Back to linux.kernel


Contents

  [PATCH] mm,vmscan: Use accurate values for zone_reclaimable() checks Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp> - 2015-10-21 14:30 +0200
    Re: [PATCH] mm,vmscan: Use accurate values for zone_reclaimable()  checks Michal Hocko <mhocko@kernel.org> - 2015-10-21 15:10 +0200
    Re: [PATCH] mm,vmscan: Use accurate values for zone_reclaimable()  checks Christoph Lameter <cl@linux.com> - 2015-10-21 16:30 +0200
      Re: [PATCH] mm,vmscan: Use accurate values for zone_reclaimable()  checks Michal Hocko <mhocko@kernel.org> - 2015-10-21 16:40 +0200
        Re: [PATCH] mm,vmscan: Use accurate values for zone_reclaimable()  checks Christoph Lameter <cl@linux.com> - 2015-10-21 16:50 +0200
          Re: [PATCH] mm,vmscan: Use accurate values for zone_reclaimable()  checks Michal Hocko <mhocko@kernel.org> - 2015-10-21 17:00 +0200
            Re: [PATCH] mm,vmscan: Use accurate values for zone_reclaimable() checks Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp> - 2015-10-21 17:40 +0200
            Re: [PATCH] mm,vmscan: Use accurate values for zone_reclaimable()  checks Christoph Lameter <cl@linux.com> - 2015-10-21 19:20 +0200
              Re: [PATCH] mm,vmscan: Use accurate values for zone_reclaimable() checks Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp> - 2015-10-22 13:40 +0200
                Re: [PATCH] mm,vmscan: Use accurate values for zone_reclaimable()  checks Christoph Lameter <cl@linux.com> - 2015-10-22 15:40 +0200
                  Re: [PATCH] mm,vmscan: Use accurate values for zone_reclaimable()  checks Tejun Heo <htejun@gmail.com> - 2015-10-22 16:20 +0200
                    Re: [PATCH] mm,vmscan: Use accurate values for zone_reclaimable()  checks Tejun Heo <htejun@gmail.com> - 2015-10-22 16:30 +0200
                      Re: [PATCH] mm,vmscan: Use accurate values for zone_reclaimable()  checks Tejun Heo <htejun@gmail.com> - 2015-10-22 16:30 +0200
                        Re: [PATCH] mm,vmscan: Use accurate values for zone_reclaimable()  checks Christoph Lameter <cl@linux.com> - 2015-10-22 16:30 +0200
                          Re: [PATCH] mm,vmscan: Use accurate values for zone_reclaimable()  checks Tejun Heo <htejun@gmail.com> - 2015-10-22 16:40 +0200
                            Re: [PATCH] mm,vmscan: Use accurate values for zone_reclaimable()  checks Christoph Lameter <cl@linux.com> - 2015-10-22 16:50 +0200
                              Re: [PATCH] mm,vmscan: Use accurate values for zone_reclaimable()  checks Tejun Heo <htejun@gmail.com> - 2015-10-22 17:20 +0200
                                Re: [PATCH] mm,vmscan: Use accurate values for zone_reclaimable()  checks Tejun Heo <htejun@gmail.com> - 2015-10-23 06:30 +0200
                      Re: [PATCH] mm,vmscan: Use accurate values for zone_reclaimable()  checks Christoph Lameter <cl@linux.com> - 2015-10-22 16:30 +0200
                    Re: [PATCH] mm,vmscan: Use accurate values for zone_reclaimable()  checks Christoph Lameter <cl@linux.com> - 2015-10-22 16:30 +0200
                    Re: [PATCH] mm,vmscan: Use accurate values for zone_reclaimable()  checks Michal Hocko <mhocko@kernel.org> - 2015-10-22 17:10 +0200
                      Re: [PATCH] mm,vmscan: Use accurate values for zone_reclaimable()  checks Tejun Heo <htejun@gmail.com> - 2015-10-22 17:20 +0200
                        Re: [PATCH] mm,vmscan: Use accurate values for zone_reclaimable()  checks Christoph Lameter <cl@linux.com> - 2015-10-22 17:40 +0200
                          Re: [PATCH] mm,vmscan: Use accurate values for zone_reclaimable()  checks Michal Hocko <mhocko@kernel.org> - 2015-10-23 10:40 +0200
                            Make vmstat deferrable again (was Re: [PATCH] mm,vmscan: Use accurate  values for zone_reclaimable() checks) Christoph Lameter <cl@linux.com> - 2015-10-23 13:50 +0200
                              Re: Make vmstat deferrable again (was Re: [PATCH] mm,vmscan: Use  accurate values for zone_reclaimable() checks) Sergey Senozhatsky <sergey.senozhatsky@gmail.com> - 2015-10-23 14:10 +0200
                                Re: Make vmstat deferrable again (was Re: [PATCH] mm,vmscan: Use  accurate values for zone_reclaimable() checks) Christoph Lameter <cl@linux.com> - 2015-10-23 16:20 +0200
                                  Re: Make vmstat deferrable again (was Re: [PATCH] mm,vmscan: Use  accurate values for zone_reclaimable() checks) Sergey Senozhatsky <sergey.senozhatsky@gmail.com> - 2015-10-23 17:00 +0200
                                    Re: Make vmstat deferrable again (was Re: [PATCH] mm,vmscan: Use  accurate values for zone_reclaimable() checks) Christoph Lameter <cl@linux.com> - 2015-10-23 18:20 +0200
                        Re: [PATCH] mm,vmscan: Use accurate values for zone_reclaimable()  checks Tejun Heo <htejun@gmail.com> - 2015-10-22 17:40 +0200
                          Re: [PATCH] mm,vmscan: Use accurate values for zone_reclaimable()  checks Michal Hocko <mhocko@kernel.org> - 2015-10-22 17:50 +0200
                            Re: [PATCH] mm,vmscan: Use accurate values for zone_reclaimable()  checks Tejun Heo <htejun@gmail.com> - 2015-10-22 20:50 +0200
                              Re: [PATCH] mm,vmscan: Use accurate values for zone_reclaimable()checks Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp> - 2015-10-22 23:50 +0200
                                Re: [PATCH] mm,vmscan: Use accurate values for  zone_reclaimable()checks Tejun Heo <htejun@gmail.com> - 2015-10-23 00:50 +0200
                                Re: [PATCH] mm,vmscan: Use accurate values for  zone_reclaimable()checks Michal Hocko <mhocko@kernel.org> - 2015-10-23 10:40 +0200
                                  Re: [PATCH] mm,vmscan: Use accurate values for  zone_reclaimable()checks Tejun Heo <htejun@gmail.com> - 2015-10-23 12:40 +0200
                              Re: [PATCH] mm,vmscan: Use accurate values for zone_reclaimable()  checks Michal Hocko <mhocko@kernel.org> - 2015-10-23 10:40 +0200
                                Re: [PATCH] mm,vmscan: Use accurate values for zone_reclaimable()  checks Tejun Heo <htejun@gmail.com> - 2015-10-23 12:40 +0200
                                  Re: [PATCH] mm,vmscan: Use accurate values for zone_reclaimable()  checks Michal Hocko <mhocko@kernel.org> - 2015-10-23 13:20 +0200
                                    Re: [PATCH] mm,vmscan: Use accurate values for zone_reclaimable() checks Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp> - 2015-10-23 14:30 +0200
                                      Re: [PATCH] mm,vmscan: Use accurate values for zone_reclaimable()  checks Tejun Heo <htejun@gmail.com> - 2015-10-23 20:30 +0200
                                        Re: [PATCH] mm,vmscan: Use accurate values for zone_reclaimable() checks Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp> - 2015-10-25 12:00 +0100
                                          Re: [PATCH] mm,vmscan: Use accurate values for zone_reclaimable()  checks Tejun Heo <htejun@gmail.com> - 2015-10-25 23:50 +0100
                                          Re: [PATCH] mm,vmscan: Use accurate values for zone_reclaimable()  checks Michal Hocko <mhocko@kernel.org> - 2015-10-27 10:30 +0100
                                            Re: [PATCH] mm,vmscan: Use accurate values for zone_reclaimable()  checks Tejun Heo <htejun@gmail.com> - 2015-10-27 12:00 +0100
                                              Re: [PATCH] mm,vmscan: Use accurate values for zone_reclaimable()  checks Michal Hocko <mhocko@kernel.org> - 2015-10-27 13:10 +0100
                                    Re: [PATCH] mm,vmscan: Use accurate values for zone_reclaimable()  checks Tejun Heo <htejun@gmail.com> - 2015-10-23 20:30 +0200
                                      Re: [PATCH] mm,vmscan: Use accurate values for zone_reclaimable()  checks Michal Hocko <mhocko@kernel.org> - 2015-10-27 10:20 +0100
                                        Re: [PATCH] mm,vmscan: Use accurate values for zone_reclaimable()  checks Tejun Heo <htejun@gmail.com> - 2015-10-27 12:00 +0100
                                        Re: [PATCH] mm,vmscan: Use accurate values for zone_reclaimable()checks Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp> - 2015-10-27 12:10 +0100
                                          Re: [PATCH] mm,vmscan: Use accurate values for  zone_reclaimable()checks Tejun Heo <htejun@gmail.com> - 2015-10-27 12:40 +0100
                        Re: [PATCH] mm,vmscan: Use accurate values for zone_reclaimable()  checks Michal Hocko <mhocko@kernel.org> - 2015-10-22 17:40 +0200

Page 2 of 3 — ← Prev page 1 [2] 3  Next page →


#1253896 — Re: [PATCH] mm,vmscan: Use accurate values for zone_reclaimable() checks

FromMichal Hocko <mhocko@kernel.org>
Date2015-10-22 17:10 +0200
SubjectRe: [PATCH] mm,vmscan: Use accurate values for zone_reclaimable() checks
Message-ID<qmpF8-678-17@gated-at.bofh.it>
In reply to#1253854
On Thu 22-10-15 23:09:44, Tejun Heo wrote:
> On Thu, Oct 22, 2015 at 08:39:11AM -0500, Christoph Lameter wrote:
> > On Thu, 22 Oct 2015, Tetsuo Handa wrote:
> > 
> > > The problem would be that the "struct task_struct" to execute vmstat_update
> > > job does not exist, and will not be able to create one on demand because we
> > > are stuck at __GFP_WAIT allocation. Therefore adding a dedicated kernel
> > > thread for vmstat_update job would work. But ...
> > 
> > Yuck. Can someone please get this major screwup out of the work queue
> > subsystem? Tejun?
> 
> Hmmm?  Just use a dedicated workqueue with WQ_MEM_RECLAIM.

Do I get it right that if vmstat_update has its own workqueue with
WQ_MEM_RECLAIM then there is a _guarantee_ that the rescuer will always
be able to process vmstat_update work from the requested CPU?

That should be sufficient because vmstat_update doesn't sleep on
allocation. I agree that this would be a more appropriate fix.

-- 
Michal Hocko
SUSE Labs
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]


#1253900 — Re: [PATCH] mm,vmscan: Use accurate values for zone_reclaimable() checks

FromTejun Heo <htejun@gmail.com>
Date2015-10-22 17:20 +0200
SubjectRe: [PATCH] mm,vmscan: Use accurate values for zone_reclaimable() checks
Message-ID<qmpON-6iz-9@gated-at.bofh.it>
In reply to#1253896
On Thu, Oct 22, 2015 at 05:06:23PM +0200, Michal Hocko wrote:
> Do I get it right that if vmstat_update has its own workqueue with
> WQ_MEM_RECLAIM then there is a _guarantee_ that the rescuer will always
> be able to process vmstat_update work from the requested CPU?

Yeah.

> That should be sufficient because vmstat_update doesn't sleep on
> allocation. I agree that this would be a more appropriate fix.

The problem seems to be reclaim path busy looping waiting for
vmstat_update and workqueue thinking that the work item must be making
forward-progress and thus not starting the next work item.

Thanks.

-- 
tejun
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]


#1253915 — Re: [PATCH] mm,vmscan: Use accurate values for zone_reclaimable() checks

FromChristoph Lameter <cl@linux.com>
Date2015-10-22 17:40 +0200
SubjectRe: [PATCH] mm,vmscan: Use accurate values for zone_reclaimable() checks
Message-ID<qmq8a-6GG-5@gated-at.bofh.it>
In reply to#1253900
Ok that also makes me rethink commit
ba4877b9ca51f80b5d30f304a46762f0509e1635 which seems to be a similar fix
this time related to idle mode not updating the counters.

Could we fix that by folding the counters before going to idle mode?

That fix seems to now create 2 separate application interuptions because
the vmstat update is not deferred anymore to occur with other events.

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]


#1254383 — Re: [PATCH] mm,vmscan: Use accurate values for zone_reclaimable() checks

FromMichal Hocko <mhocko@kernel.org>
Date2015-10-23 10:40 +0200
SubjectRe: [PATCH] mm,vmscan: Use accurate values for zone_reclaimable() checks
Message-ID<qmG3g-4qs-7@gated-at.bofh.it>
In reply to#1253915
On Thu 22-10-15 10:33:20, Christoph Lameter wrote:
> Ok that also makes me rethink commit
> ba4877b9ca51f80b5d30f304a46762f0509e1635 which seems to be a similar fix
> this time related to idle mode not updating the counters.
> 
> Could we fix that by folding the counters before going to idle mode?

This would work as well.

-- 
Michal Hocko
SUSE Labs
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]


#1254512 — Make vmstat deferrable again (was Re: [PATCH] mm,vmscan: Use accurate values for zone_reclaimable() checks)

FromChristoph Lameter <cl@linux.com>
Date2015-10-23 13:50 +0200
SubjectMake vmstat deferrable again (was Re: [PATCH] mm,vmscan: Use accurate values for zone_reclaimable() checks)
Message-ID<qmJ19-lc-27@gated-at.bofh.it>
In reply to#1254383
On Fri, 23 Oct 2015, Michal Hocko wrote:

> On Thu 22-10-15 10:33:20, Christoph Lameter wrote:
> > Ok that also makes me rethink commit
> > ba4877b9ca51f80b5d30f304a46762f0509e1635 which seems to be a similar fix
> > this time related to idle mode not updating the counters.
> >
> > Could we fix that by folding the counters before going to idle mode?
>
> This would work as well.

Is this ok?


Subject: Fix vmstat: make vmstat_updater deferrable again and shut down on idle

Currently the vmstat updater is not deferrable as a result of commit
ba4877b9ca51f80b5d30f304a46762f0509e1635. This in turn can cause multiple
interruptions of the applications because the vmstat updater may run at
different times than tick processing. No good.

Make vmstate_update deferrable again and provide a function that
shuts down the vmstat updater when we go idle by folding the differentials.
Shut it down from the load average calculation logic introduced by nohz.

Note that the shepherd thread will continue scanning the differentials
from another processor and will reenable the vmstat workers if it
detects any changes.

Fixes: ba4877b9ca51f80b5d30f304a46762f0509e1635 (do not use deferrable delay)
Signed-off-by: Christoph Lameter <cl@linux.com>

Index: linux/mm/vmstat.c
===================================================================
--- linux.orig/mm/vmstat.c
+++ linux/mm/vmstat.c
@@ -1395,6 +1395,20 @@ static void vmstat_update(struct work_st
 }

 /*
+ * Switch off vmstat processing and then fold all the remaining differentials
+ * until the diffs stay at zero. The function is used by NOHZ and can only be
+ * invoked when tick processing is not active.
+ */
+void quiet_vmstat(void)
+{
+	do {
+		if (!cpumask_test_and_set_cpu(smp_processor_id(), cpu_stat_off))
+			cancel_delayed_work(this_cpu_ptr(&vmstat_work));
+
+	} while (refresh_cpu_vm_stats());
+}
+
+/*
  * Check if the diffs for a certain cpu indicate that
  * an update is needed.
  */
@@ -1426,7 +1440,7 @@ static bool need_update(int cpu)
  */
 static void vmstat_shepherd(struct work_struct *w);

-static DECLARE_DELAYED_WORK(shepherd, vmstat_shepherd);
+static DECLARE_DEFERRABLE_WORK(shepherd, vmstat_shepherd);

 static void vmstat_shepherd(struct work_struct *w)
 {
Index: linux/include/linux/vmstat.h
===================================================================
--- linux.orig/include/linux/vmstat.h
+++ linux/include/linux/vmstat.h
@@ -211,6 +211,7 @@ extern void __inc_zone_state(struct zone
 extern void dec_zone_state(struct zone *, enum zone_stat_item);
 extern void __dec_zone_state(struct zone *, enum zone_stat_item);

+void quiet_vmstat(void);
 void cpu_vm_stats_fold(int cpu);
 void refresh_zone_stat_thresholds(void);

@@ -272,6 +273,7 @@ static inline void __dec_zone_page_state
 static inline void refresh_cpu_vm_stats(int cpu) { }
 static inline void refresh_zone_stat_thresholds(void) { }
 static inline void cpu_vm_stats_fold(int cpu) { }
+static inline void quiet_vmstat(void) { }

 static inline void drain_zonestat(struct zone *zone,
 			struct per_cpu_pageset *pset) { }
Index: linux/kernel/sched/loadavg.c
===================================================================
--- linux.orig/kernel/sched/loadavg.c
+++ linux/kernel/sched/loadavg.c
@@ -191,6 +191,8 @@ void calc_load_enter_idle(void)

 		atomic_long_add(delta, &calc_load_idle[idx]);
 	}
+	/* Fold the current vmstat counters and disable vmstat updater */
+	quiet_vmstat();
 }

 void calc_load_exit_idle(void)
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]


#1254522 — Re: Make vmstat deferrable again (was Re: [PATCH] mm,vmscan: Use accurate values for zone_reclaimable() checks)

FromSergey Senozhatsky <sergey.senozhatsky@gmail.com>
Date2015-10-23 14:10 +0200
SubjectRe: Make vmstat deferrable again (was Re: [PATCH] mm,vmscan: Use accurate values for zone_reclaimable() checks)
Message-ID<qmJku-X9-15@gated-at.bofh.it>
In reply to#1254512
On (10/23/15 06:43), Christoph Lameter wrote:
> Is this ok?

kernel/sched/loadavg.c: In function ‘calc_load_enter_idle’:
kernel/sched/loadavg.c:195:2: error: implicit declaration of function ‘quiet_vmstat’ [-Werror=implicit-function-declaration]
  quiet_vmstat();
    ^

> Subject: Fix vmstat: make vmstat_updater deferrable again and shut down on idle
> 
> Currently the vmstat updater is not deferrable as a result of commit
> ba4877b9ca51f80b5d30f304a46762f0509e1635. This in turn can cause multiple
> interruptions of the applications because the vmstat updater may run at
> different times than tick processing. No good.
> 
> Make vmstate_update deferrable again and provide a function that
> shuts down the vmstat updater when we go idle by folding the differentials.
> Shut it down from the load average calculation logic introduced by nohz.
> 
> Note that the shepherd thread will continue scanning the differentials
> from another processor and will reenable the vmstat workers if it
> detects any changes.
> 
> Fixes: ba4877b9ca51f80b5d30f304a46762f0509e1635 (do not use deferrable delay)
> Signed-off-by: Christoph Lameter <cl@linux.com>
> 
> Index: linux/mm/vmstat.c
> ===================================================================
> --- linux.orig/mm/vmstat.c
> +++ linux/mm/vmstat.c
> @@ -1395,6 +1395,20 @@ static void vmstat_update(struct work_st
>  }
> 
>  /*
> + * Switch off vmstat processing and then fold all the remaining differentials
> + * until the diffs stay at zero. The function is used by NOHZ and can only be
> + * invoked when tick processing is not active.
> + */
> +void quiet_vmstat(void)
> +{
> +	do {
> +		if (!cpumask_test_and_set_cpu(smp_processor_id(), cpu_stat_off))
> +			cancel_delayed_work(this_cpu_ptr(&vmstat_work));

shouldn't preemption be disable for smp_processor_id() here?

	-ss
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]


#1254637 — Re: Make vmstat deferrable again (was Re: [PATCH] mm,vmscan: Use accurate values for zone_reclaimable() checks)

FromChristoph Lameter <cl@linux.com>
Date2015-10-23 16:20 +0200
SubjectRe: Make vmstat deferrable again (was Re: [PATCH] mm,vmscan: Use accurate values for zone_reclaimable() checks)
Message-ID<qmLmi-3Q4-23@gated-at.bofh.it>
In reply to#1254522

[Multipart message — attachments visible in raw view] — view raw

On Fri, 23 Oct 2015, Sergey Senozhatsky wrote:

> On (10/23/15 06:43), Christoph Lameter wrote:
> > Is this ok?
>
> kernel/sched/loadavg.c: In function ‘calc_load_enter_idle’:
> kernel/sched/loadavg.c:195:2: error: implicit declaration of function ‘quiet_vmstat’ [-Werror=implicit-function-declaration]
>   quiet_vmstat();
>     ^

Oww... Not good to do that in the scheduler. Ok new patch follows that
does the call from tick_nohz_stop_sched_tick. Hopefully that is the right
location to call quiet_vmstat().

> > +		if (!cpumask_test_and_set_cpu(smp_processor_id(), cpu_stat_off))
> > +			cancel_delayed_work(this_cpu_ptr(&vmstat_work));
>
> shouldn't preemption be disable for smp_processor_id() here?

Preemption is disabled when quiet_vmstat() is called.



Subject: Fix vmstat: make vmstat_updater deferrable again and shut down on idle V2

V1->V2
 - Call vmstat_quiet from tick_nohz_stop_sched_tick() instead.

Currently the vmstat updater is not deferrable as a result of commit
ba4877b9ca51f80b5d30f304a46762f0509e1635. This in turn can cause multiple
interruptions of the applications because the vmstat updater may run at
different times than tick processing. No good.

Make vmstate_update deferrable again and provide a function that
shuts down the vmstat updater when we go idle by folding the differentials.
Shut it down from the load average calculation logic introduced by nohz.

Note that the shepherd thread will continue scanning the differentials
from another processor and will reenable the vmstat workers if it
detects any changes.

Fixes: ba4877b9ca51f80b5d30f304a46762f0509e1635 (do not use deferrable delay)
Signed-off-by: Christoph Lameter <cl@linux.com>

Index: linux/mm/vmstat.c
===================================================================
--- linux.orig/mm/vmstat.c
+++ linux/mm/vmstat.c
@@ -1395,6 +1395,20 @@ static void vmstat_update(struct work_st
 }

 /*
+ * Switch off vmstat processing and then fold all the remaining differentials
+ * until the diffs stay at zero. The function is used by NOHZ and can only be
+ * invoked when tick processing is not active.
+ */
+void quiet_vmstat(void)
+{
+	do {
+		if (!cpumask_test_and_set_cpu(smp_processor_id(), cpu_stat_off))
+			cancel_delayed_work(this_cpu_ptr(&vmstat_work));
+
+	} while (refresh_cpu_vm_stats());
+}
+
+/*
  * Check if the diffs for a certain cpu indicate that
  * an update is needed.
  */
@@ -1426,7 +1440,7 @@ static bool need_update(int cpu)
  */
 static void vmstat_shepherd(struct work_struct *w);

-static DECLARE_DELAYED_WORK(shepherd, vmstat_shepherd);
+static DECLARE_DEFERRABLE_WORK(shepherd, vmstat_shepherd);

 static void vmstat_shepherd(struct work_struct *w)
 {
Index: linux/include/linux/vmstat.h
===================================================================
--- linux.orig/include/linux/vmstat.h
+++ linux/include/linux/vmstat.h
@@ -211,6 +211,7 @@ extern void __inc_zone_state(struct zone
 extern void dec_zone_state(struct zone *, enum zone_stat_item);
 extern void __dec_zone_state(struct zone *, enum zone_stat_item);

+void quiet_vmstat(void);
 void cpu_vm_stats_fold(int cpu);
 void refresh_zone_stat_thresholds(void);

@@ -272,6 +273,7 @@ static inline void __dec_zone_page_state
 static inline void refresh_cpu_vm_stats(int cpu) { }
 static inline void refresh_zone_stat_thresholds(void) { }
 static inline void cpu_vm_stats_fold(int cpu) { }
+static inline void quiet_vmstat(void) { }

 static inline void drain_zonestat(struct zone *zone,
 			struct per_cpu_pageset *pset) { }
Index: linux/kernel/time/tick-sched.c
===================================================================
--- linux.orig/kernel/time/tick-sched.c
+++ linux/kernel/time/tick-sched.c
@@ -667,6 +667,7 @@ static ktime_t tick_nohz_stop_sched_tick
 	 */
 	if (!ts->tick_stopped) {
 		nohz_balance_enter_idle(cpu);
+		quiet_vmstat();
 		calc_load_enter_idle();

 		ts->last_tick = hrtimer_get_expires(&ts->sched_timer);

[toc] | [prev] | [next] | [standalone]


#1254654 — Re: Make vmstat deferrable again (was Re: [PATCH] mm,vmscan: Use accurate values for zone_reclaimable() checks)

FromSergey Senozhatsky <sergey.senozhatsky@gmail.com>
Date2015-10-23 17:00 +0200
SubjectRe: Make vmstat deferrable again (was Re: [PATCH] mm,vmscan: Use accurate values for zone_reclaimable() checks)
Message-ID<qmLZ1-4zk-21@gated-at.bofh.it>
In reply to#1254637
On (10/23/15 09:12), Christoph Lameter wrote:
[..]
> > > +		if (!cpumask_test_and_set_cpu(smp_processor_id(), cpu_stat_off))
> > > +			cancel_delayed_work(this_cpu_ptr(&vmstat_work));
> >
> > shouldn't preemption be disable for smp_processor_id() here?
> 
> Preemption is disabled when quiet_vmstat() is called.
> 

cond_resched()

[   29.607725] BUG: sleeping function called from invalid context at mm/vmstat.c:487
[   29.607729] in_atomic(): 1, irqs_disabled(): 1, pid: 0, name: swapper/7
[   29.607731] no locks held by swapper/7/0.
[   29.607732] irq event stamp: 48932
[   29.607733] hardirqs last  enabled at (48931): [<ffffffff813b246a>] _raw_spin_unlock_irq+0x2c/0x37
[   29.607739] hardirqs last disabled at (48932): [<ffffffff810a3fec>] tick_nohz_idle_enter+0x3c/0x5f
[   29.607743] softirqs last  enabled at (48924): [<ffffffff81041fd8>] __do_softirq+0x2bb/0x3a9
[   29.607747] softirqs last disabled at (48893): [<ffffffff810422a7>] irq_exit+0x41/0x95
[   29.607752] CPU: 7 PID: 0 Comm: swapper/7 Not tainted 4.3.0-rc6-next-20151022-dbg-00003-g01184ff-dirty #261
[   29.607754]  0000000000000000 ffff88041dae7da0 ffffffff811dd4f3 ffff88041dacd100
[   29.607756]  ffff88041dae7dc8 ffffffff8105f144 ffffffff8169f800 0000000000000000
[   29.607759]  0000000000000007 ffff88041dae7e70 ffffffff811040b1 0000000000000002
[   29.607761] Call Trace:
[   29.607767]  [<ffffffff811dd4f3>] dump_stack+0x4b/0x63
[   29.607770]  [<ffffffff8105f144>] ___might_sleep+0x1e7/0x1ee
[   29.607773]  [<ffffffff811040b1>] refresh_cpu_vm_stats+0x8b/0xb5
[   29.607776]  [<ffffffff81104f4c>] quiet_vmstat+0x3a/0x41
[   29.607778]  [<ffffffff810a3ccf>] __tick_nohz_idle_enter+0x292/0x410
[   29.607781]  [<ffffffff810a4007>] tick_nohz_idle_enter+0x57/0x5f
[   29.607784]  [<ffffffff81076d8b>] cpu_startup_entry+0x36/0x330
[   29.607788]  [<ffffffff81028821>] start_secondary+0xf3/0xf6



by the way, tick_nohz_stop_sched_tick() receives cpu from __tick_nohz_idle_enter().
do you want to pass it to quiet_vmstat()?

	if (!ts->tick_stopped) {
		nohz_balance_enter_idle(cpu);
-		quiet_vmstat();
+		quiet_vmstat(cpu);
		calc_load_enter_idle();

	-ss
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]


#1254697 — Re: Make vmstat deferrable again (was Re: [PATCH] mm,vmscan: Use accurate values for zone_reclaimable() checks)

FromChristoph Lameter <cl@linux.com>
Date2015-10-23 18:20 +0200
SubjectRe: Make vmstat deferrable again (was Re: [PATCH] mm,vmscan: Use accurate values for zone_reclaimable() checks)
Message-ID<qmNeq-6xZ-21@gated-at.bofh.it>
In reply to#1254654
On Fri, 23 Oct 2015, Sergey Senozhatsky wrote:

> by the way, tick_nohz_stop_sched_tick() receives cpu from __tick_nohz_idle_enter().
> do you want to pass it to quiet_vmstat()?

No this is quite wrong at this point. quiet_vmstat() needs to be called
from the cpu going into idle state.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]


#1253917 — Re: [PATCH] mm,vmscan: Use accurate values for zone_reclaimable() checks

FromTejun Heo <htejun@gmail.com>
Date2015-10-22 17:40 +0200
SubjectRe: [PATCH] mm,vmscan: Use accurate values for zone_reclaimable() checks
Message-ID<qmq8a-6GG-19@gated-at.bofh.it>
In reply to#1253900
On Thu, Oct 22, 2015 at 05:35:59PM +0200, Michal Hocko wrote:
> But that shouldn't happen because the allocation path does cond_resched
> even when nothing is really reclaimable (e.g. wait_iff_congested from
> __alloc_pages_slowpath).

cond_resched() isn't enough.  The work item should go !RUNNING, not
just yielding.

-- 
tejun
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]


#1253933 — Re: [PATCH] mm,vmscan: Use accurate values for zone_reclaimable() checks

FromMichal Hocko <mhocko@kernel.org>
Date2015-10-22 17:50 +0200
SubjectRe: [PATCH] mm,vmscan: Use accurate values for zone_reclaimable() checks
Message-ID<qmqhQ-6SK-29@gated-at.bofh.it>
In reply to#1253917
On Fri 23-10-15 00:37:03, Tejun Heo wrote:
> On Thu, Oct 22, 2015 at 05:35:59PM +0200, Michal Hocko wrote:
> > But that shouldn't happen because the allocation path does cond_resched
> > even when nothing is really reclaimable (e.g. wait_iff_congested from
> > __alloc_pages_slowpath).
> 
> cond_resched() isn't enough.  The work item should go !RUNNING, not
> just yielding.

I am confused. What makes rescuer to not run? Nothing seems to be
hogging CPUs, we are just out of workers which are loopin in the
allocator but that is preemptible context.
-- 
Michal Hocko
SUSE Labs
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]


#1254080 — Re: [PATCH] mm,vmscan: Use accurate values for zone_reclaimable() checks

FromTejun Heo <htejun@gmail.com>
Date2015-10-22 20:50 +0200
SubjectRe: [PATCH] mm,vmscan: Use accurate values for zone_reclaimable() checks
Message-ID<qmt61-2wa-15@gated-at.bofh.it>
In reply to#1253933
On Thu, Oct 22, 2015 at 05:49:22PM +0200, Michal Hocko wrote:
> I am confused. What makes rescuer to not run? Nothing seems to be
> hogging CPUs, we are just out of workers which are loopin in the
> allocator but that is preemptible context.

It's concurrency management.  Workqueue thinks that the pool is making
positive forward progress and doesn't schedule anything else for
execution while that work item is burning cpu cycles.

Thanks.

-- 
tejun
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]


#1254189 — Re: [PATCH] mm,vmscan: Use accurate values for zone_reclaimable()checks

FromTetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp>
Date2015-10-22 23:50 +0200
SubjectRe: [PATCH] mm,vmscan: Use accurate values for zone_reclaimable()checks
Message-ID<qmvUe-6C3-9@gated-at.bofh.it>
In reply to#1254080
Tejun Heo wrote:
> On Thu, Oct 22, 2015 at 05:49:22PM +0200, Michal Hocko wrote:
> > I am confused. What makes rescuer to not run? Nothing seems to be
> > hogging CPUs, we are just out of workers which are loopin in the
> > allocator but that is preemptible context.
> 
> It's concurrency management.  Workqueue thinks that the pool is making
> positive forward progress and doesn't schedule anything else for
> execution while that work item is burning cpu cycles.

Then, isn't below change easier to backport which will also alleviate
needlessly burning CPU cycles?

--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -3385,6 +3385,7 @@ retry:
 	((gfp_mask & __GFP_REPEAT) && pages_reclaimed < (1 << order))) {
 		/* Wait for some write requests to complete then retry */
 		wait_iff_congested(ac->preferred_zone, BLK_RW_ASYNC, HZ/50);
+		schedule_timeout_uninterruptible(1);
 		goto retry;
 	}
 
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]


#1254216 — Re: [PATCH] mm,vmscan: Use accurate values for zone_reclaimable()checks

FromTejun Heo <htejun@gmail.com>
Date2015-10-23 00:50 +0200
SubjectRe: [PATCH] mm,vmscan: Use accurate values for zone_reclaimable()checks
Message-ID<qmwQi-7Ye-11@gated-at.bofh.it>
In reply to#1254189
Hello,

On Fri, Oct 23, 2015 at 06:42:43AM +0900, Tetsuo Handa wrote:
> Then, isn't below change easier to backport which will also alleviate
> needlessly burning CPU cycles?
> 
> --- a/mm/page_alloc.c
> +++ b/mm/page_alloc.c
> @@ -3385,6 +3385,7 @@ retry:
>  	((gfp_mask & __GFP_REPEAT) && pages_reclaimed < (1 << order))) {
>  		/* Wait for some write requests to complete then retry */
>  		wait_iff_congested(ac->preferred_zone, BLK_RW_ASYNC, HZ/50);
> +		schedule_timeout_uninterruptible(1);
>  		goto retry;
>  	}

Yeah, that works too.  It should still be put on a dedicated wq with
MEM_RECLAIM tho.

Thanks.

-- 
tejun
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]


#1254385 — Re: [PATCH] mm,vmscan: Use accurate values for zone_reclaimable()checks

FromMichal Hocko <mhocko@kernel.org>
Date2015-10-23 10:40 +0200
SubjectRe: [PATCH] mm,vmscan: Use accurate values for zone_reclaimable()checks
Message-ID<qmG3g-4qs-11@gated-at.bofh.it>
In reply to#1254189
On Fri 23-10-15 06:42:43, Tetsuo Handa wrote:
> Tejun Heo wrote:
> > On Thu, Oct 22, 2015 at 05:49:22PM +0200, Michal Hocko wrote:
> > > I am confused. What makes rescuer to not run? Nothing seems to be
> > > hogging CPUs, we are just out of workers which are loopin in the
> > > allocator but that is preemptible context.
> > 
> > It's concurrency management.  Workqueue thinks that the pool is making
> > positive forward progress and doesn't schedule anything else for
> > execution while that work item is burning cpu cycles.
> 
> Then, isn't below change easier to backport which will also alleviate
> needlessly burning CPU cycles?

This is quite obscure. If the vmstat_update fix needs workqueue tweaks
as well then I would vote for your original patch which is clear,
straightforward and easy to backport.

If WQ_MEM_RECLAIM can really guarantee one worker as described in the
documentation then I agree that fixing vmstat is a better fix. But that
doesn't seem to be the case currently.
 
> --- a/mm/page_alloc.c
> +++ b/mm/page_alloc.c
> @@ -3385,6 +3385,7 @@ retry:
>  	((gfp_mask & __GFP_REPEAT) && pages_reclaimed < (1 << order))) {
>  		/* Wait for some write requests to complete then retry */
>  		wait_iff_congested(ac->preferred_zone, BLK_RW_ASYNC, HZ/50);
> +		schedule_timeout_uninterruptible(1);
>  		goto retry;
>  	}
>  
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

-- 
Michal Hocko
SUSE Labs
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]


#1254472 — Re: [PATCH] mm,vmscan: Use accurate values for zone_reclaimable()checks

FromTejun Heo <htejun@gmail.com>
Date2015-10-23 12:40 +0200
SubjectRe: [PATCH] mm,vmscan: Use accurate values for zone_reclaimable()checks
Message-ID<qmHVo-7dR-31@gated-at.bofh.it>
In reply to#1254385
On Fri, Oct 23, 2015 at 10:36:12AM +0200, Michal Hocko wrote:
> If WQ_MEM_RECLAIM can really guarantee one worker as described in the
> documentation then I agree that fixing vmstat is a better fix. But that
> doesn't seem to be the case currently.

It does.

-- 
tejun
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]


#1254382 — Re: [PATCH] mm,vmscan: Use accurate values for zone_reclaimable() checks

FromMichal Hocko <mhocko@kernel.org>
Date2015-10-23 10:40 +0200
SubjectRe: [PATCH] mm,vmscan: Use accurate values for zone_reclaimable() checks
Message-ID<qmG3g-4qs-3@gated-at.bofh.it>
In reply to#1254080
On Fri 23-10-15 03:42:26, Tejun Heo wrote:
> On Thu, Oct 22, 2015 at 05:49:22PM +0200, Michal Hocko wrote:
> > I am confused. What makes rescuer to not run? Nothing seems to be
> > hogging CPUs, we are just out of workers which are loopin in the
> > allocator but that is preemptible context.
> 
> It's concurrency management.  Workqueue thinks that the pool is making
> positive forward progress and doesn't schedule anything else for
> execution while that work item is burning cpu cycles.

Ohh, OK I can see wq_worker_sleeping now. I've missed your point in
other email, sorry about that. But now I am wondering whether this
is an intended behavior. The documentation says:
  WQ_MEM_RECLAIM

        All wq which might be used in the memory reclaim paths _MUST_
        have this flag set.  The wq is guaranteed to have at least one
        execution context regardless of memory pressure.

Which doesn't seem to be true currently, right? Now I can see your patch
to introduce WQ_IMMEDIATE but I am wondering which WQ_MEM_RECLAIM users
could do without WQ_IMMEDIATE? I mean all current workers might be
looping in the page allocator and it seems possible that WQ_MEM_RECLAIM
work items might be waiting behind them so they cannot help to relieve
the memory pressure. This doesn't sound right to me. Or I am completely
confused and still fail to understand what is WQ_MEM_RECLAIM supposed to
be used for.
-- 
Michal Hocko
SUSE Labs
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]


#1254463 — Re: [PATCH] mm,vmscan: Use accurate values for zone_reclaimable() checks

FromTejun Heo <htejun@gmail.com>
Date2015-10-23 12:40 +0200
SubjectRe: [PATCH] mm,vmscan: Use accurate values for zone_reclaimable() checks
Message-ID<qmHVn-7dR-5@gated-at.bofh.it>
In reply to#1254382
Hello, Michal.

On Fri, Oct 23, 2015 at 10:33:16AM +0200, Michal Hocko wrote:
> Ohh, OK I can see wq_worker_sleeping now. I've missed your point in
> other email, sorry about that. But now I am wondering whether this
> is an intended behavior. The documentation says:

This is.

>   WQ_MEM_RECLAIM
> 
>         All wq which might be used in the memory reclaim paths _MUST_
>         have this flag set.  The wq is guaranteed to have at least one
>         execution context regardless of memory pressure.
> 
> Which doesn't seem to be true currently, right? Now I can see your patch

It is true.

> to introduce WQ_IMMEDIATE but I am wondering which WQ_MEM_RECLAIM users
> could do without WQ_IMMEDIATE? I mean all current workers might be
> looping in the page allocator and it seems possible that WQ_MEM_RECLAIM
> work items might be waiting behind them so they cannot help to relieve
> the memory pressure. This doesn't sound right to me. Or I am completely
> confused and still fail to understand what is WQ_MEM_RECLAIM supposed to
> be used for.

It guarantees that there always is enough execution resource to
execute a work item from that workqueue.  The problem here is not lack
of execution resource but concurrency management misunderstanding the
situation.  This also can be fixed by teaching concurrency management
to be a bit smarter - e.g. if a work item is burning a lot of CPU
cycles continuously or pool hasn't finished a work item over a certain
amount of time, automatically ignore the in-flight work item for the
purpose of concurrency management; however, this sort of inter-work
item busy waits are so extremely rare and undesirable that I'm not
sure the added complexity would be worthwhile.

Thanks.

-- 
tejun
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]


#1254490 — Re: [PATCH] mm,vmscan: Use accurate values for zone_reclaimable() checks

FromMichal Hocko <mhocko@kernel.org>
Date2015-10-23 13:20 +0200
SubjectRe: [PATCH] mm,vmscan: Use accurate values for zone_reclaimable() checks
Message-ID<qmIy5-8dX-9@gated-at.bofh.it>
In reply to#1254463
On Fri 23-10-15 19:36:30, Tejun Heo wrote:
> Hello, Michal.
> 
> On Fri, Oct 23, 2015 at 10:33:16AM +0200, Michal Hocko wrote:
> > Ohh, OK I can see wq_worker_sleeping now. I've missed your point in
> > other email, sorry about that. But now I am wondering whether this
> > is an intended behavior. The documentation says:
> 
> This is.
> 
> >   WQ_MEM_RECLAIM
> > 
> >         All wq which might be used in the memory reclaim paths _MUST_
> >         have this flag set.  The wq is guaranteed to have at least one
> >         execution context regardless of memory pressure.
> > 
> > Which doesn't seem to be true currently, right? Now I can see your patch
> 
> It is true.
> 
> > to introduce WQ_IMMEDIATE but I am wondering which WQ_MEM_RECLAIM users
> > could do without WQ_IMMEDIATE? I mean all current workers might be
> > looping in the page allocator and it seems possible that WQ_MEM_RECLAIM
> > work items might be waiting behind them so they cannot help to relieve
> > the memory pressure. This doesn't sound right to me. Or I am completely
> > confused and still fail to understand what is WQ_MEM_RECLAIM supposed to
> > be used for.
> 
> It guarantees that there always is enough execution resource to
> execute a work item from that workqueue. 

OK, strictly speaking the rescuer is there but it is kind of pointless
if it doesn't fire up and do a work.

> The problem here is not lack
> of execution resource but concurrency management misunderstanding the
> situation. 

And this sounds like a bug to me.

> This also can be fixed by teaching concurrency management
> to be a bit smarter - e.g. if a work item is burning a lot of CPU
> cycles continuously or pool hasn't finished a work item over a certain
> amount of time, automatically ignore the in-flight work item for the
> purpose of concurrency management; however, this sort of inter-work
> item busy waits are so extremely rare and undesirable that I'm not
> sure the added complexity would be worthwhile.

Don't we have some IO related paths which would suffer from the same
problem. I haven't checked all the WQ_MEM_RECLAIM users but from the
name I would expect they _do_ participate in the reclaim and so they
should be able to make a progress. Now if your new IMMEDIATE flag will
guarantee that then I would argue that it should be implicit for
WQ_MEM_RECLAIM otherwise we always risk a similar situation. What would
be a counter argument for doing that?
-- 
Michal Hocko
SUSE Labs
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]


#1254541

FromTetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp>
Date2015-10-23 14:30 +0200
Message-ID<qmJDR-1jR-19@gated-at.bofh.it>
In reply to#1254490
Michal Hocko wrote:
> On Fri 23-10-15 19:36:30, Tejun Heo wrote:
> > Hello, Michal.
> > 
> > On Fri, Oct 23, 2015 at 10:33:16AM +0200, Michal Hocko wrote:
> > > Ohh, OK I can see wq_worker_sleeping now. I've missed your point in
> > > other email, sorry about that. But now I am wondering whether this
> > > is an intended behavior. The documentation says:
> > 
> > This is.
> > 
> > >   WQ_MEM_RECLAIM
> > > 
> > >         All wq which might be used in the memory reclaim paths _MUST_
> > >         have this flag set.  The wq is guaranteed to have at least one
> > >         execution context regardless of memory pressure.
> > > 
> > > Which doesn't seem to be true currently, right? Now I can see your patch
> > 
> > It is true.
> > 
> > > to introduce WQ_IMMEDIATE but I am wondering which WQ_MEM_RECLAIM users
> > > could do without WQ_IMMEDIATE? I mean all current workers might be
> > > looping in the page allocator and it seems possible that WQ_MEM_RECLAIM
> > > work items might be waiting behind them so they cannot help to relieve
> > > the memory pressure. This doesn't sound right to me. Or I am completely
> > > confused and still fail to understand what is WQ_MEM_RECLAIM supposed to
> > > be used for.
> > 
> > It guarantees that there always is enough execution resource to
> > execute a work item from that workqueue. 
> 
> OK, strictly speaking the rescuer is there but it is kind of pointless
> if it doesn't fire up and do a work.
> 
> > The problem here is not lack
> > of execution resource but concurrency management misunderstanding the
> > situation. 
> 
> And this sounds like a bug to me.
> 
> > This also can be fixed by teaching concurrency management
> > to be a bit smarter - e.g. if a work item is burning a lot of CPU
> > cycles continuously or pool hasn't finished a work item over a certain
> > amount of time, automatically ignore the in-flight work item for the
> > purpose of concurrency management; however, this sort of inter-work
> > item busy waits are so extremely rare and undesirable that I'm not
> > sure the added complexity would be worthwhile.
> 
> Don't we have some IO related paths which would suffer from the same
> problem. I haven't checked all the WQ_MEM_RECLAIM users but from the
> name I would expect they _do_ participate in the reclaim and so they
> should be able to make a progress. Now if your new IMMEDIATE flag will
> guarantee that then I would argue that it should be implicit for
> WQ_MEM_RECLAIM otherwise we always risk a similar situation. What would
> be a counter argument for doing that?

WQ_MEM_RECLAIM only guarantees that a "struct task_struct" is preallocated
in order to avoid failing to allocate it on demand due to a GFP_KERNEL
allocation? Is this correct?

WQ_CPU_INTENSIVE only guarantees that work items don't participate in
concurrency management in order to avoid failing to wake up a "struct
task_struct" which will process the work items? Is this correct?

Is Michal's question "does it make sense to use WQ_MEM_RECLAIM without
WQ_CPU_INTENSIVE"? In other words, any "struct task_struct" which calls
rescuer_thread() must imply WQ_CPU_INTENSIVE in order to avoid failing to
wake up due to being participated in concurrency management?
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]


Page 2 of 3 — ← Prev page 1 [2] 3  Next page →

Back to top | Article view | linux.kernel


csiph-web