Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]


Groups > linux.kernel > #1330486

Re: Crashes with 874bbfe600a6 in 3.18.25

From Mike Galbraith <umgwanakikbuti@gmail.com>
Newsgroups linux.kernel
Subject Re: Crashes with 874bbfe600a6 in 3.18.25
Date 2016-02-09 18:10 +0100
Message-ID <r0jXA-76q-29@gated-at.bofh.it> (permalink)
References (6 earlier) <qYVNF-5wj-7@gated-at.bofh.it> <qYVNF-5wj-5@gated-at.bofh.it> <r0iyx-5W6-137@gated-at.bofh.it> <r0juz-6EK-43@gated-at.bofh.it> <r0jNW-6Mt-41@gated-at.bofh.it>
Organization linux.* mail to news gateway

Show all headers | View raw


On Tue, 2016-02-09 at 11:50 -0500, Tejun Heo wrote:
> Hello,
> 
> On Tue, Feb 09, 2016 at 08:39:15AM -0800, Linus Torvalds wrote:
> > > A niggling question remaining is when is it gonna be killed?
> > 
> > It probably should be killed sooner rather than later.
> > 
> > Just document that if you need something to run on a _particular_
> > cpu,
> > you need to use "schedule_delayed_work_on()" and "add_timer_on()".
> 
> I'll queue a patch to put unbound work items on foreign cpus (maybe
> every Nth to reduce perf impact).  Wanted to align it to rc1 and then
> let it get tested during the devel cycle but missed this window.  It's
> a bit late in devel cycle but we can still do it in this cycle.

Or do something like the below, and get guinea pigs for free. 

workqueue: schedule WORK_CPU_UNBOUND work on wq_unbound_cpumask CPUs

WORK_CPU_UNBOUND work items queued to a bound workqueue always run
locally.  This is a good thing normally, but not when the user has
asked us to keep unbound work away from certain CPUs.  Round robin
these to wq_unbound_cpumask CPUs instead, as perturbation avoidance
trumps performance.

Signed-off-by: Mike Galbraith <umgwanakikbuti@gmail.com>
---
 kernel/workqueue.c |   27 ++++++++++++++++++++++++++-
 1 file changed, 26 insertions(+), 1 deletion(-)

--- a/kernel/workqueue.c
+++ b/kernel/workqueue.c
@@ -303,6 +303,9 @@ static bool workqueue_freezing;		/* PL:
 
 static cpumask_var_t wq_unbound_cpumask; /* PL: low level cpumask for all unbound wqs */
 
+/* CPU where WORK_CPU_UNBOUND work was last round robin scheduled from this CPU */
+static DEFINE_PER_CPU(unsigned int, wq_unbound_rr_cpu_last);
+
 /* the per-cpu worker pools */
 static DEFINE_PER_CPU_SHARED_ALIGNED(struct worker_pool [NR_STD_WORKER_POOLS],
 				     cpu_worker_pools);
@@ -1298,6 +1301,28 @@ static bool is_chained_work(struct workq
 	return worker && worker->current_pwq->wq == wq;
 }
 
+/*
+ * When queueing WORK_CPU_UNBOUND work to a !WQ_UNBOUND queue, round
+ * robin among wq_unbound_cpumask to avoid perturbing sensitive tasks.
+ */
+static unsigned int select_round_robin_cpu(unsigned int cpu)
+{
+	int new_cpu;
+
+	if (cpumask_test_cpu(cpu, wq_unbound_cpumask))
+		return cpu;
+	if (cpumask_empty(wq_unbound_cpumask))
+		return cpu;
+	new_cpu = __this_cpu_read(wq_unbound_rr_cpu_last);
+	new_cpu = cpumask_next_and(new_cpu, wq_unbound_cpumask, cpu_online_mask);
+	if (unlikely(new_cpu >= nr_cpu_ids))
+		new_cpu = cpumask_first_and(wq_unbound_cpumask, cpu_online_mask);
+	if (unlikely(WARN_ON_ONCE(new_cpu >= nr_cpu_ids)))
+		return cpu;
+	__this_cpu_write(wq_unbound_rr_cpu_last, new_cpu);
+	return new_cpu;
+}
+
 static void __queue_work(int cpu, struct workqueue_struct *wq,
 			 struct work_struct *work)
 {
@@ -1323,7 +1348,7 @@ static void __queue_work(int cpu, struct
 		return;
 retry:
 	if (req_cpu == WORK_CPU_UNBOUND)
-		cpu = raw_smp_processor_id();
+		cpu = select_round_robin_cpu(raw_smp_processor_id());
 
 	/* pwq which will be used unless @work is executing elsewhere */
 	if (!(wq->flags & WQ_UNBOUND))

Back to linux.kernel | Previous | NextPrevious in thread | Next in thread | Find similar | Unroll thread


Thread

Re: Crashes with 874bbfe600a6 in 3.18.25 Jiri Slaby <jslaby@suse.cz> - 2016-02-03 10:40 +0100
  Re: Crashes with 874bbfe600a6 in 3.18.25 Thomas Gleixner <tglx@linutronix.de> - 2016-02-03 11:50 +0100
  Re: Crashes with 874bbfe600a6 in 3.18.25 Michal Hocko <mhocko@kernel.org> - 2016-02-03 13:30 +0100
    Re: Crashes with 874bbfe600a6 in 3.18.25 Tejun Heo <tj@kernel.org> - 2016-02-03 17:30 +0100
      Re: Crashes with 874bbfe600a6 in 3.18.25 Michal Hocko <mhocko@kernel.org> - 2016-02-03 17:50 +0100
        Re: Crashes with 874bbfe600a6 in 3.18.25 Tejun Heo <tj@kernel.org> - 2016-02-03 18:10 +0100
          Re: Crashes with 874bbfe600a6 in 3.18.25 Michal Hocko <mhocko@kernel.org> - 2016-02-04 07:40 +0100
            Re: Crashes with 874bbfe600a6 in 3.18.25 Michal Hocko <mhocko@kernel.org> - 2016-02-04 08:50 +0100
      Re: Crashes with 874bbfe600a6 in 3.18.25 Mike Galbraith <umgwanakikbuti@gmail.com> - 2016-02-03 18:10 +0100
        Re: Crashes with 874bbfe600a6 in 3.18.25 Tejun Heo <tj@kernel.org> - 2016-02-03 18:10 +0100
          Re: Crashes with 874bbfe600a6 in 3.18.25 Tejun Heo <tj@kernel.org> - 2016-02-03 18:20 +0100
          Re: Crashes with 874bbfe600a6 in 3.18.25 Mike Galbraith <umgwanakikbuti@gmail.com> - 2016-02-03 18:20 +0100
          Re: Crashes with 874bbfe600a6 in 3.18.25 Mike Galbraith <umgwanakikbuti@gmail.com> - 2016-02-04 03:10 +0100
            Re: Crashes with 874bbfe600a6 in 3.18.25 Tejun Heo <tj@kernel.org> - 2016-02-05 17:50 +0100
              Re: Crashes with 874bbfe600a6 in 3.18.25 Mike Galbraith <umgwanakikbuti@gmail.com> - 2016-02-05 21:50 +0100
                Re: Crashes with 874bbfe600a6 in 3.18.25 Tejun Heo <tj@kernel.org> - 2016-02-05 22:00 +0100
                Re: Crashes with 874bbfe600a6 in 3.18.25 Tejun Heo <tj@kernel.org> - 2016-02-05 22:10 +0100
                Re: Crashes with 874bbfe600a6 in 3.18.25 Henrique de Moraes Holschuh <hmh@hmh.eng.br> - 2016-02-06 14:10 +0100
                Re: Crashes with 874bbfe600a6 in 3.18.25 Mike Galbraith <umgwanakikbuti@gmail.com> - 2016-02-07 06:30 +0100
                Re: Crashes with 874bbfe600a6 in 3.18.25 Mike Galbraith <umgwanakikbuti@gmail.com> - 2016-02-07 07:10 +0100
                Re: Crashes with 874bbfe600a6 in 3.18.25 Mike Galbraith <umgwanakikbuti@gmail.com> - 2016-02-09 16:40 +0100
                Re: Crashes with 874bbfe600a6 in 3.18.25 Linus Torvalds <torvalds@linux-foundation.org> - 2016-02-09 17:40 +0100
                Re: Crashes with 874bbfe600a6 in 3.18.25 Tejun Heo <tj@kernel.org> - 2016-02-09 18:00 +0100
                Re: Crashes with 874bbfe600a6 in 3.18.25 Mike Galbraith <umgwanakikbuti@gmail.com> - 2016-02-09 18:10 +0100
                Re: Crashes with 874bbfe600a6 in 3.18.25 Mike Galbraith <umgwanakikbuti@gmail.com> - 2016-02-09 19:00 +0100
                Re: Crashes with 874bbfe600a6 in 3.18.25 Mike Galbraith <umgwanakikbuti@gmail.com> - 2016-02-09 19:10 +0100
                Re: Crashes with 874bbfe600a6 in 3.18.25 Tejun Heo <tj@kernel.org> - 2016-02-09 19:30 +0100
                Re: Crashes with 874bbfe600a6 in 3.18.25 Tejun Heo <tj@kernel.org> - 2016-02-09 19:00 +0100
                Re: Crashes with 874bbfe600a6 in 3.18.25 Linus Torvalds <torvalds@linux-foundation.org> - 2016-02-09 18:10 +0100
                Re: Crashes with 874bbfe600a6 in 3.18.25 Tejun Heo <tj@kernel.org> - 2016-02-09 19:00 +0100
                Re: Crashes with 874bbfe600a6 in 3.18.25 Linus Torvalds <torvalds@linux-foundation.org> - 2016-02-09 19:10 +0100
                Re: Crashes with 874bbfe600a6 in 3.18.25 Mike Galbraith <umgwanakikbuti@gmail.com> - 2016-02-05 22:10 +0100
          Re: Crashes with 874bbfe600a6 in 3.18.25 Mike Galbraith <umgwanakikbuti@gmail.com> - 2016-02-04 11:10 +0100
            Re: Crashes with 874bbfe600a6 in 3.18.25 Thomas Gleixner <tglx@linutronix.de> - 2016-02-04 11:50 +0100
              Re: Crashes with 874bbfe600a6 in 3.18.25 Mike Galbraith <umgwanakikbuti@gmail.com> - 2016-02-04 12:10 +0100
              Re: Crashes with 874bbfe600a6 in 3.18.25 Jan Kara <jack@suse.cz> - 2016-02-04 12:30 +0100
                Re: Crashes with 874bbfe600a6 in 3.18.25 Daniel Bilik <daniel.bilik@neosystem.cz> - 2016-02-04 18:00 +0100
                Re: Crashes with 874bbfe600a6 in 3.18.25 Mike Galbraith <umgwanakikbuti@gmail.com> - 2016-02-05 03:50 +0100
                Re: Crashes with 874bbfe600a6 in 3.18.25 Daniel Bilik <daniel.bilik@neosystem.cz> - 2016-02-05 09:20 +0100
                Re: Crashes with 874bbfe600a6 in 3.18.25 Mike Galbraith <umgwanakikbuti@gmail.com> - 2016-02-05 09:40 +0100
      Re: Crashes with 874bbfe600a6 in 3.18.25 Thomas Gleixner <tglx@linutronix.de> - 2016-02-03 19:50 +0100
        Re: Crashes with 874bbfe600a6 in 3.18.25 Thomas Gleixner <tglx@linutronix.de> - 2016-02-03 20:10 +0100
          Re: Crashes with 874bbfe600a6 in 3.18.25 Tejun Heo <tj@kernel.org> - 2016-02-03 20:20 +0100
        Re: Crashes with 874bbfe600a6 in 3.18.25 Tejun Heo <tj@kernel.org> - 2016-02-03 20:10 +0100
      Re: Crashes with 874bbfe600a6 in 3.18.25 Mike Galbraith <umgwanakikbuti@gmail.com> - 2016-02-05 06:50 +0100

csiph-web