Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1306483 > unrolled thread

[RFC][PATCH 00/12] various perf fixes

Started byPeter Zijlstra <peterz@infradead.org>
First post2016-01-11 17:40 +0100
Last post2016-01-12 14:40 +0100
Articles 20 on this page of 25 — 5 participants

Back to article view | Back to linux.kernel

Contents

  [RFC][PATCH 00/12] various perf fixes Peter Zijlstra <peterz@infradead.org> - 2016-01-11 17:40 +0100
    [RFC][PATCH 06/12] perf: Use task_ctx_sched_out() Peter Zijlstra <peterz@infradead.org> - 2016-01-11 17:40 +0100
    [RFC][PATCH 05/12] perf: Fix enable_on_exec event scheduling Peter Zijlstra <peterz@infradead.org> - 2016-01-11 17:40 +0100
    [RFC][PATCH 10/12] perf: Fix task context scheduling Peter Zijlstra <peterz@infradead.org> - 2016-01-11 17:40 +0100
    [RFC][PATCH 07/12] perf: Simplify/fix perf_event_enable() event scheduling Peter Zijlstra <peterz@infradead.org> - 2016-01-11 17:40 +0100
    [RFC][PATCH 04/12] perf: Remove stale comment Peter Zijlstra <peterz@infradead.org> - 2016-01-11 17:40 +0100
    [RFC][PATCH 11/12] perf: Specialize perf_event_exit_task() Peter Zijlstra <peterz@infradead.org> - 2016-01-11 17:40 +0100
    Re: [RFC][PATCH 00/12] various perf fixes Dmitry Vyukov <dvyukov@google.com> - 2016-01-11 19:50 +0100
      Re: [RFC][PATCH 00/12] various perf fixes Andi Kleen <andi@firstfloor.org> - 2016-01-11 21:00 +0100
      Re: [RFC][PATCH 00/12] various perf fixes Peter Zijlstra <peterz@infradead.org> - 2016-01-11 23:10 +0100
        Re: [RFC][PATCH 00/12] various perf fixes Ingo Molnar <mingo@kernel.org> - 2016-01-12 11:00 +0100
    Re: [RFC][PATCH 00/12] various perf fixes Ingo Molnar <mingo@kernel.org> - 2016-01-12 11:20 +0100
      Re: [RFC][PATCH 00/12] various perf fixes Dmitry Vyukov <dvyukov@google.com> - 2016-01-12 12:00 +0100
        Re: [RFC][PATCH 00/12] various perf fixes Dmitry Vyukov <dvyukov@google.com> - 2016-01-12 12:10 +0100
          Re: [RFC][PATCH 00/12] various perf fixes Dmitry Vyukov <dvyukov@google.com> - 2016-01-12 12:10 +0100
            Re: [RFC][PATCH 00/12] various perf fixes Dmitry Vyukov <dvyukov@google.com> - 2016-01-12 12:30 +0100
              Re: [RFC][PATCH 00/12] various perf fixes Dmitry Vyukov <dvyukov@google.com> - 2016-01-12 12:40 +0100
              Re: [RFC][PATCH 00/12] various perf fixes Peter Zijlstra <peterz@infradead.org> - 2016-01-12 13:10 +0100
              Re: [RFC][PATCH 00/12] various perf fixes Alexander Shishkin <alexander.shishkin@linux.intel.com> - 2016-01-13 16:20 +0100
                Re: [RFC][PATCH 00/12] various perf fixes Dmitry Vyukov <dvyukov@google.com> - 2016-01-13 16:30 +0100
                  Re: [RFC][PATCH 00/12] various perf fixes Alexander Shishkin <alexander.shishkin@linux.intel.com> - 2016-01-13 16:40 +0100
              Re: [RFC][PATCH 00/12] various perf fixes Peter Zijlstra <peterz@infradead.org> - 2016-01-14 10:40 +0100
                Re: [RFC][PATCH 00/12] various perf fixes Dmitry Vyukov <dvyukov@google.com> - 2016-01-14 11:10 +0100
      Re: [RFC][PATCH 00/12] various perf fixes Peter Zijlstra <peterz@infradead.org> - 2016-01-12 14:20 +0100
        Re: [RFC][PATCH 00/12] various perf fixes Ingo Molnar <mingo@kernel.org> - 2016-01-12 14:40 +0100

Page 1 of 2  [1] 2  Next page →

#1306483 — [RFC][PATCH 00/12] various perf fixes

FromPeter Zijlstra <peterz@infradead.org>
Date2016-01-11 17:40 +0100
Subject[RFC][PATCH 00/12] various perf fixes
Message-ID<qPNFD-4Vc-3@gated-at.bofh.it>
Hi,

I've been hunting perf bugs for the past few weeks. This resulted in this pile
of patches, that mostly seems to work -- I still get an occasional fail so
something is still off.

But I've been sitting on this for far too long, so here goes.

(I've not at all tested perf-cgroup)

The code compiles in between patches, but I've not bothered trying to run/test
any intermediate stage. There's just too many inter-related fail.

Alexander, Stephane, could you guys please have a hard look at this?

Andi, Dmitry, patch 10 should explain the getting stuck in
perf_install_in_context() forever thing you both have observed.

[toc] | [next] | [standalone]

#1306484 — [RFC][PATCH 06/12] perf: Use task_ctx_sched_out()

FromPeter Zijlstra <peterz@infradead.org>
Date2016-01-11 17:40 +0100
Subject[RFC][PATCH 06/12] perf: Use task_ctx_sched_out()
Message-ID<qPNFG-4Vc-43@gated-at.bofh.it>
In reply to#1306483
We have a function that does exactly what we want here, use it. This
reduces the amount of cpuctx->task_ctx muckery.

Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
---
 kernel/events/core.c |    3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

--- a/kernel/events/core.c
+++ b/kernel/events/core.c
@@ -2545,8 +2545,7 @@ static void perf_event_context_sched_out
 
 	if (do_switch) {
 		raw_spin_lock(&ctx->lock);
-		ctx_sched_out(ctx, cpuctx, EVENT_ALL);
-		cpuctx->task_ctx = NULL;
+		task_ctx_sched_out(cpuctx, ctx);
 		raw_spin_unlock(&ctx->lock);
 	}
 }

[toc] | [prev] | [next] | [standalone]

#1306485 — [RFC][PATCH 05/12] perf: Fix enable_on_exec event scheduling

FromPeter Zijlstra <peterz@infradead.org>
Date2016-01-11 17:40 +0100
Subject[RFC][PATCH 05/12] perf: Fix enable_on_exec event scheduling
Message-ID<qPNFG-4Vc-41@gated-at.bofh.it>
In reply to#1306483
There are two problems with the current enable_on_exec event
scheduling:

  - the newly enabled events will be immediately scheduled
    irrespective of their ctx event list order.

  - there's a hole in the ctx->lock between scheduling the events
    out and putting them back on.

Esp. the latter issue is a real problem because a hole in event
scheduling leaves the thing in an observable inconsistent state,
confusing things.

Fix both issues by first doing the enable iteration and at the end,
when there are newly enabled events, reschedule the ctx in one go.

Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
---
 kernel/events/core.c |   47 +++++++++++++++++++++++++++--------------------
 1 file changed, 27 insertions(+), 20 deletions(-)

--- a/kernel/events/core.c
+++ b/kernel/events/core.c
@@ -2036,7 +2036,8 @@ static void add_event_to_ctx(struct perf
 	event->tstamp_stopped = tstamp;
 }
 
-static void task_ctx_sched_out(struct perf_event_context *ctx);
+static void task_ctx_sched_out(struct perf_cpu_context *cpuctx,
+			       struct perf_event_context *ctx);
 static void
 ctx_sched_in(struct perf_event_context *ctx,
 	     struct perf_cpu_context *cpuctx,
@@ -2067,6 +2068,17 @@ static void ___perf_install_in_context(v
 	add_event_to_ctx(event, ctx);
 }
 
+static void ctx_resched(struct perf_cpu_context *cpuctx,
+			struct perf_event_context *task_ctx)
+{
+	perf_pmu_disable(cpuctx->ctx.pmu);
+	if (task_ctx)
+		task_ctx_sched_out(cpuctx, task_ctx);
+	cpu_ctx_sched_out(cpuctx, EVENT_ALL);
+	perf_event_sched_in(cpuctx, task_ctx, current);
+	perf_pmu_enable(cpuctx->ctx.pmu);
+}
+
 /*
  * Cross CPU call to install and enable a performance event
  *
@@ -2087,7 +2099,7 @@ static int  __perf_install_in_context(vo
 	 * If there was an active task_ctx schedule it out.
 	 */
 	if (task_ctx)
-		task_ctx_sched_out(task_ctx);
+		task_ctx_sched_out(cpuctx, task_ctx);
 
 	/*
 	 * If the context we're installing events in is not the
@@ -2629,10 +2641,9 @@ void __perf_event_task_sched_out(struct
 		perf_cgroup_sched_out(task, next);
 }
 
-static void task_ctx_sched_out(struct perf_event_context *ctx)
+static void task_ctx_sched_out(struct perf_cpu_context *cpuctx,
+			       struct perf_event_context *ctx)
 {
-	struct perf_cpu_context *cpuctx = __get_cpu_context(ctx);
-
 	if (!cpuctx->task_ctx)
 		return;
 
@@ -3096,34 +3107,30 @@ static int event_enable_on_exec(struct p
 static void perf_event_enable_on_exec(int ctxn)
 {
 	struct perf_event_context *ctx, *clone_ctx = NULL;
+	struct perf_cpu_context *cpuctx;
 	struct perf_event *event;
 	unsigned long flags;
 	int enabled = 0;
-	int ret;
 
 	local_irq_save(flags);
 	ctx = current->perf_event_ctxp[ctxn];
 	if (!ctx || !ctx->nr_events)
 		goto out;
 
-	raw_spin_lock(&ctx->lock);
-	task_ctx_sched_out(ctx);
-
-	list_for_each_entry(event, &ctx->event_list, event_entry) {
-		ret = event_enable_on_exec(event, ctx);
-		if (ret)
-			enabled = 1;
-	}
+	cpuctx = __get_cpu_context(ctx);
+	perf_ctx_lock(cpuctx, ctx);
+	list_for_each_entry(event, &ctx->event_list, event_entry)
+		enabled |= event_enable_on_exec(event, ctx);
 
 	/*
-	 * Unclone this context if we enabled any event.
+	 * Unclone and reschedule this context if we enabled any event.
 	 */
-	if (enabled)
+	if (enabled) {
 		clone_ctx = unclone_ctx(ctx);
+		ctx_resched(cpuctx, ctx);
+	}
+	perf_ctx_unlock(cpuctx, ctx);
 
-	raw_spin_unlock(&ctx->lock);
-
-	perf_event_context_sched_in(ctx, ctx->task);
 out:
 	local_irq_restore(flags);
 
@@ -8737,7 +8744,7 @@ static void perf_event_exit_task_context
 	 * incremented the context's refcount before we do put_ctx below.
 	 */
 	raw_spin_lock(&child_ctx->lock);
-	task_ctx_sched_out(child_ctx);
+	task_ctx_sched_out(__get_cpu_context(child_ctx), child_ctx);
 	child->perf_event_ctxp[ctxn] = NULL;
 
 	/*

[toc] | [prev] | [next] | [standalone]

#1306486 — [RFC][PATCH 10/12] perf: Fix task context scheduling

FromPeter Zijlstra <peterz@infradead.org>
Date2016-01-11 17:40 +0100
Subject[RFC][PATCH 10/12] perf: Fix task context scheduling
Message-ID<qPNFG-4Vc-53@gated-at.bofh.it>
In reply to#1306483
There is a very nasty problem wrt disabling the perf task scheduling
hooks.

Currently we {set,clear} ctx->is_active on every
__perf_event_task_sched_{in,out}, _however_ this means that if we
disable these calls we'll have task contexts with ->is_active set that
are not active and 'active' task contexts without ->is_active set.

This can result in event_function_call() looping on the ctx->is_active
condition basically indefinitely.

Resolve this by changing things such that contexts without events do
not set ->is_active like we used to. From this invariant it trivially
follows that if there are no (task) events, every task ctx is inactive
and disabling the context switch hooks is harmless.

This leaves two places that need attention (and already had
accumulated weird and wonderful hacks to work around, without
recognising this actual problem).

Namely:

 - perf_install_in_context() will need to deal with installing events
   in an inactive context, meaning it cannot rely on ctx-is_active for
   its IPIs.

 - perf_remove_from_context() will have to mark a context as inactive
   when it removes the last event.

For specific detail, see the patch/comments.

Cc: Andi Kleen <andi@firstfloor.org>
Cc: Dmitry Vyukov <dvyukov@google.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
---
 kernel/events/core.c |  155 +++++++++++++++++++++++++++++----------------------
 1 file changed, 91 insertions(+), 64 deletions(-)

--- a/kernel/events/core.c
+++ b/kernel/events/core.c
@@ -126,6 +126,38 @@ static int cpu_function_call(int cpu, re
 	return data.ret;
 }
 
+/*
+ * On task ctx scheduling...
+ *
+ * When !ctx->nr_events a task context will not be scheduled. This means
+ * we can disable the scheduler hooks (for performance) without leaving
+ * pending task ctx state.
+ *
+ * This however results in two special cases:
+ *
+ *  - removing the last event from a task ctx; this is relatively straight
+ *    forward and is done in __perf_remove_from_context.
+ *
+ *  - adding the first event to a task ctx; this is tricky because we cannot
+ *    rely on ctx->is_active and therefore cannot use event_function_call().
+ *    See perf_install_in_context().
+ *
+ * This is because we need a ctx->lock serialized variable (ctx->is_active)
+ * to reliably determine if a particular task/context is scheduled in. The
+ * task_curr() use in task_function_call() is racy in that a remote context
+ * switch is not a single atomic operation.
+ *
+ * As is, the situation is 'safe' because we set rq->curr before we do the
+ * actual context switch. This means that task_curr() will fail early, but
+ * we'll continue spinning on ctx->is_active until we've passed
+ * perf_event_task_sched_out().
+ *
+ * Without this ctx->lock serialized variable we could have race where we find
+ * the task (and hence the context) would not be active while in fact they are.
+ *
+ * If ctx->nr_events, then ctx->is_active and cpuctx->task_ctx are set.
+ */
+
 static void event_function_call(struct perf_event *event,
 				int (*active)(void *),
 				void (*inactive)(void *),
@@ -1686,9 +1718,13 @@ static int __perf_remove_from_context(vo
 	if (re->detach_group)
 		perf_group_detach(event);
 	list_del_event(event, ctx);
-	if (!ctx->nr_events && cpuctx->task_ctx == ctx) {
+
+	if (!ctx->nr_events && ctx->is_active) {
 		ctx->is_active = 0;
-		cpuctx->task_ctx = NULL;
+		if (ctx->task) {
+			WARN_ON_ONCE(cpuctx->task_ctx != ctx);
+			cpuctx->task_ctx = NULL;
+		}
 	}
 	raw_spin_unlock(&ctx->lock);
 
@@ -2056,18 +2092,6 @@ static void perf_event_sched_in(struct p
 		ctx_sched_in(ctx, cpuctx, EVENT_FLEXIBLE, task);
 }
 
-static void ___perf_install_in_context(void *info)
-{
-	struct perf_event *event = info;
-	struct perf_event_context *ctx = event->ctx;
-
-	/*
-	 * Since the task isn't running, its safe to add the event, us holding
-	 * the ctx->lock ensures the task won't get scheduled in.
-	 */
-	add_event_to_ctx(event, ctx);
-}
-
 static void ctx_resched(struct perf_cpu_context *cpuctx,
 			struct perf_event_context *task_ctx)
 {
@@ -2086,55 +2110,27 @@ static void ctx_resched(struct perf_cpu_
  */
 static int  __perf_install_in_context(void *info)
 {
-	struct perf_event *event = info;
-	struct perf_event_context *ctx = event->ctx;
+	struct perf_event_context *ctx = info;
 	struct perf_cpu_context *cpuctx = __get_cpu_context(ctx);
 	struct perf_event_context *task_ctx = cpuctx->task_ctx;
-	struct task_struct *task = current;
 
-	perf_ctx_lock(cpuctx, task_ctx);
-	perf_pmu_disable(cpuctx->ctx.pmu);
-
-	/*
-	 * If there was an active task_ctx schedule it out.
-	 */
-	if (task_ctx)
-		task_ctx_sched_out(cpuctx, task_ctx);
+	if (ctx->task) {
+		/*
+		 * If we hit the 'wrong' task, we've since scheduled and
+		 * everything should be sorted, nothing to do!
+		 */
+		if (ctx->task != current)
+			return 0;
 
-	/*
-	 * If the context we're installing events in is not the
-	 * active task_ctx, flip them.
-	 */
-	if (ctx->task && task_ctx != ctx) {
-		if (task_ctx)
-			raw_spin_unlock(&task_ctx->lock);
-		raw_spin_lock(&ctx->lock);
+		/*
+		 * If task_ctx is set, it had better be to us.
+		 */
+		WARN_ON_ONCE(cpuctx->task_ctx != ctx && cpuctx->task_ctx);
 		task_ctx = ctx;
 	}
 
-	if (task_ctx) {
-		cpuctx->task_ctx = task_ctx;
-		task = task_ctx->task;
-	}
-
-	cpu_ctx_sched_out(cpuctx, EVENT_ALL);
-
-	update_context_time(ctx);
-	/*
-	 * update cgrp time only if current cgrp
-	 * matches event->cgrp. Must be done before
-	 * calling add_event_to_ctx()
-	 */
-	update_cgrp_time_from_event(event);
-
-	add_event_to_ctx(event, ctx);
-
-	/*
-	 * Schedule everything back in
-	 */
-	perf_event_sched_in(cpuctx, task_ctx, task);
-
-	perf_pmu_enable(cpuctx->ctx.pmu);
+	perf_ctx_lock(cpuctx, task_ctx);
+	ctx_resched(cpuctx, task_ctx);
 	perf_ctx_unlock(cpuctx, task_ctx);
 
 	return 0;
@@ -2148,14 +2144,38 @@ perf_install_in_context(struct perf_even
 			struct perf_event *event,
 			int cpu)
 {
+	struct task_struct *task = NULL;
+
 	lockdep_assert_held(&ctx->mutex);
 
 	event->ctx = ctx;
 	if (event->cpu != -1)
 		event->cpu = cpu;
 
-	event_function_call(event, __perf_install_in_context,
-			    ___perf_install_in_context, event);
+	/*
+	 * Installing events is tricky because we cannot rely on ctx->is_active
+	 * to be set in case this is the nr_events 0 -> 1 transition.
+	 *
+	 * So what we do is we add the event to the list here, which will allow
+	 * a future context switch to DTRT and then send a racy IPI. If the IPI
+	 * fails to hit the right task, this means a context switch must have
+	 * happened and that will have taken care of business.
+	 */
+	raw_spin_lock_irq(&ctx->lock);
+	update_context_time(ctx);
+	/*
+	 * Update cgrp time only if current cgrp matches event->cgrp.
+	 * Must be done before calling add_event_to_ctx().
+	 */
+	update_cgrp_time_from_event(event);
+	add_event_to_ctx(event, ctx);
+	task = ctx->task;
+	raw_spin_unlock_irq(&ctx->lock);
+
+	if (task)
+		task_function_call(task, __perf_install_in_context, ctx);
+	else
+		cpu_function_call(cpu, __perf_install_in_context, ctx);
 }
 
 /*
@@ -2328,6 +2348,16 @@ static void ctx_sched_out(struct perf_ev
 
 	lockdep_assert_held(&ctx->lock);
 
+	if (likely(!ctx->nr_events)) {
+		/*
+		 * See __perf_remove_from_context().
+		 */
+		WARN_ON_ONCE(ctx->is_active);
+		if (ctx->task)
+			WARN_ON_ONCE(cpuctx->task_ctx);
+		return;
+	}
+
 	ctx->is_active &= ~event_type;
 	if (ctx->task) {
 		WARN_ON_ONCE(cpuctx->task_ctx != ctx);
@@ -2335,9 +2365,6 @@ static void ctx_sched_out(struct perf_ev
 			cpuctx->task_ctx = NULL;
 	}
 
-	if (likely(!ctx->nr_events))
-		return;
-
 	update_context_time(ctx);
 	update_cgrp_time_from_cpuctx(cpuctx);
 	if (!ctx->nr_active)
@@ -2716,6 +2743,9 @@ ctx_sched_in(struct perf_event_context *
 
 	lockdep_assert_held(&ctx->lock);
 
+	if (likely(!ctx->nr_events))
+		return;
+
 	ctx->is_active |= event_type;
 	if (ctx->task) {
 		if (!is_active)
@@ -2724,9 +2754,6 @@ ctx_sched_in(struct perf_event_context *
 			WARN_ON_ONCE(cpuctx->task_ctx != ctx);
 	}
 
-	if (likely(!ctx->nr_events))
-		return;
-
 	now = perf_clock();
 	ctx->timestamp = now;
 	perf_cgroup_set_timestamp(task, ctx);

[toc] | [prev] | [next] | [standalone]

#1306488 — [RFC][PATCH 07/12] perf: Simplify/fix perf_event_enable() event scheduling

FromPeter Zijlstra <peterz@infradead.org>
Date2016-01-11 17:40 +0100
Subject[RFC][PATCH 07/12] perf: Simplify/fix perf_event_enable() event scheduling
Message-ID<qPNFG-4Vc-51@gated-at.bofh.it>
In reply to#1306483
Like enable_on_exec, perf_event_enable() event scheduling has problems
respecting the context hierarchy when trying to schedule events (for
example, it will try and add a pinned event without first removing
existing flexible events).

So simplify it by using the new ctx_resched() call which will DTRT.

Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
---
 kernel/events/core.c |   31 +++++--------------------------
 1 file changed, 5 insertions(+), 26 deletions(-)

--- a/kernel/events/core.c
+++ b/kernel/events/core.c
@@ -2188,7 +2188,7 @@ static int __perf_event_enable(void *inf
 	struct perf_event_context *ctx = event->ctx;
 	struct perf_event *leader = event->group_leader;
 	struct perf_cpu_context *cpuctx = __get_cpu_context(ctx);
-	int err;
+	struct perf_event_context *task_ctx = cpuctx->task_ctx;
 
 	/*
 	 * There's a time window between 'ctx->is_active' check
@@ -2202,7 +2202,8 @@ static int __perf_event_enable(void *inf
 	if (!ctx->is_active)
 		return -EINVAL;
 
-	raw_spin_lock(&ctx->lock);
+	perf_ctx_lock(cpuctx, task_ctx);
+	WARN_ON_ONCE(&cpuctx->ctx != ctx && task_ctx != ctx);
 	update_context_time(ctx);
 
 	if (event->state >= PERF_EVENT_STATE_INACTIVE)
@@ -2228,32 +2229,10 @@ static int __perf_event_enable(void *inf
 	if (leader != event && leader->state != PERF_EVENT_STATE_ACTIVE)
 		goto unlock;
 
-	if (!group_can_go_on(event, cpuctx, 1)) {
-		err = -EEXIST;
-	} else {
-		if (event == leader)
-			err = group_sched_in(event, cpuctx, ctx);
-		else
-			err = event_sched_in(event, cpuctx, ctx);
-	}
-
-	if (err) {
-		/*
-		 * If this event can't go on and it's part of a
-		 * group, then the whole group has to come off.
-		 */
-		if (leader != event) {
-			group_sched_out(leader, cpuctx, ctx);
-			perf_mux_hrtimer_restart(cpuctx);
-		}
-		if (leader->attr.pinned) {
-			update_group_times(leader);
-			leader->state = PERF_EVENT_STATE_ERROR;
-		}
-	}
+	ctx_resched(cpuctx, task_ctx);
 
 unlock:
-	raw_spin_unlock(&ctx->lock);
+	perf_ctx_unlock(cpuctx, task_ctx);
 
 	return 0;
 }

[toc] | [prev] | [next] | [standalone]

#1306490 — [RFC][PATCH 04/12] perf: Remove stale comment

FromPeter Zijlstra <peterz@infradead.org>
Date2016-01-11 17:40 +0100
Subject[RFC][PATCH 04/12] perf: Remove stale comment
Message-ID<qPNFG-4Vc-61@gated-at.bofh.it>
In reply to#1306483
The comment here is horribly out of date, remove it.

Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
---
 kernel/events/core.c |    7 -------
 1 file changed, 7 deletions(-)

--- a/kernel/events/core.c
+++ b/kernel/events/core.c
@@ -2130,13 +2130,6 @@ static int  __perf_install_in_context(vo
 
 /*
  * Attach a performance event to a context
- *
- * First we add the event to the list with the hardware enable bit
- * in event->hw_config cleared.
- *
- * If the event is attached to a task which is on a CPU we use a smp
- * call to enable it in the task context. The task might have been
- * scheduled away, but we check this in the smp call again.
  */
 static void
 perf_install_in_context(struct perf_event_context *ctx,

[toc] | [prev] | [next] | [standalone]

#1306491 — [RFC][PATCH 11/12] perf: Specialize perf_event_exit_task()

FromPeter Zijlstra <peterz@infradead.org>
Date2016-01-11 17:40 +0100
Subject[RFC][PATCH 11/12] perf: Specialize perf_event_exit_task()
Message-ID<qPNFG-4Vc-57@gated-at.bofh.it>
In reply to#1306483
The perf_remove_from_context() usage in __perf_event_exit_task() is
different from the other usages in that this site has already
detached and scheduled out the task context.

This will stand in the way of stronger assertions checking the (task)
context scheduling invariants.

Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
---
 kernel/events/core.c |   18 +++++++++++-------
 1 file changed, 11 insertions(+), 7 deletions(-)

--- a/kernel/events/core.c
+++ b/kernel/events/core.c
@@ -8726,7 +8726,13 @@ __perf_event_exit_task(struct perf_event
 	 * Do destroy all inherited groups, we don't care about those
 	 * and being thorough is better.
 	 */
-	perf_remove_from_context(child_event, !!child_event->parent);
+	raw_spin_lock_irq(&child_ctx->lock);
+	WARN_ON_ONCE(child_ctx->is_active);
+
+	if (!!child_event->parent)
+		perf_group_detach(child_event);
+	list_del_event(child_event, child_ctx);
+	raw_spin_unlock_irq(&child_ctx->lock);
 
 	/*
 	 * It can happen that the parent exits first, and has events
@@ -8746,17 +8752,15 @@ static void perf_event_exit_task_context
 {
 	struct perf_event *child_event, *next;
 	struct perf_event_context *child_ctx, *clone_ctx = NULL;
-	unsigned long flags;
 
 	if (likely(!child->perf_event_ctxp[ctxn]))
 		return;
 
-	local_irq_save(flags);
+	local_irq_disable();
+	WARN_ON_ONCE(child != current);
 	/*
 	 * We can't reschedule here because interrupts are disabled,
-	 * and either child is current or it is a task that can't be
-	 * scheduled, so we are now safe from rescheduling changing
-	 * our context.
+	 * and child must be current.
 	 */
 	child_ctx = rcu_dereference_raw(child->perf_event_ctxp[ctxn]);
 
@@ -8776,7 +8780,7 @@ static void perf_event_exit_task_context
 	 */
 	clone_ctx = unclone_ctx(child_ctx);
 	update_context_time(child_ctx);
-	raw_spin_unlock_irqrestore(&child_ctx->lock, flags);
+	raw_spin_unlock_irq(&child_ctx->lock);
 
 	if (clone_ctx)
 		put_ctx(clone_ctx);

[toc] | [prev] | [next] | [standalone]

#1306600

FromDmitry Vyukov <dvyukov@google.com>
Date2016-01-11 19:50 +0100
Message-ID<qPPHs-6ex-9@gated-at.bofh.it>
In reply to#1306483
On Mon, Jan 11, 2016 at 5:24 PM, Peter Zijlstra <peterz@infradead.org> wrote:
>
> Hi,
>
> I've been hunting perf bugs for the past few weeks. This resulted in this pile
> of patches, that mostly seems to work -- I still get an occasional fail so
> something is still off.
>
> But I've been sitting on this for far too long, so here goes.
>
> (I've not at all tested perf-cgroup)
>
> The code compiles in between patches, but I've not bothered trying to run/test
> any intermediate stage. There's just too many inter-related fail.
>
> Alexander, Stephane, could you guys please have a hard look at this?
>
> Andi, Dmitry, patch 10 should explain the getting stuck in
> perf_install_in_context() forever thing you both have observed.


I've tried to apply this series, but it failed exactly on patch 10.
Among other things I don't have event_function_call() function at all.
I am on the latest Linus tree afd2ff9b7e1b367172f18ba7f693dfb62bdcb2dc
(Jan 10, 4.4). What other patches do I need to apply this?

[toc] | [prev] | [next] | [standalone]

#1306636

FromAndi Kleen <andi@firstfloor.org>
Date2016-01-11 21:00 +0100
Message-ID<qPQNc-6Wu-23@gated-at.bofh.it>
In reply to#1306600
> 
> I've tried to apply this series, but it failed exactly on patch 10.
> Among other things I don't have event_function_call() function at all.
> I am on the latest Linus tree afd2ff9b7e1b367172f18ba7f693dfb62bdcb2dc
> (Jan 10, 4.4). What other patches do I need to apply this?

Likely perf/core from
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip

-Andi

-- 
ak@linux.intel.com -- Speaking for myself only.

[toc] | [prev] | [next] | [standalone]

#1306760

FromPeter Zijlstra <peterz@infradead.org>
Date2016-01-11 23:10 +0100
Message-ID<qPSP1-7w-41@gated-at.bofh.it>
In reply to#1306600
On Mon, Jan 11, 2016 at 07:44:55PM +0100, Dmitry Vyukov wrote:
> 
> I've tried to apply this series, but it failed exactly on patch 10.
> Among other things I don't have event_function_call() function at all.
> I am on the latest Linus tree afd2ff9b7e1b367172f18ba7f693dfb62bdcb2dc
> (Jan 10, 4.4). What other patches do I need to apply this?

As Andi said, its based on tip/perf/core

I've put up a git branch with these patches in on top of the latest tip,
I've not yet compiled or tested this exact tree, but they did apply
cleanly.

git://git.kernel.org/pub/scm/linux/kernel/git/peterz/queue.git perf/core

[toc] | [prev] | [next] | [standalone]

#1307181

FromIngo Molnar <mingo@kernel.org>
Date2016-01-12 11:00 +0100
Message-ID<qQ3U8-7AD-35@gated-at.bofh.it>
In reply to#1306760
* Peter Zijlstra <peterz@infradead.org> wrote:

> On Mon, Jan 11, 2016 at 07:44:55PM +0100, Dmitry Vyukov wrote:
> > 
> > I've tried to apply this series, but it failed exactly on patch 10.
> > Among other things I don't have event_function_call() function at all.
> > I am on the latest Linus tree afd2ff9b7e1b367172f18ba7f693dfb62bdcb2dc
> > (Jan 10, 4.4). What other patches do I need to apply this?
> 
> As Andi said, its based on tip/perf/core
> 
> I've put up a git branch with these patches in on top of the latest tip,
> I've not yet compiled or tested this exact tree, but they did apply
> cleanly.
> 
> git://git.kernel.org/pub/scm/linux/kernel/git/peterz/queue.git perf/core

The patches will also apply cleanly to Linus's very latest upstream tree (commit 
5cb52b5e1654 and later) - which has perf/core merged.

Thanks,

	Ingo

[toc] | [prev] | [next] | [standalone]

#1307209

FromIngo Molnar <mingo@kernel.org>
Date2016-01-12 11:20 +0100
Message-ID<qQ4dt-7Y1-15@gated-at.bofh.it>
In reply to#1306483
* Peter Zijlstra <peterz@infradead.org> wrote:

> Hi,
> 
> I've been hunting perf bugs for the past few weeks. This resulted in this pile
> of patches, that mostly seems to work -- I still get an occasional fail so
> something is still off.
> 
> But I've been sitting on this for far too long, so here goes.
> 
> (I've not at all tested perf-cgroup)
> 
> The code compiles in between patches, but I've not bothered trying to run/test
> any intermediate stage. There's just too many inter-related fail.
> 
> Alexander, Stephane, could you guys please have a hard look at this?
> 
> Andi, Dmitry, patch 10 should explain the getting stuck in
> perf_install_in_context() forever thing you both have observed.

Btw., if there's no test failures I plan to apply and push this to Linus fairly 
soon, so guys please give it all the review and testing you can.

Thanks,

	Ingo

[toc] | [prev] | [next] | [standalone]

#1307256

FromDmitry Vyukov <dvyukov@google.com>
Date2016-01-12 12:00 +0100
Message-ID<qQ4Qa-8ey-9@gated-at.bofh.it>
In reply to#1307209
On Tue, Jan 12, 2016 at 11:11 AM, Ingo Molnar <mingo@kernel.org> wrote:
>
> * Peter Zijlstra <peterz@infradead.org> wrote:
>
>> Hi,
>>
>> I've been hunting perf bugs for the past few weeks. This resulted in this pile
>> of patches, that mostly seems to work -- I still get an occasional fail so
>> something is still off.
>>
>> But I've been sitting on this for far too long, so here goes.
>>
>> (I've not at all tested perf-cgroup)
>>
>> The code compiles in between patches, but I've not bothered trying to run/test
>> any intermediate stage. There's just too many inter-related fail.
>>
>> Alexander, Stephane, could you guys please have a hard look at this?
>>
>> Andi, Dmitry, patch 10 should explain the getting stuck in
>> perf_install_in_context() forever thing you both have observed.
>
> Btw., if there's no test failures I plan to apply and push this to Linus fairly
> soon, so guys please give it all the review and testing you can.


Yes, I was able to apply it on top of the latest Linus tree. Thanks.
Now testing perf with these patches at full capacity.

[toc] | [prev] | [next] | [standalone]

#1307270

FromDmitry Vyukov <dvyukov@google.com>
Date2016-01-12 12:10 +0100
Message-ID<qQ4ZP-5t-15@gated-at.bofh.it>
In reply to#1307256
On Tue, Jan 12, 2016 at 11:57 AM, Dmitry Vyukov <dvyukov@google.com> wrote:
> On Tue, Jan 12, 2016 at 11:11 AM, Ingo Molnar <mingo@kernel.org> wrote:
>>
>> * Peter Zijlstra <peterz@infradead.org> wrote:
>>
>>> Hi,
>>>
>>> I've been hunting perf bugs for the past few weeks. This resulted in this pile
>>> of patches, that mostly seems to work -- I still get an occasional fail so
>>> something is still off.
>>>
>>> But I've been sitting on this for far too long, so here goes.
>>>
>>> (I've not at all tested perf-cgroup)
>>>
>>> The code compiles in between patches, but I've not bothered trying to run/test
>>> any intermediate stage. There's just too many inter-related fail.
>>>
>>> Alexander, Stephane, could you guys please have a hard look at this?
>>>
>>> Andi, Dmitry, patch 10 should explain the getting stuck in
>>> perf_install_in_context() forever thing you both have observed.
>>
>> Btw., if there's no test failures I plan to apply and push this to Linus fairly
>> soon, so guys please give it all the review and testing you can.
>
>
> Yes, I was able to apply it on top of the latest Linus tree. Thanks.
> Now testing perf with these patches at full capacity.

Hit this in a minute:

------------[ cut here ]------------
WARNING: CPU: 0 PID: 12905 at kernel/events/core.c:2651
task_ctx_sched_out+0x8d/0xa0()
Modules linked in:
CPU: 0 PID: 12905 Comm: syz-executor Not tainted 4.4.0+ #224
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
 00000000ffffffff ffff88003290f4e8 ffffffff8290d92d 0000000000000000
 ffff8800348c5f00 ffffffff85fc05c0 ffff88003290f528 ffffffff813429e9
 ffffffff81617ffd ffffffff85fc05c0 0000000000000a5b ffff880035c09188
Call Trace:
 [<     inline     >] __dump_stack lib/dump_stack.c:15
 [<ffffffff8290d92d>] dump_stack+0x6f/0xa2 lib/dump_stack.c:50
 [<ffffffff813429e9>] warn_slowpath_common+0xd9/0x140 kernel/panic.c:483
 [<ffffffff81342c19>] warn_slowpath_null+0x29/0x30 kernel/panic.c:516
 [<ffffffff81617ffd>] task_ctx_sched_out+0x8d/0xa0 kernel/events/core.c:2651
 [<     inline     >] perf_event_context_sched_out kernel/events/core.c:2550
 [<ffffffff81622263>] __perf_event_task_sched_out+0x5d3/0x1230
kernel/events/core.c:2634
 [<     inline     >] perf_event_task_sched_out include/linux/perf_event.h:948
 [<     inline     >] prepare_task_switch kernel/sched/core.c:2613
 [<     inline     >] context_switch kernel/sched/core.c:2771
 [<ffffffff85e7d2d7>] __schedule+0xb27/0x1c50 kernel/sched/core.c:3282
 [<ffffffff85e7eed2>] preempt_schedule_common+0x42/0x70 kernel/sched/core.c:3352
 [<ffffffff85e7ef17>] _cond_resched+0x17/0x20 kernel/sched/core.c:4720
 [<     inline     >] __wait_for_common kernel/sched/completion.c:90
 [<     inline     >] wait_for_common kernel/sched/completion.c:101
 [<ffffffff85e8158b>] wait_for_completion+0x8b/0x300
kernel/sched/completion.c:122
 [<ffffffff8149255e>] __wait_rcu_gp+0x12e/0x1a0 kernel/rcu/update.c:365
 [<ffffffff81498218>] synchronize_sched.part.58+0x88/0xb0 kernel/rcu/tree.c:3220
 [<ffffffff814a19a3>] synchronize_sched+0xa3/0x120 kernel/rcu/tree.c:3217
 [<     inline     >] synchronize_rcu include/linux/rcupdate.h:320
 [<ffffffff817dd6be>] namespace_unlock+0xee/0x100 fs/namespace.c:1349
 [<ffffffff817e671e>] drop_collected_mounts+0x8e/0xa0 fs/namespace.c:1762
 [<ffffffff817eaf1c>] put_mnt_ns+0x4c/0x70 fs/namespace.c:3149
 [<ffffffff813a7594>] free_nsproxy+0x44/0x1d0 kernel/nsproxy.c:161
 [<ffffffff813a7992>] switch_task_namespaces+0xa2/0xc0 kernel/nsproxy.c:213
 [<ffffffff813a79c7>] exit_task_namespaces+0x17/0x20 kernel/nsproxy.c:218
 [<ffffffff8134bb69>] do_exit+0x8b9/0x2b80 kernel/exit.c:749
 [<ffffffff8134dfa8>] do_group_exit+0x108/0x330 kernel/exit.c:880
 [<ffffffff81371214>] get_signal+0x5e4/0x1500 kernel/signal.c:2307
 [<ffffffff81193db3>] do_signal+0x83/0x1c90 arch/x86/kernel/signal.c:712
 [<ffffffff81006685>] exit_to_usermode_loop+0x1a5/0x210
arch/x86/entry/common.c:247
 [<     inline     >] prepare_exit_to_usermode arch/x86/entry/common.c:282
 [<ffffffff8100851a>] syscall_return_slowpath+0x2ba/0x340
arch/x86/entry/common.c:344
 [<ffffffff85e8dba2>] int_ret_from_sys_call+0x25/0x9f
arch/x86/entry/entry_64.S:281
---[ end trace 93985d046a082b72 ]---

[toc] | [prev] | [next] | [standalone]

#1307276

FromDmitry Vyukov <dvyukov@google.com>
Date2016-01-12 12:10 +0100
Message-ID<qQ4ZQ-5t-33@gated-at.bofh.it>
In reply to#1307270
On Tue, Jan 12, 2016 at 12:00 PM, Dmitry Vyukov <dvyukov@google.com> wrote:
> On Tue, Jan 12, 2016 at 11:57 AM, Dmitry Vyukov <dvyukov@google.com> wrote:
>> On Tue, Jan 12, 2016 at 11:11 AM, Ingo Molnar <mingo@kernel.org> wrote:
>>>
>>> * Peter Zijlstra <peterz@infradead.org> wrote:
>>>
>>>> Hi,
>>>>
>>>> I've been hunting perf bugs for the past few weeks. This resulted in this pile
>>>> of patches, that mostly seems to work -- I still get an occasional fail so
>>>> something is still off.
>>>>
>>>> But I've been sitting on this for far too long, so here goes.
>>>>
>>>> (I've not at all tested perf-cgroup)
>>>>
>>>> The code compiles in between patches, but I've not bothered trying to run/test
>>>> any intermediate stage. There's just too many inter-related fail.
>>>>
>>>> Alexander, Stephane, could you guys please have a hard look at this?
>>>>
>>>> Andi, Dmitry, patch 10 should explain the getting stuck in
>>>> perf_install_in_context() forever thing you both have observed.
>>>
>>> Btw., if there's no test failures I plan to apply and push this to Linus fairly
>>> soon, so guys please give it all the review and testing you can.
>>
>>
>> Yes, I was able to apply it on top of the latest Linus tree. Thanks.
>> Now testing perf with these patches at full capacity.
>
> Hit this in a minute:
>
> ------------[ cut here ]------------
> WARNING: CPU: 0 PID: 12905 at kernel/events/core.c:2651
> task_ctx_sched_out+0x8d/0xa0()
> Modules linked in:
> CPU: 0 PID: 12905 Comm: syz-executor Not tainted 4.4.0+ #224
> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
>  00000000ffffffff ffff88003290f4e8 ffffffff8290d92d 0000000000000000
>  ffff8800348c5f00 ffffffff85fc05c0 ffff88003290f528 ffffffff813429e9
>  ffffffff81617ffd ffffffff85fc05c0 0000000000000a5b ffff880035c09188
> Call Trace:
>  [<     inline     >] __dump_stack lib/dump_stack.c:15
>  [<ffffffff8290d92d>] dump_stack+0x6f/0xa2 lib/dump_stack.c:50
>  [<ffffffff813429e9>] warn_slowpath_common+0xd9/0x140 kernel/panic.c:483
>  [<ffffffff81342c19>] warn_slowpath_null+0x29/0x30 kernel/panic.c:516
>  [<ffffffff81617ffd>] task_ctx_sched_out+0x8d/0xa0 kernel/events/core.c:2651
>  [<     inline     >] perf_event_context_sched_out kernel/events/core.c:2550
>  [<ffffffff81622263>] __perf_event_task_sched_out+0x5d3/0x1230
> kernel/events/core.c:2634
>  [<     inline     >] perf_event_task_sched_out include/linux/perf_event.h:948
>  [<     inline     >] prepare_task_switch kernel/sched/core.c:2613
>  [<     inline     >] context_switch kernel/sched/core.c:2771
>  [<ffffffff85e7d2d7>] __schedule+0xb27/0x1c50 kernel/sched/core.c:3282
>  [<ffffffff85e7eed2>] preempt_schedule_common+0x42/0x70 kernel/sched/core.c:3352
>  [<ffffffff85e7ef17>] _cond_resched+0x17/0x20 kernel/sched/core.c:4720
>  [<     inline     >] __wait_for_common kernel/sched/completion.c:90
>  [<     inline     >] wait_for_common kernel/sched/completion.c:101
>  [<ffffffff85e8158b>] wait_for_completion+0x8b/0x300
> kernel/sched/completion.c:122
>  [<ffffffff8149255e>] __wait_rcu_gp+0x12e/0x1a0 kernel/rcu/update.c:365
>  [<ffffffff81498218>] synchronize_sched.part.58+0x88/0xb0 kernel/rcu/tree.c:3220
>  [<ffffffff814a19a3>] synchronize_sched+0xa3/0x120 kernel/rcu/tree.c:3217
>  [<     inline     >] synchronize_rcu include/linux/rcupdate.h:320
>  [<ffffffff817dd6be>] namespace_unlock+0xee/0x100 fs/namespace.c:1349
>  [<ffffffff817e671e>] drop_collected_mounts+0x8e/0xa0 fs/namespace.c:1762
>  [<ffffffff817eaf1c>] put_mnt_ns+0x4c/0x70 fs/namespace.c:3149
>  [<ffffffff813a7594>] free_nsproxy+0x44/0x1d0 kernel/nsproxy.c:161
>  [<ffffffff813a7992>] switch_task_namespaces+0xa2/0xc0 kernel/nsproxy.c:213
>  [<ffffffff813a79c7>] exit_task_namespaces+0x17/0x20 kernel/nsproxy.c:218
>  [<ffffffff8134bb69>] do_exit+0x8b9/0x2b80 kernel/exit.c:749
>  [<ffffffff8134dfa8>] do_group_exit+0x108/0x330 kernel/exit.c:880
>  [<ffffffff81371214>] get_signal+0x5e4/0x1500 kernel/signal.c:2307
>  [<ffffffff81193db3>] do_signal+0x83/0x1c90 arch/x86/kernel/signal.c:712
>  [<ffffffff81006685>] exit_to_usermode_loop+0x1a5/0x210
> arch/x86/entry/common.c:247
>  [<     inline     >] prepare_exit_to_usermode arch/x86/entry/common.c:282
>  [<ffffffff8100851a>] syscall_return_slowpath+0x2ba/0x340
> arch/x86/entry/common.c:344
>  [<ffffffff85e8dba2>] int_ret_from_sys_call+0x25/0x9f
> arch/x86/entry/entry_64.S:281
> ---[ end trace 93985d046a082b72 ]---


And this:

------------[ cut here ]------------
WARNING: CPU: 3 PID: 24027 at kernel/events/core.c:2744
ctx_sched_in+0x253/0x1390()
Modules linked in:
CPU: 3 PID: 24027 Comm: syz-executor Not tainted 4.4.0+ #224
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
 00000000ffffffff ffff8800620079d0 ffffffff8290d92d 0000000000000000
 ffff880065652f80 ffffffff85fc05c0 ffff880062007a10 ffffffff813429e9
 ffffffff8161bd13 ffffffff85fc05c0 0000000000000ab8 0000000000000003
Call Trace:
 [<     inline     >] __dump_stack lib/dump_stack.c:15
 [<ffffffff8290d92d>] dump_stack+0x6f/0xa2 lib/dump_stack.c:50
 [<ffffffff813429e9>] warn_slowpath_common+0xd9/0x140 kernel/panic.c:483
 [<ffffffff81342c19>] warn_slowpath_null+0x29/0x30 kernel/panic.c:516
 [<ffffffff8161bd13>] ctx_sched_in+0x253/0x1390 kernel/events/core.c:2744
 [<ffffffff8161ce97>] perf_event_sched_in+0x47/0xa0 kernel/events/core.c:2106
 [<ffffffff8161cfcc>] ctx_resched+0xdc/0x1d0 kernel/events/core.c:2119
 [<ffffffff8161db6b>] __perf_install_in_context+0x12b/0x1b0
kernel/events/core.c:2150
 [<ffffffff8160c36f>] remote_function+0x12f/0x1b0 kernel/events/core.c:74
 [<ffffffff814eab83>] generic_exec_single+0x253/0x450 kernel/smp.c:156
 [<ffffffff814eaf96>] smp_call_function_single+0x216/0x340 kernel/smp.c:300
 [<ffffffff81606b7c>] task_function_call+0x11c/0x130 kernel/events/core.c:101
 [<ffffffff81611bc8>] perf_install_in_context+0x318/0x640
kernel/events/core.c:2193
 [<ffffffff81628a20>] SYSC_perf_event_open+0x1720/0x1fa0
kernel/events/core.c:8457
 [<ffffffff8162fad9>] SyS_perf_event_open+0x39/0x50 kernel/events/core.c:8153
 [<ffffffff85e8da36>] entry_SYSCALL_64_fastpath+0x16/0x7a
arch/x86/entry/entry_64.S:185
---[ end trace 3930c39e9993a16d ]---

[toc] | [prev] | [next] | [standalone]

#1307285

FromDmitry Vyukov <dvyukov@google.com>
Date2016-01-12 12:30 +0100
Message-ID<qQ5jb-cc-1@gated-at.bofh.it>
In reply to#1307276
On Tue, Jan 12, 2016 at 12:01 PM, Dmitry Vyukov <dvyukov@google.com> wrote:
> On Tue, Jan 12, 2016 at 12:00 PM, Dmitry Vyukov <dvyukov@google.com> wrote:
>> On Tue, Jan 12, 2016 at 11:57 AM, Dmitry Vyukov <dvyukov@google.com> wrote:
>>> On Tue, Jan 12, 2016 at 11:11 AM, Ingo Molnar <mingo@kernel.org> wrote:
>>>>
>>>> * Peter Zijlstra <peterz@infradead.org> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> I've been hunting perf bugs for the past few weeks. This resulted in this pile
>>>>> of patches, that mostly seems to work -- I still get an occasional fail so
>>>>> something is still off.
>>>>>
>>>>> But I've been sitting on this for far too long, so here goes.
>>>>>
>>>>> (I've not at all tested perf-cgroup)
>>>>>
>>>>> The code compiles in between patches, but I've not bothered trying to run/test
>>>>> any intermediate stage. There's just too many inter-related fail.
>>>>>
>>>>> Alexander, Stephane, could you guys please have a hard look at this?
>>>>>
>>>>> Andi, Dmitry, patch 10 should explain the getting stuck in
>>>>> perf_install_in_context() forever thing you both have observed.
>>>>
>>>> Btw., if there's no test failures I plan to apply and push this to Linus fairly
>>>> soon, so guys please give it all the review and testing you can.
>>>
>>>
>>> Yes, I was able to apply it on top of the latest Linus tree. Thanks.
>>> Now testing perf with these patches at full capacity.
>>
>> Hit this in a minute:
>>
>> ------------[ cut here ]------------
>> WARNING: CPU: 0 PID: 12905 at kernel/events/core.c:2651
>> task_ctx_sched_out+0x8d/0xa0()
>> Modules linked in:
>> CPU: 0 PID: 12905 Comm: syz-executor Not tainted 4.4.0+ #224
>> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
>>  00000000ffffffff ffff88003290f4e8 ffffffff8290d92d 0000000000000000
>>  ffff8800348c5f00 ffffffff85fc05c0 ffff88003290f528 ffffffff813429e9
>>  ffffffff81617ffd ffffffff85fc05c0 0000000000000a5b ffff880035c09188
>> Call Trace:
>>  [<     inline     >] __dump_stack lib/dump_stack.c:15
>>  [<ffffffff8290d92d>] dump_stack+0x6f/0xa2 lib/dump_stack.c:50
>>  [<ffffffff813429e9>] warn_slowpath_common+0xd9/0x140 kernel/panic.c:483
>>  [<ffffffff81342c19>] warn_slowpath_null+0x29/0x30 kernel/panic.c:516
>>  [<ffffffff81617ffd>] task_ctx_sched_out+0x8d/0xa0 kernel/events/core.c:2651
>>  [<     inline     >] perf_event_context_sched_out kernel/events/core.c:2550
>>  [<ffffffff81622263>] __perf_event_task_sched_out+0x5d3/0x1230
>> kernel/events/core.c:2634
>>  [<     inline     >] perf_event_task_sched_out include/linux/perf_event.h:948
>>  [<     inline     >] prepare_task_switch kernel/sched/core.c:2613
>>  [<     inline     >] context_switch kernel/sched/core.c:2771
>>  [<ffffffff85e7d2d7>] __schedule+0xb27/0x1c50 kernel/sched/core.c:3282
>>  [<ffffffff85e7eed2>] preempt_schedule_common+0x42/0x70 kernel/sched/core.c:3352
>>  [<ffffffff85e7ef17>] _cond_resched+0x17/0x20 kernel/sched/core.c:4720
>>  [<     inline     >] __wait_for_common kernel/sched/completion.c:90
>>  [<     inline     >] wait_for_common kernel/sched/completion.c:101
>>  [<ffffffff85e8158b>] wait_for_completion+0x8b/0x300
>> kernel/sched/completion.c:122
>>  [<ffffffff8149255e>] __wait_rcu_gp+0x12e/0x1a0 kernel/rcu/update.c:365
>>  [<ffffffff81498218>] synchronize_sched.part.58+0x88/0xb0 kernel/rcu/tree.c:3220
>>  [<ffffffff814a19a3>] synchronize_sched+0xa3/0x120 kernel/rcu/tree.c:3217
>>  [<     inline     >] synchronize_rcu include/linux/rcupdate.h:320
>>  [<ffffffff817dd6be>] namespace_unlock+0xee/0x100 fs/namespace.c:1349
>>  [<ffffffff817e671e>] drop_collected_mounts+0x8e/0xa0 fs/namespace.c:1762
>>  [<ffffffff817eaf1c>] put_mnt_ns+0x4c/0x70 fs/namespace.c:3149
>>  [<ffffffff813a7594>] free_nsproxy+0x44/0x1d0 kernel/nsproxy.c:161
>>  [<ffffffff813a7992>] switch_task_namespaces+0xa2/0xc0 kernel/nsproxy.c:213
>>  [<ffffffff813a79c7>] exit_task_namespaces+0x17/0x20 kernel/nsproxy.c:218
>>  [<ffffffff8134bb69>] do_exit+0x8b9/0x2b80 kernel/exit.c:749
>>  [<ffffffff8134dfa8>] do_group_exit+0x108/0x330 kernel/exit.c:880
>>  [<ffffffff81371214>] get_signal+0x5e4/0x1500 kernel/signal.c:2307
>>  [<ffffffff81193db3>] do_signal+0x83/0x1c90 arch/x86/kernel/signal.c:712
>>  [<ffffffff81006685>] exit_to_usermode_loop+0x1a5/0x210
>> arch/x86/entry/common.c:247
>>  [<     inline     >] prepare_exit_to_usermode arch/x86/entry/common.c:282
>>  [<ffffffff8100851a>] syscall_return_slowpath+0x2ba/0x340
>> arch/x86/entry/common.c:344
>>  [<ffffffff85e8dba2>] int_ret_from_sys_call+0x25/0x9f
>> arch/x86/entry/entry_64.S:281
>> ---[ end trace 93985d046a082b72 ]---
>
>
> And this:
>
> ------------[ cut here ]------------
> WARNING: CPU: 3 PID: 24027 at kernel/events/core.c:2744
> ctx_sched_in+0x253/0x1390()
> Modules linked in:
> CPU: 3 PID: 24027 Comm: syz-executor Not tainted 4.4.0+ #224
> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
>  00000000ffffffff ffff8800620079d0 ffffffff8290d92d 0000000000000000
>  ffff880065652f80 ffffffff85fc05c0 ffff880062007a10 ffffffff813429e9
>  ffffffff8161bd13 ffffffff85fc05c0 0000000000000ab8 0000000000000003
> Call Trace:
>  [<     inline     >] __dump_stack lib/dump_stack.c:15
>  [<ffffffff8290d92d>] dump_stack+0x6f/0xa2 lib/dump_stack.c:50
>  [<ffffffff813429e9>] warn_slowpath_common+0xd9/0x140 kernel/panic.c:483
>  [<ffffffff81342c19>] warn_slowpath_null+0x29/0x30 kernel/panic.c:516
>  [<ffffffff8161bd13>] ctx_sched_in+0x253/0x1390 kernel/events/core.c:2744
>  [<ffffffff8161ce97>] perf_event_sched_in+0x47/0xa0 kernel/events/core.c:2106
>  [<ffffffff8161cfcc>] ctx_resched+0xdc/0x1d0 kernel/events/core.c:2119
>  [<ffffffff8161db6b>] __perf_install_in_context+0x12b/0x1b0
> kernel/events/core.c:2150
>  [<ffffffff8160c36f>] remote_function+0x12f/0x1b0 kernel/events/core.c:74
>  [<ffffffff814eab83>] generic_exec_single+0x253/0x450 kernel/smp.c:156
>  [<ffffffff814eaf96>] smp_call_function_single+0x216/0x340 kernel/smp.c:300
>  [<ffffffff81606b7c>] task_function_call+0x11c/0x130 kernel/events/core.c:101
>  [<ffffffff81611bc8>] perf_install_in_context+0x318/0x640
> kernel/events/core.c:2193
>  [<ffffffff81628a20>] SYSC_perf_event_open+0x1720/0x1fa0
> kernel/events/core.c:8457
>  [<ffffffff8162fad9>] SyS_perf_event_open+0x39/0x50 kernel/events/core.c:8153
>  [<ffffffff85e8da36>] entry_SYSCALL_64_fastpath+0x16/0x7a
> arch/x86/entry/entry_64.S:185
> ---[ end trace 3930c39e9993a16d ]---


Whole assortment:

2016/01/12 11:57:23 qemu-11: saving crash 'WARNING: CPU: 0 PID: 12905
at kernel/events/core.c:2651 task_ctx_sched_out+0x8d/0xa0()' to
crash-qemu-11-1452596243752451147
2016/01/12 11:59:50 qemu-0: saving crash 'WARNING: CPU: 3 PID: 24027
at kernel/events/core.c:2744 ctx_sched_in+0x253/0x1390()' to
crash-qemu-0-1452596390210686680
2016/01/12 12:05:30 qemu-18: saving crash 'WARNING: CPU: 2 PID: 10497
at kernel/events/core.c:2145 __perf_install_in_context+0x177/0x1b0()'
to crash-qemu-18-1452596730362194373
2016/01/12 12:08:26 qemu-12: saving crash 'WARNING: CPU: 2 PID: 26946
at kernel/events/core.c:213 event_function+0x4ba/0x590()' to
crash-qemu-12-1452596906889159952
2016/01/12 12:11:09 qemu-3: saving crash 'WARNING: CPU: 1 PID: 27863
at kernel/events/core.c:213 event_function+0x4ba/0x590()' to
crash-qemu-3-1452597069411271690
2016/01/12 12:13:38 qemu-5: saving crash 'WARNING: CPU: 1 PID: 18036
at kernel/events/core.c:213 event_function+0x4ba/0x590()' to
crash-qemu-5-1452597218740474180
2016/01/12 12:14:00 qemu-2: saving crash 'WARNING: CPU: 1 PID: 27287
at kernel/events/core.c:213 event_function+0x4ba/0x590()' to
crash-qemu-2-1452597240810824249
2016/01/12 12:14:21 qemu-13: saving crash 'WARNING: CPU: 0 PID: 24151
at kernel/events/core.c:213 event_function+0x4ba/0x590()' to
crash-qemu-13-1452597261384610361
2016/01/12 12:15:40 qemu-12: saving crash 'WARNING: CPU: 2 PID: 1802
at kernel/events/core.c:1698 event_sched_out.isra.86+0x79c/0xf40()' to
crash-qemu-12-1452597340496838168
2016/01/12 12:16:17 qemu-1: saving crash 'WARNING: CPU: 2 PID: 5975 at
kernel/events/core.c:213 event_function+0x4ba/0x590()' to
crash-qemu-1-1452597377189059606
2016/01/12 12:16:51 qemu-2: saving crash 'WARNING: CPU: 2 PID: 8948 at
kernel/events/core.c:1698 event_sched_out.isra.86+0x79c/0xf40()' to
crash-qemu-2-1452597411120105533
2016/01/12 12:17:56 qemu-1: saving crash 'WARNING: CPU: 2 PID: 30319
at kernel/events/core.c:1698 event_sched_out.isra.86+0x79c/0xf40()' to
crash-qemu-1-1452597476734445707
2016/01/12 12:18:52 qemu-7: saving crash 'WARNING: CPU: 1 PID: 31877
at kernel/events/core.c:213 event_function+0x4ba/0x590()' to
crash-qemu-7-1452597532508063377
2016/01/12 12:19:07 qemu-13: saving crash 'WARNING: CPU: 1 PID: 964 at
kernel/events/core.c:2145 __perf_install_in_context+0x177/0x1b0()' to
crash-qemu-13-1452597547020900569


Peter,

What do you think if we work on making syzkaller work for you locally?
I think it will be more efficient than testing on my side. I am ready
to provide any necessary help. If you are interested please drop a
email to syzkaller@googlegroups.com.

[toc] | [prev] | [next] | [standalone]

#1307297

FromDmitry Vyukov <dvyukov@google.com>
Date2016-01-12 12:40 +0100
Message-ID<qQ5sS-fW-5@gated-at.bofh.it>
In reply to#1307285
On Tue, Jan 12, 2016 at 12:26 PM, Dmitry Vyukov <dvyukov@google.com> wrote:
> Whole assortment:
>
> 2016/01/12 11:57:23 qemu-11: saving crash 'WARNING: CPU: 0 PID: 12905
> at kernel/events/core.c:2651 task_ctx_sched_out+0x8d/0xa0()' to
> crash-qemu-11-1452596243752451147
> 2016/01/12 11:59:50 qemu-0: saving crash 'WARNING: CPU: 3 PID: 24027
> at kernel/events/core.c:2744 ctx_sched_in+0x253/0x1390()' to
> crash-qemu-0-1452596390210686680
> 2016/01/12 12:05:30 qemu-18: saving crash 'WARNING: CPU: 2 PID: 10497
> at kernel/events/core.c:2145 __perf_install_in_context+0x177/0x1b0()'
> to crash-qemu-18-1452596730362194373
> 2016/01/12 12:08:26 qemu-12: saving crash 'WARNING: CPU: 2 PID: 26946
> at kernel/events/core.c:213 event_function+0x4ba/0x590()' to
> crash-qemu-12-1452596906889159952
> 2016/01/12 12:11:09 qemu-3: saving crash 'WARNING: CPU: 1 PID: 27863
> at kernel/events/core.c:213 event_function+0x4ba/0x590()' to
> crash-qemu-3-1452597069411271690
> 2016/01/12 12:13:38 qemu-5: saving crash 'WARNING: CPU: 1 PID: 18036
> at kernel/events/core.c:213 event_function+0x4ba/0x590()' to
> crash-qemu-5-1452597218740474180
> 2016/01/12 12:14:00 qemu-2: saving crash 'WARNING: CPU: 1 PID: 27287
> at kernel/events/core.c:213 event_function+0x4ba/0x590()' to
> crash-qemu-2-1452597240810824249
> 2016/01/12 12:14:21 qemu-13: saving crash 'WARNING: CPU: 0 PID: 24151
> at kernel/events/core.c:213 event_function+0x4ba/0x590()' to
> crash-qemu-13-1452597261384610361
> 2016/01/12 12:15:40 qemu-12: saving crash 'WARNING: CPU: 2 PID: 1802
> at kernel/events/core.c:1698 event_sched_out.isra.86+0x79c/0xf40()' to
> crash-qemu-12-1452597340496838168
> 2016/01/12 12:16:17 qemu-1: saving crash 'WARNING: CPU: 2 PID: 5975 at
> kernel/events/core.c:213 event_function+0x4ba/0x590()' to
> crash-qemu-1-1452597377189059606
> 2016/01/12 12:16:51 qemu-2: saving crash 'WARNING: CPU: 2 PID: 8948 at
> kernel/events/core.c:1698 event_sched_out.isra.86+0x79c/0xf40()' to
> crash-qemu-2-1452597411120105533
> 2016/01/12 12:17:56 qemu-1: saving crash 'WARNING: CPU: 2 PID: 30319
> at kernel/events/core.c:1698 event_sched_out.isra.86+0x79c/0xf40()' to
> crash-qemu-1-1452597476734445707
> 2016/01/12 12:18:52 qemu-7: saving crash 'WARNING: CPU: 1 PID: 31877
> at kernel/events/core.c:213 event_function+0x4ba/0x590()' to
> crash-qemu-7-1452597532508063377
> 2016/01/12 12:19:07 qemu-13: saving crash 'WARNING: CPU: 1 PID: 964 at
> kernel/events/core.c:2145 __perf_install_in_context+0x177/0x1b0()' to
> crash-qemu-13-1452597547020900569
>
>
> Peter,
>
> What do you think if we work on making syzkaller work for you locally?
> I think it will be more efficient than testing on my side. I am ready
> to provide any necessary help. If you are interested please drop a
> email to syzkaller@googlegroups.com.


Also got 2 different stalls, so it's not just WARNINGS.
Difficult to say whether the situation become better or worse, because
it was actively failing in different ways before as well...

[toc] | [prev] | [next] | [standalone]

#1307335

FromPeter Zijlstra <peterz@infradead.org>
Date2016-01-12 13:10 +0100
Message-ID<qQ5VV-FV-39@gated-at.bofh.it>
In reply to#1307285
On Tue, Jan 12, 2016 at 12:26:27PM +0100, Dmitry Vyukov wrote:
> Peter,
> 
> What do you think if we work on making syzkaller work for you locally?
> I think it will be more efficient than testing on my side. I am ready
> to provide any necessary help. If you are interested please drop a
> email to syzkaller@googlegroups.com.

I did manage to hit some fail locally (as stated in the cover letter),
let me try and hunt that down before I try that.

I think you hit some of the same, but way faster than I did :-)

Thanks for testing though!

[toc] | [prev] | [next] | [standalone]

#1308498

FromAlexander Shishkin <alexander.shishkin@linux.intel.com>
Date2016-01-13 16:20 +0100
Message-ID<qQvnk-1xE-27@gated-at.bofh.it>
In reply to#1307285
Dmitry Vyukov <dvyukov@google.com> writes:

> 2016/01/12 12:08:26 qemu-12: saving crash 'WARNING: CPU: 2 PID: 26946
> at kernel/events/core.c:213 event_function+0x4ba/0x590()' to

If you still have the code, can you check which of the WARN_ON_ONCE()s
is this?

Thanks,
--
Alex

[toc] | [prev] | [next] | [standalone]

#1308506

FromDmitry Vyukov <dvyukov@google.com>
Date2016-01-13 16:30 +0100
Message-ID<qQvx1-1Cf-23@gated-at.bofh.it>
In reply to#1308498
On Wed, Jan 13, 2016 at 4:18 PM, Alexander Shishkin
<alexander.shishkin@linux.intel.com> wrote:
> Dmitry Vyukov <dvyukov@google.com> writes:
>
>> 2016/01/12 12:08:26 qemu-12: saving crash 'WARNING: CPU: 2 PID: 26946
>> at kernel/events/core.c:213 event_function+0x4ba/0x590()' to
>
> If you still have the code, can you check which of the WARN_ON_ONCE()s
> is this?

Here is my core.c
https://gist.github.com/dvyukov/3b3a4993ade37e344636


>
> Thanks,
> --
> Alex

[toc] | [prev] | [next] | [standalone]

Page 1 of 2  [1] 2  Next page →

Back to top | Article view | linux.kernel

csiph-web