Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1265656 > unrolled thread

Re: [PATCH v4 3/3] sched: optimize migration by forcing rmb() and updating to be called once

Back to article view | Back to linux.kernel

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: [PATCH v4 3/3] sched: optimize migration by forcing rmb() and  updating to be called once Peter Zijlstra <peterz@infradead.org> - 2015-11-09 14:30 +0100
    Re: [PATCH v4 3/3] sched: optimize migration by forcing rmb() and  updating to be called once Byungchul Park <byungchul.park@lge.com> - 2015-11-10 02:20 +0100
      Re: [PATCH v4 3/3] sched: optimize migration by forcing rmb() and  updating to be called once Peter Zijlstra <peterz@infradead.org> - 2015-11-10 13:20 +0100
        Re: [PATCH v4 3/3] sched: optimize migration by forcing rmb() and  updating to be called once Byungchul Park <byungchul.park@lge.com> - 2015-11-11 01:00 +0100
          Re: [PATCH v4 3/3] sched: optimize migration by forcing rmb() and  updating to be called once Byungchul Park <byungchul.park@lge.com> - 2015-11-11 11:20 +0100
          Re: [PATCH v4 3/3] sched: optimize migration by forcing rmb() and  updating to be called once Peter Zijlstra <peterz@infradead.org> - 2015-11-16 14:00 +0100
            Re: [PATCH v4 3/3] sched: optimize migration by forcing rmb() and  updating to be called once Byungchul Park <byungchul.park@lge.com> - 2015-11-17 01:50 +0100
              Re: [PATCH v4 3/3] sched: optimize migration by forcing rmb() and  updating to be called once Peter Zijlstra <peterz@infradead.org> - 2015-11-17 12:30 +0100
                Re: [PATCH v4 3/3] sched: optimize migration by forcing rmb() and  updating to be called once Byungchul Park <byungchul.park@lge.com> - 2015-11-18 00:40 +0100
                  Re: [PATCH v4 3/3] sched: optimize migration by forcing rmb() and  updating to be called once Peter Zijlstra <peterz@infradead.org> - 2015-11-18 01:00 +0100
                    Re: [PATCH v4 3/3] sched: optimize migration by forcing rmb() and  updating to be called once Byungchul Park <byungchul.park@lge.com> - 2015-11-18 01:10 +0100

#1265656 — Re: [PATCH v4 3/3] sched: optimize migration by forcing rmb() and updating to be called once

On Sat, Oct 24, 2015 at 01:16:21AM +0900, byungchul.park@lge.com wrote:
> +++ b/kernel/sched/core.c
> @@ -1264,6 +1264,8 @@ EXPORT_SYMBOL_GPL(set_cpus_allowed_ptr);
>  
>  void set_task_cpu(struct task_struct *p, unsigned int new_cpu)
>  {
> +	unsigned int prev_cpu = task_cpu(p);
> +
>  #ifdef CONFIG_SCHED_DEBUG
>  	/*
>  	 * We should never call set_task_cpu() on a blocked task,
> @@ -1289,15 +1291,14 @@ void set_task_cpu(struct task_struct *p, unsigned int new_cpu)
>  #endif
>  
>  	trace_sched_migrate_task(p, new_cpu);
> +	__set_task_cpu(p, new_cpu);
>  
> -	if (task_cpu(p) != new_cpu) {
> +	if (prev_cpu != new_cpu) {
>  		if (p->sched_class->migrate_task_rq)
> -			p->sched_class->migrate_task_rq(p, new_cpu);
> +			p->sched_class->migrate_task_rq(p, prev_cpu);
>  		p->se.nr_migrations++;
>  		perf_event_task_migrate(p);
>  	}
> -
> -	__set_task_cpu(p, new_cpu);
>  }

I don't think this is safe, see the comment in __set_task_cpu(). We want
that to be last.

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [next] | [standalone]

#1266136

On Mon, Nov 09, 2015 at 02:29:14PM +0100, Peter Zijlstra wrote:
> On Sat, Oct 24, 2015 at 01:16:21AM +0900, byungchul.park@lge.com wrote:
> > +++ b/kernel/sched/core.c
> > @@ -1264,6 +1264,8 @@ EXPORT_SYMBOL_GPL(set_cpus_allowed_ptr);
> >  
> >  void set_task_cpu(struct task_struct *p, unsigned int new_cpu)
> >  {
> > +	unsigned int prev_cpu = task_cpu(p);
> > +
> >  #ifdef CONFIG_SCHED_DEBUG
> >  	/*
> >  	 * We should never call set_task_cpu() on a blocked task,
> > @@ -1289,15 +1291,14 @@ void set_task_cpu(struct task_struct *p, unsigned int new_cpu)
> >  #endif
> >  
> >  	trace_sched_migrate_task(p, new_cpu);
> > +	__set_task_cpu(p, new_cpu);
> >  
> > -	if (task_cpu(p) != new_cpu) {
> > +	if (prev_cpu != new_cpu) {
> >  		if (p->sched_class->migrate_task_rq)
> > -			p->sched_class->migrate_task_rq(p, new_cpu);
> > +			p->sched_class->migrate_task_rq(p, prev_cpu);
> >  		p->se.nr_migrations++;
> >  		perf_event_task_migrate(p);
> >  	}
> > -
> > -	__set_task_cpu(p, new_cpu);
> >  }
> 
> I don't think this is safe, see the comment in __set_task_cpu(). We want
> that to be last.

I am sorry but I don't understand what you said. I checked the comment in 
__set_task_cpu().

	/*
	 * After ->cpu is set up to a new value, task_rq_lock(p, ...) can be
	 * successfuly executed on another CPU. We must ensure that updates of
	 * per-task data have been completed by this moment.
	 */

Of course, ->cpu should be set up to a new value for task_rq_lock() to be
executed successfully on another CPU. Is this the case? Is there something
i missed? I think it would be ok if task->pi_lock can work correctly within
"if" statement in set_task_cpu(). Is there problem to do that?

> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1266452

On Tue, Nov 10, 2015 at 10:09:05AM +0900, Byungchul Park wrote:
> On Mon, Nov 09, 2015 at 02:29:14PM +0100, Peter Zijlstra wrote:
> > On Sat, Oct 24, 2015 at 01:16:21AM +0900, byungchul.park@lge.com wrote:
> > > +++ b/kernel/sched/core.c
> > > @@ -1264,6 +1264,8 @@ EXPORT_SYMBOL_GPL(set_cpus_allowed_ptr);
> > >  
> > >  void set_task_cpu(struct task_struct *p, unsigned int new_cpu)
> > >  {
> > > +	unsigned int prev_cpu = task_cpu(p);
> > > +
> > >  #ifdef CONFIG_SCHED_DEBUG
> > >  	/*
> > >  	 * We should never call set_task_cpu() on a blocked task,
> > > @@ -1289,15 +1291,14 @@ void set_task_cpu(struct task_struct *p, unsigned int new_cpu)
> > >  #endif
> > >  
> > >  	trace_sched_migrate_task(p, new_cpu);
> > > +	__set_task_cpu(p, new_cpu);
> > >  
> > > -	if (task_cpu(p) != new_cpu) {
> > > +	if (prev_cpu != new_cpu) {
> > >  		if (p->sched_class->migrate_task_rq)
> > > -			p->sched_class->migrate_task_rq(p, new_cpu);
> > > +			p->sched_class->migrate_task_rq(p, prev_cpu);
> > >  		p->se.nr_migrations++;
> > >  		perf_event_task_migrate(p);
> > >  	}
> > > -
> > > -	__set_task_cpu(p, new_cpu);
> > >  }
> > 
> > I don't think this is safe, see the comment in __set_task_cpu(). We want
> > that to be last.
> 
> I am sorry but I don't understand what you said. I checked the comment in 
> __set_task_cpu().
> 
> 	/*
> 	 * After ->cpu is set up to a new value, task_rq_lock(p, ...) can be
> 	 * successfuly executed on another CPU. We must ensure that updates of
> 	 * per-task data have been completed by this moment.
> 	 */
> 
> Of course, ->cpu should be set up to a new value for task_rq_lock() to be
> executed successfully on another CPU. Is this the case? Is there something
> i missed? I think it would be ok if task->pi_lock can work correctly within
> "if" statement in set_task_cpu(). Is there problem to do that?

So the problem is that as soon as that ->cpu store comes through, the
other rq->lock can happen, even though we might still hold a rq->lock
thinking we're serialized.

Take for instance move_queued_tasks(), it does:

	dequeue_task(rq, p, 0);
	p->on_rq = TASK_ON_RQ_MIGRATING;
	set_task_cpu(p, new_cpu) {
	  __set_task_cpu();

^^^ here holding rq->lock is insufficient and the below:

	  p->sched_class->migrate_task_rq()

would no longer be serialized by rq->lock.

	}
	raw_spin_unlock(&rq->lock);


--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1266877

On Tue, Nov 10, 2015 at 01:16:47PM +0100, Peter Zijlstra wrote:
> On Tue, Nov 10, 2015 at 10:09:05AM +0900, Byungchul Park wrote:
> > On Mon, Nov 09, 2015 at 02:29:14PM +0100, Peter Zijlstra wrote:
> > > On Sat, Oct 24, 2015 at 01:16:21AM +0900, byungchul.park@lge.com wrote:
> > > > +++ b/kernel/sched/core.c
> > > > @@ -1264,6 +1264,8 @@ EXPORT_SYMBOL_GPL(set_cpus_allowed_ptr);
> > > >  
> > > >  void set_task_cpu(struct task_struct *p, unsigned int new_cpu)
> > > >  {
> > > > +	unsigned int prev_cpu = task_cpu(p);
> > > > +
> > > >  #ifdef CONFIG_SCHED_DEBUG
> > > >  	/*
> > > >  	 * We should never call set_task_cpu() on a blocked task,
> > > > @@ -1289,15 +1291,14 @@ void set_task_cpu(struct task_struct *p, unsigned int new_cpu)
> > > >  #endif
> > > >  
> > > >  	trace_sched_migrate_task(p, new_cpu);
> > > > +	__set_task_cpu(p, new_cpu);
> > > >  
> > > > -	if (task_cpu(p) != new_cpu) {
> > > > +	if (prev_cpu != new_cpu) {
> > > >  		if (p->sched_class->migrate_task_rq)
> > > > -			p->sched_class->migrate_task_rq(p, new_cpu);
> > > > +			p->sched_class->migrate_task_rq(p, prev_cpu);
> > > >  		p->se.nr_migrations++;
> > > >  		perf_event_task_migrate(p);
> > > >  	}
> > > > -
> > > > -	__set_task_cpu(p, new_cpu);
> > > >  }
> > > 
> > > I don't think this is safe, see the comment in __set_task_cpu(). We want
> > > that to be last.
> > 
> > I am sorry but I don't understand what you said. I checked the comment in 
> > __set_task_cpu().
> > 
> > 	/*
> > 	 * After ->cpu is set up to a new value, task_rq_lock(p, ...) can be
> > 	 * successfuly executed on another CPU. We must ensure that updates of
> > 	 * per-task data have been completed by this moment.
> > 	 */
> > 
> > Of course, ->cpu should be set up to a new value for task_rq_lock() to be
> > executed successfully on another CPU. Is this the case? Is there something
> > i missed? I think it would be ok if task->pi_lock can work correctly within
> > "if" statement in set_task_cpu(). Is there problem to do that?
> 
> So the problem is that as soon as that ->cpu store comes through, the
> other rq->lock can happen, even though we might still hold a rq->lock
> thinking we're serialized.
> 
> Take for instance move_queued_tasks(), it does:
> 
> 	dequeue_task(rq, p, 0);
> 	p->on_rq = TASK_ON_RQ_MIGRATING;
> 	set_task_cpu(p, new_cpu) {
> 	  __set_task_cpu();
> 
> ^^^ here holding rq->lock is insufficient and the below:
> 
> 	  p->sched_class->migrate_task_rq()

Thank you for explaning in detail, but this's why i asked you.
Yes, rq->lock is insufficient in this place as you said, but
should migrate_task_rq() be serialized by rq->lock? I might have
agreed with you if the migrate_task_rq() should be serialized by
rq->lock, but I think it's not the case. I think it would be of
if task->pi_lock can work correcly within *if statement* in 
set_task_cpu(). Wrong?

> 
> would no longer be serialized by rq->lock.
> 
> 	}
> 	raw_spin_unlock(&rq->lock);
> 
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1267085

On Wed, Nov 11, 2015 at 08:51:47AM +0900, Byungchul Park wrote:
> On Tue, Nov 10, 2015 at 01:16:47PM +0100, Peter Zijlstra wrote:
> > On Tue, Nov 10, 2015 at 10:09:05AM +0900, Byungchul Park wrote:
> > > On Mon, Nov 09, 2015 at 02:29:14PM +0100, Peter Zijlstra wrote:
> > > > On Sat, Oct 24, 2015 at 01:16:21AM +0900, byungchul.park@lge.com wrote:
> > > > > +++ b/kernel/sched/core.c
> > > > > @@ -1264,6 +1264,8 @@ EXPORT_SYMBOL_GPL(set_cpus_allowed_ptr);
> > > > >  
> > > > >  void set_task_cpu(struct task_struct *p, unsigned int new_cpu)
> > > > >  {
> > > > > +	unsigned int prev_cpu = task_cpu(p);
> > > > > +
> > > > >  #ifdef CONFIG_SCHED_DEBUG
> > > > >  	/*
> > > > >  	 * We should never call set_task_cpu() on a blocked task,
> > > > > @@ -1289,15 +1291,14 @@ void set_task_cpu(struct task_struct *p, unsigned int new_cpu)
> > > > >  #endif
> > > > >  
> > > > >  	trace_sched_migrate_task(p, new_cpu);
> > > > > +	__set_task_cpu(p, new_cpu);
> > > > >  
> > > > > -	if (task_cpu(p) != new_cpu) {
> > > > > +	if (prev_cpu != new_cpu) {
> > > > >  		if (p->sched_class->migrate_task_rq)
> > > > > -			p->sched_class->migrate_task_rq(p, new_cpu);
> > > > > +			p->sched_class->migrate_task_rq(p, prev_cpu);
> > > > >  		p->se.nr_migrations++;
> > > > >  		perf_event_task_migrate(p);
> > > > >  	}
> > > > > -
> > > > > -	__set_task_cpu(p, new_cpu);
> > > > >  }
> > > > 
> > > > I don't think this is safe, see the comment in __set_task_cpu(). We want
> > > > that to be last.
> > > 
> > > I am sorry but I don't understand what you said. I checked the comment in 
> > > __set_task_cpu().
> > > 
> > > 	/*
> > > 	 * After ->cpu is set up to a new value, task_rq_lock(p, ...) can be
> > > 	 * successfuly executed on another CPU. We must ensure that updates of
> > > 	 * per-task data have been completed by this moment.
> > > 	 */
> > > 
> > > Of course, ->cpu should be set up to a new value for task_rq_lock() to be
> > > executed successfully on another CPU. Is this the case? Is there something
> > > i missed? I think it would be ok if task->pi_lock can work correctly within
> > > "if" statement in set_task_cpu(). Is there problem to do that?
> > 
> > So the problem is that as soon as that ->cpu store comes through, the
> > other rq->lock can happen, even though we might still hold a rq->lock
> > thinking we're serialized.
> > 
> > Take for instance move_queued_tasks(), it does:
> > 
> > 	dequeue_task(rq, p, 0);
> > 	p->on_rq = TASK_ON_RQ_MIGRATING;
> > 	set_task_cpu(p, new_cpu) {
> > 	  __set_task_cpu();
> > 
> > ^^^ here holding rq->lock is insufficient and the below:
> > 
> > 	  p->sched_class->migrate_task_rq()
> 
> Thank you for explaning in detail, but this's why i asked you.
> Yes, rq->lock is insufficient in this place as you said, but
> should migrate_task_rq() be serialized by rq->lock? I might have
> agreed with you if the migrate_task_rq() should be serialized by
> rq->lock, but I think it's not the case. I think it would be of

rq->lock, but I think it's not the case. I think it would be *ok*

(sorry for typo)

> if task->pi_lock can work correcly within *if statement* in 
> set_task_cpu(). Wrong?
> 
> > 
> > would no longer be serialized by rq->lock.
> > 
> > 	}
> > 	raw_spin_unlock(&rq->lock);
> > 
> > 
> > --
> > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> > Please read the FAQ at  http://www.tux.org/lkml/
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1270089

On Wed, Nov 11, 2015 at 08:51:47AM +0900, Byungchul Park wrote:
> On Tue, Nov 10, 2015 at 01:16:47PM +0100, Peter Zijlstra wrote:
> > So the problem is that as soon as that ->cpu store comes through, the
> > other rq->lock can happen, even though we might still hold a rq->lock
> > thinking we're serialized.
> > 
> > Take for instance move_queued_tasks(), it does:
> > 
> > 	dequeue_task(rq, p, 0);
> > 	p->on_rq = TASK_ON_RQ_MIGRATING;
> > 	set_task_cpu(p, new_cpu) {
> > 	  __set_task_cpu();
> > 
> > ^^^ here holding rq->lock is insufficient and the below:
> > 
> > 	  p->sched_class->migrate_task_rq()
> 
> Thank you for explaning in detail, but this's why i asked you.

> Yes, rq->lock is insufficient in this place as you said, but
> should migrate_task_rq() be serialized by rq->lock? I might have
> agreed with you if the migrate_task_rq() should be serialized by
> rq->lock, but I think it's not the case. I think it would be of
> if task->pi_lock can work correcly within *if statement* in 
> set_task_cpu(). Wrong?

So currently, set_task_cpu() is serialized by:

 - p->pi_lock; on wakeup
 - rq->lock; otherwise

(see the #ifdef CONFIG_LOCKDEP comment in set_task_cpu())

This means that sched_class::migrate_task() cannot indeed rely on
rq->lock for full serialization, however it still means that
task_rq_lock() will fully serialize against the thing.

By changing this, it no longer will.

Even without that; I think such a change, if correct, is very fragile
and prone to creating problems later on, and sets bad precedent.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1270724

On Mon, Nov 16, 2015 at 01:53:51PM +0100, Peter Zijlstra wrote:
> On Wed, Nov 11, 2015 at 08:51:47AM +0900, Byungchul Park wrote:
> > On Tue, Nov 10, 2015 at 01:16:47PM +0100, Peter Zijlstra wrote:
> > > So the problem is that as soon as that ->cpu store comes through, the
> > > other rq->lock can happen, even though we might still hold a rq->lock
> > > thinking we're serialized.
> > > 
> > > Take for instance move_queued_tasks(), it does:
> > > 
> > > 	dequeue_task(rq, p, 0);
> > > 	p->on_rq = TASK_ON_RQ_MIGRATING;
> > > 	set_task_cpu(p, new_cpu) {
> > > 	  __set_task_cpu();
> > > 
> > > ^^^ here holding rq->lock is insufficient and the below:
> > > 
> > > 	  p->sched_class->migrate_task_rq()
> > 
> > Thank you for explaning in detail, but this's why i asked you.
> 
> > Yes, rq->lock is insufficient in this place as you said, but
> > should migrate_task_rq() be serialized by rq->lock? I might have
> > agreed with you if the migrate_task_rq() should be serialized by
> > rq->lock, but I think it's not the case. I think it would be of
> > if task->pi_lock can work correcly within *if statement* in 
> > set_task_cpu(). Wrong?
> 
> So currently, set_task_cpu() is serialized by:
> 
>  - p->pi_lock; on wakeup
>  - rq->lock; otherwise
> 
> (see the #ifdef CONFIG_LOCKDEP comment in set_task_cpu())

I already read the comment.. Then do you mean the comment above
migrate_task_rq_fair() is wrong and should be fixed? I thought the 
comment above migrate_task_rq_fair() is correct rather than
CONFIG_LOCKDEP comment in set_task_cpu(), when I read it. I think
these two comments are conflict each other a little bit, so one of
those should be fixed.

* the comment above migrate_task_rq_fair() describes it like,
Caller SHOULD HOLD (&p->pi_lock)

* the CONFIG_LOCKDEP comment in set_task_cpu() describes it like,
Caller SHOULD HOLD (&p->pi_lock || &rq->lock)

> 
> This means that sched_class::migrate_task() cannot indeed rely on
> rq->lock for full serialization, however it still means that
> task_rq_lock() will fully serialize against the thing.

Yes I also think this is true.

> 
> By changing this, it no longer will.

???

> 
> Even without that; I think such a change, if correct, is very fragile
> and prone to creating problems later on, and sets bad precedent.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1271094

On Tue, Nov 17, 2015 at 09:44:16AM +0900, Byungchul Park wrote:

> > So currently, set_task_cpu() is serialized by:
> > 
> >  - p->pi_lock; on wakeup
> >  - rq->lock; otherwise
> > 
> > (see the #ifdef CONFIG_LOCKDEP comment in set_task_cpu())
> 
> I already read the comment.. Then do you mean the comment above
> migrate_task_rq_fair() is wrong and should be fixed? 

Looks that way, I'm not sure we always hold pi_lock there. But I'm low
on sleep, so I could have overlooked something.

See for example move_queued_task(), we call set_task_cpu() with rq->lock
held, but no pi_lock.

> I thought the comment above migrate_task_rq_fair() is correct rather
> than CONFIG_LOCKDEP comment in set_task_cpu(), when I read it. I think
> these two comments are conflict each other a little bit, so one of
> those should be fixed.

Agreed.

> * the comment above migrate_task_rq_fair() describes it like,
> Caller SHOULD HOLD (&p->pi_lock)
> 
> * the CONFIG_LOCKDEP comment in set_task_cpu() describes it like,
> Caller SHOULD HOLD (&p->pi_lock || &rq->lock)

Indeed.

> > 
> > This means that sched_class::migrate_task() cannot indeed rely on
> > rq->lock for full serialization, however it still means that
> > task_rq_lock() will fully serialize against the thing.
> 
> Yes I also think this is true.
> 
> > 
> > By changing this, it no longer will.
> 
> ???

I meant, if you call __set_task_cpu() before
sched_class::migrate_task_rq(), in that case task_rq_lock() will no
longer fully serialize against set_task_cpu().

Because once you've called __set_task_cpu(), task_rq_lock() will acquire
the _other_ rq->lock. And we cannot rely on our rq->lock to serialize
things.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1271741

On Tue, Nov 17, 2015 at 12:21:49PM +0100, Peter Zijlstra wrote:
> 
> Looks that way, I'm not sure we always hold pi_lock there. But I'm low
> on sleep, so I could have overlooked something.
> 
> See for example move_queued_task(), we call set_task_cpu() with rq->lock
> held, but no pi_lock.

Indeed.

> 
> > I thought the comment above migrate_task_rq_fair() is correct rather
> > than CONFIG_LOCKDEP comment in set_task_cpu(), when I read it. I think
> > these two comments are conflict each other a little bit, so one of
> > those should be fixed.
> 
> Agreed.

Which one do you think to be fixed? The one above migrate_task_rq_fair()?
I wonder if it would be ok even it does not hold pi_lock in
migrate_task_rq_fair(). If you say *no problem*, I will try to fix the
comment.

> 
> I meant, if you call __set_task_cpu() before
> sched_class::migrate_task_rq(), in that case task_rq_lock() will no
> longer fully serialize against set_task_cpu().
> 
> Because once you've called __set_task_cpu(), task_rq_lock() will acquire
> the _other_ rq->lock. And we cannot rely on our rq->lock to serialize
> things.

I agree with you if migtrate_task_rq() can be serialized by rq->lock
without holding pi_lock. (even though I am still wondering..)

But I thought it was no problem if migrate_task_rq() was serialized only
by pi_lock as the comment above the migrate_task_rq() describes, because
breaking rq->lock does not affect the sericalization by pi_lock.

I would appreciate it if you would answer my questions.

> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1271748

On Wed, Nov 18, 2015 at 08:37:00AM +0900, Byungchul Park wrote:

> Which one do you think to be fixed? The one above migrate_task_rq_fair()?
> I wonder if it would be ok even it does not hold pi_lock in
> migrate_task_rq_fair(). If you say *no problem*, I will try to fix the
> comment.

The one above migrate_task_rq_fair() is obviously broken, as
demonstrated by the move_queued_task() case.

Also, pretty much all runnable task migration code will not take
pi_lock, see also {pull,push}_{rt,dl}_task().

Note that this is done very much by design, task_rq_lock() is the thing
that fully serializes a task's scheduler state. Runnable tasks use
rq->lock, waking tasks use pi_lock.

> > I meant, if you call __set_task_cpu() before
> > sched_class::migrate_task_rq(), in that case task_rq_lock() will no
> > longer fully serialize against set_task_cpu().
> > 
> > Because once you've called __set_task_cpu(), task_rq_lock() will acquire
> > the _other_ rq->lock. And we cannot rely on our rq->lock to serialize
> > things.
> 
> I agree with you if migtrate_task_rq() can be serialized by rq->lock
> without holding pi_lock. (even though I am still wondering..)

move_queued_task() illustrates this.

> But I thought it was no problem if migrate_task_rq() was serialized only
> by pi_lock as the comment above the migrate_task_rq() describes, because
> breaking rq->lock does not affect the sericalization by pi_lock.

Right, but per the above, we cannot assume pi_lock is in fact held over
this.

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1271750

On Wed, Nov 18, 2015 at 12:55:10AM +0100, Peter Zijlstra wrote:
> On Wed, Nov 18, 2015 at 08:37:00AM +0900, Byungchul Park wrote:
> 
> > Which one do you think to be fixed? The one above migrate_task_rq_fair()?
> > I wonder if it would be ok even it does not hold pi_lock in
> > migrate_task_rq_fair(). If you say *no problem*, I will try to fix the
> > comment.
> 
> The one above migrate_task_rq_fair() is obviously broken, as
> demonstrated by the move_queued_task() case.
> 
> Also, pretty much all runnable task migration code will not take
> pi_lock, see also {pull,push}_{rt,dl}_task().
> 
> Note that this is done very much by design, task_rq_lock() is the thing
> that fully serializes a task's scheduler state. Runnable tasks use
> rq->lock, waking tasks use pi_lock.
> 
> > > I meant, if you call __set_task_cpu() before
> > > sched_class::migrate_task_rq(), in that case task_rq_lock() will no
> > > longer fully serialize against set_task_cpu().
> > > 
> > > Because once you've called __set_task_cpu(), task_rq_lock() will acquire
> > > the _other_ rq->lock. And we cannot rely on our rq->lock to serialize
> > > things.
> > 
> > I agree with you if migtrate_task_rq() can be serialized by rq->lock
> > without holding pi_lock. (even though I am still wondering..)
> 
> move_queued_task() illustrates this.
> 
> > But I thought it was no problem if migrate_task_rq() was serialized only
> > by pi_lock as the comment above the migrate_task_rq() describes, because
> > breaking rq->lock does not affect the sericalization by pi_lock.
> 
> Right, but per the above, we cannot assume pi_lock is in fact held over
> this.

Thank you.

> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web