Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1693199 > unrolled thread

Re: [PATCH 0/2] sparc64: Use low latency path to resume idle cpu

Back to article view | Back to linux.kernel

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: [PATCH 0/2] sparc64: Use low latency path to resume idle cpu David Miller <davem@davemloft.net> - 2017-07-20 22:00 +0200
    Re: [PATCH 0/2] sparc64: Use low latency path to resume idle cpu Vijay Kumar <vijay.ac.kumar@oracle.com> - 2017-07-21 04:50 +0200
      Re: [PATCH 0/2] sparc64: Use low latency path to resume idle cpu David Miller <davem@davemloft.net> - 2017-07-21 05:00 +0200
        Re: [PATCH 0/2] sparc64: Use low latency path to resume idle cpu Vijay Kumar <vijay.ac.kumar@oracle.com> - 2017-07-21 05:40 +0200
          Re: [PATCH 0/2] sparc64: Use low latency path to resume idle cpu David Miller <davem@davemloft.net> - 2017-07-21 05:50 +0200
            Re: [PATCH 0/2] sparc64: Use low latency path to resume idle cpu Vijay Kumar <vijay.ac.kumar@oracle.com> - 2017-07-21 06:20 +0200

#1693199 — Re: [PATCH 0/2] sparc64: Use low latency path to resume idle cpu

From: Vijay Kumar <vijay.ac.kumar@oracle.com>
Date: Sat,  8 Jul 2017 14:23:42 -0600

> cpu_poke is a low latency path to resume the target cpu if suspended
> using cpu_yield. Use cpu poke to resume cpu if supported by hypervisor.
> 
> 	     hackbench results (lower is better):
> Number of		
> Process:		w/o fix		with fix
> 1  			0.012		 0.010
> 10			0.021		 0.019
> 100			0.151		 0.148

So this only works for a cpu which has yielded.

The kernel sends reschedule events to both idle and non-idle cpus.
That's why you have to have that fallback code to still send the
mondo IPI right?

For the case where POKE works, it seems like completely unnecessary
overhead to set the PIL interrupt.  Just disable local cpu interrupts
and call schedule_ipi() directly.

I bet that improves your benchmark even more.

[toc] | [next] | [standalone]

#1693355

On 7/20/2017 2:57 PM, David Miller wrote:
> From: Vijay Kumar <vijay.ac.kumar@oracle.com>
> Date: Sat,  8 Jul 2017 14:23:42 -0600
>
>> cpu_poke is a low latency path to resume the target cpu if suspended
>> using cpu_yield. Use cpu poke to resume cpu if supported by hypervisor.
>>
>> 	     hackbench results (lower is better):
>> Number of		
>> Process:		w/o fix		with fix
>> 1  			0.012		 0.010
>> 10			0.021		 0.019
>> 100			0.151		 0.148
> So this only works for a cpu which has yielded.
>
> The kernel sends reschedule events to both idle and non-idle cpus.
> That's why you have to have that fallback code to still send the
> mondo IPI right?
That is correct.

>
> For the case where POKE works, it seems like completely unnecessary
> overhead to set the PIL interrupt.  Just disable local cpu interrupts
> and call schedule_ipi() directly.
>
> I bet that improves your benchmark even more.

I had same thoughts initially but I had to go with this approach as 
scheduler_ipi is wrapped with irq_enter() and irq_exit(). Whereas POKE 
resumes the cpu in process context.

Comments in scheduler_ipi():

          * Not all reschedule IPI handlers call irq_enter/irq_exit, since
          * traditionally all their work was done from the interrupt return
          * path. Now that we actually do some work, we need to make sure
          * we do call them.
          *
          * Some archs already do call them, luckily irq_enter/exit nest
          * properly.
          *
          * Arguably we should visit all archs and update all handlers,
          * however a fair share of IPIs are still resched only so this 
would
          * somewhat pessimize the simple resched case.
          */
         irq_enter();


-Vijay

[toc] | [prev] | [next] | [standalone]

#1693357

From: Vijay Kumar <vijay.ac.kumar@oracle.com>
Date: Thu, 20 Jul 2017 21:44:24 -0500

> I had same thoughts initially but I had to go with this approach as
> scheduler_ipi is wrapped with irq_enter() and irq_exit(). Whereas POKE
> resumes the cpu in process context.
> 
> Comments in scheduler_ipi():
> 
>          * Not all reschedule IPI handlers call irq_enter/irq_exit, since
>          * traditionally all their work was done from the interrupt return
>          * path. Now that we actually do some work, we need to make sure
>          * we do call them.
>          *
>          * Some archs already do call them, luckily irq_enter/exit nest
>          * properly.
>          *
>          * Arguably we should visit all archs and update all handlers,
>          * however a fair share of IPIs are still resched only so this would
>          * somewhat pessimize the simple resched case.
>          */
>         irq_enter();
> 

I still think we should be able to fake the state such that this
direct schedule_ipi() call will work.

I could be wrong :)

[toc] | [prev] | [next] | [standalone]

#1693367

On 7/20/2017 9:55 PM, David Miller wrote:
> From: Vijay Kumar <vijay.ac.kumar@oracle.com>
> Date: Thu, 20 Jul 2017 21:44:24 -0500
>
>> I had same thoughts initially but I had to go with this approach as
>> scheduler_ipi is wrapped with irq_enter() and irq_exit(). Whereas POKE
>> resumes the cpu in process context.
>>
>> Comments in scheduler_ipi():
>>
>>           * Not all reschedule IPI handlers call irq_enter/irq_exit, since
>>           * traditionally all their work was done from the interrupt return
>>           * path. Now that we actually do some work, we need to make sure
>>           * we do call them.
>>           *
>>           * Some archs already do call them, luckily irq_enter/exit nest
>>           * properly.
>>           *
>>           * Arguably we should visit all archs and update all handlers,
>>           * however a fair share of IPIs are still resched only so this would
>>           * somewhat pessimize the simple resched case.
>>           */
>>          irq_enter();
>>
> I still think we should be able to fake the state such that this
> direct schedule_ipi() call will work.
>
> I could be wrong :)
I can give a try :). But looks to me one thing that will go wrong is irq 
accounting done in __irq_enter() and rcu_irq_enter().

Thanks,
Vijay

[toc] | [prev] | [next] | [standalone]

#1693372

From: Vijay Kumar <vijay.ac.kumar@oracle.com>
Date: Thu, 20 Jul 2017 22:36:42 -0500

> I can give a try :). But looks to me one thing that will go wrong is
> irq accounting done in __irq_enter() and rcu_irq_enter().

Actually, the bigger problem is that scheduler_ipi() can raise a
software interrupt, and nothing will invoke it.

It's turning quite ugly to avoid the IRQ overhead, I must admit.
So ignore this for now.

In the longer term a probably cleaner way to do this is to have
a special direct version of scheduler_ipi() that invokes all the
necessary work, even the rebalance softirq, directly rather than
indirectly.

[toc] | [prev] | [next] | [standalone]

#1693383

On 7/20/2017 10:45 PM, David Miller wrote:
> From: Vijay Kumar <vijay.ac.kumar@oracle.com>
> Date: Thu, 20 Jul 2017 22:36:42 -0500
>
>> I can give a try :). But looks to me one thing that will go wrong is
>> irq accounting done in __irq_enter() and rcu_irq_enter().
> Actually, the bigger problem is that scheduler_ipi() can raise a
> software interrupt, and nothing will invoke it.
Yes, I see your point.

> It's turning quite ugly to avoid the IRQ overhead, I must admit.
> So ignore this for now.
>
> In the longer term a probably cleaner way to do this is to have
> a special direct version of scheduler_ipi() that invokes all the
> necessary work, even the rebalance softirq, directly rather than
> indirectly.

Sure. Thanks.

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web