Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > linux.kernel > #1542833
| From | Mark Rutland <mark.rutland@arm.com> |
|---|---|
| Newsgroups | linux.kernel |
| Subject | Re: [RFC v2.1 1/5] rcu: Introduce for_each_leaf_node_cpu() |
| Date | 2016-12-15 16:40 +0100 |
| Message-ID | <sOGiu-7uh-23@gated-at.bofh.it> (permalink) |
| References | <sOuhj-8sr-3@gated-at.bofh.it> <sOG8O-7rb-37@gated-at.bofh.it> |
| Organization | linux.* mail to news gateway |
On Thu, Dec 15, 2016 at 11:21:05PM +0800, Boqun Feng wrote:
> There are some places inside RCU core, where we need to iterate all mask
> (->qsmask, ->expmask, etc) bits in a leaf node, in order to iterate all
> corresponding CPUs. The current code iterates all possible CPUs in this
> leaf node and then checks with the mask to see whether the bit is set.
>
> However, given the fact that most bits in cpu_possible_mask are set but
> rare bits in an RCU leaf node mask are set(in other words, ->qsmask and
> its friends are usually more sparse than cpu_possible_mask), it's better
> to iterate in the other way, that is iterating mask bits in a leaf node.
> By doing so, we can save several checks in the loop, moreover, that fast
> path checking(e.g. ->qsmask == 0) could then be consolidated into the
> loop logic.
>
> This patch introduce for_each_leaf_node_cpu() to iterate mask bits in a
> more efficient way.
>
> By design, The CPUs whose bits are set in the leaf node masks should be
> a subset of possible CPUs, so we don't need extra check with
> cpu_possible(), however, a WARN_ON_ONCE() is put to check whether there
> are some nasty cases we miss, and we skip that "impossible" CPU in that
> case.
>
> Signed-off-by: Boqun Feng <boqun.feng@gmail.com>
Acked-by: Mark Rutland <mark.rutland@arm.com>
Thanks,
Mark.
> ---
> kernel/rcu/tree.h | 19 +++++++++++++++++++
> 1 file changed, 19 insertions(+)
>
> diff --git a/kernel/rcu/tree.h b/kernel/rcu/tree.h
> index c0a4bf8f1ed0..b35da5b5dab1 100644
> --- a/kernel/rcu/tree.h
> +++ b/kernel/rcu/tree.h
> @@ -295,6 +295,25 @@ struct rcu_node {
> cpu <= rnp->grphi; \
> cpu = cpumask_next((cpu), cpu_possible_mask))
>
> +
> +#define MASK_BITS(mask) (BITS_PER_BYTE * sizeof(mask))
> +/*
> + * Iterate over all CPUs a leaf RCU node which are still masked in
> + * @mask.
> + *
> + * Note @rnp has to be a leaf node and @mask has to belong to @rnp. And we
> + * assume that no CPU is masked in @mask but not set in cpu_possible_mask. IOW,
> + * masks of a leaf node never set a bit for an "impossible" CPU.
> + */
> +#define for_each_leaf_node_cpu(rnp, mask, cpu) \
> + for ((cpu) = (rnp)->grplo + find_first_bit(&(mask), MASK_BITS(mask)); \
> + (cpu) <= (rnp)->grphi; \
> + (cpu) = (rnp)->grplo + find_next_bit(&(mask), MASK_BITS(mask), \
> + (cpu) - (rnp)->grplo + 1)) \
> + if (WARN_ON_ONCE(!cpu_possible(cpu))) \
> + continue; \
> + else
> +
> /*
> * Union to allow "aggregate OR" operation on the need for a quiescent
> * state by the normal and expedited grace periods.
> --
> 2.10.2
>
Back to linux.kernel | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread
[RFC v2 0/5] rcu: Introduce for_each_leaf_node_cpu() Boqun Feng <boqun.feng@gmail.com> - 2016-12-15 03:50 +0100
[RFC v2 1/5] rcu: Introduce for_each_leaf_node_cpu() Boqun Feng <boqun.feng@gmail.com> - 2016-12-15 03:50 +0100
Re: [RFC v2 1/5] rcu: Introduce for_each_leaf_node_cpu() Mark Rutland <mark.rutland@arm.com> - 2016-12-15 12:50 +0100
Re: [RFC v2 1/5] rcu: Introduce for_each_leaf_node_cpu() Boqun Feng <boqun.feng@gmail.com> - 2016-12-15 15:40 +0100
Re: [RFC v2 1/5] rcu: Introduce for_each_leaf_node_cpu() Boqun Feng <boqun.feng@gmail.com> - 2016-12-15 16:20 +0100
Re: [RFC v2 1/5] rcu: Introduce for_each_leaf_node_cpu() Mark Rutland <mark.rutland@arm.com> - 2016-12-15 16:20 +0100
[RFC v2.1 1/5] rcu: Introduce for_each_leaf_node_cpu() Boqun Feng <boqun.feng@gmail.com> - 2016-12-15 16:30 +0100
Re: [RFC v2.1 1/5] rcu: Introduce for_each_leaf_node_cpu() Mark Rutland <mark.rutland@arm.com> - 2016-12-15 16:40 +0100
[RFC v2 2/5] rcu: Use for_each_leaf_node_cpu() in RCU stall checking Boqun Feng <boqun.feng@gmail.com> - 2016-12-15 03:50 +0100
[RFC v2 4/5] rcu: Use for_each_leaf_node_cpu() in force_qs_rnp() Boqun Feng <boqun.feng@gmail.com> - 2016-12-15 03:50 +0100
Re: [RFC v2 4/5] rcu: Use for_each_leaf_node_cpu() in force_qs_rnp() Mark Rutland <mark.rutland@arm.com> - 2016-12-15 13:10 +0100
Re: [RFC v2 4/5] rcu: Use for_each_leaf_node_cpu() in force_qs_rnp() Boqun Feng <boqun.feng@gmail.com> - 2016-12-15 15:50 +0100
Re: [RFC v2 4/5] rcu: Use for_each_leaf_node_cpu() in force_qs_rnp() Colin Ian King <colin.king@canonical.com> - 2016-12-15 16:00 +0100
Re: [RFC v2 4/5] rcu: Use for_each_leaf_node_cpu() in force_qs_rnp() Boqun Feng <boqun.feng@gmail.com> - 2016-12-19 16:20 +0100
Re: [RFC v2 4/5] rcu: Use for_each_leaf_node_cpu() in force_qs_rnp() "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2016-12-20 06:10 +0100
Re: [RFC v2 4/5] rcu: Use for_each_leaf_node_cpu() in force_qs_rnp() Boqun Feng <boqun.feng@gmail.com> - 2016-12-20 07:00 +0100
Re: [RFC v2 4/5] rcu: Use for_each_leaf_node_cpu() in force_qs_rnp() Boqun Feng <boqun.feng@gmail.com> - 2016-12-20 09:20 +0100
Re: [RFC v2 4/5] rcu: Use for_each_leaf_node_cpu() in force_qs_rnp() "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2016-12-20 16:40 +0100
Re: [RFC v2 4/5] rcu: Use for_each_leaf_node_cpu() in force_qs_rnp() "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2016-12-20 16:30 +0100
Re: [RFC v2 4/5] rcu: Use for_each_leaf_node_cpu() in force_qs_rnp() Boqun Feng <boqun.feng@gmail.com> - 2016-12-21 03:40 +0100
Re: [RFC v2 4/5] rcu: Use for_each_leaf_node_cpu() in force_qs_rnp() "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2016-12-21 04:50 +0100
Re: [RFC v2 4/5] rcu: Use for_each_leaf_node_cpu() in force_qs_rnp() Boqun Feng <boqun.feng@gmail.com> - 2016-12-21 05:20 +0100
Re: [RFC v2 4/5] rcu: Use for_each_leaf_node_cpu() in force_qs_rnp() "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2016-12-21 17:50 +0100
Re: [RFC v2 4/5] rcu: Use for_each_leaf_node_cpu() in force_qs_rnp() Boqun Feng <boqun.feng@gmail.com> - 2016-12-22 02:10 +0100
[RFC v2 5/5] rcu: Use for_each_leaf_node_cpu() in online CPU iteration Boqun Feng <boqun.feng@gmail.com> - 2016-12-15 03:50 +0100
[RFC v2 3/5] rcu: Use for_each_leaf_node_cpu() in ->expmask iteration Boqun Feng <boqun.feng@gmail.com> - 2016-12-15 03:50 +0100
csiph-web