Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1340971 > unrolled thread

Re: [PATCH v11 08/10] dt, numa: Add NUMA dt binding implementation.

Back to article view | Back to linux.kernel

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: [PATCH v11 08/10] dt, numa: Add NUMA dt binding implementation. Rob Herring <robh@kernel.org> - 2016-02-23 20:40 +0100
    Re: [PATCH v11 08/10] dt, numa: Add NUMA dt binding implementation. Will Deacon <will.deacon@arm.com> - 2016-02-26 19:30 +0100
      Re: [PATCH v11 08/10] dt, numa: Add NUMA dt binding implementation. Rob Herring <robh@kernel.org> - 2016-03-01 18:00 +0100
    Re: [PATCH v11 08/10] dt, numa: Add NUMA dt binding implementation. Rob Herring <robh@kernel.org> - 2016-03-01 17:50 +0100
      Re: [PATCH v11 08/10] dt, numa: Add NUMA dt binding implementation. Rob Herring <robh@kernel.org> - 2016-03-01 18:50 +0100

#1340971 — Re: [PATCH v11 08/10] dt, numa: Add NUMA dt binding implementation.

On Fri, Feb 19, 2016 at 05:13:17PM -0800, David Daney wrote:
> From: Ganapatrao Kulkarni <gkulkarni@caviumnetworks.com>
> 
> ADD device tree node parsing for NUMA topology using device
> "numa-node-id" property distance-map.

I still want an adequate explanation why NUMA setup cannot be done with 
an unflattened tree. PowerPC manages to do that and should have a 
similar init flow being memblock based, so I would expect arm64 can too.

Rob

[toc] | [next] | [standalone]

#1344543

On Thu, Feb 25, 2016 at 05:26:34PM -0800, David Daney wrote:
> On 02/23/2016 11:36 AM, Rob Herring wrote:
> >On Fri, Feb 19, 2016 at 05:13:17PM -0800, David Daney wrote:
> >>From: Ganapatrao Kulkarni <gkulkarni@caviumnetworks.com>
> >>
> >>ADD device tree node parsing for NUMA topology using device
> >>"numa-node-id" property distance-map.
> >
> >I still want an adequate explanation why NUMA setup cannot be done with
> >an unflattened tree. PowerPC manages to do that and should have a
> >similar init flow being memblock based, so I would expect arm64 can too.
> 
> Many things could be done.  Really, we want to know what *should* be done.
> 
> In the context of the current arm64 memory initialization we (more or less)
> do:
> 
>  1) early_init_fdt_scan_reserved_mem();
>  2) memory_present()
>  3) sparse_init()
>  4) other things
>  5) unflatten_device_tree()
> 
> We are already reading information out of the FDT at #1.
> 
> This patch set adds a step between 1 and 2 where we read NUMA information
> out of the FDT.
> 
> Hypothetically, it might be possible to rewrite the arm64 setup code so that
> the ordering was different, and the NUMA setup was done on the unflattened
> tree, but that would certainly be a much more invasive patch.

I just looked at what PPC get up to, and there's really not an obvious
way we could do that on arm64. They run whole swathes of the kernel
with the MMU off directly from head.S to parse the flattened tree and
get memblock up really early. On arm64, the head.S environment is
considerably more hostile, and I don't think we'd want to do that (not
to mention the interaction with EFI stub).

So I'm perfectly happy for this to operate on the flattened tree.

Will

[toc] | [prev] | [next] | [standalone]

#1346755

On Fri, Feb 26, 2016 at 12:27 PM, Will Deacon <will.deacon@arm.com> wrote:
> On Thu, Feb 25, 2016 at 05:26:34PM -0800, David Daney wrote:
>> On 02/23/2016 11:36 AM, Rob Herring wrote:
>> >On Fri, Feb 19, 2016 at 05:13:17PM -0800, David Daney wrote:
>> >>From: Ganapatrao Kulkarni <gkulkarni@caviumnetworks.com>
>> >>
>> >>ADD device tree node parsing for NUMA topology using device
>> >>"numa-node-id" property distance-map.
>> >
>> >I still want an adequate explanation why NUMA setup cannot be done with
>> >an unflattened tree. PowerPC manages to do that and should have a
>> >similar init flow being memblock based, so I would expect arm64 can too.
>>
>> Many things could be done.  Really, we want to know what *should* be done.
>>
>> In the context of the current arm64 memory initialization we (more or less)
>> do:
>>
>>  1) early_init_fdt_scan_reserved_mem();
>>  2) memory_present()
>>  3) sparse_init()
>>  4) other things
>>  5) unflatten_device_tree()
>>
>> We are already reading information out of the FDT at #1.
>>
>> This patch set adds a step between 1 and 2 where we read NUMA information
>> out of the FDT.
>>
>> Hypothetically, it might be possible to rewrite the arm64 setup code so that
>> the ordering was different, and the NUMA setup was done on the unflattened
>> tree, but that would certainly be a much more invasive patch.
>
> I just looked at what PPC get up to, and there's really not an obvious
> way we could do that on arm64. They run whole swathes of the kernel
> with the MMU off directly from head.S to parse the flattened tree and
> get memblock up really early. On arm64, the head.S environment is
> considerably more hostile, and I don't think we'd want to do that (not
> to mention the interaction with EFI stub).
>
> So I'm perfectly happy for this to operate on the flattened tree.

Of course you are. The code is not in your tree. I'm all for keeping
things out of /arch, but just want to understand what are the
dependencies which aren't clearly spelled out here.

Rob

[toc] | [prev] | [next] | [standalone]

#1346750

On Thu, Feb 25, 2016 at 7:26 PM, David Daney <ddaney@caviumnetworks.com> wrote:
> On 02/23/2016 11:36 AM, Rob Herring wrote:
>>
>> On Fri, Feb 19, 2016 at 05:13:17PM -0800, David Daney wrote:
>>>
>>> From: Ganapatrao Kulkarni <gkulkarni@caviumnetworks.com>
>>>
>>> ADD device tree node parsing for NUMA topology using device
>>> "numa-node-id" property distance-map.
>>
>>
>> I still want an adequate explanation why NUMA setup cannot be done with
>> an unflattened tree. PowerPC manages to do that and should have a
>> similar init flow being memblock based, so I would expect arm64 can too.
>
>
> Many things could be done.  Really, we want to know what *should* be done.
>
> In the context of the current arm64 memory initialization we (more or less)
> do:
>
>  1) early_init_fdt_scan_reserved_mem();
>  2) memory_present()
>  3) sparse_init()
>  4) other things
>  5) unflatten_device_tree()
>
> We are already reading information out of the FDT at #1.
>
> This patch set adds a step between 1 and 2 where we read NUMA information
> out of the FDT.

The dependency on unflattening is that memblock is up and we can
allocate a chunk from it. Isn't that dependency met by step 1 or is
there a dependency on sparsemem (or something else)?

Rob

[toc] | [prev] | [next] | [standalone]

#1346809

On Tue, Mar 1, 2016 at 10:57 AM, David Daney <ddaney@caviumnetworks.com> wrote:
> On 03/01/2016 08:47 AM, Rob Herring wrote:
>>
>> On Thu, Feb 25, 2016 at 7:26 PM, David Daney <ddaney@caviumnetworks.com>
>> wrote:
>>>
>>> On 02/23/2016 11:36 AM, Rob Herring wrote:
>>>>
>>>>
>>>> On Fri, Feb 19, 2016 at 05:13:17PM -0800, David Daney wrote:
>>>>>
>>>>>
>>>>> From: Ganapatrao Kulkarni <gkulkarni@caviumnetworks.com>
>>>>>
>>>>> ADD device tree node parsing for NUMA topology using device
>>>>> "numa-node-id" property distance-map.
>>>>
>>>>
>>>>
>>>> I still want an adequate explanation why NUMA setup cannot be done with
>>>> an unflattened tree. PowerPC manages to do that and should have a
>>>> similar init flow being memblock based, so I would expect arm64 can too.
>>>
>>>
>>>
>>> Many things could be done.  Really, we want to know what *should* be
>>> done.
>>>
>>> In the context of the current arm64 memory initialization we (more or
>>> less)
>>> do:
>>>
>>>   1) early_init_fdt_scan_reserved_mem();
>>>   2) memory_present()
>>>   3) sparse_init()
>>>   4) other things
>>>   5) unflatten_device_tree()
>>>
>>> We are already reading information out of the FDT at #1.
>>>
>>> This patch set adds a step between 1 and 2 where we read NUMA information
>>> out of the FDT.
>>
>>
>> The dependency on unflattening is that memblock is up and we can
>> allocate a chunk from it. Isn't that dependency met by step 1
>
>
> No.

Really, because it seems that numa_alloc_distance is essentially doing
a memblock alloc and that happens before memory_present.

>
>> or is
>> there a dependency on sparsemem (or something else)?
>
>
> Will Deacon talked about this over here:
>
> https://lkml.org/lkml/2016/2/26/782

I'm not saying to move memblock setup earlier nor before the MMU is
on, so I don't see how Will's reply is relevant other than PPC doesn't
serve as an example. Maybe PPC should be ignored because I think maybe
NUMA is only used on non-FDT systems.

In any case, no one has clearly explained what the dependencies are or
what happens if you moved the unflattening up sooner. You told me what
the current order is which doesn't equate to dependencies. For
example, step 4 may or may not be a dependency of step 5. These are
the dependencies I'm aware of:

memblock dependent on DT memory and reserved-memory parsing
unflattening dependent on memblock_alloc()
sparsemem dependent on NUMA parsing and memblock

What am I missing from here?

Rob

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web