Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.kernel > #69347 > unrolled thread

Bug#981186: linux: Enable CMN-600 interconnect on arm64

Back to article view | Back to linux.debian.kernel

Contents

  Bug#981186: linux: Enable CMN-600 interconnect on arm64 Wookey <wookey@debian.org> - 2021-01-27 14:10 +0100
    Bug#981186: linux: Enable CMN-600 interconnect on arm64 Noah Meyerhans <noahm@debian.org> - 2021-02-02 22:40 +0100
    Bug#981186: linux: Enable CMN-600 interconnect on arm64 Wookey <wookey@wookware.org> - 2021-03-17 03:30 +0100
    Processed: Re: Bug#981186: linux: Enable CMN-600 interconnect on  arm64 "Debian Bug Tracking System" <owner@bugs.debian.org> - 2021-03-17 15:00 +0100
    Processed: Re: Bug#981186: linux: Enable CMN-600 interconnect on  arm64 "Debian Bug Tracking System" <owner@bugs.debian.org> - 2021-03-17 15:00 +0100
    Bug#981186: linux: Enable CMN-600 interconnect on arm64 Vincent Blut <vincent.debian@free.fr> - 2021-03-17 15:00 +0100
      Bug#981186: linux: Enable CMN-600 interconnect on arm64 Wookey <wookey@wookware.org> - 2021-03-17 17:00 +0100
        Bug#981186: linux: Enable CMN-600 interconnect on arm64 Vincent Blut <vincent.debian@free.fr> - 2021-03-17 19:50 +0100
          Bug#981186: linux: Enable CMN-600 interconnect on arm64 Wookey <wookey@wookware.org> - 2021-03-17 20:30 +0100
            Bug#981186: linux: Enable CMN-600 interconnect on arm64 Vincent Blut <vincent.debian@free.fr> - 2021-03-17 22:10 +0100
              Bug#981186: linux: Enable CMN-600 interconnect on arm64 Salvatore Bonaccorso <carnil@debian.org> - 2021-03-19 10:00 +0100

#69347 — Bug#981186: linux: Enable CMN-600 interconnect on arm64

[Multipart message — attachments visible in raw view] — view raw

Source: linux
Version: Please enable ARM CMN-600 power management on arm64
Severity: normal
Tags: patch

Current arm hardware such as graviton2 (AWS arm64 hardware) has
'Coherent Mesh Network' interconnect (between components in a
soc). It's important that support for this is built in the kernel so
it can be used.

This requires CONFIG_ARM_CMN=y

This explains what the feature is:
https://www.arm.com/products/silicon-ip-system/corelink-interconnect/cmn-600
Graviton2:
https://www.anandtech.com/show/15578/cloud-clash-amazon-graviton2-arm-against-intel-and-amd

patch attached:
--- debian/config/arm64/config~	2021-01-27 04:34:51.359552398 +0000
+++ debian/config/arm64/config	2021-01-27 04:53:11.922998842 +0000
@@ -942,6 +942,7 @@
 CONFIG_ARM_CCI400_PMU=y
 CONFIG_ARM_CCI5xx_PMU=y
 CONFIG_ARM_CCN=y
+CONFIG_ARM_CMN=y
 CONFIG_QCOM_L2_PMU=y
 CONFIG_QCOM_L3_PMU=y
 CONFIG_XGENE_PMU=y

[toc] | [next] | [standalone]

#69420

On Wed, Jan 27, 2021 at 12:57:07PM +0000, Wookey wrote:
> Current arm hardware such as graviton2 (AWS arm64 hardware) has
> 'Coherent Mesh Network' interconnect (between components in a
> soc). It's important that support for this is built in the kernel so
> it can be used.
> 
> This requires CONFIG_ARM_CMN=y

To be precise, this driver is needed for perf event monitoring of this
interconnect.  The interconnect itself is always in use.

On Amazon EC2, these PMU events are only exposed on the bare-metal
instances (e.g. m6g.metal), not the VMs.

We should still enable support for this driver, in any case.

noah

[toc] | [prev] | [next] | [standalone]

#69836

There is hardware publicly available to buy now that uses the CMN-600 
interconnect so we really should turn it on for this stable release if 
at all possible.

The Ampere ALTRA:https://store.avantek.co.uk/arm-servers.html


Wookey
-- 
Principal hats:  Linaro, Debian, Wookware, ARM
http://wookware.org/

[toc] | [prev] | [next] | [standalone]

#69837 — Processed: Re: Bug#981186: linux: Enable CMN-600 interconnect on arm64

Processing control commands:

> tags -1 moreinfo
Bug #981186 [src:linux] linux: Enable CMN-600 interconnect on arm64
Ignoring request to alter tags of bug #981186 to the same tags previously set

-- 
981186: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=981186
Debian Bug Tracking System
Contact owner@bugs.debian.org with problems

[toc] | [prev] | [next] | [standalone]

#69838 — Processed: Re: Bug#981186: linux: Enable CMN-600 interconnect on arm64

Processing control commands:

> tags -1 moreinfo
Bug #981186 [src:linux] linux: Enable CMN-600 interconnect on arm64
Added tag(s) moreinfo.

-- 
981186: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=981186
Debian Bug Tracking System
Contact owner@bugs.debian.org with problems

[toc] | [prev] | [next] | [standalone]

#69839

[Multipart message — attachments visible in raw view] — view raw

Control: tags -1 moreinfo

Hi Wookey,

Le 2021-01-27 12:57, Wookey a écrit :
> Source: linux
> Version: Please enable ARM CMN-600 power management on arm64
> Severity: normal
> Tags: patch
> 
> Current arm hardware such as graviton2 (AWS arm64 hardware) has
> 'Coherent Mesh Network' interconnect (between components in a
> soc). It's important that support for this is built in the kernel so
> it can be used.
> 
> This requires CONFIG_ARM_CMN=y

Does it really have to be built-in instead of being provided as a module? Last I
checked, Fedora and Ubuntu provide it as a module.

> This explains what the feature is:
> https://www.arm.com/products/silicon-ip-system/corelink-interconnect/cmn-600
> Graviton2:
> https://www.anandtech.com/show/15578/cloud-clash-amazon-graviton2-arm-against-intel-and-amd
> 
> patch attached:
> --- debian/config/arm64/config~	2021-01-27 04:34:51.359552398 +0000
> +++ debian/config/arm64/config	2021-01-27 04:53:11.922998842 +0000
> @@ -942,6 +942,7 @@
>  CONFIG_ARM_CCI400_PMU=y
>  CONFIG_ARM_CCI5xx_PMU=y
>  CONFIG_ARM_CCN=y
> +CONFIG_ARM_CMN=y
>  CONFIG_QCOM_L2_PMU=y
>  CONFIG_QCOM_L3_PMU=y
>  CONFIG_XGENE_PMU=y

> --- debian/config/arm64/config~	2021-01-27 04:34:51.359552398 +0000
> +++ debian/config/arm64/config	2021-01-27 04:53:11.922998842 +0000
> @@ -942,6 +942,7 @@
>  CONFIG_ARM_CCI400_PMU=y
>  CONFIG_ARM_CCI5xx_PMU=y
>  CONFIG_ARM_CCN=y
> +CONFIG_ARM_CMN=y
>  CONFIG_QCOM_L2_PMU=y
>  CONFIG_QCOM_L3_PMU=y
>  CONFIG_XGENE_PMU=y

Cheers,
Vincent

[toc] | [prev] | [next] | [standalone]

#69842

[Multipart message — attachments visible in raw view] — view raw

On 2021-03-17 14:52 +0100, Vincent Blut wrote:
> Control: tags -1 moreinfo
>
> Hi Wookey,
>
> Le 2021-01-27 12:57, Wookey a écrit :
> > Source: linux
> > Version: Please enable ARM CMN-600 power management on arm64
> > Severity: normal
> > Tags: patch
> >
> > Current arm hardware such as graviton2 (AWS arm64 hardware) has
> > 'Coherent Mesh Network' interconnect (between components in a
> > soc). It's important that support for this is built in the kernel so
> > it can be used.
> >
> > This requires CONFIG_ARM_CMN=y
>
> Does it really have to be built-in instead of being provided as a module? Last I
> checked, Fedora and Ubuntu provide it as a module.

No it should really be a module. Perf is driven from userspace so you
never need to use it before modules can be loaded.

I just did it like this because the other settings here are set as
built-ins too and this seemed less disruptive. (If we make it a module
we'll need to make sure it gets included in the right module package -
I'm not sure if that need tweaking somewhere else in the build system)

I see that
CONFIG_THUNDERX2_PMU=y
CONFIG_ARM_SMMU_V3_PMU=y
are also set as builtins. That's probably wrong too.
This should be a module (whic is the upstream default - I'm not sure why
it's coming out as a built-in in the debian build):
CONFIG_THUNDERX2_PMU=m

I'm not sure about CONFIG_ARM_SMMU_V3_PMU=y as it's an architectureal
feature. Best to leave it as a built-in for now.

I also checked the state of the other perf configs with the arm kernel team
and got feedback that we have all the ones that should sensibly be set set once
CONFIG_ARM_CMN=m
and
CONFIG_THUNDERX2_PMU=m
is added

Upstream enables
CONFIG_FSL_IMX8_DDR_PMU=m
by default too. IMX8 hardware is available so we should probably turn this on too

Do you want me to knock up a patch for this or is that enough info?

Wookey
--
Principal hats:  Linaro, Debian, Wookware, ARM
http://wookware.org/

[toc] | [prev] | [next] | [standalone]

#69843

[Multipart message — attachments visible in raw view] — view raw

Le 2021-03-17 15:49, Wookey a écrit :
> On 2021-03-17 14:52 +0100, Vincent Blut wrote:
> > Control: tags -1 moreinfo
> >
> > Hi Wookey,
> >
> > Le 2021-01-27 12:57, Wookey a écrit :
> > > Source: linux
> > > Version: Please enable ARM CMN-600 power management on arm64
> > > Severity: normal
> > > Tags: patch
> > >
> > > Current arm hardware such as graviton2 (AWS arm64 hardware) has
> > > 'Coherent Mesh Network' interconnect (between components in a
> > > soc). It's important that support for this is built in the kernel so
> > > it can be used.
> > >
> > > This requires CONFIG_ARM_CMN=y
> >
> > Does it really have to be built-in instead of being provided as a module? Last I
> > checked, Fedora and Ubuntu provide it as a module.
> 
> No it should really be a module. Perf is driven from userspace so you
> never need to use it before modules can be loaded.

Agreed.

> I just did it like this because the other settings here are set as
> built-ins too and this seemed less disruptive. (If we make it a module
> we'll need to make sure it gets included in the right module package -
> I'm not sure if that need tweaking somewhere else in the build system)
> 
> I see that
> CONFIG_THUNDERX2_PMU=y
> CONFIG_ARM_SMMU_V3_PMU=y
> are also set as builtins. That's probably wrong too.

It seems your arm64 kernel config deviates from the one we provide in Debian.
CONFIG_THUNDERX2_PMU is compiled as a module while CONFIG_ARM_SMMU_V3_PMU is
not set, at least in linux 5.10.19-1.

> […]

> I also checked the state of the other perf configs with the arm kernel team
> and got feedback that we have all the ones that should sensibly be set set once
> CONFIG_ARM_CMN=m
> and
> CONFIG_THUNDERX2_PMU=m
> is added

This means updating the arm64 kernel config to only include ARM_CMN as a module.
To me it is acceptable for Bullseye as this seems uncontroversial, but note that
I can't speak for the kernel team.

> Upstream enables
> CONFIG_FSL_IMX8_DDR_PMU=m
> by default too. IMX8 hardware is available so we should probably turn this on too

Contrary to Ubuntu, we do not provide support for the i.MX8M SoC family,
so enabling this option in the arm64 kernel config is not an option, right?

[toc] | [prev] | [next] | [standalone]

#69844

[Multipart message — attachments visible in raw view] — view raw

On 2021-03-17 19:43 +0100, Vincent Blut wrote:
> Le 2021-03-17 15:49, Wookey a écrit :
> > On 2021-03-17 14:52 +0100, Vincent Blut wrote:
> > > Le 2021-01-27 12:57, Wookey a écrit :
> > > > Version: Please enable ARM CMN-600 power management on arm64
> > > >
> > > > This requires CONFIG_ARM_CMN=y
> > >
> > > Does it really have to be built-in instead of being provided as a module? Last I
> > > checked, Fedora and Ubuntu provide it as a module.
> > 
> > No it should really be a module. Perf is driven from userspace so you
> > never need to use it before modules can be loaded.
> 
> Agreed.

> > I see that
> > CONFIG_THUNDERX2_PMU=y
> > CONFIG_ARM_SMMU_V3_PMU=y
> > are also set as builtins. That's probably wrong too.
> 
> It seems your arm64 kernel config deviates from the one we provide in Debian.
> CONFIG_THUNDERX2_PMU is compiled as a module while CONFIG_ARM_SMMU_V3_PMU is
> not set, at least in linux 5.10.19-1.

Hmm. I was looking at the (built, with CONFIG_ARM_CMN=y) sources for
5.10.9-1 and the (unbuilt) sources for 5.10.19-1. So yes, slightly
different and the built version is not up to date any more.

If we already have CONFIG_THUNDERX2_PMU=m already then that's great
(Ah yes - that's the upstream default).  Adding
CONFIG_ARM_SMMU_V3_PMU=m would be good too. Adding it as a module
should be pretty harmless then at least it's available? I'll set off a
build now to check it works.

> > […]
> 
> > I also checked the state of the other perf configs with the arm kernel team
> > and got feedback that we have all the ones that should sensibly be set set once
> > CONFIG_ARM_CMN=m
> > and
> > CONFIG_THUNDERX2_PMU=m
> > is added
> 
> This means updating the arm64 kernel config to only include ARM_CMN as a module.
> To me it is acceptable for Bullseye as this seems uncontroversial, but note that
> I can't speak for the kernel team.

Will you ask them, or should I? 

It seems like prodding someone would be good as this was filed back on 27th
jan and there have been uploads since, so I guess no-one has noticed till now.

> > Upstream enables
> > CONFIG_FSL_IMX8_DDR_PMU=m
> > by default too. IMX8 hardware is available so we should probably turn this on too
> 
> Contrary to Ubuntu, we do not provide support for the i.MX8M SoC family,
> so enabling this option in the arm64 kernel config is not an option, right?

Ah OK. I didn't realise IMX8 was not enabled in the debian kernel (A
subject for a different bug). In that case, no this is not
appropriate.

Wookey
-- 
Principal hats:  Linaro, Debian, Wookware, ARM
http://wookware.org/

[toc] | [prev] | [next] | [standalone]

#69845

[Multipart message — attachments visible in raw view] — view raw

Le 2021-03-17 19:24, Wookey a écrit :
> On 2021-03-17 19:43 +0100, Vincent Blut wrote:
> > Le 2021-03-17 15:49, Wookey a écrit :
> > > On 2021-03-17 14:52 +0100, Vincent Blut wrote:
> > > > Le 2021-01-27 12:57, Wookey a écrit :
> > > > > Version: Please enable ARM CMN-600 power management on arm64
> > > > >
> > > > > This requires CONFIG_ARM_CMN=y
> > > >
> > > > Does it really have to be built-in instead of being provided as a module? Last I
> > > > checked, Fedora and Ubuntu provide it as a module.
> > > 
> > > No it should really be a module. Perf is driven from userspace so you
> > > never need to use it before modules can be loaded.
> > 
> > Agreed.
> 
> > > I see that
> > > CONFIG_THUNDERX2_PMU=y
> > > CONFIG_ARM_SMMU_V3_PMU=y
> > > are also set as builtins. That's probably wrong too.
> > 
> > It seems your arm64 kernel config deviates from the one we provide in Debian.
> > CONFIG_THUNDERX2_PMU is compiled as a module while CONFIG_ARM_SMMU_V3_PMU is
> > not set, at least in linux 5.10.19-1.
> 
> Hmm. I was looking at the (built, with CONFIG_ARM_CMN=y) sources for
> 5.10.9-1 and the (unbuilt) sources for 5.10.19-1. So yes, slightly
> different and the built version is not up to date any more.
> 
> If we already have CONFIG_THUNDERX2_PMU=m already then that's great
> (Ah yes - that's the upstream default).  Adding
> CONFIG_ARM_SMMU_V3_PMU=m would be good too. Adding it as a module
> should be pretty harmless then at least it's available? I'll set off a
> build now to check it works.

Enabling ARM_SMMU_V3_PMU as a module should be harmless, indeed.

> > > […]
> > 
> > > I also checked the state of the other perf configs with the arm kernel team
> > > and got feedback that we have all the ones that should sensibly be set set once
> > > CONFIG_ARM_CMN=m
> > > and
> > > CONFIG_THUNDERX2_PMU=m
> > > is added
> > 
> > This means updating the arm64 kernel config to only include ARM_CMN as a module.
> > To me it is acceptable for Bullseye as this seems uncontroversial, but note that
> > I can't speak for the kernel team.
> 
> Will you ask them, or should I?

I can send merge requests to enable ARM_CMN and ARM_SMMU_V3_PMU if you wish.

> It seems like prodding someone would be good as this was filed back on 27th
> jan and there have been uploads since, so I guess no-one has noticed till now.

I have been contributing for some time to help the kernel team, but I must admit
I didn't notice this one (and probably many others).

> > > Upstream enables
> > > CONFIG_FSL_IMX8_DDR_PMU=m
> > > by default too. IMX8 hardware is available so we should probably turn this on too
>
> > Contrary to Ubuntu, we do not provide support for the i.MX8M SoC family,
> > so enabling this option in the arm64 kernel config is not an option, right?
> 
> Ah OK. I didn't realise IMX8 was not enabled in the debian kernel (A
> subject for a different bug). In that case, no this is not
> appropriate.

I wanted to work on this a few months ago, but sadly I was unable to obtain a
i.MX8 SBC.

> Wookey
> -- 
> Principal hats:  Linaro, Debian, Wookware, ARM
> http://wookware.org/

Cheers,
Vincent

[toc] | [prev] | [next] | [standalone]

#69861

Hi Vincent, Wookey,

On Wed, Mar 17, 2021 at 10:02:25PM +0100, Vincent Blut wrote:
> Le 2021-03-17 19:24, Wookey a écrit :
> > On 2021-03-17 19:43 +0100, Vincent Blut wrote:
> > > Le 2021-03-17 15:49, Wookey a écrit :
> > > > On 2021-03-17 14:52 +0100, Vincent Blut wrote:
> > > > > Le 2021-01-27 12:57, Wookey a écrit :
> > > > > > Version: Please enable ARM CMN-600 power management on arm64
> > > > > >
> > > > > > This requires CONFIG_ARM_CMN=y
> > > > >
> > > > > Does it really have to be built-in instead of being provided as a module? Last I
> > > > > checked, Fedora and Ubuntu provide it as a module.
> > > > 
> > > > No it should really be a module. Perf is driven from userspace so you
> > > > never need to use it before modules can be loaded.
> > > 
> > > Agreed.
> > 
> > > > I see that
> > > > CONFIG_THUNDERX2_PMU=y
> > > > CONFIG_ARM_SMMU_V3_PMU=y
> > > > are also set as builtins. That's probably wrong too.
> > > 
> > > It seems your arm64 kernel config deviates from the one we provide in Debian.
> > > CONFIG_THUNDERX2_PMU is compiled as a module while CONFIG_ARM_SMMU_V3_PMU is
> > > not set, at least in linux 5.10.19-1.
> > 
> > Hmm. I was looking at the (built, with CONFIG_ARM_CMN=y) sources for
> > 5.10.9-1 and the (unbuilt) sources for 5.10.19-1. So yes, slightly
> > different and the built version is not up to date any more.
> > 
> > If we already have CONFIG_THUNDERX2_PMU=m already then that's great
> > (Ah yes - that's the upstream default).  Adding
> > CONFIG_ARM_SMMU_V3_PMU=m would be good too. Adding it as a module
> > should be pretty harmless then at least it's available? I'll set off a
> > build now to check it works.
> 
> Enabling ARM_SMMU_V3_PMU as a module should be harmless, indeed.
> 
> > > > […]
> > > 
> > > > I also checked the state of the other perf configs with the arm kernel team
> > > > and got feedback that we have all the ones that should sensibly be set set once
> > > > CONFIG_ARM_CMN=m
> > > > and
> > > > CONFIG_THUNDERX2_PMU=m
> > > > is added
> > > 
> > > This means updating the arm64 kernel config to only include ARM_CMN as a module.
> > > To me it is acceptable for Bullseye as this seems uncontroversial, but note that
> > > I can't speak for the kernel team.
> > 
> > Will you ask them, or should I?
> 
> I can send merge requests to enable ARM_CMN and ARM_SMMU_V3_PMU if you wish.
> 
> > It seems like prodding someone would be good as this was filed back on 27th
> > jan and there have been uploads since, so I guess no-one has noticed till now.
> 
> I have been contributing for some time to help the kernel team, but I must admit
> I didn't notice this one (and probably many others).

So just to confirm, were you be able to test with those two changes?

Regards,
Salvatore

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.kernel

csiph-web