Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1645189 > unrolled thread

[PATCH v10 00/10] coresight: enable debug module

Back to article view | Back to linux.kernel

Contents

  [PATCH v10 00/10] coresight: enable debug module Leo Yan <leo.yan@linaro.org> - 2017-05-19 06:30 +0200
    [PATCH v10 05/10] coresight: of_get_coresight_platform_data: Add missing of_node_put Leo Yan <leo.yan@linaro.org> - 2017-05-19 06:30 +0200
    [PATCH v10 02/10] doc: Add documentation for Coresight CPU debug Leo Yan <leo.yan@linaro.org> - 2017-05-19 06:30 +0200
      Re: [PATCH v10 02/10] doc: Add documentation for Coresight CPU debug Liviu Dudau <liviu.dudau@arm.com> - 2017-05-22 12:20 +0200
        Re: [PATCH v10 02/10] doc: Add documentation for Coresight CPU debug Leo Yan <leo.yan@linaro.org> - 2017-05-22 12:40 +0200
    [PATCH v10 10/10] arm64: dts: juno: Add Coresight CPU debug nodes Leo Yan <leo.yan@linaro.org> - 2017-05-19 06:30 +0200
      Re: [PATCH v10 10/10] arm64: dts: juno: Add Coresight CPU debug nodes Sudeep Holla <sudeep.holla@arm.com> - 2017-05-19 19:50 +0200
        Re: [PATCH v10 10/10] arm64: dts: juno: Add Coresight CPU debug nodes Suzuki K Poulose <Suzuki.Poulose@arm.com> - 2017-05-19 20:00 +0200
    [PATCH v10 04/10] MAINTAINERS: update file entries for Coresight subsystem Leo Yan <leo.yan@linaro.org> - 2017-05-19 06:30 +0200
    [PATCH v10 03/10] doc: Add coresight_cpu_debug.enable to kernel-parameters.txt Leo Yan <leo.yan@linaro.org> - 2017-05-19 06:30 +0200
    [PATCH v10 06/10] coresight: refactor with function of_coresight_get_cpu Leo Yan <leo.yan@linaro.org> - 2017-05-19 06:30 +0200

#1645189 — [PATCH v10 00/10] coresight: enable debug module

ARMv8 architecture reference manual (ARM DDI 0487A.k) Chapter H7 "The
Sample-based Profiling Extension" has description for sampling
registers, we can utilize these registers to check program counter
value with combined CPU exception level, secure state, etc. So this is
helpful for CPU lockup bugs, e.g. if one CPU has run into infinite loop
with IRQ disabled; the 'hang' CPU cannot switch context and handle any
interrupt, so it cannot handle SMP call for stack dump, etc.

This patch series is to enable coresight debug module with sample-based
registers and register call back notifier for PCSR register dumping
when panic happens, so we can see below dumping info for panic; and
this patch series has considered the conditions for access permission
for debug registers self, so this can avoid access debug registers when
CPU power domain is off; the driver also try to figure out the CPU is
in secure or non-secure state.

Patch 0001 is to document the dt binding; patch 0002 adds one detailed
document to describe the Coresight debug module implementation, the
clock and power domain impaction on the driver, some examples for usage.

Patch 0003 is to document boot parameters used in kernel command line.

Patch 0004 is to add file entries for MAINTAINERS.

Patch 0005 is used to fix the func of_get_coresight_platform_data()
doesn't properly drop the reference to the CPU node pointer; and
patch 0006 is refactor to add new function of_coresight_get_cpu().

Patch 0007 is the driver for CPU debug module.

Patch 0008 in this series are to enable debug unit on 96boards Hikey,
Patch 0009 is to enable debug on 96boards DB410c. Have verified on both
two boards.
Patch 0010 is to add Coresight CPU debug nodes for Juno r0, r1 & r2.

We can enable debugging with two methods, adding parameters into kernel
command line for build-in module:
  coresight_cpu_debug.enable=1

Or we can wait the system has booted up to use debugfs nodes to enable
debugging:
  # echo 1 > /sys/kernel/debug/coresight_cpu_debug/enable

As result we can get below log after input command:
echo c > /proc/sysrq-trigger:

ARM external debug module:
coresight-cpu-debug 850000.debug: CPU[0]:
coresight-cpu-debug 850000.debug:  EDPRSR:  00000001 (Power:On DLK:Unlock)
coresight-cpu-debug 850000.debug:  EDPCSR:  [<ffff00000808e9bc>] handle_IPI+0x174/0x1d8
coresight-cpu-debug 850000.debug:  EDCIDSR: 00000000
coresight-cpu-debug 850000.debug:  EDVIDSR: 90000000 (State:Non-secure Mode:EL1/0 Width:64bits VMID:0)
coresight-cpu-debug 852000.debug: CPU[1]:
coresight-cpu-debug 852000.debug:  EDPRSR:  00000001 (Power:On DLK:Unlock)
coresight-cpu-debug 852000.debug:  EDPCSR:  [<ffff0000087fab34>] debug_notifier_call+0x23c/0x358
coresight-cpu-debug 852000.debug:  EDCIDSR: 00000000
coresight-cpu-debug 852000.debug:  EDVIDSR: 90000000 (State:Non-secure Mode:EL1/0 Width:64bits VMID:0)


[...]

Changes from v9:
* Used dev_xyz() to replace pr_xyz() for print log.
* Added DT binding patch for Juno shared by Suzuki.

Changes from v8:
* According to Mathieu suggestions to split the doc into two patches,
  one is for kernel parameter and another is for driver documentation.
* Add file entries to MAINTAINERS.
* According to Mathieu suggestions, refined functions
  debug_enable_func()/debug_disable_func().

Changes from v7:
* Fix operator priority bug.
* Minor sequence adjustment for function debug_func_exit().

Changes from v6:
* According to Suzuki and Mathieu suggestions, refined debug module
  driver to install panic notifier when insmod module; refined function
  debug_force_cpu_powered_up() for CPU power state checking; some minor
  fixing for output log, adding comments for memory barrier, code
  alignment.

Changes from v5:
* According to Suzuki and Mathieu suggestions, refined debug module
  driver to drop unused structure members, refactored initialization
  code to distinguish hardware implementation features, refactored
  flow for forcing CPU powered up, supported pm_runtime operations.
* Added one new doc file: Documentation/trace/coresight-cpu-debug.txt,
  which is used to describe detailed info for implementation, clock
  and power domain impaction on debug module, and exmaples for common
  usage.
* Removed "idle constraints" from debug driver.

Changes from v4:
* This version is mainly credit to ARM colleagues many contribution
  ideas for better quality (Thanks a lot Suzuki, Mike and Sudeep!).
* According to Suzuki suggestion, refined debug module driver to avoid
  memory leak for drvdata struct, handle PCSAMPLE_MODE=1, use flag
  drvdata.pc_has_offset to indicate if PCSR has offset, minor fixes.
* According to Mathieu suggestion, refined dt binding description.
* Changed driver to support module mode;
* According to Mike suggestion and very appreciate the pseudo code,
  added support to force CPU powered up with register EDPRCR;
* According to discussions, added command line and debugfs nodes to
  support enabling debugging for boot time, or later can dynamically
  enable/disable debugging by debugfs.
* According to Rob Herring suggestion, one minor fixes in DT binding.
* According to Stephen Boyd suggestion, add const quality to structure
  device_node. And used use of_cpu_device_node_get() to replace
  of_get_cpu_node() in patch 0003.

Changes from v3:
* Added Suzuki K Poulose's patch to fix issue for the func
  of_get_coresight_platform_data() doesn't properly drop the reference
  to the CPU node pointer.
* According to Suzuki suggestion, added code to handl the corner case
  for ARMv8 CPU with aarch32 mode.
* According to Suzuki suggestion, changed compatible string to
  "arm,coresight-cpu-debug".
* According to Mathieu suggestion, added "power-domains" as optional
  properties.

Changes from v2:
* According to Mathieu Poirier suggestion, applied some minor fixes.
* Added two extra patches for enabling debug module on Hikey.

Changes from v1:
* According to Mike Leach suggestion, removed the binding for debug
  module clocks which have been directly provided by CPU clocks.
* According to Mathieu Poirier suggestion, added function
  of_coresight_get_cpu() and some minor refactors for debug module
  driver.

Changes from RFC:
* According to Mike Leach suggestion, added check for EDPRSR to avoid
  lockup; added supporting EDVIDSR and EDCIDSR registers.
* According to Mark Rutland and Mathieu Poirier suggestion, rewrote
  the documentation for DT binding.
* According to Mark and Mathieu suggestion, refined debug driver.

Leo Yan (8):
  coresight: bindings for CPU debug module
  doc: Add documentation for Coresight CPU debug
  doc: Add coresight_cpu_debug.enable to kernel-parameters.txt
  MAINTAINERS: update file entries for Coresight subsystem
  coresight: refactor with function of_coresight_get_cpu
  coresight: add support for CPU debug module
  arm64: dts: hi6220: register debug module
  arm64: dts: qcom: msm8916: Add debug unit

Suzuki K Poulose (2):
  coresight: of_get_coresight_platform_data: Add missing of_node_put
  arm64: dts: juno: Add Coresight CPU debug nodes

 Documentation/admin-guide/kernel-parameters.txt    |   7 +
 .../bindings/arm/coresight-cpu-debug.txt           |  49 ++
 Documentation/trace/coresight-cpu-debug.txt        | 174 ++++++
 MAINTAINERS                                        |   2 +
 arch/arm64/boot/dts/arm/juno-base.dtsi             |  54 ++
 arch/arm64/boot/dts/arm/juno-r1.dts                |  24 +
 arch/arm64/boot/dts/arm/juno-r2.dts                |  24 +
 arch/arm64/boot/dts/arm/juno.dts                   |  24 +
 arch/arm64/boot/dts/hisilicon/hi6220.dtsi          |  64 ++
 arch/arm64/boot/dts/qcom/msm8916.dtsi              |  32 +
 drivers/hwtracing/coresight/Kconfig                |  14 +
 drivers/hwtracing/coresight/Makefile               |   1 +
 drivers/hwtracing/coresight/coresight-cpu-debug.c  | 696 +++++++++++++++++++++
 drivers/hwtracing/coresight/of_coresight.c         |  40 +-
 include/linux/coresight.h                          |   3 +
 15 files changed, 1196 insertions(+), 12 deletions(-)
 create mode 100644 Documentation/devicetree/bindings/arm/coresight-cpu-debug.txt
 create mode 100644 Documentation/trace/coresight-cpu-debug.txt
 create mode 100644 drivers/hwtracing/coresight/coresight-cpu-debug.c

-- 
2.7.4

[toc] | [next] | [standalone]

#1645190 — [PATCH v10 05/10] coresight: of_get_coresight_platform_data: Add missing of_node_put

From: Suzuki K Poulose <suzuki.poulose@arm.com>

The of_get_coresight_platform_data iterates over the possible CPU nodes
to find a given cpu phandle. However it does not drop the reference
to the node pointer returned by the of_get_coresight_platform_data.

This patch also introduces another minor fix is to use
of_cpu_device_node_get() to replace of_get_cpu_node().

Cc: Mathieu Poirier <mathieu.poirier@linaro.org>
Reviewed-by: Suzuki K Poulose <suzuki.poulose@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
[Leo: minor tweaks for of_get_coresight_platform_data]
Signed-off-by: Leo Yan <leo.yan@linaro.org>
---
 drivers/hwtracing/coresight/of_coresight.c | 17 ++++++++++-------
 1 file changed, 10 insertions(+), 7 deletions(-)

diff --git a/drivers/hwtracing/coresight/of_coresight.c b/drivers/hwtracing/coresight/of_coresight.c
index 2749853..225b2dd 100644
--- a/drivers/hwtracing/coresight/of_coresight.c
+++ b/drivers/hwtracing/coresight/of_coresight.c
@@ -109,7 +109,8 @@ of_get_coresight_platform_data(struct device *dev,
 	struct coresight_platform_data *pdata;
 	struct of_endpoint endpoint, rendpoint;
 	struct device *rdev;
-	struct device_node *dn;
+	bool found;
+	struct device_node *dn, *np;
 	struct device_node *ep = NULL;
 	struct device_node *rparent = NULL;
 	struct device_node *rport = NULL;
@@ -176,17 +177,19 @@ of_get_coresight_platform_data(struct device *dev,
 		} while (ep);
 	}
 
-	/* Affinity defaults to CPU0 */
-	pdata->cpu = 0;
 	dn = of_parse_phandle(node, "cpu", 0);
-	for (cpu = 0; dn && cpu < nr_cpu_ids; cpu++) {
-		if (dn == of_get_cpu_node(cpu, NULL)) {
-			pdata->cpu = cpu;
+	for_each_possible_cpu(cpu) {
+		np = of_cpu_device_node_get(cpu);
+		found = (dn == np);
+		of_node_put(np);
+		if (found)
 			break;
-		}
 	}
 	of_node_put(dn);
 
+	/* Affinity to CPU0 if no cpu nodes are found */
+	pdata->cpu = found ? cpu : 0;
+
 	return pdata;
 }
 EXPORT_SYMBOL_GPL(of_get_coresight_platform_data);
-- 
2.7.4

[toc] | [prev] | [next] | [standalone]

#1645191 — [PATCH v10 02/10] doc: Add documentation for Coresight CPU debug

Add detailed documentation for Coresight CPU debug driver, which
contains the info for driver implementation, Mike Leach excellent
summary for "clock and power domain". At the end some examples on how
to enable the debugging functionality are provided.

Suggested-by: Mike Leach <mike.leach@linaro.org>
Reviewed-by: Mathieu Poirier <mathieu.poirier@linaro.org>
Signed-off-by: Leo Yan <leo.yan@linaro.org>
---
 Documentation/trace/coresight-cpu-debug.txt | 174 ++++++++++++++++++++++++++++
 1 file changed, 174 insertions(+)
 create mode 100644 Documentation/trace/coresight-cpu-debug.txt

diff --git a/Documentation/trace/coresight-cpu-debug.txt b/Documentation/trace/coresight-cpu-debug.txt
new file mode 100644
index 0000000..f0c3f0f
--- /dev/null
+++ b/Documentation/trace/coresight-cpu-debug.txt
@@ -0,0 +1,174 @@
+		Coresight CPU Debug Module
+		==========================
+
+   Author:   Leo Yan <leo.yan@linaro.org>
+   Date:     April 5th, 2017
+
+Introduction
+------------
+
+Coresight CPU debug module is defined in ARMv8-a architecture reference manual
+(ARM DDI 0487A.k) Chapter 'Part H: External debug', the CPU can integrate
+debug module and it is mainly used for two modes: self-hosted debug and
+external debug. Usually the external debug mode is well known as the external
+debugger connects with SoC from JTAG port; on the other hand the program can
+explore debugging method which rely on self-hosted debug mode, this document
+is to focus on this part.
+
+The debug module provides sample-based profiling extension, which can be used
+to sample CPU program counter, secure state and exception level, etc; usually
+every CPU has one dedicated debug module to be connected. Based on self-hosted
+debug mechanism, Linux kernel can access these related registers from mmio
+region when the kernel panic happens. The callback notifier for kernel panic
+will dump related registers for every CPU; finally this is good for assistant
+analysis for panic.
+
+
+Implementation
+--------------
+
+- During driver registration, use EDDEVID and EDDEVID1 two device ID
+  registers to decide if sample-based profiling is implemented or not. On some
+  platforms this hardware feature is fully or partialy implemented; and if
+  this feature is not supported then registration will fail.
+
+- When write this doc, the debug driver mainly relies on three sampling
+  registers. The kernel panic callback notifier gathers info from EDPCSR
+  EDVIDSR and EDCIDSR; from EDPCSR we can get program counter, EDVIDSR has
+  information for secure state, exception level, bit width, etc; EDCIDSR is
+  context ID value which contains the sampled value of CONTEXTIDR_EL1.
+
+- The driver supports CPU running mode with either AArch64 or AArch32. The
+  registers naming convention is a bit different between them, AArch64 uses
+  'ED' for register prefix (ARM DDI 0487A.k, chapter H9.1) and AArch32 uses
+  'DBG' as prefix (ARM DDI 0487A.k, chapter G5.1). The driver is unified to
+  use AArch64 naming convention.
+
+- ARMv8-a (ARM DDI 0487A.k) and ARMv7-a (ARM DDI 0406C.b) have different
+  register bits definition. So the driver consolidates two difference:
+
+  If PCSROffset=0b0000, on ARMv8-a the feature of EDPCSR is not implemented;
+  but ARMv7-a defines "PCSR samples are offset by a value that depends on the
+  instruction set state". For ARMv7-a, the driver checks furthermore if CPU
+  runs with ARM or thumb instruction set and calibrate PCSR value, the
+  detailed description for offset is in ARMv7-a ARM (ARM DDI 0406C.b) chapter
+  C11.11.34 "DBGPCSR, Program Counter Sampling Register".
+
+  If PCSROffset=0b0010, ARMv8-a defines "EDPCSR implemented, and samples have
+  no offset applied and do not sample the instruction set state in AArch32
+  state". So on ARMv8 if EDDEVID1.PCSROffset is 0b0010 and the CPU operates
+  in AArch32 state, EDPCSR is not sampled; when the CPU operates in AArch64
+  state EDPCSR is sampled and no offset are applied.
+
+
+Clock and power domain
+----------------------
+
+Before accessing debug registers, we should ensure the clock and power domain
+have been enabled properly. In ARMv8-a ARM (ARM DDI 0487A.k) chapter 'H9.1
+Debug registers', the debug registers are spread into two domains: the debug
+domain and the CPU domain.
+
+                                +---------------+
+                                |               |
+                                |               |
+                     +----------+--+            |
+          dbg_clk -->|          |**|            |<-- cpu_clk
+                     |    Debug |**|   CPU      |
+           dbg_pd -->|          |**|            |<-- cpu_pd
+                     +----------+--+            |
+                                |               |
+                                |               |
+                                +---------------+
+
+For debug domain, the user uses DT binding "clocks" and "power-domains" to
+specify the corresponding clock source and power supply for the debug logic.
+The driver calls the pm_runtime_{put|get} operations as needed to handle the
+debug power domain.
+
+For CPU domain, the different SoC designs have different power management
+schemes and finally this heavily impacts external debug module. So we can
+divide into below cases:
+
+- On systems with a sane power controller which can behave correctly with
+  respect to CPU power domain, the CPU power domain can be controlled by
+  register EDPRCR in driver. The driver firstly writes bit EDPRCR.COREPURQ
+  to power up the CPU, and then writes bit EDPRCR.CORENPDRQ for emulation
+  of CPU power down. As result, this can ensure the CPU power domain is
+  powered on properly during the period when access debug related registers;
+
+- Some designs will power down an entire cluster if all CPUs on the cluster
+  are powered down - including the parts of the debug registers that should
+  remain powered in the debug power domain. The bits in EDPRCR are not
+  respected in these cases, so these designs do not support debug over
+  power down in the way that the CoreSight / Debug designers anticipated.
+  This means that even checking EDPRSR has the potential to cause a bus hang
+  if the target register is unpowered.
+
+  In this case, accessing to the debug registers while they are not powered
+  is a recipe for disaster; so we need preventing CPU low power states at boot
+  time or when user enable module at the run time. Please see chapter
+  "How to use the module" for detailed usage info for this.
+
+
+Device Tree Bindings
+--------------------
+
+See Documentation/devicetree/bindings/arm/coresight-cpu-debug.txt for details.
+
+
+How to use the module
+---------------------
+
+If you want to enable debugging functionality at boot time, you can add
+"coresight_cpu_debug.enable=1" to the kernel command line parameter.
+
+The driver also can work as module, so can enable the debugging when insmod
+module:
+# insmod coresight_cpu_debug.ko debug=1
+
+When boot time or insmod module you have not enabled the debugging, the driver
+uses the debugfs file system to provide a knob to dynamically enable or disable
+debugging:
+
+To enable it, write a '1' into /sys/kernel/debug/coresight_cpu_debug/enable:
+# echo 1 > /sys/kernel/debug/coresight_cpu_debug/enable
+
+To disable it, write a '0' into /sys/kernel/debug/coresight_cpu_debug/enable:
+# echo 0 > /sys/kernel/debug/coresight_cpu_debug/enable
+
+As explained in chapter "Clock and power domain", if you are working on one
+platform which has idle states to power off debug logic and the power
+controller cannot work well for the request from EDPRCR, then you should
+firstly constraint CPU idle states before enable CPU debugging feature; so can
+ensure the accessing to debug logic.
+
+If you want to limit idle states at boot time, you can use "nohlt" or
+"cpuidle.off=1" in the kernel command line.
+
+At the runtime you can disable idle states with below methods:
+
+Set latency request to /dev/cpu_dma_latency to disable all CPUs specific idle
+states (if latency = 0uS then disable all idle states):
+# echo "what_ever_latency_you_need_in_uS" > /dev/cpu_dma_latency
+
+Disable specific CPU's specific idle state:
+# echo 1 > /sys/devices/system/cpu/cpu$cpu/cpuidle/state$state/disable
+
+
+Output format
+-------------
+
+Here is an example of the debugging output format:
+
+ARM external debug module:
+coresight-cpu-debug 850000.debug: CPU[0]:
+coresight-cpu-debug 850000.debug:  EDPRSR:  00000001 (Power:On DLK:Unlock)
+coresight-cpu-debug 850000.debug:  EDPCSR:  [<ffff00000808e9bc>] handle_IPI+0x174/0x1d8
+coresight-cpu-debug 850000.debug:  EDCIDSR: 00000000
+coresight-cpu-debug 850000.debug:  EDVIDSR: 90000000 (State:Non-secure Mode:EL1/0 Width:64bits VMID:0)
+coresight-cpu-debug 852000.debug: CPU[1]:
+coresight-cpu-debug 852000.debug:  EDPRSR:  00000001 (Power:On DLK:Unlock)
+coresight-cpu-debug 852000.debug:  EDPCSR:  [<ffff0000087fab34>] debug_notifier_call+0x23c/0x358
+coresight-cpu-debug 852000.debug:  EDCIDSR: 00000000
+coresight-cpu-debug 852000.debug:  EDVIDSR: 90000000 (State:Non-secure Mode:EL1/0 Width:64bits VMID:0)
-- 
2.7.4

[toc] | [prev] | [next] | [standalone]

#1646701 — Re: [PATCH v10 02/10] doc: Add documentation for Coresight CPU debug

On Fri, May 19, 2017 at 12:25:49PM +0800, Leo Yan wrote:
> Add detailed documentation for Coresight CPU debug driver, which
> contains the info for driver implementation, Mike Leach excellent
> summary for "clock and power domain". At the end some examples on how
> to enable the debugging functionality are provided.

Hi Leo,

Below are my minor suggestions to improve readability of the documentation.

Thanks,
Liviu

> 
> Suggested-by: Mike Leach <mike.leach@linaro.org>
> Reviewed-by: Mathieu Poirier <mathieu.poirier@linaro.org>
> Signed-off-by: Leo Yan <leo.yan@linaro.org>
> ---
>  Documentation/trace/coresight-cpu-debug.txt | 174 ++++++++++++++++++++++++++++
>  1 file changed, 174 insertions(+)
>  create mode 100644 Documentation/trace/coresight-cpu-debug.txt
> 
> diff --git a/Documentation/trace/coresight-cpu-debug.txt b/Documentation/trace/coresight-cpu-debug.txt
> new file mode 100644
> index 0000000..f0c3f0f
> --- /dev/null
> +++ b/Documentation/trace/coresight-cpu-debug.txt
> @@ -0,0 +1,174 @@
> +		Coresight CPU Debug Module
> +		==========================
> +
> +   Author:   Leo Yan <leo.yan@linaro.org>
> +   Date:     April 5th, 2017
> +
> +Introduction
> +------------
> +
> +Coresight CPU debug module is defined in ARMv8-a architecture reference manual
> +(ARM DDI 0487A.k) Chapter 'Part H: External debug', the CPU can integrate
> +debug module and it is mainly used for two modes: self-hosted debug and
> +external debug. Usually the external debug mode is well known as the external
> +debugger connects with SoC from JTAG port; on the other hand the program can
> +explore debugging method which rely on self-hosted debug mode, this document
> +is to focus on this part.
> +
> +The debug module provides sample-based profiling extension, which can be used
> +to sample CPU program counter, secure state and exception level, etc; usually
> +every CPU has one dedicated debug module to be connected. Based on self-hosted
> +debug mechanism, Linux kernel can access these related registers from mmio
> +region when the kernel panic happens. The callback notifier for kernel panic
> +will dump related registers for every CPU; finally this is good for assistant
> +analysis for panic.
> +
> +
> +Implementation
> +--------------
> +
> +- During driver registration, use EDDEVID and EDDEVID1 two device ID

During driver registration, it uses EDDEVID and EDDEVID1 - two device ID

> +  registers to decide if sample-based profiling is implemented or not. On some
> +  platforms this hardware feature is fully or partialy implemented; and if
> +  this feature is not supported then registration will fail.
> +
> +- When write this doc, the debug driver mainly relies on three sampling
> +  registers. The kernel panic callback notifier gathers info from EDPCSR
> +  EDVIDSR and EDCIDSR; from EDPCSR we can get program counter, EDVIDSR has

At the time this documentation was writen, the debug driver mainly relies on
information gathered by the kernel panic callback notifier from three sampling
registers: EDPCSR, EDVIDSR and EDCIDSR: ....

> +  information for secure state, exception level, bit width, etc; EDCIDSR is
> +  context ID value which contains the sampled value of CONTEXTIDR_EL1.
> +
> +- The driver supports CPU running mode with either AArch64 or AArch32. The

... supports a CPU running in either AArch64 or AArch32 mode. The

> +  registers naming convention is a bit different between them, AArch64 uses
> +  'ED' for register prefix (ARM DDI 0487A.k, chapter H9.1) and AArch32 uses
> +  'DBG' as prefix (ARM DDI 0487A.k, chapter G5.1). The driver is unified to
> +  use AArch64 naming convention.
> +
> +- ARMv8-a (ARM DDI 0487A.k) and ARMv7-a (ARM DDI 0406C.b) have different
> +  register bits definition. So the driver consolidates two difference:
> +
> +  If PCSROffset=0b0000, on ARMv8-a the feature of EDPCSR is not implemented;
> +  but ARMv7-a defines "PCSR samples are offset by a value that depends on the
> +  instruction set state". For ARMv7-a, the driver checks furthermore if CPU
> +  runs with ARM or thumb instruction set and calibrate PCSR value, the
> +  detailed description for offset is in ARMv7-a ARM (ARM DDI 0406C.b) chapter
> +  C11.11.34 "DBGPCSR, Program Counter Sampling Register".
> +
> +  If PCSROffset=0b0010, ARMv8-a defines "EDPCSR implemented, and samples have
> +  no offset applied and do not sample the instruction set state in AArch32
> +  state". So on ARMv8 if EDDEVID1.PCSROffset is 0b0010 and the CPU operates
> +  in AArch32 state, EDPCSR is not sampled; when the CPU operates in AArch64
> +  state EDPCSR is sampled and no offset are applied.
> +
> +
> +Clock and power domain
> +----------------------
> +
> +Before accessing debug registers, we should ensure the clock and power domain
> +have been enabled properly. In ARMv8-a ARM (ARM DDI 0487A.k) chapter 'H9.1
> +Debug registers', the debug registers are spread into two domains: the debug
> +domain and the CPU domain.
> +
> +                                +---------------+
> +                                |               |
> +                                |               |
> +                     +----------+--+            |
> +          dbg_clk -->|          |**|            |<-- cpu_clk
> +                     |    Debug |**|   CPU      |
> +           dbg_pd -->|          |**|            |<-- cpu_pd
> +                     +----------+--+            |
> +                                |               |
> +                                |               |
> +                                +---------------+
> +
> +For debug domain, the user uses DT binding "clocks" and "power-domains" to
> +specify the corresponding clock source and power supply for the debug logic.
> +The driver calls the pm_runtime_{put|get} operations as needed to handle the
> +debug power domain.
> +
> +For CPU domain, the different SoC designs have different power management
> +schemes and finally this heavily impacts external debug module. So we can
> +divide into below cases:
> +
> +- On systems with a sane power controller which can behave correctly with
> +  respect to CPU power domain, the CPU power domain can be controlled by
> +  register EDPRCR in driver. The driver firstly writes bit EDPRCR.COREPURQ
> +  to power up the CPU, and then writes bit EDPRCR.CORENPDRQ for emulation
> +  of CPU power down. As result, this can ensure the CPU power domain is
> +  powered on properly during the period when access debug related registers;
> +
> +- Some designs will power down an entire cluster if all CPUs on the cluster
> +  are powered down - including the parts of the debug registers that should
> +  remain powered in the debug power domain. The bits in EDPRCR are not
> +  respected in these cases, so these designs do not support debug over
> +  power down in the way that the CoreSight / Debug designers anticipated.
> +  This means that even checking EDPRSR has the potential to cause a bus hang
> +  if the target register is unpowered.
> +
> +  In this case, accessing to the debug registers while they are not powered
> +  is a recipe for disaster; so we need preventing CPU low power states at boot
> +  time or when user enable module at the run time. Please see chapter
> +  "How to use the module" for detailed usage info for this.
> +
> +
> +Device Tree Bindings
> +--------------------
> +
> +See Documentation/devicetree/bindings/arm/coresight-cpu-debug.txt for details.
> +
> +
> +How to use the module
> +---------------------
> +
> +If you want to enable debugging functionality at boot time, you can add
> +"coresight_cpu_debug.enable=1" to the kernel command line parameter.
> +
> +The driver also can work as module, so can enable the debugging when insmod
> +module:
> +# insmod coresight_cpu_debug.ko debug=1
> +
> +When boot time or insmod module you have not enabled the debugging, the driver
> +uses the debugfs file system to provide a knob to dynamically enable or disable
> +debugging:
> +
> +To enable it, write a '1' into /sys/kernel/debug/coresight_cpu_debug/enable:
> +# echo 1 > /sys/kernel/debug/coresight_cpu_debug/enable
> +
> +To disable it, write a '0' into /sys/kernel/debug/coresight_cpu_debug/enable:
> +# echo 0 > /sys/kernel/debug/coresight_cpu_debug/enable
> +
> +As explained in chapter "Clock and power domain", if you are working on one
> +platform which has idle states to power off debug logic and the power
> +controller cannot work well for the request from EDPRCR, then you should
> +firstly constraint CPU idle states before enable CPU debugging feature; so can
> +ensure the accessing to debug logic.
> +
> +If you want to limit idle states at boot time, you can use "nohlt" or
> +"cpuidle.off=1" in the kernel command line.
> +
> +At the runtime you can disable idle states with below methods:
> +
> +Set latency request to /dev/cpu_dma_latency to disable all CPUs specific idle
> +states (if latency = 0uS then disable all idle states):
> +# echo "what_ever_latency_you_need_in_uS" > /dev/cpu_dma_latency
> +
> +Disable specific CPU's specific idle state:
> +# echo 1 > /sys/devices/system/cpu/cpu$cpu/cpuidle/state$state/disable
> +
> +
> +Output format
> +-------------
> +
> +Here is an example of the debugging output format:
> +
> +ARM external debug module:
> +coresight-cpu-debug 850000.debug: CPU[0]:
> +coresight-cpu-debug 850000.debug:  EDPRSR:  00000001 (Power:On DLK:Unlock)
> +coresight-cpu-debug 850000.debug:  EDPCSR:  [<ffff00000808e9bc>] handle_IPI+0x174/0x1d8
> +coresight-cpu-debug 850000.debug:  EDCIDSR: 00000000
> +coresight-cpu-debug 850000.debug:  EDVIDSR: 90000000 (State:Non-secure Mode:EL1/0 Width:64bits VMID:0)
> +coresight-cpu-debug 852000.debug: CPU[1]:
> +coresight-cpu-debug 852000.debug:  EDPRSR:  00000001 (Power:On DLK:Unlock)
> +coresight-cpu-debug 852000.debug:  EDPCSR:  [<ffff0000087fab34>] debug_notifier_call+0x23c/0x358
> +coresight-cpu-debug 852000.debug:  EDCIDSR: 00000000
> +coresight-cpu-debug 852000.debug:  EDVIDSR: 90000000 (State:Non-secure Mode:EL1/0 Width:64bits VMID:0)
> -- 
> 2.7.4
> 

-- 
====================
| I would like to |
| fix the world,  |
| but they're not |
| giving me the   |
 \ source code!  /
  ---------------
    ¯\_(ツ)_/¯

[toc] | [prev] | [next] | [standalone]

#1646711 — Re: [PATCH v10 02/10] doc: Add documentation for Coresight CPU debug

On Mon, May 22, 2017 at 11:16:00AM +0100, Liviu Dudau wrote:
> On Fri, May 19, 2017 at 12:25:49PM +0800, Leo Yan wrote:
> > Add detailed documentation for Coresight CPU debug driver, which
> > contains the info for driver implementation, Mike Leach excellent
> > summary for "clock and power domain". At the end some examples on how
> > to enable the debugging functionality are provided.
> 
> Hi Leo,
> 
> Below are my minor suggestions to improve readability of the documentation.

Thanks, Liviu. Accept all suggestions and will spin a new version.

[...]

Thanks,
Leo Yan

[toc] | [prev] | [next] | [standalone]

#1645192 — [PATCH v10 10/10] arm64: dts: juno: Add Coresight CPU debug nodes

From: Suzuki K Poulose <suzuki.poulose@arm.com>

Add Coresight CPU debug nodes for Juno r0, r1 & r2. The CPU
debug areas are mapped at the same address for all revisions,
like the ETM, even though the CPUs have changed from r1 to r2.

Cc: Sudeep Holla <sudeep.holla@arm.com>
Cc: Leo Yan <leo.yan@linaro.org>
Cc: Mathieu Poirier <mathieu.porier@linaro.org>
Cc: Liviu Dudau <liviu.dudau@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
 arch/arm64/boot/dts/arm/juno-base.dtsi | 54 ++++++++++++++++++++++++++++++++++
 arch/arm64/boot/dts/arm/juno-r1.dts    | 24 +++++++++++++++
 arch/arm64/boot/dts/arm/juno-r2.dts    | 24 +++++++++++++++
 arch/arm64/boot/dts/arm/juno.dts       | 24 +++++++++++++++
 4 files changed, 126 insertions(+)

diff --git a/arch/arm64/boot/dts/arm/juno-base.dtsi b/arch/arm64/boot/dts/arm/juno-base.dtsi
index bfe7d68..784a80a 100644
--- a/arch/arm64/boot/dts/arm/juno-base.dtsi
+++ b/arch/arm64/boot/dts/arm/juno-base.dtsi
@@ -216,6 +216,15 @@
 		};
 	};
 
+	cpu_debug0: cpu_debug@22010000 {
+		compatible = "arm,coresight-cpu-debug", "arm,primecell";
+		reg = <0 0x22010000 0 0x1000>;
+
+		clocks = <&soc_smc50mhz>;
+		clock-names = "apb_pclk";
+		power-domains = <&scpi_devpd 0>;
+	};
+
 	funnel@220c0000 { /* cluster0 funnel */
 		compatible = "arm,coresight-funnel", "arm,primecell";
 		reg = <0 0x220c0000 0 0x1000>;
@@ -266,6 +275,15 @@
 		};
 	};
 
+	cpu_debug1: cpu_debug@22110000 {
+		compatible = "arm,coresight-cpu-debug", "arm,primecell";
+		reg = <0 0x22110000 0 0x1000>;
+
+		clocks = <&soc_smc50mhz>;
+		clock-names = "apb_pclk";
+		power-domains = <&scpi_devpd 0>;
+	};
+
 	etm2: etm@23040000 {
 		compatible = "arm,coresight-etm4x", "arm,primecell";
 		reg = <0 0x23040000 0 0x1000>;
@@ -280,6 +298,15 @@
 		};
 	};
 
+	cpu_debug2: cpu_debug@23010000 {
+		compatible = "arm,coresight-cpu-debug", "arm,primecell";
+		reg = <0 0x23010000 0 0x1000>;
+
+		clocks = <&soc_smc50mhz>;
+		clock-names = "apb_pclk";
+		power-domains = <&scpi_devpd 0>;
+	};
+
 	funnel@230c0000 { /* cluster1 funnel */
 		compatible = "arm,coresight-funnel", "arm,primecell";
 		reg = <0 0x230c0000 0 0x1000>;
@@ -344,6 +371,15 @@
 		};
 	};
 
+	cpu_debug3: cpu_debug@23110000 {
+		compatible = "arm,coresight-cpu-debug", "arm,primecell";
+		reg = <0 0x23110000 0 0x1000>;
+
+		clocks = <&soc_smc50mhz>;
+		clock-names = "apb_pclk";
+		power-domains = <&scpi_devpd 0>;
+	};
+
 	etm4: etm@23240000 {
 		compatible = "arm,coresight-etm4x", "arm,primecell";
 		reg = <0 0x23240000 0 0x1000>;
@@ -358,6 +394,15 @@
 		};
 	};
 
+	cpu_debug4: cpu_debug@23210000 {
+		compatible = "arm,coresight-cpu-debug", "arm,primecell";
+		reg = <0 0x23210000 0 0x1000>;
+
+		clocks = <&soc_smc50mhz>;
+		clock-names = "apb_pclk";
+		power-domains = <&scpi_devpd 0>;
+	};
+
 	etm5: etm@23340000 {
 		compatible = "arm,coresight-etm4x", "arm,primecell";
 		reg = <0 0x23340000 0 0x1000>;
@@ -372,6 +417,15 @@
 		};
 	};
 
+	cpu_debug5: cpu_debug@23310000 {
+		compatible = "arm,coresight-cpu-debug", "arm,primecell";
+		reg = <0 0x23310000 0 0x1000>;
+
+		clocks = <&soc_smc50mhz>;
+		clock-names = "apb_pclk";
+		power-domains = <&scpi_devpd 0>;
+	};
+
 	replicator@20120000 {
 		compatible = "qcom,coresight-replicator1x", "arm,primecell";
 		reg = <0 0x20120000 0 0x1000>;
diff --git a/arch/arm64/boot/dts/arm/juno-r1.dts b/arch/arm64/boot/dts/arm/juno-r1.dts
index 0e8943a..aed6389 100644
--- a/arch/arm64/boot/dts/arm/juno-r1.dts
+++ b/arch/arm64/boot/dts/arm/juno-r1.dts
@@ -281,3 +281,27 @@
 &stm_out_port {
 	remote-endpoint = <&csys1_funnel_in_port0>;
 };
+
+&cpu_debug0 {
+	cpu = <&A57_0>;
+};
+
+&cpu_debug1 {
+	cpu = <&A57_1>;
+};
+
+&cpu_debug2 {
+	cpu = <&A53_0>;
+};
+
+&cpu_debug3 {
+	cpu = <&A53_1>;
+};
+
+&cpu_debug4 {
+	cpu = <&A53_2>;
+};
+
+&cpu_debug5 {
+	cpu = <&A53_3>;
+};
diff --git a/arch/arm64/boot/dts/arm/juno-r2.dts b/arch/arm64/boot/dts/arm/juno-r2.dts
index 405e2fb..b39b6d6 100644
--- a/arch/arm64/boot/dts/arm/juno-r2.dts
+++ b/arch/arm64/boot/dts/arm/juno-r2.dts
@@ -281,3 +281,27 @@
 &stm_out_port {
 	remote-endpoint = <&csys1_funnel_in_port0>;
 };
+
+&cpu_debug0 {
+	cpu = <&A72_0>;
+};
+
+&cpu_debug1 {
+	cpu = <&A72_1>;
+};
+
+&cpu_debug2 {
+	cpu = <&A53_0>;
+};
+
+&cpu_debug3 {
+	cpu = <&A53_1>;
+};
+
+&cpu_debug4 {
+	cpu = <&A53_2>;
+};
+
+&cpu_debug5 {
+	cpu = <&A53_3>;
+};
diff --git a/arch/arm64/boot/dts/arm/juno.dts b/arch/arm64/boot/dts/arm/juno.dts
index 0220494..c9236c4 100644
--- a/arch/arm64/boot/dts/arm/juno.dts
+++ b/arch/arm64/boot/dts/arm/juno.dts
@@ -268,3 +268,27 @@
 		};
 	};
 };
+
+&cpu_debug0 {
+	cpu = <&A57_0>;
+};
+
+&cpu_debug1 {
+	cpu = <&A57_1>;
+};
+
+&cpu_debug2 {
+	cpu = <&A53_0>;
+};
+
+&cpu_debug3 {
+	cpu = <&A53_1>;
+};
+
+&cpu_debug4 {
+	cpu = <&A53_2>;
+};
+
+&cpu_debug5 {
+	cpu = <&A53_3>;
+};
-- 
2.7.4

[toc] | [prev] | [next] | [standalone]

#1645791 — Re: [PATCH v10 10/10] arm64: dts: juno: Add Coresight CPU debug nodes

Hi Suzuki, Leo,

On 19/05/17 05:25, Leo Yan wrote:
> From: Suzuki K Poulose <suzuki.poulose@arm.com>
> 
> Add Coresight CPU debug nodes for Juno r0, r1 & r2. The CPU
> debug areas are mapped at the same address for all revisions,
> like the ETM, even though the CPUs have changed from r1 to r2.
> 
> Cc: Sudeep Holla <sudeep.holla@arm.com>
> Cc: Leo Yan <leo.yan@linaro.org>
> Cc: Mathieu Poirier <mathieu.porier@linaro.org>
> Cc: Liviu Dudau <liviu.dudau@arm.com>
> Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>

ARM-SoC team expect DTS to be routed via platform tree.
So I have queued this for v4.13[1]

-- 
Regards,
Sudeep

[1] https://git.kernel.org/sudeep.holla/linux/h/for-next/juno-updates

[toc] | [prev] | [next] | [standalone]

#1645800 — Re: [PATCH v10 10/10] arm64: dts: juno: Add Coresight CPU debug nodes

On 19/05/17 18:44, Sudeep Holla wrote:
> Hi Suzuki, Leo,
>
> On 19/05/17 05:25, Leo Yan wrote:
>> From: Suzuki K Poulose <suzuki.poulose@arm.com>
>>
>> Add Coresight CPU debug nodes for Juno r0, r1 & r2. The CPU
>> debug areas are mapped at the same address for all revisions,
>> like the ETM, even though the CPUs have changed from r1 to r2.
>>
>> Cc: Sudeep Holla <sudeep.holla@arm.com>
>> Cc: Leo Yan <leo.yan@linaro.org>
>> Cc: Mathieu Poirier <mathieu.porier@linaro.org>
>> Cc: Liviu Dudau <liviu.dudau@arm.com>
>> Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
>
> ARM-SoC team expect DTS to be routed via platform tree.
> So I have queued this for v4.13[1]

Sudeep

Thanks for picking that up.

Suzuki

[toc] | [prev] | [next] | [standalone]

#1645193 — [PATCH v10 04/10] MAINTAINERS: update file entries for Coresight subsystem

Update document file entries for Coresight debug module.

Reviewed-by: Mathieu Poirier <mathieu.poirier@linaro.org>
Signed-off-by: Leo Yan <leo.yan@linaro.org>
---
 MAINTAINERS | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/MAINTAINERS b/MAINTAINERS
index f7d568b..d443258 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1208,7 +1208,9 @@ L:	linux-arm-kernel@lists.infradead.org (moderated for non-subscribers)
 S:	Maintained
 F:	drivers/hwtracing/coresight/*
 F:	Documentation/trace/coresight.txt
+F:	Documentation/trace/coresight-cpu-debug.txt
 F:	Documentation/devicetree/bindings/arm/coresight.txt
+F:	Documentation/devicetree/bindings/arm/coresight-cpu-debug.txt
 F:	Documentation/ABI/testing/sysfs-bus-coresight-devices-*
 F:	tools/perf/arch/arm/util/pmu.c
 F:	tools/perf/arch/arm/util/auxtrace.c
-- 
2.7.4

[toc] | [prev] | [next] | [standalone]

#1645194 — [PATCH v10 03/10] doc: Add coresight_cpu_debug.enable to kernel-parameters.txt

Add coresight_cpu_debug.enable to kernel-parameters.txt, this flag is
used to enable/disable the CPU sampling based debugging.

Reviewed-by: Mathieu Poirier <mathieu.poirier@linaro.org>
Signed-off-by: Leo Yan <leo.yan@linaro.org>
---
 Documentation/admin-guide/kernel-parameters.txt | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index 15f79c2..ff67ad7 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -649,6 +649,13 @@
 			/proc/<pid>/coredump_filter.
 			See also Documentation/filesystems/proc.txt.
 
+	coresight_cpu_debug.enable
+			[ARM,ARM64]
+			Format: <bool>
+			Enable/disable the CPU sampling based debugging.
+			0: default value, disable debugging
+			1: enable debugging at boot time
+
 	cpuidle.off=1	[CPU_IDLE]
 			disable the cpuidle sub-system
 
-- 
2.7.4

[toc] | [prev] | [next] | [standalone]

#1645195 — [PATCH v10 06/10] coresight: refactor with function of_coresight_get_cpu

This is refactor to add function of_coresight_get_cpu(), so it's used to
retrieve CPU id for coresight component. Finally can use it as a common
function for multiple places.

Suggested-by: Mathieu Poirier <mathieu.poirier@linaro.org>
Reviewed-by: Suzuki K Poulose <suzuki.poulose@arm.com>
Signed-off-by: Leo Yan <leo.yan@linaro.org>
---
 drivers/hwtracing/coresight/of_coresight.c | 43 +++++++++++++++++++-----------
 include/linux/coresight.h                  |  3 +++
 2 files changed, 31 insertions(+), 15 deletions(-)

diff --git a/drivers/hwtracing/coresight/of_coresight.c b/drivers/hwtracing/coresight/of_coresight.c
index 225b2dd..a187941 100644
--- a/drivers/hwtracing/coresight/of_coresight.c
+++ b/drivers/hwtracing/coresight/of_coresight.c
@@ -101,16 +101,40 @@ static int of_coresight_alloc_memory(struct device *dev,
 	return 0;
 }
 
+int of_coresight_get_cpu(const struct device_node *node)
+{
+	int cpu;
+	bool found;
+	struct device_node *dn, *np;
+
+	dn = of_parse_phandle(node, "cpu", 0);
+
+	/* Affinity defaults to CPU0 */
+	if (!dn)
+		return 0;
+
+	for_each_possible_cpu(cpu) {
+		np = of_cpu_device_node_get(cpu);
+		found = (dn == np);
+		of_node_put(np);
+		if (found)
+			break;
+	}
+	of_node_put(dn);
+
+	/* Affinity to CPU0 if no cpu nodes are found */
+	return found ? cpu : 0;
+}
+EXPORT_SYMBOL_GPL(of_coresight_get_cpu);
+
 struct coresight_platform_data *
 of_get_coresight_platform_data(struct device *dev,
 			       const struct device_node *node)
 {
-	int i = 0, ret = 0, cpu;
+	int i = 0, ret = 0;
 	struct coresight_platform_data *pdata;
 	struct of_endpoint endpoint, rendpoint;
 	struct device *rdev;
-	bool found;
-	struct device_node *dn, *np;
 	struct device_node *ep = NULL;
 	struct device_node *rparent = NULL;
 	struct device_node *rport = NULL;
@@ -177,18 +201,7 @@ of_get_coresight_platform_data(struct device *dev,
 		} while (ep);
 	}
 
-	dn = of_parse_phandle(node, "cpu", 0);
-	for_each_possible_cpu(cpu) {
-		np = of_cpu_device_node_get(cpu);
-		found = (dn == np);
-		of_node_put(np);
-		if (found)
-			break;
-	}
-	of_node_put(dn);
-
-	/* Affinity to CPU0 if no cpu nodes are found */
-	pdata->cpu = found ? cpu : 0;
+	pdata->cpu = of_coresight_get_cpu(node);
 
 	return pdata;
 }
diff --git a/include/linux/coresight.h b/include/linux/coresight.h
index bf0aa50..d950dad 100644
--- a/include/linux/coresight.h
+++ b/include/linux/coresight.h
@@ -263,10 +263,13 @@ static inline int coresight_timeout(void __iomem *addr, u32 offset,
 #endif
 
 #ifdef CONFIG_OF
+extern int of_coresight_get_cpu(const struct device_node *node);
 extern struct coresight_platform_data *
 of_get_coresight_platform_data(struct device *dev,
 			       const struct device_node *node);
 #else
+static inline int of_coresight_get_cpu(const struct device_node *node)
+{ return 0; }
 static inline struct coresight_platform_data *of_get_coresight_platform_data(
 	struct device *dev, const struct device_node *node) { return NULL; }
 #endif
-- 
2.7.4

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web