Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1217338 > unrolled thread

[PATCH] ARM: fix alignement of __bug_table section entries

Started byRobert Jarzmik <robert.jarzmik@free.fr>
First post2015-09-02 08:30 +0200
Last post2015-09-07 02:00 +0200
Articles 11 — 3 participants

Back to article view | Back to linux.kernel

Contents

  [PATCH] ARM: fix alignement of __bug_table section entries Robert Jarzmik <robert.jarzmik@free.fr> - 2015-09-02 08:30 +0200
    Re: [PATCH] ARM: fix alignement of __bug_table section entries Dave Martin <Dave.Martin@arm.com> - 2015-09-02 12:50 +0200
      Re: [PATCH] ARM: fix alignement of __bug_table section entries Robert Jarzmik <robert.jarzmik@free.fr> - 2015-09-05 16:00 +0200
        Re: [PATCH] ARM: fix alignement of __bug_table section entries Russell King - ARM Linux <linux@arm.linux.org.uk> - 2015-09-05 16:30 +0200
          Re: [PATCH] ARM: fix alignement of __bug_table section entries Robert Jarzmik <robert.jarzmik@free.fr> - 2015-09-05 19:20 +0200
            Re: [PATCH] ARM: fix alignement of __bug_table section entries Russell King - ARM Linux <linux@arm.linux.org.uk> - 2015-09-05 22:40 +0200
              Re: [PATCH] ARM: fix alignement of __bug_table section entries Robert Jarzmik <robert.jarzmik@free.fr> - 2015-09-06 00:20 +0200
              Re: [PATCH] ARM: fix alignement of __bug_table section entries Robert Jarzmik <robert.jarzmik@free.fr> - 2015-09-06 19:40 +0200
                Re: [PATCH] ARM: fix alignement of __bug_table section entries Russell King - ARM Linux <linux@arm.linux.org.uk> - 2015-09-06 21:50 +0200
                  Re: [PATCH] ARM: fix alignement of __bug_table section entries Robert Jarzmik <robert.jarzmik@free.fr> - 2015-09-06 23:40 +0200
                    Re: [PATCH] ARM: fix alignement of __bug_table section entries Russell King - ARM Linux <linux@arm.linux.org.uk> - 2015-09-07 02:00 +0200

#1217338 — [PATCH] ARM: fix alignement of __bug_table section entries

FromRobert Jarzmik <robert.jarzmik@free.fr>
Date2015-09-02 08:30 +0200
Subject[PATCH] ARM: fix alignement of __bug_table section entries
Message-ID<q49It-5mx-7@gated-at.bofh.it>
On old ARM chips, unaligned accesses to memory are not trapped and
fixed.  On module load, symbols are relocated, and the relocation of
__bug_table symbols is done on a u32 basis. Yet the section is not
aligned to a multiple of 4 address, but to a multiple of 2.

This triggers an Oops on pxa architecture, where address 0xbf0021ea
is the first relocation in the __bug_table section :
  apply_relocate(): pxa3xx_nand: section 13 reloc 0 sym ''
  Unable to handle kernel paging request at virtual address bf0021ea
  pgd = e1cd0000
  [bf0021ea] *pgd=c1cce851, *pte=c1cde04f, *ppte=c1cde01f
  Internal error: Oops: 23 [#1] ARM
  Modules linked in:
  CPU: 0 PID: 606 Comm: insmod Not tainted 4.2.0-rc8-next-20150828-cm-x300+ #887
  Hardware name: CM-X300 module
  task: e1c68700 ti: e1c3e000 task.ti: e1c3e000
  PC is at apply_relocate+0x2f4/0x3d4
  LR is at 0xbf0021ea
  pc : [<c000e7c8>]    lr : [<bf0021ea>]    psr: 80000013
  sp : e1c3fe30  ip : 60000013  fp : e49e8c60
  r10: e49e8fa8  r9 : 00000000  r8 : e49e7c58
  r7 : e49e8c38  r6 : e49e8a58  r5 : e49e8920  r4 : e49e8918
  r3 : bf0021ea  r2 : bf007034  r1 : 00000000  r0 : bf000000
  Flags: Nzcv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment none
  Control: 0000397f  Table: c1cd0018  DAC: 00000051
  Process insmod (pid: 606, stack limit = 0xe1c3e198)
  [<c000e7c8>] (apply_relocate) from [<c005ce5c>] (load_module+0x1248/0x1f5c)
  [<c005ce5c>] (load_module) from [<c005dc54>] (SyS_init_module+0xe4/0x170)
  [<c005dc54>] (SyS_init_module) from [<c000a420>] (ret_fast_syscall+0x0/0x38)

Fix this by ensuring entries in __bug_table are all aligned to at least
of multiple of 4. This transforms a module section  __bug_table as :
-   [12] __bug_table       PROGBITS        00000000 002232 000018 00   A  0   0  1
+   [12] __bug_table       PROGBITS        00000000 002232 000018 00   A  0   0  4

Signed-off-by: Robert Jarzmik <robert.jarzmik@free.fr>
---
 arch/arm/include/asm/bug.h | 1 +
 1 file changed, 1 insertion(+)

diff --git a/arch/arm/include/asm/bug.h b/arch/arm/include/asm/bug.h
index b274bde24905..e7335a92144e 100644
--- a/arch/arm/include/asm/bug.h
+++ b/arch/arm/include/asm/bug.h
@@ -40,6 +40,7 @@ do {								\
 		"2:\t.asciz " #__file "\n" 			\
 		".popsection\n" 				\
 		".pushsection __bug_table,\"a\"\n"		\
+		".align 2\n"					\
 		"3:\t.word 1b, 2b\n"				\
 		"\t.hword " #__line ", 0\n"			\
 		".popsection");					\
-- 
2.1.4

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [next] | [standalone]

#1217510

FromDave Martin <Dave.Martin@arm.com>
Date2015-09-02 12:50 +0200
Message-ID<q4dM5-2DA-5@gated-at.bofh.it>
In reply to#1217338
On Wed, Sep 02, 2015 at 08:23:29AM +0200, Robert Jarzmik wrote:
> On old ARM chips, unaligned accesses to memory are not trapped and
> fixed.  On module load, symbols are relocated, and the relocation of
> __bug_table symbols is done on a u32 basis. Yet the section is not
> aligned to a multiple of 4 address, but to a multiple of 2.
> 
> This triggers an Oops on pxa architecture, where address 0xbf0021ea
> is the first relocation in the __bug_table section :
>   apply_relocate(): pxa3xx_nand: section 13 reloc 0 sym ''
>   Unable to handle kernel paging request at virtual address bf0021ea
>   pgd = e1cd0000
>   [bf0021ea] *pgd=c1cce851, *pte=c1cde04f, *ppte=c1cde01f
>   Internal error: Oops: 23 [#1] ARM
>   Modules linked in:
>   CPU: 0 PID: 606 Comm: insmod Not tainted 4.2.0-rc8-next-20150828-cm-x300+ #887
>   Hardware name: CM-X300 module
>   task: e1c68700 ti: e1c3e000 task.ti: e1c3e000
>   PC is at apply_relocate+0x2f4/0x3d4
>   LR is at 0xbf0021ea
>   pc : [<c000e7c8>]    lr : [<bf0021ea>]    psr: 80000013
>   sp : e1c3fe30  ip : 60000013  fp : e49e8c60
>   r10: e49e8fa8  r9 : 00000000  r8 : e49e7c58
>   r7 : e49e8c38  r6 : e49e8a58  r5 : e49e8920  r4 : e49e8918
>   r3 : bf0021ea  r2 : bf007034  r1 : 00000000  r0 : bf000000
>   Flags: Nzcv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment none
>   Control: 0000397f  Table: c1cd0018  DAC: 00000051
>   Process insmod (pid: 606, stack limit = 0xe1c3e198)
>   [<c000e7c8>] (apply_relocate) from [<c005ce5c>] (load_module+0x1248/0x1f5c)
>   [<c005ce5c>] (load_module) from [<c005dc54>] (SyS_init_module+0xe4/0x170)
>   [<c005dc54>] (SyS_init_module) from [<c000a420>] (ret_fast_syscall+0x0/0x38)
> 
> Fix this by ensuring entries in __bug_table are all aligned to at least
> of multiple of 4. This transforms a module section  __bug_table as :
> -   [12] __bug_table       PROGBITS        00000000 002232 000018 00   A  0   0  1
> +   [12] __bug_table       PROGBITS        00000000 002232 000018 00   A  0   0  4
> 
> Signed-off-by: Robert Jarzmik <robert.jarzmik@free.fr>
> ---
>  arch/arm/include/asm/bug.h | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/arch/arm/include/asm/bug.h b/arch/arm/include/asm/bug.h
> index b274bde24905..e7335a92144e 100644
> --- a/arch/arm/include/asm/bug.h
> +++ b/arch/arm/include/asm/bug.h
> @@ -40,6 +40,7 @@ do {								\
>  		"2:\t.asciz " #__file "\n" 			\
>  		".popsection\n" 				\
>  		".pushsection __bug_table,\"a\"\n"		\
> +		".align 2\n"					\
>  		"3:\t.word 1b, 2b\n"				\
>  		"\t.hword " #__line ", 0\n"			\
>  		".popsection");					\

Reviewed-by: Dave Martin <Dave.Martin@arm.com>

I added the .align in my recent patches implementing BUG for arm64,
but didn't touch arch/arm.

When referring to the arm code I did notice that there was no .align.
I'd concluded that the linker script layout and lack of bug reports
meant the arm code was alignment-safe in practice, but I guess I was
mistaken...

Cheers
---Dave

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1219554

FromRobert Jarzmik <robert.jarzmik@free.fr>
Date2015-09-05 16:00 +0200
Message-ID<q5maB-2gW-13@gated-at.bofh.it>
In reply to#1217510
Dave Martin <Dave.Martin@arm.com> writes:

> On Wed, Sep 02, 2015 at 08:23:29AM +0200, Robert Jarzmik wrote:
>> On old ARM chips, unaligned accesses to memory are not trapped and
>> fixed.  On module load, symbols are relocated, and the relocation of
>> __bug_table symbols is done on a u32 basis. Yet the section is not
>> aligned to a multiple of 4 address, but to a multiple of 2.

Hi Russell,

I dug deeper, and got another stack, unrelated to modules insertion but related
to alignement fault (see [1] for reference). As before, this didn't happen on
v4.1, but happens on linux-next.
I'm wondering if alignement fixup does work in my case and if I understand it.

This time I took my JTAG to have a look at the flow, in arch/arm/mm/alignment.c,
where I added the small chunk in [2], which gave in my case :
    RJK: fault=4 instr=0x00000000 instrptr=0xc02b37c8 thumb_mode=0 tinstr=0x0000

The instruction (as seen with vmlinux disassembly or JTAG memory dump) is  :
    0xc02b37c8 <+372>:	b2 50 c6 10	strhne	r5, [r6], #2

I must admit I fail to see how the following "fixup:" label can be reached with
a "missed" copy_from_user() (fault == 4).

This is probably what happened to me with the modules __bug_table section, and
it will continue to happen until I understand why this copy fails. It's really
odd nobody but me faces this issue.

Cheers.

-- 
Robert

[1] New stack
=============
  #0: pxa2xx-ac97 (Wolfson WM9713,WM9714)
RJK: fault=4 instr=0x00000000 instrptr=0xc02b37c8 thumb_mode=0 tinstr=0x0000
&Unable to handle kernel paging request at virtual address c3057661
&pgd = c0004000
"[c3057661] *pgd=a300040e(bad)
Internal error: Oops: 803 [#1] PREEMPT ARM
Modules linked in:
CPU: 0 PID: 1 Comm: swapper Not tainted 4.2.0-rc8-next-20150828+ #900
Hardware name: MIO A701
task: c3858bc0 ti: c385c000 task.ti: c385c000
PC is at doc_read_data_area+0x174/0x370
LR is at doc_read_page_getbytes+0x58/0x78
pc : [<c02b37c8>]    lr : [<c02b3a1c>]    psr: a8000013
sp : c385d8e0  ip : c07142b8  fp : c385d91c
r10: c3aac540  r9 : c070ffc0  r8 : c385c000
@QGi
r7 : 00000002  r6 : c3057661  r5 : 0000c1e5  r4 : 0000000b
r3 : 0000000a  r2 : 0000000b  r1 : c3057661  r0 : 00000000
Flags: NzCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment none
Control: 0000397f  Table: a0004000  DAC: 00000053
Process swapper (pid: 1, stack limit = 0xc385c198)
Stack: (0xc385d8e0 to 0xc385e000)
d8e0: 00000000 c02b3a34 00000001 0000000a c385d914 c3057660 0000000c c3aac540
... chop chop ...
dfe0: 00000000 00000000 00000000 00000000 00000013 00000000 f3ff8cdd 4cf3d76f
[<c02b37c8>] (doc_read_data_area) from [<c02b3a1c>] (doc_read_page_getbytes+0x58/0x78)
[<c02b3a1c>] (doc_read_page_getbytes) from [<c02b5f20>] (doc_read_oob+0x22c/0x75c)
[<c02b5f20>] (doc_read_oob) from [<c02b64b4>] (doc_read+0x64/0x74)
[<c02b64b4>] (doc_read) from [<c02ade08>] (part_read+0x58/0x9c)
[<c02ade08>] (part_read) from [<c02aa8d8>] (mtd_read+0x88/0xbc)
[<c02aa8d8>] (mtd_read) from [<c02c35ec>] (ubi_io_read+0x16c/0x358)
[<c02c35ec>] (ubi_io_read) from [<c02c0450>] (ubi_eba_read_leb+0x384/0x4d4)
[<c02c0450>] (ubi_eba_read_leb) from [<c02bf944>] (ubi_leb_read+0xd4/0x134)
[<c02bf944>] (ubi_leb_read) from [<c0185e9c>] (ubifs_leb_read+0x3c/0xa8)s
[<c0185e9c>] (ubifs_leb_read) from [<c019fd94>] (ubifs_read_nnode+0xec/0x200)
[<c019fd94>] (ubifs_read_nnode) from [<c01a03a0>] (ubifs_lpt_lookup_dirty+0x38/0x330)
[<c01a03a0>] (ubifs_lpt_lookup_dirty) from [<c0190448>] (ubifs_replay_journal+0x3c/0x1b38)
[<c0190448>] (ubifs_replay_journal) from [<c0182c38>] (ubifs_mount+0x1444/0x2410)
[<c0182c38>] (ubifs_mount) from [<c00eb604>] (mount_fs+0x24/0xb0)
[<c00eb604>] (mount_fs) from [<c01081d0>] (vfs_kern_mount+0x58/0x124)
[<c01081d0>] (vfs_kern_mount) from [<c010b384>] (do_mount+0xb40/0xd10)
[<c010b384>] (do_mount) from [<c010b8a8>] (SyS_mount+0x84/0xb0)
[<c010b8a8>] (SyS_mount) from [<c068319c>] (mount_block_root+0x12c/0x2dc)
[<c068319c>] (mount_block_root) from [<c0683564>] (prepare_namespace+0x98/0x1bc)
[<c0683564>] (prepare_namespace) from [<c0682eb4>] (kernel_init_freeable+0x188/0x1d4)
[<c0682eb4>] (kernel_init_freeable) from [<c04a2610>] (kernel_init+0x18/0xfc)
[<c04a2610>] (kernel_init) from [<c000a5ec>] (ret_from_fork+0x14/0x28)
Code: 1a000060 e51b3030 e3560000 e2877002 (10c650b2) 
---[ end trace a0bcca195299a22d ]---

[2] Debug patch
===============
diff --git a/arch/arm/mm/alignment.c b/arch/arm/mm/alignment.c
index 2c0c541c60ca..b0897da5456c 100644
--- a/arch/arm/mm/alignment.c
+++ b/arch/arm/mm/alignment.c
@@ -787,6 +787,8 @@ do_alignment(unsigned long addr, unsigned int fsr, struct pt_regs *regs)
                instr = __mem_to_opcode_arm(instr);
        }
 
+       pr_info("RJK: fault=%d instr=0x%08lx instrptr=0x%08lx thumb_mode=%lu tinstr=0x%04x\n",
+               fault, instr, instrptr, thumb_mode(regs), tinstr);
        if (fault) {
                type = TYPE_FAULT;
                goto bad_or_fault;

[3] Abort stack
===============
#0  panic (fmt=0xc385d6c4 <incomplete sequence \344>) at kernel/panic.c:72
#1  0xc000e788 in oops_end (regs=<optimized out>, signr=<optimized out>, flags=<optimized out>) at arch/arm/kernel/traps.c:311
#2  die (str=0x68000013 "", regs=<optimized out>, err=-1014638958) at arch/arm/kernel/traps.c:333
#3  0xc00137c0 in __do_kernel_fault (mm=0xc06e8c04 <init_mm>, addr=3271915105, fsr=2051, regs=0xc385d890) at arch/arm/mm/fault.c:150
#4  0xc0013b10 in do_bad_area (addr=<optimized out>, fsr=<optimized out>, regs=<optimized out>) at arch/arm/mm/fault.c:198
#5  0xc00166c8 in do_alignment (addr=3271915105, fsr=2051, regs=0xc385d890) at arch/arm/mm/alignment.c:900
#6  0xc0009264 in do_DataAbort (addr=3271915105, fsr=2051, regs=0xc385d890) at arch/arm/mm/fault.c:550
#7  0xc000f024 in __dabt_svc () at arch/arm/kernel/entry-armv.S:204

[4] frame #6 regs dump
======================
$1 = {uregs = {0, 3271915105, 11, 10, 11, 49637, 3271915105, 2, 3280322560, 3228630976, 3282748736, 3280328988, 3228648120, 3280328928, 3224058396, 3224057800, 2818572307, 
    4294967295}}
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1219563

FromRussell King - ARM Linux <linux@arm.linux.org.uk>
Date2015-09-05 16:30 +0200
Message-ID<q5mDD-33W-1@gated-at.bofh.it>
In reply to#1219554
On Sat, Sep 05, 2015 at 03:48:38PM +0200, Robert Jarzmik wrote:
> Dave Martin <Dave.Martin@arm.com> writes:
> 
> > On Wed, Sep 02, 2015 at 08:23:29AM +0200, Robert Jarzmik wrote:
> >> On old ARM chips, unaligned accesses to memory are not trapped and
> >> fixed.  On module load, symbols are relocated, and the relocation of
> >> __bug_table symbols is done on a u32 basis. Yet the section is not
> >> aligned to a multiple of 4 address, but to a multiple of 2.
> 
> Hi Russell,
> 
> I dug deeper, and got another stack, unrelated to modules insertion but related
> to alignement fault (see [1] for reference). As before, this didn't happen on
> v4.1, but happens on linux-next.
> I'm wondering if alignement fixup does work in my case and if I understand it.
> 
> This time I took my JTAG to have a look at the flow, in arch/arm/mm/alignment.c,
> where I added the small chunk in [2], which gave in my case :
>     RJK: fault=4 instr=0x00000000 instrptr=0xc02b37c8 thumb_mode=0 tinstr=0x0000

Right, so as fault is nonzero, this means that we were unable to read the
instruction.  That seems mad though - the instruction pointer is certainly
valid, and as we're using probe_kernel_address(), that switches to the
kernel "segment" before trying to read kernel addresses.  That should
mean that __copy_from_user_inatomic() is able to read the instruction.

I think this is the root cause of the issue.

> The instruction (as seen with vmlinux disassembly or JTAG memory dump) is  :
>     0xc02b37c8 <+372>:	b2 50 c6 10	strhne	r5, [r6], #2
> 
> I must admit I fail to see how the following "fixup:" label can be reached with
> a "missed" copy_from_user() (fault == 4).

We can't fix up the fault if we failed to read the instruction causing the
fault - because we've no idea what register(s) will need updating.

> [1] New stack
> =============
>   #0: pxa2xx-ac97 (Wolfson WM9713,WM9714)
> RJK: fault=4 instr=0x00000000 instrptr=0xc02b37c8 thumb_mode=0 tinstr=0x0000
> &Unable to handle kernel paging request at virtual address c3057661
> &pgd = c0004000
> "[c3057661] *pgd=a300040e(bad)
> Internal error: Oops: 803 [#1] PREEMPT ARM
> Modules linked in:
> CPU: 0 PID: 1 Comm: swapper Not tainted 4.2.0-rc8-next-20150828+ #900
> Hardware name: MIO A701
> task: c3858bc0 ti: c385c000 task.ti: c385c000
> PC is at doc_read_data_area+0x174/0x370
> LR is at doc_read_page_getbytes+0x58/0x78
> pc : [<c02b37c8>]    lr : [<c02b3a1c>]    psr: a8000013
> sp : c385d8e0  ip : c07142b8  fp : c385d91c
> r10: c3aac540  r9 : c070ffc0  r8 : c385c000
> @QGi
> r7 : 00000002  r6 : c3057661  r5 : 0000c1e5  r4 : 0000000b
> r3 : 0000000a  r2 : 0000000b  r1 : c3057661  r0 : 00000000
> Flags: NzCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment none

It seems you have SW_DOMAIN_PAN enabled.

> Control: 0000397f  Table: a0004000  DAC: 00000053

And the DACR for the parent context shows that user no-access, kernel
manager-access (iow, in the doc_read_data_area() function).  I have to
wonder why that would be the case - I can't find anything that would
set the kernel to manager-access.  There's no get_ds() or KERNEL_DS
reference in fs/ubifs or drivers/mtd.

> [3] Abort stack
> ===============
> #0  panic (fmt=0xc385d6c4 <incomplete sequence \344>) at kernel/panic.c:72
> #1  0xc000e788 in oops_end (regs=<optimized out>, signr=<optimized out>, flags=<optimized out>) at arch/arm/kernel/traps.c:311
> #2  die (str=0x68000013 "", regs=<optimized out>, err=-1014638958) at arch/arm/kernel/traps.c:333
> #3  0xc00137c0 in __do_kernel_fault (mm=0xc06e8c04 <init_mm>, addr=3271915105, fsr=2051, regs=0xc385d890) at arch/arm/mm/fault.c:150
> #4  0xc0013b10 in do_bad_area (addr=<optimized out>, fsr=<optimized out>, regs=<optimized out>) at arch/arm/mm/fault.c:198
> #5  0xc00166c8 in do_alignment (addr=3271915105, fsr=2051, regs=0xc385d890) at arch/arm/mm/alignment.c:900
> #6  0xc0009264 in do_DataAbort (addr=3271915105, fsr=2051, regs=0xc385d890) at arch/arm/mm/fault.c:550
> #7  0xc000f024 in __dabt_svc () at arch/arm/kernel/entry-armv.S:204

I'm afraid this isn't useful, and I think (as seems to be typical with gdb)
some of those values are completely wrong.  There's no way "str" would
be 0x68000013 in frame 2 for example.

-- 
FTTC broadband for 0.8mile line: currently at 9.6Mbps down 400kbps up
according to speedtest.net.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1219587

FromRobert Jarzmik <robert.jarzmik@free.fr>
Date2015-09-05 19:20 +0200
Message-ID<q5pia-6Se-7@gated-at.bofh.it>
In reply to#1219563
Russell King - ARM Linux <linux@arm.linux.org.uk> writes:

> On Sat, Sep 05, 2015 at 03:48:38PM +0200, Robert Jarzmik wrote:
>> This time I took my JTAG to have a look at the flow, in arch/arm/mm/alignment.c,
>> where I added the small chunk in [2], which gave in my case :
>>     RJK: fault=4 instr=0x00000000 instrptr=0xc02b37c8 thumb_mode=0 tinstr=0x0000
>
> Right, so as fault is nonzero, this means that we were unable to read the
> instruction.  That seems mad though - the instruction pointer is certainly
> valid, and as we're using probe_kernel_address(), that switches to the
> kernel "segment" before trying to read kernel addresses.  That should
> mean that __copy_from_user_inatomic() is able to read the instruction.
>
> I think this is the root cause of the issue.

And there is more madness to come : I tried to "reread" the instruction [1] a
second time if the first result was 4 :
RJK: fault=4 instr=0x00000000(@c385d72c) instrptr=0xc02b39e8 thumb_mode=0 tinstr=0x0000
RJK: reread instruction: [0xc02b39e8] = 0x10c650b2: 0

Guess what, the second probe_kernel_address() with the same parameters returns
0, and everything works. It's insane.

>> Flags: NzCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment none
> It seems you have SW_DOMAIN_PAN enabled.
That's the default arch/arm/Kconfig implies.
And ... this is what also _is_ the cause of this behavior : removing
SW_DOMAIN_PAN makes all my pxa boards work again !!!

Moreover, this is consistent with the fact that this commit is in linux-next but
not in v4.1 :
    a5e090acbf54 ("ARM: software-based priviledged-no-access support")

So the issue is around this SW_DOMAIN_PAN, at least on PXA.

--
Robert

[1]
@@ -787,6 +798,15 @@ do_alignment(unsigned long addr, unsigned int fsr, struct pt_regs *regs)
                instr = __mem_to_opcode_arm(instr);
        }
 
+       pr_info("RJK: fault=%d instr=0x%08lx(@%p) instrptr=0x%08lx thumb_mode=%lu tinstr=0x%04x\n",
+               fault, instr, &instr, instrptr, thumb_mode(regs), tinstr);
+       if (fault == 4 && !thumb_mode(regs)) {
+               fault = probe_kernel_address(instrptr, instr);
+               pr_info("RJK: reread instruction: [0x%08lx] = 0x%08lx: %u\n",
+                       instrptr, instr, fault);
+               rjk_debug_point(instrptr);
+       }
+
        if (fault) {
                type = TYPE_FAULT;
                goto bad_or_fault;

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1219644

FromRussell King - ARM Linux <linux@arm.linux.org.uk>
Date2015-09-05 22:40 +0200
Message-ID<q5spH-2MT-11@gated-at.bofh.it>
In reply to#1219587
On Sat, Sep 05, 2015 at 07:10:49PM +0200, Robert Jarzmik wrote:
> Russell King - ARM Linux <linux@arm.linux.org.uk> writes:
> 
> > On Sat, Sep 05, 2015 at 03:48:38PM +0200, Robert Jarzmik wrote:
> >> This time I took my JTAG to have a look at the flow, in arch/arm/mm/alignment.c,
> >> where I added the small chunk in [2], which gave in my case :
> >>     RJK: fault=4 instr=0x00000000 instrptr=0xc02b37c8 thumb_mode=0 tinstr=0x0000
> >
> > Right, so as fault is nonzero, this means that we were unable to read the
> > instruction.  That seems mad though - the instruction pointer is certainly
> > valid, and as we're using probe_kernel_address(), that switches to the
> > kernel "segment" before trying to read kernel addresses.  That should
> > mean that __copy_from_user_inatomic() is able to read the instruction.
> >
> > I think this is the root cause of the issue.
> 
> And there is more madness to come : I tried to "reread" the instruction [1] a
> second time if the first result was 4 :
> RJK: fault=4 instr=0x00000000(@c385d72c) instrptr=0xc02b39e8 thumb_mode=0 tinstr=0x0000
> RJK: reread instruction: [0xc02b39e8] = 0x10c650b2: 0
> 
> Guess what, the second probe_kernel_address() with the same parameters returns
> 0, and everything works. It's insane.
> 
> >> Flags: NzCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment none
> > It seems you have SW_DOMAIN_PAN enabled.
> That's the default arch/arm/Kconfig implies.
> And ... this is what also _is_ the cause of this behavior : removing
> SW_DOMAIN_PAN makes all my pxa boards work again !!!
> 
> Moreover, this is consistent with the fact that this commit is in linux-next but
> not in v4.1 :
>     a5e090acbf54 ("ARM: software-based priviledged-no-access support")
> 
> So the issue is around this SW_DOMAIN_PAN, at least on PXA.

Is it only PXA which seems to be affected?

If so, you may need to add:

	mrc p15, 0, \rd, c2, c0, 0
	mov \rd, \rd
	sub pc, pc, #4

to the places we update the domain access register to ensure that the
Xscale pipeline stalls to allow the CP15 DACR update to hit.

-- 
FTTC broadband for 0.8mile line: currently at 9.6Mbps down 400kbps up
according to speedtest.net.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1219651

FromRobert Jarzmik <robert.jarzmik@free.fr>
Date2015-09-06 00:20 +0200
Message-ID<q5tYu-55g-13@gated-at.bofh.it>
In reply to#1219644
Russell King - ARM Linux <linux@arm.linux.org.uk> writes:

>> Moreover, this is consistent with the fact that this commit is in linux-next but
>> not in v4.1 :
>>     a5e090acbf54 ("ARM: software-based priviledged-no-access support")
>> 
>> So the issue is around this SW_DOMAIN_PAN, at least on PXA.
>
> Is it only PXA which seems to be affected?
Sorry I don't know, I only own pxa platforms.

> If so, you may need to add:
>
> 	mrc p15, 0, \rd, c2, c0, 0
> 	mov \rd, \rd
> 	sub pc, pc, #4
>
> to the places we update the domain access register to ensure that the
> Xscale pipeline stalls to allow the CP15 DACR update to hit.
Okay, I'll try that.

By the way, the ARMv5 manual states in chapter "B4.5.1 MMU Fault" that for a
DACR update, a "PrefetchFlush" operation has to be done (chapter B2.6.3
PrefetchFlush CP15 register 7), quoting :
    Changes to the Domain Access Control register are synchronized by performing
    a PrefetchFlush operation (or as result of an exception or exception
    return). See Changes to CP15 registers and the memory order model on page
    B2-24 for details.

Cheers.

-- 
Robert
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1219838

FromRobert Jarzmik <robert.jarzmik@free.fr>
Date2015-09-06 19:40 +0200
Message-ID<q5M53-5w9-11@gated-at.bofh.it>
In reply to#1219644
Russell King - ARM Linux <linux@arm.linux.org.uk> writes:

> On Sat, Sep 05, 2015 at 07:10:49PM +0200, Robert Jarzmik wrote:
>> Russell King - ARM Linux <linux@arm.linux.org.uk> writes:
>> So the issue is around this SW_DOMAIN_PAN, at least on PXA.
>
> If so, you may need to add:
>
> 	mrc p15, 0, \rd, c2, c0, 0
> 	mov \rd, \rd
> 	sub pc, pc, #4
>
> to the places we update the domain access register to ensure that the
> Xscale pipeline stalls to allow the CP15 DACR update to hit.
Nope, that didn't work.
I have tried 2 different patches :
 - in [1], your proposed solution
 - in [2], a PrefetchFlush as adviced by ARM Architecture Reference Manual

None of them worked. I confirmed by disassembling __dabt_svc my changes hit the
abort routine, and they did. I'll continue next week by trying to have a closer
look at the SW_DOMAIN_PAN commit (a5e090acbf545), and take time to walk through
the whole Oops for information I have missed.

Cheers.

-- 
Robert

[1] Approach 1 : translation table sync
=======================================
diff --cc arch/arm/include/asm/assembler.h
index 7bbf325a4f31,7bbf325a4f31..6bb46198fd08
--- a/arch/arm/include/asm/assembler.h
+++ b/arch/arm/include/asm/assembler.h
@@@ -449,6 -449,6 +449,12 @@@ THUMB(    orr     \reg , \reg , #PSR_T_BIT        
  #endif
        .endm
  
++      .macro  dacr_sync, rd
++      mrc     p15, 0, \rd, c2, c0, 0
++      mov     \rd, \rd
++      sub     pc, pc, #4
++      .endm
++
        .macro  uaccess_disable, tmp, isb=1
  #ifdef CONFIG_CPU_SW_DOMAIN_PAN
        /*
@@@ -457,6 -457,6 +463,7 @@@
         */
        mov     \tmp, #DACR_UACCESS_DISABLE
        mcr     p15, 0, \tmp, c3, c0, 0         @ Set domain register
++      dacr_sync \tmp
        .if     \isb
        instr_sync
        .endif
@@@ -471,6 -471,6 +478,7 @@@
         */
        mov     \tmp, #DACR_UACCESS_ENABLE
        mcr     p15, 0, \tmp, c3, c0, 0
++      dacr_sync \tmp
        .if     \isb
        instr_sync
        .endif
@@@ -488,6 -488,6 +496,7 @@@
  #ifdef CONFIG_CPU_SW_DOMAIN_PAN
        ldr     r0, [sp, #S_FRAME_SIZE]
        mcr     p15, 0, r0, c3, c0, 0
++      dacr_sync r0
  #endif
        .endm

[2] Approach 2 : PrefetchFlush
==============================
diff --git a/arch/arm/include/asm/assembler.h b/arch/arm/include/asm/assembler.h
index 7bbf325a4f31..2152d43d7ede 100644
--- a/arch/arm/include/asm/assembler.h
+++ b/arch/arm/include/asm/assembler.h
@@ -449,6 +449,10 @@ THUMB(     orr     \reg , \reg , #PSR_T_BIT        )
 #endif
        .endm
 
+       .macro  dacr_sync, rd
+       mcr     p15, 0, \rd, c7, c5, 4
+       .endm
+
        .macro  uaccess_disable, tmp, isb=1
 #ifdef CONFIG_CPU_SW_DOMAIN_PAN
        /*
@@ -457,6 +461,7 @@ THUMB(      orr     \reg , \reg , #PSR_T_BIT        )
         */
        mov     \tmp, #DACR_UACCESS_DISABLE
        mcr     p15, 0, \tmp, c3, c0, 0         @ Set domain register
+       dacr_sync \tmp
        .if     \isb
        instr_sync
        .endif
@@ -471,6 +476,7 @@ THUMB(      orr     \reg , \reg , #PSR_T_BIT        )
         */
        mov     \tmp, #DACR_UACCESS_ENABLE
        mcr     p15, 0, \tmp, c3, c0, 0
+       dacr_sync \tmp
        .if     \isb
        instr_sync
        .endif
@@ -488,6 +494,7 @@ THUMB(      orr     \reg , \reg , #PSR_T_BIT        )
 #ifdef CONFIG_CPU_SW_DOMAIN_PAN
        ldr     r0, [sp, #S_FRAME_SIZE]
        mcr     p15, 0, r0, c3, c0, 0
+       dacr_sync r0
 #endif
        .endm
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1219850

FromRussell King - ARM Linux <linux@arm.linux.org.uk>
Date2015-09-06 21:50 +0200
Message-ID<q5O6S-8ne-7@gated-at.bofh.it>
In reply to#1219838
On Sun, Sep 06, 2015 at 07:25:01PM +0200, Robert Jarzmik wrote:
> Russell King - ARM Linux <linux@arm.linux.org.uk> writes:
> 
> > On Sat, Sep 05, 2015 at 07:10:49PM +0200, Robert Jarzmik wrote:
> >> Russell King - ARM Linux <linux@arm.linux.org.uk> writes:
> >> So the issue is around this SW_DOMAIN_PAN, at least on PXA.
> >
> > If so, you may need to add:
> >
> > 	mrc p15, 0, \rd, c2, c0, 0
> > 	mov \rd, \rd
> > 	sub pc, pc, #4
> >
> > to the places we update the domain access register to ensure that the
> > Xscale pipeline stalls to allow the CP15 DACR update to hit.
> Nope, that didn't work.
> I have tried 2 different patches :
>  - in [1], your proposed solution
>  - in [2], a PrefetchFlush as adviced by ARM Architecture Reference Manual
> 
> None of them worked. I confirmed by disassembling __dabt_svc my changes hit the
> abort routine, and they did. I'll continue next week by trying to have a closer
> look at the SW_DOMAIN_PAN commit (a5e090acbf545), and take time to walk through
> the whole Oops for information I have missed.
> 
> Cheers.
> 
> -- 
> Robert
> 
> [1] Approach 1 : translation table sync
> =======================================
> diff --cc arch/arm/include/asm/assembler.h
> index 7bbf325a4f31,7bbf325a4f31..6bb46198fd08
> --- a/arch/arm/include/asm/assembler.h
> +++ b/arch/arm/include/asm/assembler.h
> @@@ -449,6 -449,6 +449,12 @@@ THUMB(    orr     \reg , \reg , #PSR_T_BIT        
>   #endif
>         .endm
>   
> ++      .macro  dacr_sync, rd
> ++      mrc     p15, 0, \rd, c2, c0, 0
> ++      mov     \rd, \rd
> ++      sub     pc, pc, #4
> ++      .endm
> ++
>         .macro  uaccess_disable, tmp, isb=1
>   #ifdef CONFIG_CPU_SW_DOMAIN_PAN
>         /*
> @@@ -457,6 -457,6 +463,7 @@@
>          */
>         mov     \tmp, #DACR_UACCESS_DISABLE
>         mcr     p15, 0, \tmp, c3, c0, 0         @ Set domain register
> ++      dacr_sync \tmp
>         .if     \isb
>         instr_sync
>         .endif
> @@@ -471,6 -471,6 +478,7 @@@
>          */
>         mov     \tmp, #DACR_UACCESS_ENABLE
>         mcr     p15, 0, \tmp, c3, c0, 0
> ++      dacr_sync \tmp
>         .if     \isb
>         instr_sync
>         .endif
> @@@ -488,6 -488,6 +496,7 @@@
>   #ifdef CONFIG_CPU_SW_DOMAIN_PAN
>         ldr     r0, [sp, #S_FRAME_SIZE]
>         mcr     p15, 0, r0, c3, c0, 0
> ++      dacr_sync r0
>   #endif
>         .endm

The important place is in arch/arm/include/asm/domain.h, which is where
we manipulate the DACR within probe_kernel_address().

-- 
FTTC broadband for 0.8mile line: currently at 9.6Mbps down 400kbps up
according to speedtest.net.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1219866

FromRobert Jarzmik <robert.jarzmik@free.fr>
Date2015-09-06 23:40 +0200
Message-ID<q5PPk-2pH-7@gated-at.bofh.it>
In reply to#1219850
Russell King - ARM Linux <linux@arm.linux.org.uk> writes:

>> [1] Approach 1 : translation table sync
>> =======================================
...
> The important place is in arch/arm/include/asm/domain.h, which is where
> we manipulate the DACR within probe_kernel_address().

Gah, silly me. But even with [1], I still get an error [2]. I have a
confirmation that I have a "Page Permission" fault on the
probe_kernel_address().

Next thing I'll check is if I can read the TLB cache for the code entry. It's a
very instructive bug for me :)

Cheers.

-- 
Robert

[1] Approach 1 : translation table sync + PrefetchFlush
=======================================================
diff --git a/arch/arm/include/asm/assembler.h b/arch/arm/include/asm/assembler.h
index 7bbf325a4f31..73d5ad456e32 100644
--- a/arch/arm/include/asm/assembler.h
+++ b/arch/arm/include/asm/assembler.h
@@ -449,6 +449,13 @@ THUMB(	orr	\reg , \reg , #PSR_T_BIT	)
 #endif
 	.endm
 
+	.macro	dacr_sync, rd
+	mrc     p15, 0, \rd, c2, c0, 0
+	mov     \rd, \rd
+	sub     pc, pc, #4
+	mcr     p15, 0, \rd, c7, c5, 4
+	.endm
+
 	.macro	uaccess_disable, tmp, isb=1
 #ifdef CONFIG_CPU_SW_DOMAIN_PAN
 	/*
@@ -457,6 +464,7 @@ THUMB(	orr	\reg , \reg , #PSR_T_BIT	)
 	 */
 	mov	\tmp, #DACR_UACCESS_DISABLE
 	mcr	p15, 0, \tmp, c3, c0, 0		@ Set domain register
+	dacr_sync \tmp
 	.if	\isb
 	instr_sync
 	.endif
@@ -471,6 +479,7 @@ THUMB(	orr	\reg , \reg , #PSR_T_BIT	)
 	 */
 	mov	\tmp, #DACR_UACCESS_ENABLE
 	mcr	p15, 0, \tmp, c3, c0, 0
+	dacr_sync \tmp
 	.if	\isb
 	instr_sync
 	.endif
@@ -488,6 +497,7 @@ THUMB(	orr	\reg , \reg , #PSR_T_BIT	)
 #ifdef CONFIG_CPU_SW_DOMAIN_PAN
 	ldr	r0, [sp, #S_FRAME_SIZE]
 	mcr	p15, 0, r0, c3, c0, 0
+	dacr_sync r0
 #endif
 	.endm
 
diff --git a/arch/arm/include/asm/domain.h b/arch/arm/include/asm/domain.h
index e878129f2fee..10c9a38636ac 100644
--- a/arch/arm/include/asm/domain.h
+++ b/arch/arm/include/asm/domain.h
@@ -97,7 +97,11 @@ static inline unsigned int get_domain(void)
 static inline void set_domain(unsigned val)
 {
 	asm volatile(
-	"mcr	p15, 0, %0, c3, c0	@ set domain"
+	"mcr	p15, 0, %0, c3, c0;	@ set domain	\
+	mrc	p15, 0, %0, c2, c0, 0;			\
+	mov	%0, %0;					\
+	sub	pc, pc, #4;				\
+	mcr     p15, 0, %0, c7, c5, 4"
 	  : : "r" (val));
 	isb();
 }
diff --git a/arch/arm/mm/alignment.c b/arch/arm/mm/alignment.c
index 9769f1eefe3b..c9c454129344 100644
--- a/arch/arm/mm/alignment.c
+++ b/arch/arm/mm/alignment.c
@@ -747,6 +747,27 @@ do_alignment_t32_to_handler(unsigned long *pinstr, struct pt_regs *regs,
 	return NULL;
 }
 
+static u32 far_read(void)
+{
+        u32 far;
+        asm("mrc        p15, 0, %0, c6, c0, 0" : "=r" (far));
+        return far;
+}
+
+static u32 fsr_read(void)
+{
+        u32 fsr;
+        asm("mrc        p15, 0, %0, c5, c0, 0" : "=r" (fsr));
+        return fsr;
+}
+
+static u32 dacr_read(void)
+{
+        u32 dacr;
+        asm("mrc        p15, 0, %0, c3, c0, 0" : "=r" (dacr));
+        return dacr;
+}
+
 static int
 do_alignment(unsigned long addr, unsigned int fsr, struct pt_regs *regs)
 {
@@ -763,6 +784,8 @@ do_alignment(unsigned long addr, unsigned int fsr, struct pt_regs *regs)
 		local_irq_enable();
 
 	instrptr = instruction_pointer(regs);
+	pr_info("RJK1: fsr=%x far=%x dacr=%x\n", fsr_read(), far_read(), dacr_read());
+	pr_info("RJK2: fsr=%x far=%x dacr=%x\n", fsr_read(), far_read(), dacr_read());
 
 	if (thumb_mode(regs)) {
 		u16 *ptr = (u16 *)(instrptr & ~1);
@@ -787,6 +810,8 @@ do_alignment(unsigned long addr, unsigned int fsr, struct pt_regs *regs)
 		instr = __mem_to_opcode_arm(instr);
 	}
 
+	pr_info("RJK3: fsr=%x far=%x dacr=%x\n", fsr_read(), far_read(), dacr_read());
+	pr_info("RJK: fault=%d instr=0x%08x instrptr=%p\n", fault, instr, instrptr);
 	if (fault) {
 		type = TYPE_FAULT;
 		goto bad_or_fault;

[2] Oops
========
# insmod /tmp/unalign.ko 
RJK1: fsr=23 far=e1c23643 dacr=51
RJK2: fsr=23 far=e1c23643 dacr=51
RJK3: fsr=2f far=bf00202c dacr=51
RJK: fault=4 instr=0x00000000 instrptr=bf00202c
Unable to handle kernel paging request at virtual address e1c23643
pgd = e1cd4000
[e1c23643] *pgd=c1c0044e(bad)
Internal error: Oops: 823 [#1] ARM
Modules linked in: unalign(+)
CPU: 0 PID: 608 Comm: insmod Not tainted 4.2.0-rc8-next-20150828-cm-x300+ #919
Hardware name: CM-X300 module
task: e1c69880 ti: e1cb0000 task.ti: e1cb0000
PC is at u_init+0x2c/0x40 [unalign]
LR is at u_init+0x14/0x40 [unalign]
pc : [<bf00202c>]    lr : [<bf002014>]    psr: a0000013
sp : e1cb1df8  ip : e1c23e00  fp : 1e3dc89c
r10: e1c23780  r9 : 00000001  r8 : 00000000
r7 : bf002000  r6 : e1ca65c0  r5 : c0b85b80  r4 : c0b85b80
r3 : e1c23640  r2 : 00000004  r1 : a0000013  r0 : 00000000
Flags: NzCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment none
Control: 0000397f  Table: c1cd4018  DAC: 00000051
Process insmod (pid: 608, stack limit = 0xe1cb0198)
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1219879

FromRussell King - ARM Linux <linux@arm.linux.org.uk>
Date2015-09-07 02:00 +0200
Message-ID<q5S0N-5r9-9@gated-at.bofh.it>
In reply to#1219866
On Sun, Sep 06, 2015 at 11:31:34PM +0200, Robert Jarzmik wrote:
> Russell King - ARM Linux <linux@arm.linux.org.uk> writes:
> 
> >> [1] Approach 1 : translation table sync
> >> =======================================
> ...
> > The important place is in arch/arm/include/asm/domain.h, which is where
> > we manipulate the DACR within probe_kernel_address().
> 
> Gah, silly me. But even with [1], I still get an error [2]. I have a
> confirmation that I have a "Page Permission" fault on the
> probe_kernel_address().

Hmm, that's not right.  If it's the DACR, then it should be a page domain
fault, not a page permission fault.

> [2] Oops
> ========
> # insmod /tmp/unalign.ko 
> RJK1: fsr=23 far=e1c23643 dacr=51
> RJK2: fsr=23 far=e1c23643 dacr=51
> RJK3: fsr=2f far=bf00202c dacr=51
> RJK: fault=4 instr=0x00000000 instrptr=bf00202c

Can you add a show_pte(current->mm, instrptr) to dump those page
table entries please?

-- 
FTTC broadband for 0.8mile line: currently at 9.6Mbps down 400kbps up
according to speedtest.net.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web