Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > linux.kernel > #1306179 > unrolled thread
| Started by | Ard Biesheuvel <ard.biesheuvel@linaro.org> |
|---|---|
| First post | 2016-01-11 14:20 +0100 |
| Last post | 2016-01-12 08:20 +0100 |
| Articles | 20 on this page of 39 — 3 participants |
Back to article view | Back to linux.kernel
[PATCH v3 00/21] arm64: implement support for KASLR Ard Biesheuvel <ard.biesheuvel@linaro.org> - 2016-01-11 14:20 +0100
[PATCH v3 14/21] arm64: [re]define SWAPPER_TABLE_[SHIFT|SIZE] for use in asm code Ard Biesheuvel <ard.biesheuvel@linaro.org> - 2016-01-11 14:30 +0100
Re: [PATCH v3 14/21] arm64: [re]define SWAPPER_TABLE_[SHIFT|SIZE] for use in asm code Ard Biesheuvel <ard.biesheuvel@linaro.org> - 2016-01-11 14:30 +0100
[PATCH v3 08/21] arm64: add support for module PLTs Ard Biesheuvel <ard.biesheuvel@linaro.org> - 2016-01-11 14:30 +0100
[PATCH v3 04/21] arm64: decouple early fixmap init from linear mapping Ard Biesheuvel <ard.biesheuvel@linaro.org> - 2016-01-11 14:30 +0100
Re: [PATCH v3 04/21] arm64: decouple early fixmap init from linear mapping Mark Rutland <mark.rutland@arm.com> - 2016-01-11 17:10 +0100
Re: [PATCH v3 04/21] arm64: decouple early fixmap init from linear mapping Ard Biesheuvel <ard.biesheuvel@linaro.org> - 2016-01-11 17:20 +0100
Re: [PATCH v3 04/21] arm64: decouple early fixmap init from linear mapping Mark Rutland <mark.rutland@arm.com> - 2016-01-11 17:30 +0100
Re: [PATCH v3 04/21] arm64: decouple early fixmap init from linear mapping Mark Rutland <mark.rutland@arm.com> - 2016-01-11 18:00 +0100
Re: [PATCH v3 04/21] arm64: decouple early fixmap init from linear mapping Ard Biesheuvel <ard.biesheuvel@linaro.org> - 2016-01-11 18:10 +0100
Re: [PATCH v3 04/21] arm64: decouple early fixmap init from linear mapping Ard Biesheuvel <ard.biesheuvel@linaro.org> - 2016-01-11 18:20 +0100
Re: [PATCH v3 04/21] arm64: decouple early fixmap init from linear mapping Mark Rutland <mark.rutland@arm.com> - 2016-01-11 18:30 +0100
[PATCH v3 14/21] arm64: redefine SWAPPER_TABLE_SHIFT for use in asm code Ard Biesheuvel <ard.biesheuvel@linaro.org> - 2016-01-11 14:30 +0100
[PATCH v3 01/21] of/fdt: make memblock minimum physical address arch configurable Ard Biesheuvel <ard.biesheuvel@linaro.org> - 2016-01-11 14:30 +0100
[PATCH v3 16/21] scripts/sortextable: add support for ET_DYN binaries Ard Biesheuvel <ard.biesheuvel@linaro.org> - 2016-01-11 14:30 +0100
[PATCH v3 17/21] arm64: add support for a relocatable kernel and KASLR Ard Biesheuvel <ard.biesheuvel@linaro.org> - 2016-01-11 14:30 +0100
[PATCH v3 11/21] arm64: avoid R_AARCH64_ABS64 relocations for Image header fields Ard Biesheuvel <ard.biesheuvel@linaro.org> - 2016-01-11 14:30 +0100
Re: [PATCH v3 11/21] arm64: avoid R_AARCH64_ABS64 relocations for Image header fields Mark Rutland <mark.rutland@arm.com> - 2016-01-13 19:20 +0100
Re: [PATCH v3 11/21] arm64: avoid R_AARCH64_ABS64 relocations for Image header fields Ard Biesheuvel <ard.biesheuvel@linaro.org> - 2016-01-13 19:50 +0100
Re: [PATCH v3 11/21] arm64: avoid R_AARCH64_ABS64 relocations for Image header fields Ard Biesheuvel <ard.biesheuvel@linaro.org> - 2016-01-14 10:00 +0100
Re: [PATCH v3 11/21] arm64: avoid R_AARCH64_ABS64 relocations for Image header fields Ard Biesheuvel <ard.biesheuvel@linaro.org> - 2016-01-14 10:10 +0100
Re: [PATCH v3 11/21] arm64: avoid R_AARCH64_ABS64 relocations for Image header fields Mark Rutland <mark.rutland@arm.com> - 2016-01-14 11:50 +0100
Re: [PATCH v3 11/21] arm64: avoid R_AARCH64_ABS64 relocations for Image header fields Ard Biesheuvel <ard.biesheuvel@linaro.org> - 2016-01-14 12:30 +0100
[PATCH v3 06/21] arm64: pgtable: implement static [pte|pmd|pud]_offset variants Ard Biesheuvel <ard.biesheuvel@linaro.org> - 2016-01-11 14:30 +0100
Re: [PATCH v3 06/21] arm64: pgtable: implement static [pte|pmd|pud]_offset variants Mark Rutland <mark.rutland@arm.com> - 2016-01-11 17:30 +0100
Re: [PATCH v3 06/21] arm64: pgtable: implement static [pte|pmd|pud]_offset variants Ard Biesheuvel <ard.biesheuvel@linaro.org> - 2016-01-11 18:30 +0100
Re: [PATCH v3 06/21] arm64: pgtable: implement static [pte|pmd|pud]_offset variants Mark Rutland <mark.rutland@arm.com> - 2016-01-11 18:40 +0100
[PATCH v3 13/21] arm64: allow kernel Image to be loaded anywhere in physical memory Ard Biesheuvel <ard.biesheuvel@linaro.org> - 2016-01-11 14:30 +0100
[PATCH v3 12/21] arm64: avoid dynamic relocations in early boot code Ard Biesheuvel <ard.biesheuvel@linaro.org> - 2016-01-11 14:30 +0100
Re: [PATCH v3 12/21] arm64: avoid dynamic relocations in early boot code Mark Rutland <mark.rutland@arm.com> - 2016-01-14 18:20 +0100
[PATCH v3 15/21] arm64: split elf relocs into a separate header. Ard Biesheuvel <ard.biesheuvel@linaro.org> - 2016-01-11 14:30 +0100
[PATCH v3 09/21] extable: add support for relative extables to search and sort routines Ard Biesheuvel <ard.biesheuvel@linaro.org> - 2016-01-11 14:30 +0100
[PATCH v3 10/21] arm64: switch to relative exception tables Ard Biesheuvel <ard.biesheuvel@linaro.org> - 2016-01-11 14:30 +0100
[PATCH v3 19/21] efi: stub: add implementation of efi_random_alloc() Ard Biesheuvel <ard.biesheuvel@linaro.org> - 2016-01-11 14:30 +0100
[PATCH v3 18/21] efi: stub: implement efi_get_random_bytes() based on EFI_RNG_PROTOCOL Ard Biesheuvel <ard.biesheuvel@linaro.org> - 2016-01-11 14:30 +0100
[PATCH v3 20/21] efi: stub: use high allocation for converted command line Ard Biesheuvel <ard.biesheuvel@linaro.org> - 2016-01-11 14:30 +0100
[PATCH v3 21/21] arm64: efi: invoke EFI_RNG_PROTOCOL to supply KASLR randomness Ard Biesheuvel <ard.biesheuvel@linaro.org> - 2016-01-11 14:30 +0100
Re: [PATCH v3 00/21] arm64: implement support for KASLR Kees Cook <keescook@chromium.org> - 2016-01-11 23:10 +0100
Re: [PATCH v3 00/21] arm64: implement support for KASLR Ard Biesheuvel <ard.biesheuvel@linaro.org> - 2016-01-12 08:20 +0100
Page 1 of 2 [1] 2 Next page →
| From | Ard Biesheuvel <ard.biesheuvel@linaro.org> |
|---|---|
| Date | 2016-01-11 14:20 +0100 |
| Subject | [PATCH v3 00/21] arm64: implement support for KASLR |
| Message-ID | <qPKy5-2SK-3@gated-at.bofh.it> |
This series implements KASLR for arm64, by building the kernel as a PIE
executable that can relocate itself at runtime, and moving it to a random
offset in the vmalloc area. v2 and up also implement physical randomization,
i.e., it allows the kernel to deal with being loaded at any physical offset
(modulo the required alignment), and invokes the EFI_RNG_PROTOCOL from the
UEFI stub to obtain random bits and perform the actual randomization of the
physical load address.
Changes since v2:
- Incorporated feedback from Marc Zyngier into the KVM patch (#5)
- Dropped the pgdir section and the patch that memblock_reserve()'s the kernel
sections at a smaller granularity. This is no longer necessary with the pgdir
section gone. This also fixes an issue spotted by James Morse where the fixmap
page tables are not zeroed correctly; these have been moved back to the .bss
section.
- Got rid of all ifdef'ery regarding the number of translation levels in the
changed .c files, by introducing new definitions in pgtable.h (#3, #6)
- Fixed KAsan support, which was broken by all earlier versions.
- Moved module region along with the virtually randomized kernel, so that module
addresses become unpredictable as well, and we only have to rely on veneers in
the PLTs when the module region is exhausted (which is somewhat more likely
since the module region is now shared with other uses of the vmalloc area)
- Added support for the 'nokaslr' command line option. This affects the
randomization performed by the stub, and results in a warning if passed while
the bootloader also presented a random seed for virtual KASLR in register x1.
- The .text/.rodata sections of the kernel are no longer aliased in the linear
region with a writable mapping.
- Added a separate image header flag for kernel images that may be loaded at any
2 MB aligned offset (+ TEXT_OFFSET)
- The KASLR displacement is now corrected if it results in the kernel image
intersecting a PUD/PMD boundary (4k and 16k/64k granule kernels, respectively)
- Split out UEFI stub random routines into separate patches.
- Implemented a weight based EFI random allocation routine so that each suitable
offset in available memory is equally likely to be selected (as suggested by
Kees Cook)
- Reused CONFIG_RELOCATABLE and CONFIG_RANDOMIZE_BASE instead of introducing
new Kconfig symbols to describe the same functionality.
- Reimplemented mem= logic so memory is clipped from the top first.
Changes since v1/RFC:
- This series now implements fully independent virtual and physical address
randomization at load time. I have recycled some patches from this series:
http://thread.gmane.org/gmane.linux.ports.arm.kernel/455151, and updated the
final UEFI stub patch to randomize the physical address as well.
- Added a patch to deal with the way KVM on arm64 makes assumptions about the
relation between kernel symbols and the linear mapping (on which the HYP
mapping is based), as these assumptions cease to be valid once we move the
kernel Image out of the linear mapping.
- Updated the module PLT patch so it works on BE kernels as well.
- Moved the constant Image header values to head.S, and updated the linker
script to provide the kernel size using R_AARCH64_ABS32 relocation rather
than a R_AARCH64_ABS64 relocation, since those are always resolved at build
time. This allows me to get rid of the post-build perl script to swab header
values on BE kernels.
- Minor style tweaks.
Notes:
- These patches apply on top of Mark Rutland's pagetable rework series:
http://thread.gmane.org/gmane.linux.ports.arm.kernel/462438
- The arm64 Image is uncompressed by default, and the Elf64_Rela format uses
24 bytes per relocation entry. This results in considerable bloat (i.e., a
couple of MBs worth of relocation data in an .init section). However, no
build time postprocessing is required, we rely fully on the toolchain to
produce the image
- We have to rely on the bootloader to supply some randomness in register x1
upon kernel entry. Since we have no decompressor, it is simply not feasible
to collect randomness in the head.S code path before mapping the kernel and
enabling the MMU.
- The EFI_RNG_PROTOCOL that is invoked in patch #13 to supply randomness on
UEFI systems is not universally available. A QEMU/KVM firmware image that
implements a pseudo-random version is available here:
http://people.linaro.org/~ard.biesheuvel/QEMU_EFI.fd.aarch64-rng.bz2
(requires access to PMCCNTR_EL0 and support for AES instructions)
See below for instructions how to run the pseudo-random version on real
hardware.
- Only mildly tested. Help appreciated.
Code can be found here:
git://git.linaro.org/people/ard.biesheuvel/linux-arm.git arm64-kaslr-v3
https://git.linaro.org/people/ard.biesheuvel/linux-arm.git/shortlog/refs/heads/arm64-kaslr-v3
Patch #1 updates the OF code to allow the minimum memblock physical address to
be overridden by the arch.
Patch #2 introduces KIMAGE_VADDR as the base of the kernel virtual region.
Patch #3 introduces dummy pud_index() and pmd_index() macros that are intended
to be optimized away if the configured number of translation levels does not
actually use them.
Patch #4 rewrites early_fixmap_init() so it does not rely on the linear mapping
(i.e., the use of phys_to_virt() is avoided)
Patch #5 updates KVM on arm64 so it can deal with kernel symbols whose addresses
are not covered by the linear mapping.
Patch #6 introduces pte_offset_kimg(), pmd_offset_kimg() and pud_offset_kimg()
that allow statically allocated page tables (i.e., by fixmap and kasan) to be
traversed before the linear mapping is installed.
Patch #7 moves the kernel virtual mapping to the vmalloc area, along with the
module region which is kept right below it, as before.
Patch #8 adds support for PLTs in modules so that relative branches can be
resolved via a PLT if the target is out of range. This is required for KASLR,
since modules may be loaded far away from the core kernel.
Patch #9 and #10 move arm64 to the a new generic relative version of the extable
implementation so that it no longer contains absolute addresses that require
fixing up at relocation time, but uses relative offsets instead.
Patch #11 reverts some changes to the Image header population code so we no
longer depend on the linker to populate the header fields. This is necessary
since the R_AARCH64_ABS64 relocations that are emitted for these fields are not
resolved at build time for PIE executables.
Patch #12 updates the code in head.S that needs to execute before relocation to
avoid the use of values that are subject to dynamic relocation. These values
will not be populated in PIE executables.
Patch #13 allows the kernel Image to be loaded anywhere in physical memory, by
decoupling PHYS_OFFSET from the base of the kernel image.
Patch #14 redefines SWAPPER_TABLE_SHIFT in a way that allows it to be used from
assembler code regardless of the number of configured translation levels.
Patch #15 (from Mark Rutland) moves the ELF relocation type #defines to a
separate file so we can use it from head.S later
Patch #16 updates scripts/sortextable.c so it accepts ET_DYN (relocatable)
executables as well as ET_EXEC (static) executables.
Patch #17 implements the core KASLR, by taking randomness supplied in register
x1 and using it to move the kernel inside the vmalloc area.
Patch #18 implements efi_get_random_bytes() based on the EFI_RNG_PROTOCOL
Patch #19 implements efi_random_alloc()
Patch #20 moves the allocation for the converted command line (UTF-16 to ASCII)
away from the base of memory. This is necessary since for parsing
Patch #21 implements the actual KASLR, by randomizing the kernel physical
address, and passing entropy in x1 so that the kernel proper can relocate itself
virtually.
Ard Biesheuvel (20):
of/fdt: make memblock minimum physical address arch configurable
arm64: introduce KIMAGE_VADDR as the virtual base of the kernel region
arm64: pgtable: add dummy pud_index() and pmd_index() definitions
arm64: decouple early fixmap init from linear mapping
arm64: kvm: deal with kernel symbols outside of linear mapping
arm64: pgtable: implement static [pte|pmd|pud]_offset variants
arm64: move kernel image to base of vmalloc area
arm64: add support for module PLTs
extable: add support for relative extables to search and sort routines
arm64: switch to relative exception tables
arm64: avoid R_AARCH64_ABS64 relocations for Image header fields
arm64: avoid dynamic relocations in early boot code
arm64: allow kernel Image to be loaded anywhere in physical memory
arm64: redefine SWAPPER_TABLE_SHIFT for use in asm code
scripts/sortextable: add support for ET_DYN binaries
arm64: add support for a relocatable kernel and KASLR
efi: stub: implement efi_get_random_bytes() based on EFI_RNG_PROTOCOL
efi: stub: add implementation of efi_random_alloc()
efi: stub: use high allocation for converted command line
arm64: efi: invoke EFI_RNG_PROTOCOL to supply KASLR randomness
Mark Rutland (1):
arm64: split elf relocs into a separate header.
Documentation/arm64/booting.txt | 34 ++++-
arch/arm/include/asm/kvm_asm.h | 2 +
arch/arm/include/asm/kvm_mmu.h | 2 +
arch/arm/kvm/arm.c | 5 +-
arch/arm/kvm/mmu.c | 8 +-
arch/arm64/Kconfig | 40 +++++
arch/arm64/Makefile | 10 +-
arch/arm64/include/asm/assembler.h | 30 +++-
arch/arm64/include/asm/boot.h | 6 +
arch/arm64/include/asm/elf.h | 54 +------
arch/arm64/include/asm/elf_relocs.h | 75 ++++++++++
arch/arm64/include/asm/futex.h | 12 +-
arch/arm64/include/asm/kasan.h | 20 +--
arch/arm64/include/asm/kernel-pgtable.h | 20 ++-
arch/arm64/include/asm/kvm_asm.h | 19 ++-
arch/arm64/include/asm/kvm_host.h | 8 +-
arch/arm64/include/asm/kvm_mmu.h | 2 +
arch/arm64/include/asm/memory.h | 38 +++--
arch/arm64/include/asm/module.h | 11 ++
arch/arm64/include/asm/pgtable.h | 22 ++-
arch/arm64/include/asm/uaccess.h | 30 ++--
arch/arm64/include/asm/virt.h | 4 -
arch/arm64/include/asm/word-at-a-time.h | 7 +-
arch/arm64/kernel/Makefile | 1 +
arch/arm64/kernel/armv8_deprecated.c | 7 +-
arch/arm64/kernel/efi-entry.S | 9 +-
arch/arm64/kernel/head.S | 155 +++++++++++++++++---
arch/arm64/kernel/image.h | 37 ++---
arch/arm64/kernel/module-plts.c | 137 +++++++++++++++++
arch/arm64/kernel/module.c | 15 +-
arch/arm64/kernel/module.lds | 4 +
arch/arm64/kernel/setup.c | 44 +++++-
arch/arm64/kernel/vmlinux.lds.S | 13 +-
arch/arm64/kvm/debug.c | 1 +
arch/arm64/kvm/hyp.S | 6 +-
arch/arm64/mm/dump.c | 12 +-
arch/arm64/mm/extable.c | 2 +-
arch/arm64/mm/init.c | 91 ++++++++++--
arch/arm64/mm/kasan_init.c | 21 ++-
arch/arm64/mm/mmu.c | 95 +++++++-----
arch/x86/include/asm/efi.h | 2 +
drivers/firmware/efi/libstub/Makefile | 2 +-
drivers/firmware/efi/libstub/arm-stub.c | 17 ++-
drivers/firmware/efi/libstub/arm64-stub.c | 67 +++++++--
drivers/firmware/efi/libstub/efi-stub-helper.c | 24 ++-
drivers/firmware/efi/libstub/efistub.h | 9 ++
drivers/firmware/efi/libstub/random.c | 120 +++++++++++++++
drivers/of/fdt.c | 5 +-
include/linux/efi.h | 5 +-
lib/extable.c | 50 +++++--
scripts/sortextable.c | 10 +-
51 files changed, 1111 insertions(+), 309 deletions(-)
create mode 100644 arch/arm64/include/asm/elf_relocs.h
create mode 100644 arch/arm64/kernel/module-plts.c
create mode 100644 arch/arm64/kernel/module.lds
create mode 100644 drivers/firmware/efi/libstub/random.c
EFI_RNG_PROTOCOL on real hardware
=================================
To test whether your UEFI implements the EFI_RNG_PROTOCOL, download the
following executable and run it from the UEFI Shell:
http://people.linaro.org/~ard.biesheuvel/RngTest.efi
FS0:\> rngtest
UEFI RNG Protocol Testing :
----------------------------
-- Locate UEFI RNG Protocol : [Fail - Status = Not Found]
If your UEFI does not implement the EFI_RNG_PROTOCOL, you can download and
install the pseudo-random version that uses the generic timer and PMCCNTR_EL0
values and permutes them using a couple of rounds of AES.
http://people.linaro.org/~ard.biesheuvel/RngDxe.efi
NOTE: not for production!! This is a quick and dirty hack to test the KASLR
code, and is not suitable for anything else.
FS0:\> rngdxe
FS0:\> rngtest
UEFI RNG Protocol Testing :
----------------------------
-- Locate UEFI RNG Protocol : [Pass]
-- Call RNG->GetInfo() interface :
>> Supported RNG Algorithm (Count = 2) :
0) 44F0DE6E-4D8C-4045-A8C7-4DD168856B9E
1) E43176D7-B6E8-4827-B784-7FFDC4B68561
-- Call RNG->GetRNG() interface :
>> RNG with default algorithm : [Pass]
>> RNG with SP800-90-HMAC-256 : [Fail - Status = Unsupported]
>> RNG with SP800-90-Hash-256 : [Fail - Status = Unsupported]
>> RNG with SP800-90-CTR-256 : [Pass]
>> RNG with X9.31-3DES : [Fail - Status = Unsupported]
>> RNG with X9.31-AES : [Fail - Status = Unsupported]
>> RNG with RAW Entropy : [Pass]
-- Random Number Generation Test with default RNG Algorithm (20 Rounds):
01) - 27
02) - 61E8
03) - 496FD8
04) - DDD793BF
05) - B6C37C8E23
06) - 4D183C604A96
07) - 9363311DB61298
08) - 5715A7294F4E436E
09) - F0D4D7BAA0DD52318E
10) - C88C6EBCF4C0474D87C3
11) - B5594602B482A643932172
12) - CA7573F704B2089B726B9CF1
13) - A93E9451CB533DCFBA87B97C33
14) - 45AA7B83DB6044F7BBAB031F0D24
15) - 3DD7A4D61F34ADCB400B5976730DCF
16) - 4DD168D21FAB8F59708330D6A9BEB021
17) - 4BBB225E61C465F174254159467E65939F
18) - 030A156C9616337A20070941E702827DA8E1
19) - AB0FC11C9A4E225011382A9D164D9D55CA2B64
20) - 72B9B4735DC445E5DA6AF88DE965B7E87CB9A23C
[toc] | [next] | [standalone]
| From | Ard Biesheuvel <ard.biesheuvel@linaro.org> |
|---|---|
| Date | 2016-01-11 14:30 +0100 |
| Subject | [PATCH v3 14/21] arm64: [re]define SWAPPER_TABLE_[SHIFT|SIZE] for use in asm code |
| Message-ID | <qPKHL-2Wj-1@gated-at.bofh.it> |
| In reply to | #1306179 |
The current definition of SWAPPER_TABLE_SHIFT can only be used in asm code if the configured number of translation levels defines PUD_SHIFT and/or PMD_SHIFT natively (4KB and 16KB/64KB granule, respectively). Otherwise, it depends on the nopmd/nopud fixup headers, which can only be included in C code. So redefine SWAPPER_TABLE_SHIFT in a way that is independent of the number of configured translation levels. Define SWAPPER_TABLE_SIZE as well, we will need it later. Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> --- arch/arm64/include/asm/kernel-pgtable.h | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/arch/arm64/include/asm/kernel-pgtable.h b/arch/arm64/include/asm/kernel-pgtable.h index daa8a7b9917a..eaac46097359 100644 --- a/arch/arm64/include/asm/kernel-pgtable.h +++ b/arch/arm64/include/asm/kernel-pgtable.h @@ -57,13 +57,14 @@ #if ARM64_SWAPPER_USES_SECTION_MAPS #define SWAPPER_BLOCK_SHIFT SECTION_SHIFT #define SWAPPER_BLOCK_SIZE SECTION_SIZE -#define SWAPPER_TABLE_SHIFT PUD_SHIFT #else #define SWAPPER_BLOCK_SHIFT PAGE_SHIFT #define SWAPPER_BLOCK_SIZE PAGE_SIZE -#define SWAPPER_TABLE_SHIFT PMD_SHIFT #endif +#define SWAPPER_TABLE_SHIFT (SWAPPER_BLOCK_SHIFT + PAGE_SHIFT - 3) +#define SWAPPER_TABLE_SIZE (1 << SWAPPER_TABLE_SHIFT) + /* The size of the initial kernel direct mapping */ #define SWAPPER_INIT_MAP_SIZE (_AC(1, UL) << SWAPPER_TABLE_SHIFT) -- 2.5.0
[toc] | [prev] | [next] | [standalone]
| From | Ard Biesheuvel <ard.biesheuvel@linaro.org> |
|---|---|
| Date | 2016-01-11 14:30 +0100 |
| Subject | Re: [PATCH v3 14/21] arm64: [re]define SWAPPER_TABLE_[SHIFT|SIZE] for use in asm code |
| Message-ID | <qPKHL-2Wj-5@gated-at.bofh.it> |
| In reply to | #1306180 |
Please disregard this patch, I accidentally sent out two versions of 14/21, and this is the wrong one. On 11 January 2016 at 14:19, Ard Biesheuvel <ard.biesheuvel@linaro.org> wrote: > The current definition of SWAPPER_TABLE_SHIFT can only be used in > asm code if the configured number of translation levels defines > PUD_SHIFT and/or PMD_SHIFT natively (4KB and 16KB/64KB granule, > respectively). Otherwise, it depends on the nopmd/nopud fixup > headers, which can only be included in C code. > > So redefine SWAPPER_TABLE_SHIFT in a way that is independent of the > number of configured translation levels. Define SWAPPER_TABLE_SIZE > as well, we will need it later. > > Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> > --- > arch/arm64/include/asm/kernel-pgtable.h | 5 +++-- > 1 file changed, 3 insertions(+), 2 deletions(-) > > diff --git a/arch/arm64/include/asm/kernel-pgtable.h b/arch/arm64/include/asm/kernel-pgtable.h > index daa8a7b9917a..eaac46097359 100644 > --- a/arch/arm64/include/asm/kernel-pgtable.h > +++ b/arch/arm64/include/asm/kernel-pgtable.h > @@ -57,13 +57,14 @@ > #if ARM64_SWAPPER_USES_SECTION_MAPS > #define SWAPPER_BLOCK_SHIFT SECTION_SHIFT > #define SWAPPER_BLOCK_SIZE SECTION_SIZE > -#define SWAPPER_TABLE_SHIFT PUD_SHIFT > #else > #define SWAPPER_BLOCK_SHIFT PAGE_SHIFT > #define SWAPPER_BLOCK_SIZE PAGE_SIZE > -#define SWAPPER_TABLE_SHIFT PMD_SHIFT > #endif > > +#define SWAPPER_TABLE_SHIFT (SWAPPER_BLOCK_SHIFT + PAGE_SHIFT - 3) > +#define SWAPPER_TABLE_SIZE (1 << SWAPPER_TABLE_SHIFT) > + > /* The size of the initial kernel direct mapping */ > #define SWAPPER_INIT_MAP_SIZE (_AC(1, UL) << SWAPPER_TABLE_SHIFT) > > -- > 2.5.0 >
[toc] | [prev] | [next] | [standalone]
| From | Ard Biesheuvel <ard.biesheuvel@linaro.org> |
|---|---|
| Date | 2016-01-11 14:30 +0100 |
| Subject | [PATCH v3 08/21] arm64: add support for module PLTs |
| Message-ID | <qPKHL-2Wj-3@gated-at.bofh.it> |
| In reply to | #1306179 |
This adds support for emitting PLTs at module load time for relative
branches that are out of range. This is a prerequisite for KASLR, which
may place the kernel and the modules anywhere in the vmalloc area,
making it more likely that branch target offsets exceed the maximum
range of +/- 128 MB.
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
arch/arm64/Kconfig | 9 ++
arch/arm64/Makefile | 6 +-
arch/arm64/include/asm/module.h | 11 ++
arch/arm64/kernel/Makefile | 1 +
arch/arm64/kernel/module-plts.c | 137 ++++++++++++++++++++
arch/arm64/kernel/module.c | 12 ++
arch/arm64/kernel/module.lds | 4 +
7 files changed, 179 insertions(+), 1 deletion(-)
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index ffa3c549a4ba..778df20bf623 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -363,6 +363,7 @@ config ARM64_ERRATUM_843419
bool "Cortex-A53: 843419: A load or store might access an incorrect address"
depends on MODULES
default y
+ select ARM64_MODULE_CMODEL_LARGE
help
This option builds kernel modules using the large memory model in
order to avoid the use of the ADRP instruction, which can cause
@@ -702,6 +703,14 @@ config ARM64_LSE_ATOMICS
endmenu
+config ARM64_MODULE_CMODEL_LARGE
+ bool
+
+config ARM64_MODULE_PLTS
+ bool
+ select ARM64_MODULE_CMODEL_LARGE
+ select HAVE_MOD_ARCH_SPECIFIC
+
endmenu
menu "Boot options"
diff --git a/arch/arm64/Makefile b/arch/arm64/Makefile
index cd822d8454c0..db462980c6be 100644
--- a/arch/arm64/Makefile
+++ b/arch/arm64/Makefile
@@ -41,10 +41,14 @@ endif
CHECKFLAGS += -D__aarch64__
-ifeq ($(CONFIG_ARM64_ERRATUM_843419), y)
+ifeq ($(CONFIG_ARM64_MODULE_CMODEL_LARGE), y)
KBUILD_CFLAGS_MODULE += -mcmodel=large
endif
+ifeq ($(CONFIG_ARM64_MODULE_PLTS),y)
+KBUILD_LDFLAGS_MODULE += -T $(srctree)/arch/arm64/kernel/module.lds
+endif
+
# Default value
head-y := arch/arm64/kernel/head.o
diff --git a/arch/arm64/include/asm/module.h b/arch/arm64/include/asm/module.h
index e80e232b730e..7b8cd3dc9d8e 100644
--- a/arch/arm64/include/asm/module.h
+++ b/arch/arm64/include/asm/module.h
@@ -20,4 +20,15 @@
#define MODULE_ARCH_VERMAGIC "aarch64"
+#ifdef CONFIG_ARM64_MODULE_PLTS
+struct mod_arch_specific {
+ struct elf64_shdr *core_plt;
+ struct elf64_shdr *init_plt;
+ int core_plt_count;
+ int init_plt_count;
+};
+#endif
+
+u64 get_module_plt(struct module *mod, void *loc, u64 val);
+
#endif /* __ASM_MODULE_H */
diff --git a/arch/arm64/kernel/Makefile b/arch/arm64/kernel/Makefile
index 474691f8b13a..f42b0fff607f 100644
--- a/arch/arm64/kernel/Makefile
+++ b/arch/arm64/kernel/Makefile
@@ -30,6 +30,7 @@ arm64-obj-$(CONFIG_COMPAT) += sys32.o kuser32.o signal32.o \
../../arm/kernel/opcodes.o
arm64-obj-$(CONFIG_FUNCTION_TRACER) += ftrace.o entry-ftrace.o
arm64-obj-$(CONFIG_MODULES) += arm64ksyms.o module.o
+arm64-obj-$(CONFIG_ARM64_MODULE_PLTS) += module-plts.o
arm64-obj-$(CONFIG_PERF_EVENTS) += perf_regs.o perf_callchain.o
arm64-obj-$(CONFIG_HW_PERF_EVENTS) += perf_event.o
arm64-obj-$(CONFIG_HAVE_HW_BREAKPOINT) += hw_breakpoint.o
diff --git a/arch/arm64/kernel/module-plts.c b/arch/arm64/kernel/module-plts.c
new file mode 100644
index 000000000000..4a8ef9ea01ee
--- /dev/null
+++ b/arch/arm64/kernel/module-plts.c
@@ -0,0 +1,137 @@
+/*
+ * Copyright (C) 2014-2015 Linaro Ltd. <ard.biesheuvel@linaro.org>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#include <linux/elf.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+
+struct plt_entry {
+ __le32 mov0; /* movn x16, #0x.... */
+ __le32 mov1; /* movk x16, #0x...., lsl #16 */
+ __le32 mov2; /* movk x16, #0x...., lsl #32 */
+ __le32 br; /* br x16 */
+} __aligned(8);
+
+static bool in_init(const struct module *mod, void *addr)
+{
+ return (u64)addr - (u64)mod->module_init < mod->init_size;
+}
+
+u64 get_module_plt(struct module *mod, void *loc, u64 val)
+{
+ struct plt_entry entry = {
+ cpu_to_le32(0x92800010 | (((~val ) & 0xffff)) << 5),
+ cpu_to_le32(0xf2a00010 | ((( val >> 16) & 0xffff)) << 5),
+ cpu_to_le32(0xf2c00010 | ((( val >> 32) & 0xffff)) << 5),
+ cpu_to_le32(0xd61f0200)
+ }, *plt;
+ int i, *count;
+
+ if (in_init(mod, loc)) {
+ plt = (struct plt_entry *)mod->arch.init_plt->sh_addr;
+ count = &mod->arch.init_plt_count;
+ } else {
+ plt = (struct plt_entry *)mod->arch.core_plt->sh_addr;
+ count = &mod->arch.core_plt_count;
+ }
+
+ /* Look for an existing entry pointing to 'val' */
+ for (i = 0; i < *count; i++)
+ if (plt[i].mov0 == entry.mov0 &&
+ plt[i].mov1 == entry.mov1 &&
+ plt[i].mov2 == entry.mov2)
+ return (u64)&plt[i];
+
+ i = (*count)++;
+ plt[i] = entry;
+ return (u64)&plt[i];
+}
+
+static int duplicate_rel(Elf64_Addr base, const Elf64_Rela *rela, int num)
+{
+ int i;
+
+ for (i = 0; i < num; i++) {
+ if (rela[i].r_info == rela[num].r_info &&
+ rela[i].r_addend == rela[num].r_addend)
+ return 1;
+ }
+ return 0;
+}
+
+/* Count how many PLT entries we may need */
+static unsigned int count_plts(Elf64_Addr base, const Elf64_Rela *rela, int num)
+{
+ unsigned int ret = 0;
+ int i;
+
+ /*
+ * Sure, this is order(n^2), but it's usually short, and not
+ * time critical
+ */
+ for (i = 0; i < num; i++)
+ switch (ELF64_R_TYPE(rela[i].r_info)) {
+ case R_AARCH64_JUMP26:
+ case R_AARCH64_CALL26:
+ if (!duplicate_rel(base, rela, i))
+ ret++;
+ break;
+ }
+ return ret;
+}
+
+int module_frob_arch_sections(Elf_Ehdr *ehdr, Elf_Shdr *sechdrs,
+ char *secstrings, struct module *mod)
+{
+ unsigned long core_plts = 0, init_plts = 0;
+ Elf64_Shdr *s, *sechdrs_end = sechdrs + ehdr->e_shnum;
+
+ /*
+ * To store the PLTs, we expand the .text section for core module code
+ * and the .init.text section for initialization code.
+ */
+ for (s = sechdrs; s < sechdrs_end; ++s)
+ if (strcmp(".core.plt", secstrings + s->sh_name) == 0)
+ mod->arch.core_plt = s;
+ else if (strcmp(".init.plt", secstrings + s->sh_name) == 0)
+ mod->arch.init_plt = s;
+
+ if (!mod->arch.core_plt || !mod->arch.init_plt) {
+ pr_err("%s: sections missing\n", mod->name);
+ return -ENOEXEC;
+ }
+
+ for (s = sechdrs + 1; s < sechdrs_end; ++s) {
+ const Elf64_Rela *rels = (void *)ehdr + s->sh_offset;
+ int numrels = s->sh_size / sizeof(Elf64_Rela);
+ Elf64_Shdr *dstsec = sechdrs + s->sh_info;
+
+ if (s->sh_type != SHT_RELA)
+ continue;
+
+ if (strstr(secstrings + s->sh_name, ".init"))
+ init_plts += count_plts(dstsec->sh_addr, rels, numrels);
+ else
+ core_plts += count_plts(dstsec->sh_addr, rels, numrels);
+ }
+
+ mod->arch.core_plt->sh_type = SHT_NOBITS;
+ mod->arch.core_plt->sh_flags = SHF_EXECINSTR | SHF_ALLOC;
+ mod->arch.core_plt->sh_addralign = L1_CACHE_BYTES;
+ mod->arch.core_plt->sh_size = core_plts * sizeof(struct plt_entry);
+ mod->arch.core_plt_count = 0;
+
+ mod->arch.init_plt->sh_type = SHT_NOBITS;
+ mod->arch.init_plt->sh_flags = SHF_EXECINSTR | SHF_ALLOC;
+ mod->arch.init_plt->sh_addralign = L1_CACHE_BYTES;
+ mod->arch.init_plt->sh_size = init_plts * sizeof(struct plt_entry);
+ mod->arch.init_plt_count = 0;
+ pr_debug("%s: core.plt=%lld, init.plt=%lld\n", __func__,
+ mod->arch.core_plt->sh_size, mod->arch.init_plt->sh_size);
+ return 0;
+}
diff --git a/arch/arm64/kernel/module.c b/arch/arm64/kernel/module.c
index 93e970231ca9..3a298b0e21bb 100644
--- a/arch/arm64/kernel/module.c
+++ b/arch/arm64/kernel/module.c
@@ -38,6 +38,11 @@ void *module_alloc(unsigned long size)
GFP_KERNEL, PAGE_KERNEL_EXEC, 0,
NUMA_NO_NODE, __builtin_return_address(0));
+ if (!p && IS_ENABLED(CONFIG_ARM64_MODULE_PLTS))
+ p = __vmalloc_node_range(size, MODULE_ALIGN, VMALLOC_START,
+ VMALLOC_END, GFP_KERNEL, PAGE_KERNEL_EXEC, 0,
+ NUMA_NO_NODE, __builtin_return_address(0));
+
if (p && (kasan_module_alloc(p, size) < 0)) {
vfree(p);
return NULL;
@@ -361,6 +366,13 @@ int apply_relocate_add(Elf64_Shdr *sechdrs,
case R_AARCH64_CALL26:
ovf = reloc_insn_imm(RELOC_OP_PREL, loc, val, 2, 26,
AARCH64_INSN_IMM_26);
+
+ if (IS_ENABLED(CONFIG_ARM64_MODULE_PLTS) &&
+ ovf == -ERANGE) {
+ val = get_module_plt(me, loc, val);
+ ovf = reloc_insn_imm(RELOC_OP_PREL, loc, val, 2,
+ 26, AARCH64_INSN_IMM_26);
+ }
break;
default:
diff --git a/arch/arm64/kernel/module.lds b/arch/arm64/kernel/module.lds
new file mode 100644
index 000000000000..3682fa107918
--- /dev/null
+++ b/arch/arm64/kernel/module.lds
@@ -0,0 +1,4 @@
+SECTIONS {
+ .core.plt : { BYTE(0) }
+ .init.plt : { BYTE(0) }
+}
--
2.5.0
[toc] | [prev] | [next] | [standalone]
| From | Ard Biesheuvel <ard.biesheuvel@linaro.org> |
|---|---|
| Date | 2016-01-11 14:30 +0100 |
| Subject | [PATCH v3 04/21] arm64: decouple early fixmap init from linear mapping |
| Message-ID | <qPKHL-2Wj-9@gated-at.bofh.it> |
| In reply to | #1306179 |
Since the early fixmap page tables are populated using pages that are
part of the static footprint of the kernel, they are covered by the
initial kernel mapping, and we can refer to them without using __va/__pa
translations, which are tied to the linear mapping.
Since the fixmap page tables are disjoint from the kernel mapping up
to the top level pgd entry, we can refer to bm_pte[] directly, and there
is no need to walk the page tables and perform __pa()/__va() translations
at each step.
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
arch/arm64/mm/mmu.c | 32 ++++++--------------
1 file changed, 9 insertions(+), 23 deletions(-)
diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c
index 7711554a94f4..75b5f0dc3bdc 100644
--- a/arch/arm64/mm/mmu.c
+++ b/arch/arm64/mm/mmu.c
@@ -570,38 +570,24 @@ void vmemmap_free(unsigned long start, unsigned long end)
#endif /* CONFIG_SPARSEMEM_VMEMMAP */
static pte_t bm_pte[PTRS_PER_PTE] __page_aligned_bss;
-#if CONFIG_PGTABLE_LEVELS > 2
static pmd_t bm_pmd[PTRS_PER_PMD] __page_aligned_bss;
-#endif
-#if CONFIG_PGTABLE_LEVELS > 3
static pud_t bm_pud[PTRS_PER_PUD] __page_aligned_bss;
-#endif
static inline pud_t * fixmap_pud(unsigned long addr)
{
- pgd_t *pgd = pgd_offset_k(addr);
-
- BUG_ON(pgd_none(*pgd) || pgd_bad(*pgd));
-
- return pud_offset(pgd, addr);
+ return (CONFIG_PGTABLE_LEVELS > 3) ? &bm_pud[pud_index(addr)]
+ : (pud_t *)pgd_offset_k(addr);
}
-static inline pmd_t * fixmap_pmd(unsigned long addr)
+static inline pte_t * fixmap_pmd(unsigned long addr)
{
- pud_t *pud = fixmap_pud(addr);
-
- BUG_ON(pud_none(*pud) || pud_bad(*pud));
-
- return pmd_offset(pud, addr);
+ return (CONFIG_PGTABLE_LEVELS > 2) ? &bm_pmd[pmd_index(addr)]
+ : (pmd_t *)pgd_offset_k(addr);
}
static inline pte_t * fixmap_pte(unsigned long addr)
{
- pmd_t *pmd = fixmap_pmd(addr);
-
- BUG_ON(pmd_none(*pmd) || pmd_bad(*pmd));
-
- return pte_offset_kernel(pmd, addr);
+ return &bm_pte[pte_index(addr)];
}
void __init early_fixmap_init(void)
@@ -613,14 +599,14 @@ void __init early_fixmap_init(void)
pgd = pgd_offset_k(addr);
pgd_populate(&init_mm, pgd, bm_pud);
- pud = pud_offset(pgd, addr);
+ pud = fixmap_pud(addr);
pud_populate(&init_mm, pud, bm_pmd);
- pmd = pmd_offset(pud, addr);
+ pmd = fixmap_pmd(addr);
pmd_populate_kernel(&init_mm, pmd, bm_pte);
/*
* The boot-ioremap range spans multiple pmds, for which
- * we are not preparted:
+ * we are not prepared:
*/
BUILD_BUG_ON((__fix_to_virt(FIX_BTMAP_BEGIN) >> PMD_SHIFT)
!= (__fix_to_virt(FIX_BTMAP_END) >> PMD_SHIFT));
--
2.5.0
[toc] | [prev] | [next] | [standalone]
| From | Mark Rutland <mark.rutland@arm.com> |
|---|---|
| Date | 2016-01-11 17:10 +0100 |
| Subject | Re: [PATCH v3 04/21] arm64: decouple early fixmap init from linear mapping |
| Message-ID | <qPNcC-4IN-19@gated-at.bofh.it> |
| In reply to | #1306184 |
On Mon, Jan 11, 2016 at 02:18:57PM +0100, Ard Biesheuvel wrote:
> Since the early fixmap page tables are populated using pages that are
> part of the static footprint of the kernel, they are covered by the
> initial kernel mapping, and we can refer to them without using __va/__pa
> translations, which are tied to the linear mapping.
>
> Since the fixmap page tables are disjoint from the kernel mapping up
> to the top level pgd entry, we can refer to bm_pte[] directly, and there
> is no need to walk the page tables and perform __pa()/__va() translations
> at each step.
>
> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> ---
> arch/arm64/mm/mmu.c | 32 ++++++--------------
> 1 file changed, 9 insertions(+), 23 deletions(-)
>
> diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c
> index 7711554a94f4..75b5f0dc3bdc 100644
> --- a/arch/arm64/mm/mmu.c
> +++ b/arch/arm64/mm/mmu.c
> @@ -570,38 +570,24 @@ void vmemmap_free(unsigned long start, unsigned long end)
> #endif /* CONFIG_SPARSEMEM_VMEMMAP */
>
> static pte_t bm_pte[PTRS_PER_PTE] __page_aligned_bss;
> -#if CONFIG_PGTABLE_LEVELS > 2
> static pmd_t bm_pmd[PTRS_PER_PMD] __page_aligned_bss;
> -#endif
> -#if CONFIG_PGTABLE_LEVELS > 3
> static pud_t bm_pud[PTRS_PER_PUD] __page_aligned_bss;
> -#endif
>
> static inline pud_t * fixmap_pud(unsigned long addr)
> {
> - pgd_t *pgd = pgd_offset_k(addr);
> -
> - BUG_ON(pgd_none(*pgd) || pgd_bad(*pgd));
> -
> - return pud_offset(pgd, addr);
> + return (CONFIG_PGTABLE_LEVELS > 3) ? &bm_pud[pud_index(addr)]
> + : (pud_t *)pgd_offset_k(addr);
If we move patch 6 earlier, we could use pud_offset_kimg here, and avoid
the cast, at the cost of passing the pgd into fixmap_pud.
Similarly for fixmap_pmd.
> }
>
> -static inline pmd_t * fixmap_pmd(unsigned long addr)
> +static inline pte_t * fixmap_pmd(unsigned long addr)
> {
> - pud_t *pud = fixmap_pud(addr);
> -
> - BUG_ON(pud_none(*pud) || pud_bad(*pud));
> -
> - return pmd_offset(pud, addr);
> + return (CONFIG_PGTABLE_LEVELS > 2) ? &bm_pmd[pmd_index(addr)]
> + : (pmd_t *)pgd_offset_k(addr);
> }
I assume the return type change was unintentional?
With STRICT_MM_TYPECHECKS:
arch/arm64/mm/mmu.c: In function 'fixmap_pmd':
arch/arm64/mm/mmu.c:604:9: warning: return from incompatible pointer type [-Wincompatible-pointer-types]
return (CONFIG_PGTABLE_LEVELS > 2) ? &bm_pmd[pmd_index(addr)]
^
arch/arm64/mm/mmu.c: In function 'early_fixmap_init':
arch/arm64/mm/mmu.c:635:6: warning: assignment from incompatible pointer type [-Wincompatible-pointer-types]
pmd = fixmap_pmd(addr);
^
arch/arm64/mm/mmu.c:645:11: warning: comparison of distinct pointer types lacks a cast
if ((pmd != fixmap_pmd(fix_to_virt(FIX_BTMAP_BEGIN)))
^
arch/arm64/mm/mmu.c:646:14: warning: comparison of distinct pointer types lacks a cast
|| pmd != fixmap_pmd(fix_to_virt(FIX_BTMAP_END))) {
^
Side note: is there any reason we can't/shouldn't make
STRICT_MM_TYPECHECKS a common config option? Or simply have it on by
default for arm64?
Having built with and without typechecks I see that it doesn't bloat the
kernel Image size, though the binary isn't quite identical:
[mark@leverpostej:~/src/linux]% ls -al *.*checks
-rwxrwxr-x 1 mark mark 9288192 Jan 11 15:40 Image.checks
-rwxrwxr-x 1 mark mark 9288192 Jan 11 15:36 Image.nochecks
-rwxrwxr-x 1 mark mark 106782024 Jan 11 15:40 vmlinux.checks
-rwxrwxr-x 1 mark mark 106688928 Jan 11 15:35 vmlinux.nochecks
Things didn't quite line up between the two images, though I'm not sure
what the underlying difference was.
Thanks,
Mark.
[toc] | [prev] | [next] | [standalone]
| From | Ard Biesheuvel <ard.biesheuvel@linaro.org> |
|---|---|
| Date | 2016-01-11 17:20 +0100 |
| Subject | Re: [PATCH v3 04/21] arm64: decouple early fixmap init from linear mapping |
| Message-ID | <qPNmi-4Mf-19@gated-at.bofh.it> |
| In reply to | #1306449 |
On 11 January 2016 at 17:09, Mark Rutland <mark.rutland@arm.com> wrote:
> On Mon, Jan 11, 2016 at 02:18:57PM +0100, Ard Biesheuvel wrote:
>> Since the early fixmap page tables are populated using pages that are
>> part of the static footprint of the kernel, they are covered by the
>> initial kernel mapping, and we can refer to them without using __va/__pa
>> translations, which are tied to the linear mapping.
>>
>> Since the fixmap page tables are disjoint from the kernel mapping up
>> to the top level pgd entry, we can refer to bm_pte[] directly, and there
>> is no need to walk the page tables and perform __pa()/__va() translations
>> at each step.
>>
>> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
>> ---
>> arch/arm64/mm/mmu.c | 32 ++++++--------------
>> 1 file changed, 9 insertions(+), 23 deletions(-)
>>
>> diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c
>> index 7711554a94f4..75b5f0dc3bdc 100644
>> --- a/arch/arm64/mm/mmu.c
>> +++ b/arch/arm64/mm/mmu.c
>> @@ -570,38 +570,24 @@ void vmemmap_free(unsigned long start, unsigned long end)
>> #endif /* CONFIG_SPARSEMEM_VMEMMAP */
>>
>> static pte_t bm_pte[PTRS_PER_PTE] __page_aligned_bss;
>> -#if CONFIG_PGTABLE_LEVELS > 2
>> static pmd_t bm_pmd[PTRS_PER_PMD] __page_aligned_bss;
>> -#endif
>> -#if CONFIG_PGTABLE_LEVELS > 3
>> static pud_t bm_pud[PTRS_PER_PUD] __page_aligned_bss;
>> -#endif
>>
>> static inline pud_t * fixmap_pud(unsigned long addr)
>> {
>> - pgd_t *pgd = pgd_offset_k(addr);
>> -
>> - BUG_ON(pgd_none(*pgd) || pgd_bad(*pgd));
>> -
>> - return pud_offset(pgd, addr);
>> + return (CONFIG_PGTABLE_LEVELS > 3) ? &bm_pud[pud_index(addr)]
>> + : (pud_t *)pgd_offset_k(addr);
>
> If we move patch 6 earlier, we could use pud_offset_kimg here, and avoid
> the cast, at the cost of passing the pgd into fixmap_pud.
>
> Similarly for fixmap_pmd.
>
Is that necessarily an improvement? I know it hides the cast, but I
think having an explicit pgd_t* to pud_t* cast that so obviously
applies to CONFIG_PGTABLE_LEVELS < 4 only is fine as well.
>> }
>>
>> -static inline pmd_t * fixmap_pmd(unsigned long addr)
>> +static inline pte_t * fixmap_pmd(unsigned long addr)
>> {
>> - pud_t *pud = fixmap_pud(addr);
>> -
>> - BUG_ON(pud_none(*pud) || pud_bad(*pud));
>> -
>> - return pmd_offset(pud, addr);
>> + return (CONFIG_PGTABLE_LEVELS > 2) ? &bm_pmd[pmd_index(addr)]
>> + : (pmd_t *)pgd_offset_k(addr);
>> }
>
> I assume the return type change was unintentional?
>
Yes. Thanks for spotting that.
> With STRICT_MM_TYPECHECKS:
>
> arch/arm64/mm/mmu.c: In function 'fixmap_pmd':
> arch/arm64/mm/mmu.c:604:9: warning: return from incompatible pointer type [-Wincompatible-pointer-types]
> return (CONFIG_PGTABLE_LEVELS > 2) ? &bm_pmd[pmd_index(addr)]
> ^
> arch/arm64/mm/mmu.c: In function 'early_fixmap_init':
> arch/arm64/mm/mmu.c:635:6: warning: assignment from incompatible pointer type [-Wincompatible-pointer-types]
> pmd = fixmap_pmd(addr);
> ^
> arch/arm64/mm/mmu.c:645:11: warning: comparison of distinct pointer types lacks a cast
> if ((pmd != fixmap_pmd(fix_to_virt(FIX_BTMAP_BEGIN)))
> ^
> arch/arm64/mm/mmu.c:646:14: warning: comparison of distinct pointer types lacks a cast
> || pmd != fixmap_pmd(fix_to_virt(FIX_BTMAP_END))) {
> ^
>
> Side note: is there any reason we can't/shouldn't make
> STRICT_MM_TYPECHECKS a common config option? Or simply have it on by
> default for arm64?
>
I wouldn't mind at all.
> Having built with and without typechecks I see that it doesn't bloat the
> kernel Image size, though the binary isn't quite identical:
>
> [mark@leverpostej:~/src/linux]% ls -al *.*checks
> -rwxrwxr-x 1 mark mark 9288192 Jan 11 15:40 Image.checks
> -rwxrwxr-x 1 mark mark 9288192 Jan 11 15:36 Image.nochecks
> -rwxrwxr-x 1 mark mark 106782024 Jan 11 15:40 vmlinux.checks
> -rwxrwxr-x 1 mark mark 106688928 Jan 11 15:35 vmlinux.nochecks
>
> Things didn't quite line up between the two images, though I'm not sure
> what the underlying difference was.
>
> Thanks,
> Mark.
[toc] | [prev] | [next] | [standalone]
| From | Mark Rutland <mark.rutland@arm.com> |
|---|---|
| Date | 2016-01-11 17:30 +0100 |
| Subject | Re: [PATCH v3 04/21] arm64: decouple early fixmap init from linear mapping |
| Message-ID | <qPNvY-4Q9-25@gated-at.bofh.it> |
| In reply to | #1306462 |
On Mon, Jan 11, 2016 at 05:15:13PM +0100, Ard Biesheuvel wrote:
> On 11 January 2016 at 17:09, Mark Rutland <mark.rutland@arm.com> wrote:
> > On Mon, Jan 11, 2016 at 02:18:57PM +0100, Ard Biesheuvel wrote:
> >> Since the early fixmap page tables are populated using pages that are
> >> part of the static footprint of the kernel, they are covered by the
> >> initial kernel mapping, and we can refer to them without using __va/__pa
> >> translations, which are tied to the linear mapping.
> >>
> >> Since the fixmap page tables are disjoint from the kernel mapping up
> >> to the top level pgd entry, we can refer to bm_pte[] directly, and there
> >> is no need to walk the page tables and perform __pa()/__va() translations
> >> at each step.
> >>
> >> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> >> ---
> >> arch/arm64/mm/mmu.c | 32 ++++++--------------
> >> 1 file changed, 9 insertions(+), 23 deletions(-)
> >>
> >> diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c
> >> index 7711554a94f4..75b5f0dc3bdc 100644
> >> --- a/arch/arm64/mm/mmu.c
> >> +++ b/arch/arm64/mm/mmu.c
> >> @@ -570,38 +570,24 @@ void vmemmap_free(unsigned long start, unsigned long end)
> >> #endif /* CONFIG_SPARSEMEM_VMEMMAP */
> >>
> >> static pte_t bm_pte[PTRS_PER_PTE] __page_aligned_bss;
> >> -#if CONFIG_PGTABLE_LEVELS > 2
> >> static pmd_t bm_pmd[PTRS_PER_PMD] __page_aligned_bss;
> >> -#endif
> >> -#if CONFIG_PGTABLE_LEVELS > 3
> >> static pud_t bm_pud[PTRS_PER_PUD] __page_aligned_bss;
> >> -#endif
> >>
> >> static inline pud_t * fixmap_pud(unsigned long addr)
> >> {
> >> - pgd_t *pgd = pgd_offset_k(addr);
> >> -
> >> - BUG_ON(pgd_none(*pgd) || pgd_bad(*pgd));
> >> -
> >> - return pud_offset(pgd, addr);
> >> + return (CONFIG_PGTABLE_LEVELS > 3) ? &bm_pud[pud_index(addr)]
> >> + : (pud_t *)pgd_offset_k(addr);
> >
> > If we move patch 6 earlier, we could use pud_offset_kimg here, and avoid
> > the cast, at the cost of passing the pgd into fixmap_pud.
> >
> > Similarly for fixmap_pmd.
> >
>
> Is that necessarily an improvement? I know it hides the cast, but I
> think having an explicit pgd_t* to pud_t* cast that so obviously
> applies to CONFIG_PGTABLE_LEVELS < 4 only is fine as well.
True; it's not a big thing either way.
> >> }
> >>
> >> -static inline pmd_t * fixmap_pmd(unsigned long addr)
> >> +static inline pte_t * fixmap_pmd(unsigned long addr)
> >> {
> >> - pud_t *pud = fixmap_pud(addr);
> >> -
> >> - BUG_ON(pud_none(*pud) || pud_bad(*pud));
> >> -
> >> - return pmd_offset(pud, addr);
> >> + return (CONFIG_PGTABLE_LEVELS > 2) ? &bm_pmd[pmd_index(addr)]
> >> + : (pmd_t *)pgd_offset_k(addr);
> >> }
> >
> > I assume the return type change was unintentional?
> >
>
> Yes. Thanks for spotting that.
With that fixed:
Reviewed-by: Mark Rutland <mark.rutland@arm.com>
> > Side note: is there any reason we can't/shouldn't make
> > STRICT_MM_TYPECHECKS a common config option? Or simply have it on by
> > default for arm64?
> >
>
> I wouldn't mind at all.
I'll dig into that a bit futher then...
Thanks,
Mark.
[toc] | [prev] | [next] | [standalone]
| From | Mark Rutland <mark.rutland@arm.com> |
|---|---|
| Date | 2016-01-11 18:00 +0100 |
| Subject | Re: [PATCH v3 04/21] arm64: decouple early fixmap init from linear mapping |
| Message-ID | <qPNZ0-53I-21@gated-at.bofh.it> |
| In reply to | #1306469 |
On Mon, Jan 11, 2016 at 04:27:38PM +0000, Mark Rutland wrote:
> On Mon, Jan 11, 2016 at 05:15:13PM +0100, Ard Biesheuvel wrote:
> > On 11 January 2016 at 17:09, Mark Rutland <mark.rutland@arm.com> wrote:
> > > On Mon, Jan 11, 2016 at 02:18:57PM +0100, Ard Biesheuvel wrote:
> > >> Since the early fixmap page tables are populated using pages that are
> > >> part of the static footprint of the kernel, they are covered by the
> > >> initial kernel mapping, and we can refer to them without using __va/__pa
> > >> translations, which are tied to the linear mapping.
> > >>
> > >> Since the fixmap page tables are disjoint from the kernel mapping up
> > >> to the top level pgd entry, we can refer to bm_pte[] directly, and there
> > >> is no need to walk the page tables and perform __pa()/__va() translations
> > >> at each step.
> > >>
> > >> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> > >> ---
> > >> arch/arm64/mm/mmu.c | 32 ++++++--------------
> > >> 1 file changed, 9 insertions(+), 23 deletions(-)
> > >>
> > >> diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c
> > >> index 7711554a94f4..75b5f0dc3bdc 100644
> > >> --- a/arch/arm64/mm/mmu.c
> > >> +++ b/arch/arm64/mm/mmu.c
> > >> @@ -570,38 +570,24 @@ void vmemmap_free(unsigned long start, unsigned long end)
> > >> #endif /* CONFIG_SPARSEMEM_VMEMMAP */
> > >>
> > >> static pte_t bm_pte[PTRS_PER_PTE] __page_aligned_bss;
> > >> -#if CONFIG_PGTABLE_LEVELS > 2
> > >> static pmd_t bm_pmd[PTRS_PER_PMD] __page_aligned_bss;
> > >> -#endif
> > >> -#if CONFIG_PGTABLE_LEVELS > 3
> > >> static pud_t bm_pud[PTRS_PER_PUD] __page_aligned_bss;
> > >> -#endif
> > >>
> > >> static inline pud_t * fixmap_pud(unsigned long addr)
> > >> {
> > >> - pgd_t *pgd = pgd_offset_k(addr);
> > >> -
> > >> - BUG_ON(pgd_none(*pgd) || pgd_bad(*pgd));
> > >> -
> > >> - return pud_offset(pgd, addr);
> > >> + return (CONFIG_PGTABLE_LEVELS > 3) ? &bm_pud[pud_index(addr)]
> > >> + : (pud_t *)pgd_offset_k(addr);
> > >
> > > If we move patch 6 earlier, we could use pud_offset_kimg here, and avoid
> > > the cast, at the cost of passing the pgd into fixmap_pud.
> > >
> > > Similarly for fixmap_pmd.
> > >
> >
> > Is that necessarily an improvement? I know it hides the cast, but I
> > think having an explicit pgd_t* to pud_t* cast that so obviously
> > applies to CONFIG_PGTABLE_LEVELS < 4 only is fine as well.
>
> True; it's not a big thing either way.
Sorry, I'm gonig to change my mind on that again. I think using
p?d_offset_kimg is preferable. e.g.
static inline pud_t * fixmap_pud(unsigned long addr)
{
pgd_t *pgd = pgd_offset_k(addr);
BUG_ON(pgd_none(*pgd) || pgd_bad(*pgd));
return pud_offset_kimg(pgd, addr);
}
static inline pmd_t * fixmap_pmd(unsigned long addr)
{
pud_t *pud = fixmap_pud(addr);
BUG_ON(pud_none(*pud) || pud_bad(*pud));
return pmd_offset_kimg(pud, addr);
}
That avoids having to check CONFIG_PGTABLE_LEVELS check and perform a cast,
avoids duplicating details about bm_{pud,pmd}, and keeps the existing structure
so it's easier to reason about the change. I was wrong about having to pass the
pgd or pud in, so callers don't need upating.
From my PoV that is preferable.
Thanks,
Mark.
[toc] | [prev] | [next] | [standalone]
| From | Ard Biesheuvel <ard.biesheuvel@linaro.org> |
|---|---|
| Date | 2016-01-11 18:10 +0100 |
| Subject | Re: [PATCH v3 04/21] arm64: decouple early fixmap init from linear mapping |
| Message-ID | <qPO8H-5nr-35@gated-at.bofh.it> |
| In reply to | #1306514 |
On 11 January 2016 at 17:51, Mark Rutland <mark.rutland@arm.com> wrote:
> On Mon, Jan 11, 2016 at 04:27:38PM +0000, Mark Rutland wrote:
>> On Mon, Jan 11, 2016 at 05:15:13PM +0100, Ard Biesheuvel wrote:
>> > On 11 January 2016 at 17:09, Mark Rutland <mark.rutland@arm.com> wrote:
>> > > On Mon, Jan 11, 2016 at 02:18:57PM +0100, Ard Biesheuvel wrote:
>> > >> Since the early fixmap page tables are populated using pages that are
>> > >> part of the static footprint of the kernel, they are covered by the
>> > >> initial kernel mapping, and we can refer to them without using __va/__pa
>> > >> translations, which are tied to the linear mapping.
>> > >>
>> > >> Since the fixmap page tables are disjoint from the kernel mapping up
>> > >> to the top level pgd entry, we can refer to bm_pte[] directly, and there
>> > >> is no need to walk the page tables and perform __pa()/__va() translations
>> > >> at each step.
>> > >>
>> > >> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
>> > >> ---
>> > >> arch/arm64/mm/mmu.c | 32 ++++++--------------
>> > >> 1 file changed, 9 insertions(+), 23 deletions(-)
>> > >>
>> > >> diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c
>> > >> index 7711554a94f4..75b5f0dc3bdc 100644
>> > >> --- a/arch/arm64/mm/mmu.c
>> > >> +++ b/arch/arm64/mm/mmu.c
>> > >> @@ -570,38 +570,24 @@ void vmemmap_free(unsigned long start, unsigned long end)
>> > >> #endif /* CONFIG_SPARSEMEM_VMEMMAP */
>> > >>
>> > >> static pte_t bm_pte[PTRS_PER_PTE] __page_aligned_bss;
>> > >> -#if CONFIG_PGTABLE_LEVELS > 2
>> > >> static pmd_t bm_pmd[PTRS_PER_PMD] __page_aligned_bss;
>> > >> -#endif
>> > >> -#if CONFIG_PGTABLE_LEVELS > 3
>> > >> static pud_t bm_pud[PTRS_PER_PUD] __page_aligned_bss;
>> > >> -#endif
>> > >>
>> > >> static inline pud_t * fixmap_pud(unsigned long addr)
>> > >> {
>> > >> - pgd_t *pgd = pgd_offset_k(addr);
>> > >> -
>> > >> - BUG_ON(pgd_none(*pgd) || pgd_bad(*pgd));
>> > >> -
>> > >> - return pud_offset(pgd, addr);
>> > >> + return (CONFIG_PGTABLE_LEVELS > 3) ? &bm_pud[pud_index(addr)]
>> > >> + : (pud_t *)pgd_offset_k(addr);
>> > >
>> > > If we move patch 6 earlier, we could use pud_offset_kimg here, and avoid
>> > > the cast, at the cost of passing the pgd into fixmap_pud.
>> > >
>> > > Similarly for fixmap_pmd.
>> > >
>> >
>> > Is that necessarily an improvement? I know it hides the cast, but I
>> > think having an explicit pgd_t* to pud_t* cast that so obviously
>> > applies to CONFIG_PGTABLE_LEVELS < 4 only is fine as well.
>>
>> True; it's not a big thing either way.
>
> Sorry, I'm gonig to change my mind on that again. I think using
> p?d_offset_kimg is preferable. e.g.
>
> static inline pud_t * fixmap_pud(unsigned long addr)
> {
> pgd_t *pgd = pgd_offset_k(addr);
>
> BUG_ON(pgd_none(*pgd) || pgd_bad(*pgd));
>
> return pud_offset_kimg(pgd, addr);
> }
>
> static inline pmd_t * fixmap_pmd(unsigned long addr)
> {
> pud_t *pud = fixmap_pud(addr);
>
> BUG_ON(pud_none(*pud) || pud_bad(*pud));
>
> return pmd_offset_kimg(pud, addr);
> }
>
> That avoids having to check CONFIG_PGTABLE_LEVELS check and perform a cast,
> avoids duplicating details about bm_{pud,pmd}, and keeps the existing structure
> so it's easier to reason about the change. I was wrong about having to pass the
> pgd or pud in, so callers don't need upating.
>
> From my PoV that is preferable.
>
OK. I think it looks better, indeed.
[toc] | [prev] | [next] | [standalone]
| From | Ard Biesheuvel <ard.biesheuvel@linaro.org> |
|---|---|
| Date | 2016-01-11 18:20 +0100 |
| Subject | Re: [PATCH v3 04/21] arm64: decouple early fixmap init from linear mapping |
| Message-ID | <qPOil-5rH-9@gated-at.bofh.it> |
| In reply to | #1306521 |
On 11 January 2016 at 18:08, Ard Biesheuvel <ard.biesheuvel@linaro.org> wrote:
> On 11 January 2016 at 17:51, Mark Rutland <mark.rutland@arm.com> wrote:
>> On Mon, Jan 11, 2016 at 04:27:38PM +0000, Mark Rutland wrote:
>>> On Mon, Jan 11, 2016 at 05:15:13PM +0100, Ard Biesheuvel wrote:
>>> > On 11 January 2016 at 17:09, Mark Rutland <mark.rutland@arm.com> wrote:
>>> > > On Mon, Jan 11, 2016 at 02:18:57PM +0100, Ard Biesheuvel wrote:
>>> > >> Since the early fixmap page tables are populated using pages that are
>>> > >> part of the static footprint of the kernel, they are covered by the
>>> > >> initial kernel mapping, and we can refer to them without using __va/__pa
>>> > >> translations, which are tied to the linear mapping.
>>> > >>
>>> > >> Since the fixmap page tables are disjoint from the kernel mapping up
>>> > >> to the top level pgd entry, we can refer to bm_pte[] directly, and there
>>> > >> is no need to walk the page tables and perform __pa()/__va() translations
>>> > >> at each step.
>>> > >>
>>> > >> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
>>> > >> ---
>>> > >> arch/arm64/mm/mmu.c | 32 ++++++--------------
>>> > >> 1 file changed, 9 insertions(+), 23 deletions(-)
>>> > >>
>>> > >> diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c
>>> > >> index 7711554a94f4..75b5f0dc3bdc 100644
>>> > >> --- a/arch/arm64/mm/mmu.c
>>> > >> +++ b/arch/arm64/mm/mmu.c
>>> > >> @@ -570,38 +570,24 @@ void vmemmap_free(unsigned long start, unsigned long end)
>>> > >> #endif /* CONFIG_SPARSEMEM_VMEMMAP */
>>> > >>
>>> > >> static pte_t bm_pte[PTRS_PER_PTE] __page_aligned_bss;
>>> > >> -#if CONFIG_PGTABLE_LEVELS > 2
>>> > >> static pmd_t bm_pmd[PTRS_PER_PMD] __page_aligned_bss;
>>> > >> -#endif
>>> > >> -#if CONFIG_PGTABLE_LEVELS > 3
>>> > >> static pud_t bm_pud[PTRS_PER_PUD] __page_aligned_bss;
>>> > >> -#endif
>>> > >>
>>> > >> static inline pud_t * fixmap_pud(unsigned long addr)
>>> > >> {
>>> > >> - pgd_t *pgd = pgd_offset_k(addr);
>>> > >> -
>>> > >> - BUG_ON(pgd_none(*pgd) || pgd_bad(*pgd));
>>> > >> -
>>> > >> - return pud_offset(pgd, addr);
>>> > >> + return (CONFIG_PGTABLE_LEVELS > 3) ? &bm_pud[pud_index(addr)]
>>> > >> + : (pud_t *)pgd_offset_k(addr);
>>> > >
>>> > > If we move patch 6 earlier, we could use pud_offset_kimg here, and avoid
>>> > > the cast, at the cost of passing the pgd into fixmap_pud.
>>> > >
>>> > > Similarly for fixmap_pmd.
>>> > >
>>> >
>>> > Is that necessarily an improvement? I know it hides the cast, but I
>>> > think having an explicit pgd_t* to pud_t* cast that so obviously
>>> > applies to CONFIG_PGTABLE_LEVELS < 4 only is fine as well.
>>>
>>> True; it's not a big thing either way.
>>
>> Sorry, I'm gonig to change my mind on that again. I think using
>> p?d_offset_kimg is preferable. e.g.
>>
>> static inline pud_t * fixmap_pud(unsigned long addr)
>> {
>> pgd_t *pgd = pgd_offset_k(addr);
>>
>> BUG_ON(pgd_none(*pgd) || pgd_bad(*pgd));
>>
>> return pud_offset_kimg(pgd, addr);
>> }
>>
>> static inline pmd_t * fixmap_pmd(unsigned long addr)
>> {
>> pud_t *pud = fixmap_pud(addr);
>>
>> BUG_ON(pud_none(*pud) || pud_bad(*pud));
>>
>> return pmd_offset_kimg(pud, addr);
>> }
>>
>> That avoids having to check CONFIG_PGTABLE_LEVELS check and perform a cast,
>> avoids duplicating details about bm_{pud,pmd}, and keeps the existing structure
>> so it's easier to reason about the change. I was wrong about having to pass the
>> pgd or pud in, so callers don't need upating.
>>
>> From my PoV that is preferable.
>>
>
> OK. I think it looks better, indeed.
... however, this does mean we have to go through a __pa() translation
and back just to get to the address of bm_pud/bm_pmd
[toc] | [prev] | [next] | [standalone]
| From | Mark Rutland <mark.rutland@arm.com> |
|---|---|
| Date | 2016-01-11 18:30 +0100 |
| Subject | Re: [PATCH v3 04/21] arm64: decouple early fixmap init from linear mapping |
| Message-ID | <qPOs2-5vv-15@gated-at.bofh.it> |
| In reply to | #1306526 |
On Mon, Jan 11, 2016 at 06:15:56PM +0100, Ard Biesheuvel wrote:
> On 11 January 2016 at 18:08, Ard Biesheuvel <ard.biesheuvel@linaro.org> wrote:
> > On 11 January 2016 at 17:51, Mark Rutland <mark.rutland@arm.com> wrote:
> >> Sorry, I'm gonig to change my mind on that again. I think using
> >> p?d_offset_kimg is preferable. e.g.
> >>
> >> static inline pud_t * fixmap_pud(unsigned long addr)
> >> {
> >> pgd_t *pgd = pgd_offset_k(addr);
> >>
> >> BUG_ON(pgd_none(*pgd) || pgd_bad(*pgd));
> >>
> >> return pud_offset_kimg(pgd, addr);
> >> }
> >>
> >> static inline pmd_t * fixmap_pmd(unsigned long addr)
> >> {
> >> pud_t *pud = fixmap_pud(addr);
> >>
> >> BUG_ON(pud_none(*pud) || pud_bad(*pud));
> >>
> >> return pmd_offset_kimg(pud, addr);
> >> }
> >>
> >> That avoids having to check CONFIG_PGTABLE_LEVELS check and perform a cast,
> >> avoids duplicating details about bm_{pud,pmd}, and keeps the existing structure
> >> so it's easier to reason about the change. I was wrong about having to pass the
> >> pgd or pud in, so callers don't need upating.
> >>
> >> From my PoV that is preferable.
> >>
> >
> > OK. I think it looks better, indeed.
>
> ... however, this does mean we have to go through a __pa() translation
> and back just to get to the address of bm_pud/bm_pmd
True, but we only do it in the case of a one-off init function, so I
don't think we'll notice the overhead.
Mark.
[toc] | [prev] | [next] | [standalone]
| From | Ard Biesheuvel <ard.biesheuvel@linaro.org> |
|---|---|
| Date | 2016-01-11 14:30 +0100 |
| Subject | [PATCH v3 14/21] arm64: redefine SWAPPER_TABLE_SHIFT for use in asm code |
| Message-ID | <qPKHL-2Wj-11@gated-at.bofh.it> |
| In reply to | #1306179 |
The current definition of SWAPPER_TABLE_SHIFT can only be used in asm code if the configured number of translation levels defines PUD_SHIFT and/or PMD_SHIFT natively (4KB and 16KB/64KB granule, respectively). Otherwise, it depends on the nopmd/nopud fixup headers, which can only be included in C code. So redefine SWAPPER_TABLE_SHIFT in a way that is independent of the number of configured translation levels. Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> --- arch/arm64/include/asm/kernel-pgtable.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/arch/arm64/include/asm/kernel-pgtable.h b/arch/arm64/include/asm/kernel-pgtable.h index dfe4bae463b7..b1c96a29fad7 100644 --- a/arch/arm64/include/asm/kernel-pgtable.h +++ b/arch/arm64/include/asm/kernel-pgtable.h @@ -57,13 +57,13 @@ #if ARM64_SWAPPER_USES_SECTION_MAPS #define SWAPPER_BLOCK_SHIFT SECTION_SHIFT #define SWAPPER_BLOCK_SIZE SECTION_SIZE -#define SWAPPER_TABLE_SHIFT PUD_SHIFT #else #define SWAPPER_BLOCK_SHIFT PAGE_SHIFT #define SWAPPER_BLOCK_SIZE PAGE_SIZE -#define SWAPPER_TABLE_SHIFT PMD_SHIFT #endif +#define SWAPPER_TABLE_SHIFT (SWAPPER_BLOCK_SHIFT + PAGE_SHIFT - 3) + /* The size of the initial kernel direct mapping */ #define SWAPPER_INIT_MAP_SIZE (_AC(1, UL) << SWAPPER_TABLE_SHIFT) -- 2.5.0
[toc] | [prev] | [next] | [standalone]
| From | Ard Biesheuvel <ard.biesheuvel@linaro.org> |
|---|---|
| Date | 2016-01-11 14:30 +0100 |
| Subject | [PATCH v3 01/21] of/fdt: make memblock minimum physical address arch configurable |
| Message-ID | <qPKHM-2Wj-15@gated-at.bofh.it> |
| In reply to | #1306179 |
By default, early_init_dt_add_memory_arch() ignores memory below
the base of the kernel image since it won't be addressable via the
linear mapping. However, this is not appropriate anymore once we
decouple the kernel text mapping from the linear mapping, so archs
may want to drop the low limit entirely. So allow the minimum to be
overridden by setting MIN_MEMBLOCK_ADDR.
Acked-by: Mark Rutland <mark.rutland@arm.com>
Acked-by: Rob Herring <robh@kernel.org>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
drivers/of/fdt.c | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/drivers/of/fdt.c b/drivers/of/fdt.c
index d2430298a309..0455564f8cbc 100644
--- a/drivers/of/fdt.c
+++ b/drivers/of/fdt.c
@@ -971,13 +971,16 @@ int __init early_init_dt_scan_chosen(unsigned long node, const char *uname,
}
#ifdef CONFIG_HAVE_MEMBLOCK
+#ifndef MIN_MEMBLOCK_ADDR
+#define MIN_MEMBLOCK_ADDR __pa(PAGE_OFFSET)
+#endif
#ifndef MAX_MEMBLOCK_ADDR
#define MAX_MEMBLOCK_ADDR ((phys_addr_t)~0)
#endif
void __init __weak early_init_dt_add_memory_arch(u64 base, u64 size)
{
- const u64 phys_offset = __pa(PAGE_OFFSET);
+ const u64 phys_offset = MIN_MEMBLOCK_ADDR;
if (!PAGE_ALIGNED(base)) {
if (size < PAGE_SIZE - (base & ~PAGE_MASK)) {
--
2.5.0
[toc] | [prev] | [next] | [standalone]
| From | Ard Biesheuvel <ard.biesheuvel@linaro.org> |
|---|---|
| Date | 2016-01-11 14:30 +0100 |
| Subject | [PATCH v3 16/21] scripts/sortextable: add support for ET_DYN binaries |
| Message-ID | <qPKHM-2Wj-19@gated-at.bofh.it> |
| In reply to | #1306179 |
Add support to scripts/sortextable for handling relocatable (PIE)
executables, whose ELF type is ET_DYN, not ET_EXEC. Other than adding
support for the new type, no changes are needed.
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
scripts/sortextable.c | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/scripts/sortextable.c b/scripts/sortextable.c
index af247c70fb66..19d83647846c 100644
--- a/scripts/sortextable.c
+++ b/scripts/sortextable.c
@@ -266,9 +266,9 @@ do_file(char const *const fname)
break;
} /* end switch */
if (memcmp(ELFMAG, ehdr->e_ident, SELFMAG) != 0
- || r2(&ehdr->e_type) != ET_EXEC
+ || (r2(&ehdr->e_type) != ET_EXEC && r2(&ehdr->e_type) != ET_DYN)
|| ehdr->e_ident[EI_VERSION] != EV_CURRENT) {
- fprintf(stderr, "unrecognized ET_EXEC file %s\n", fname);
+ fprintf(stderr, "unrecognized ET_EXEC/ET_DYN file %s\n", fname);
fail_file();
}
@@ -304,7 +304,7 @@ do_file(char const *const fname)
if (r2(&ehdr->e_ehsize) != sizeof(Elf32_Ehdr)
|| r2(&ehdr->e_shentsize) != sizeof(Elf32_Shdr)) {
fprintf(stderr,
- "unrecognized ET_EXEC file: %s\n", fname);
+ "unrecognized ET_EXEC/ET_DYN file: %s\n", fname);
fail_file();
}
do32(ehdr, fname, custom_sort);
@@ -314,7 +314,7 @@ do_file(char const *const fname)
if (r2(&ghdr->e_ehsize) != sizeof(Elf64_Ehdr)
|| r2(&ghdr->e_shentsize) != sizeof(Elf64_Shdr)) {
fprintf(stderr,
- "unrecognized ET_EXEC file: %s\n", fname);
+ "unrecognized ET_EXEC/ET_DYN file: %s\n", fname);
fail_file();
}
do64(ghdr, fname, custom_sort);
--
2.5.0
[toc] | [prev] | [next] | [standalone]
| From | Ard Biesheuvel <ard.biesheuvel@linaro.org> |
|---|---|
| Date | 2016-01-11 14:30 +0100 |
| Subject | [PATCH v3 17/21] arm64: add support for a relocatable kernel and KASLR |
| Message-ID | <qPKHM-2Wj-21@gated-at.bofh.it> |
| In reply to | #1306179 |
This adds support for runtime relocation of the kernel Image, by
building it as a PIE (ET_DYN) executable and applying the dynamic
relocations in the early boot code.
On top of this, support for KASLR is implemented, based on entropy
provided by the bootloader in register x1 at kernel entry. Depending
on the size of the address space (VA_BITS) and the page size, the
entropy in the virtual displacement is up to 13 bits (16k/2 levels)
and up to 25 bits (all 4 levels), with the caveat that displacements
that result in the kernel image straddling a 1GB/32MB/512MB alignment
boundary (for 4KB/16KB/64KB granule kernels, respectively) are not
allowed.
The same virtual offset is applied to the module region: this gives
almost the same security benefits, and keeps the modules in close
proximity to the kernel so we only have to rely on branches via PLTs
once the module region is exhausted (which is slightly more likely
to occur, as the relocated module region is shared with other uses
of the vmalloc area)
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
Documentation/arm64/booting.txt | 16 +++-
arch/arm64/Kconfig | 26 ++++++
arch/arm64/Makefile | 4 +
arch/arm64/include/asm/elf_relocs.h | 2 +
arch/arm64/include/asm/memory.h | 3 +
arch/arm64/kernel/head.S | 94 +++++++++++++++++++-
arch/arm64/kernel/module.c | 3 +-
arch/arm64/kernel/setup.c | 38 ++++++--
arch/arm64/kernel/vmlinux.lds.S | 9 ++
9 files changed, 180 insertions(+), 15 deletions(-)
diff --git a/Documentation/arm64/booting.txt b/Documentation/arm64/booting.txt
index 67484067ce4f..0bd5ea83a54f 100644
--- a/Documentation/arm64/booting.txt
+++ b/Documentation/arm64/booting.txt
@@ -115,13 +115,25 @@ Header notes:
accessible
1 - 2MB aligned base may be anywhere in physical
memory
- Bits 4-63: Reserved.
+ Bit 4: Virtual address space layout randomization (KASLR)
+ 0 - kernel will execute from a fixed virtual offset
+ that is decided at compile time, register x1 should
+ be zero at kernel entry
+ 1 - kernel will execute from a virtual offset that is
+ randomized based on the contents of register x1 at
+ kernel entry
+ Bits 5-63: Reserved.
- When image_size is zero, a bootloader should attempt to keep as much
memory as possible free for use by the kernel immediately after the
end of the kernel image. The amount of space required will vary
depending on selected features, and is effectively unbound.
+- It is up to the bootloader to decide whether a KASLR capable kernel should
+ boot with randomization enabled. If this is the case, register x1 should
+ contain a strong random value. If the bootloader passes 'nokaslr' on the
+ kernel command line to disable randomization, it must also pass 0 in x1.
+
The Image must be placed text_offset bytes from a 2MB aligned base
address anywhere in usable system RAM and called there. The region
between the 2 MB aligned base address and the start of the image has no
@@ -145,7 +157,7 @@ Before jumping into the kernel, the following conditions must be met:
- Primary CPU general-purpose register settings
x0 = physical address of device tree blob (dtb) in system RAM.
- x1 = 0 (reserved for future use)
+ x1 = 0, unless bit 4 is set in the Image header
x2 = 0 (reserved for future use)
x3 = 0 (reserved for future use)
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index 778df20bf623..7fa5b74ee80d 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -711,6 +711,32 @@ config ARM64_MODULE_PLTS
select ARM64_MODULE_CMODEL_LARGE
select HAVE_MOD_ARCH_SPECIFIC
+config RELOCATABLE
+ bool
+ help
+ This builds the kernel as a Position Independent Executable (PIE),
+ which retains all relocation metadata required to relocate the
+ kernel binary at runtime to a different virtual address than the
+ address it was linked at.
+ Since AArch64 uses the RELA relocation format, this requires a
+ relocation pass at runtime even if the kernel is loaded at the
+ same address it was linked at.
+
+config RANDOMIZE_BASE
+ bool "Randomize the address of the kernel image"
+ select ARM64_MODULE_PLTS
+ select RELOCATABLE
+ help
+ Randomizes the virtual address at which the kernel image is
+ loaded, as a security feature that deters exploit attempts
+ relying on knowledge of the location of kernel internals.
+
+ It is the bootloader's job to provide entropy, by passing a
+ random value in x1 at kernel entry.
+
+ If unsure, say N.
+
+
endmenu
menu "Boot options"
diff --git a/arch/arm64/Makefile b/arch/arm64/Makefile
index db462980c6be..c3eaa03f9020 100644
--- a/arch/arm64/Makefile
+++ b/arch/arm64/Makefile
@@ -15,6 +15,10 @@ CPPFLAGS_vmlinux.lds = -DTEXT_OFFSET=$(TEXT_OFFSET)
OBJCOPYFLAGS :=-O binary -R .note -R .note.gnu.build-id -R .comment -S
GZFLAGS :=-9
+ifneq ($(CONFIG_RELOCATABLE),)
+LDFLAGS_vmlinux += -pie
+endif
+
KBUILD_DEFCONFIG := defconfig
# Check for binutils support for specific extensions
diff --git a/arch/arm64/include/asm/elf_relocs.h b/arch/arm64/include/asm/elf_relocs.h
index 3f6b93099011..e1316de840a5 100644
--- a/arch/arm64/include/asm/elf_relocs.h
+++ b/arch/arm64/include/asm/elf_relocs.h
@@ -69,5 +69,7 @@
#define R_AARCH64_MOVW_PREL_G2_NC 292
#define R_AARCH64_MOVW_PREL_G3 293
+#define R_AARCH64_RELATIVE 1027
+
#endif /* __ASM_ELF_RELOCS_H */
diff --git a/arch/arm64/include/asm/memory.h b/arch/arm64/include/asm/memory.h
index 758fb4a503ef..422a30a5f328 100644
--- a/arch/arm64/include/asm/memory.h
+++ b/arch/arm64/include/asm/memory.h
@@ -122,6 +122,9 @@ extern phys_addr_t memstart_addr;
/* PHYS_OFFSET - the physical address of the start of memory. */
#define PHYS_OFFSET ({ memstart_addr; })
+/* the virtual base of the kernel image (minus TEXT_OFFSET) */
+extern u64 kimage_vaddr;
+
/* the offset between the kernel virtual and physical mappings */
extern u64 kimage_voffset;
diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S
index d66aee595170..4bf6a5c9a24e 100644
--- a/arch/arm64/kernel/head.S
+++ b/arch/arm64/kernel/head.S
@@ -29,6 +29,7 @@
#include <asm/asm-offsets.h>
#include <asm/cache.h>
#include <asm/cputype.h>
+#include <asm/elf_relocs.h>
#include <asm/kernel-pgtable.h>
#include <asm/memory.h>
#include <asm/pgtable-hwdef.h>
@@ -61,9 +62,16 @@
#define __HEAD_FLAG_PAGE_SIZE ((PAGE_SHIFT - 10) / 2)
+#ifdef CONFIG_RANDOMIZE_BASE
+#define __HEAD_FLAG_KASLR 1
+#else
+#define __HEAD_FLAG_KASLR 0
+#endif
+
#define __HEAD_FLAGS ((__HEAD_FLAG_BE << 0) | \
(__HEAD_FLAG_PAGE_SIZE << 1) | \
- (__HEAD_FLAG_PHYS_BASE << 3))
+ (__HEAD_FLAG_PHYS_BASE << 3) | \
+ (__HEAD_FLAG_KASLR << 4))
/*
* Kernel startup entry point.
@@ -234,6 +242,7 @@ ENTRY(stext)
*/
ldr x27, 0f // address to jump to after
// MMU has been enabled
+ add x27, x27, x23 // add KASLR displacement
adr_l lr, __enable_mmu // return (PIC) address
b __cpu_setup // initialise processor
ENDPROC(stext)
@@ -245,6 +254,7 @@ ENDPROC(stext)
*/
preserve_boot_args:
mov x21, x0 // x21=FDT
+ mov x22, x1 // x22=random seed
adr_l x0, boot_args // record the contents of
stp x21, x1, [x0] // x0 .. x3 at kernel entry
@@ -328,6 +338,40 @@ __create_page_tables:
adrp x26, swapper_pg_dir
mov x27, lr
+#ifdef CONFIG_RANDOMIZE_BASE
+ /*
+ * Mask off the bits of the random value stored in x22 so it can serve
+ * as a KASLR displacement value which will move the kernel image to a
+ * random offset in the lower half of the VMALLOC area (VA_BITS - 2).
+ * Even if we could randomize at page granularity for 16k and 64k
+ * granule kernels, let's always preserve the 2 MB (21 bit) alignment
+ * and not interfere with the ability to use ranges of contiguous PTEs.
+ */
+ .set RANDOM_WIDTH, VA_BITS - 2
+ .set RANDOM_ALIGN, 21
+
+ mov x10, ((1 << (RANDOM_WIDTH - RANDOM_ALIGN)) - 1) << RANDOM_ALIGN
+ and x23, x22, x10
+
+ /*
+ * The kernel Image should not extend across a 1GB/32MB/512MB alignment
+ * boundary (for 4KB/16KB/64KB granule kernels, respectively). If this
+ * happens, increase the KASLR displacement in x23 by the size of the
+ * kernel image.
+ */
+ ldr w8, kernel_img_size
+ mov x11, KIMAGE_VADDR & ((1 << SWAPPER_TABLE_SHIFT) - 1)
+ add x11, x11, x23
+ add x9, x8, x11
+ eor x9, x9, x11
+ tbz x9, SWAPPER_TABLE_SHIFT, 0f
+ add x23, x23, x8
+ and x23, x23, x10
+0:
+#else
+ mov x23, xzr
+#endif
+
/*
* Invalidate the idmap and swapper page tables to avoid potential
* dirty cache lines being evicted.
@@ -405,6 +449,7 @@ __create_page_tables:
*/
mov x0, x26 // swapper_pg_dir
ldr x5, =KIMAGE_VADDR
+ add x5, x5, x23 // add KASLR displacement
create_pgd_entry x0, x5, x3, x6
ldr w6, kernel_img_size
add x6, x6, x5
@@ -446,13 +491,52 @@ __mmap_switched:
bl __pi_memset
dsb ishst // Make zero page visible to PTW
+
+#ifdef CONFIG_RELOCATABLE
+
+ /*
+ * Iterate over each entry in the relocation table, and apply the
+ * relocations in place.
+ */
+ adr_l x8, __dynsym_start // start of symbol table
+ adr_l x9, __reloc_start // start of reloc table
+ adr_l x10, __reloc_end // end of reloc table
+
+0: cmp x9, x10
+ b.hs 2f
+ ldp x11, x12, [x9], #24
+ ldr x13, [x9, #-8]
+ cmp w12, #R_AARCH64_RELATIVE
+ b.ne 1f
+ add x13, x13, x23 // relocate
+ str x13, [x11, x23]
+ b 0b
+
+1: cmp w12, #R_AARCH64_ABS64
+ b.ne 0b
+ add x12, x12, x12, lsl #1 // symtab offset: 24x top word
+ add x12, x8, x12, lsr #(32 - 3) // ... shifted into bottom word
+ ldrsh w14, [x12, #6] // Elf64_Sym::st_shndx
+ ldr x15, [x12, #8] // Elf64_Sym::st_value
+ cmp w14, #-0xf // SHN_ABS (0xfff1) ?
+ add x14, x15, x23 // relocate
+ csel x15, x14, x15, ne
+ add x15, x13, x15
+ str x15, [x11, x23]
+ b 0b
+
+2: adr_l x8, kimage_vaddr // make relocated kimage_vaddr
+ dc cvac, x8 // value visible to secondaries
+ dsb sy // with MMU off
+#endif
+
adr_l sp, initial_sp, x4
mov x4, sp
and x4, x4, #~(THREAD_SIZE - 1)
msr sp_el0, x4 // Save thread_info
str_l x21, __fdt_pointer, x5 // Save FDT pointer
- ldr x0, =KIMAGE_VADDR // Save the offset between
+ ldr_l x0, kimage_vaddr // Save the offset between
sub x24, x0, x24 // the kernel virtual and
str_l x24, kimage_voffset, x0 // physical mappings
@@ -468,6 +552,10 @@ ENDPROC(__mmap_switched)
* hotplug and needs to have the same protections as the text region
*/
.section ".text","ax"
+
+ENTRY(kimage_vaddr)
+ .quad _text - TEXT_OFFSET
+
/*
* If we're fortunate enough to boot at EL2, ensure that the world is
* sane before dropping to EL1.
@@ -628,7 +716,7 @@ ENTRY(secondary_startup)
adrp x26, swapper_pg_dir
bl __cpu_setup // initialise processor
- ldr x8, =KIMAGE_VADDR
+ ldr x8, kimage_vaddr
ldr w9, 0f
sub x27, x8, w9, sxtw // address to jump to after enabling the MMU
b __enable_mmu
diff --git a/arch/arm64/kernel/module.c b/arch/arm64/kernel/module.c
index 3a298b0e21bb..d38662028200 100644
--- a/arch/arm64/kernel/module.c
+++ b/arch/arm64/kernel/module.c
@@ -34,7 +34,8 @@ void *module_alloc(unsigned long size)
{
void *p;
- p = __vmalloc_node_range(size, MODULE_ALIGN, MODULES_VADDR, MODULES_END,
+ p = __vmalloc_node_range(size, MODULE_ALIGN,
+ kimage_vaddr - MODULES_VSIZE, kimage_vaddr,
GFP_KERNEL, PAGE_KERNEL_EXEC, 0,
NUMA_NO_NODE, __builtin_return_address(0));
diff --git a/arch/arm64/kernel/setup.c b/arch/arm64/kernel/setup.c
index c67ba4453ec6..f8111894447c 100644
--- a/arch/arm64/kernel/setup.c
+++ b/arch/arm64/kernel/setup.c
@@ -288,16 +288,41 @@ static inline void __init relocate_initrd(void)
}
#endif
+static bool nokaslr;
+static int __init early_nokaslr(char *p)
+{
+ nokaslr = true;
+ return 0;
+}
+early_param("nokaslr", early_nokaslr);
+
+static void check_boot_args(void)
+{
+ if ((!IS_ENABLED(CONFIG_RANDOMIZE_BASE) && boot_args[1]) ||
+ boot_args[2] || boot_args[3]) {
+ pr_err("WARNING: x1-x3 nonzero in violation of boot protocol:\n"
+ "\tx1: %016llx\n\tx2: %016llx\n\tx3: %016llx\n"
+ "This indicates a broken bootloader or old kernel\n",
+ boot_args[1], boot_args[2], boot_args[3]);
+ }
+ if (IS_ENABLED(CONFIG_RANDOMIZE_BASE) && boot_args[1] && nokaslr) {
+ pr_err("WARNING: found KASLR entropy in x1 but 'nokaslr' was passed on the commmand line:\n"
+ "\tx1: %016llx\n"
+ "This indicates a broken bootloader\n",
+ boot_args[1]);
+ }
+}
+
u64 __cpu_logical_map[NR_CPUS] = { [0 ... NR_CPUS-1] = INVALID_HWID };
void __init setup_arch(char **cmdline_p)
{
static struct vm_struct vmlinux_vm;
- vmlinux_vm.addr = (void *)KIMAGE_VADDR;
- vmlinux_vm.size = round_up((u64)_end - KIMAGE_VADDR,
+ vmlinux_vm.addr = (void *)kimage_vaddr;
+ vmlinux_vm.size = round_up((u64)_end - kimage_vaddr,
SWAPPER_BLOCK_SIZE);
- vmlinux_vm.phys_addr = __pa(KIMAGE_VADDR);
+ vmlinux_vm.phys_addr = __pa(kimage_vaddr);
vmlinux_vm.flags = VM_MAP;
vmlinux_vm.caller = setup_arch;
@@ -366,12 +391,7 @@ void __init setup_arch(char **cmdline_p)
conswitchp = &dummy_con;
#endif
#endif
- if (boot_args[1] || boot_args[2] || boot_args[3]) {
- pr_err("WARNING: x1-x3 nonzero in violation of boot protocol:\n"
- "\tx1: %016llx\n\tx2: %016llx\n\tx3: %016llx\n"
- "This indicates a broken bootloader or old kernel\n",
- boot_args[1], boot_args[2], boot_args[3]);
- }
+ check_boot_args();
}
static int __init arm64_device_init(void)
diff --git a/arch/arm64/kernel/vmlinux.lds.S b/arch/arm64/kernel/vmlinux.lds.S
index ced0dedcabcc..eddd234d7721 100644
--- a/arch/arm64/kernel/vmlinux.lds.S
+++ b/arch/arm64/kernel/vmlinux.lds.S
@@ -148,6 +148,15 @@ SECTIONS
.altinstr_replacement : {
*(.altinstr_replacement)
}
+ .rela : ALIGN(8) {
+ __reloc_start = .;
+ *(.rela .rela*)
+ __reloc_end = .;
+ }
+ .dynsym : ALIGN(8) {
+ __dynsym_start = .;
+ *(.dynsym)
+ }
. = ALIGN(PAGE_SIZE);
__init_end = .;
--
2.5.0
[toc] | [prev] | [next] | [standalone]
| From | Ard Biesheuvel <ard.biesheuvel@linaro.org> |
|---|---|
| Date | 2016-01-11 14:30 +0100 |
| Subject | [PATCH v3 11/21] arm64: avoid R_AARCH64_ABS64 relocations for Image header fields |
| Message-ID | <qPKHM-2Wj-23@gated-at.bofh.it> |
| In reply to | #1306179 |
Unfortunately, the current way of using the linker to emit build time constants into the Image header will no longer work once we switch to the use of PIE executables. The reason is that such constants are emitted into the binary using R_AARCH64_ABS64 relocations, which we will resolve at runtime, not at build time, and the places targeted by those relocations will contain zeroes before that. So move back to assembly time constants or R_AARCH64_ABS32 relocations (which, interestingly enough, do get resolved at build time) Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> --- arch/arm64/include/asm/assembler.h | 15 ++++++++ arch/arm64/kernel/head.S | 17 +++++++-- arch/arm64/kernel/image.h | 37 ++++++-------------- 3 files changed, 40 insertions(+), 29 deletions(-) diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h index d8bfcc1ce923..e211af783a3d 100644 --- a/arch/arm64/include/asm/assembler.h +++ b/arch/arm64/include/asm/assembler.h @@ -222,4 +222,19 @@ lr .req x30 // link register .size __pi_##x, . - x; \ ENDPROC(x) + .macro le16, val + .byte \val & 0xff + .byte (\val >> 8) & 0xff + .endm + + .macro le32, val + le16 \val + le16 (\val >> 16) + .endm + + .macro le64, val + le32 \val + le32 (\val >> 32) + .endm + #endif /* __ASM_ASSEMBLER_H */ diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S index 350515276541..211f75e673f4 100644 --- a/arch/arm64/kernel/head.S +++ b/arch/arm64/kernel/head.S @@ -51,6 +51,17 @@ #define KERNEL_START _text #define KERNEL_END _end +#ifdef CONFIG_CPU_BIG_ENDIAN +#define __HEAD_FLAG_BE 1 +#else +#define __HEAD_FLAG_BE 0 +#endif + +#define __HEAD_FLAG_PAGE_SIZE ((PAGE_SHIFT - 10) / 2) + +#define __HEAD_FLAGS ((__HEAD_FLAG_BE << 0) | \ + (__HEAD_FLAG_PAGE_SIZE << 1)) + /* * Kernel startup entry point. * --------------------------- @@ -83,9 +94,9 @@ efi_head: b stext // branch to kernel start, magic .long 0 // reserved #endif - .quad _kernel_offset_le // Image load offset from start of RAM, little-endian - .quad _kernel_size_le // Effective size of kernel image, little-endian - .quad _kernel_flags_le // Informative flags, little-endian + le64 TEXT_OFFSET // Image load offset from start of RAM, little-endian + .long _kernel_size_le, 0 // Effective size of kernel image, little-endian + le64 __HEAD_FLAGS // Informative flags, little-endian .quad 0 // reserved .quad 0 // reserved .quad 0 // reserved diff --git a/arch/arm64/kernel/image.h b/arch/arm64/kernel/image.h index bc2abb8b1599..bb6b0e69d0a4 100644 --- a/arch/arm64/kernel/image.h +++ b/arch/arm64/kernel/image.h @@ -26,41 +26,26 @@ * There aren't any ELF relocations we can use to endian-swap values known only * at link time (e.g. the subtraction of two symbol addresses), so we must get * the linker to endian-swap certain values before emitting them. + * Note that this will not work for 64-bit values: these are resolved using + * R_AARCH64_ABS64 relocations, which are fixed up at runtime rather than at + * build time when building the PIE executable (for KASLR). */ #ifdef CONFIG_CPU_BIG_ENDIAN -#define DATA_LE64(data) \ - ((((data) & 0x00000000000000ff) << 56) | \ - (((data) & 0x000000000000ff00) << 40) | \ - (((data) & 0x0000000000ff0000) << 24) | \ - (((data) & 0x00000000ff000000) << 8) | \ - (((data) & 0x000000ff00000000) >> 8) | \ - (((data) & 0x0000ff0000000000) >> 24) | \ - (((data) & 0x00ff000000000000) >> 40) | \ - (((data) & 0xff00000000000000) >> 56)) +#define DATA_LE32(data) \ + ((((data) & 0x000000ff) << 24) | \ + (((data) & 0x0000ff00) << 8) | \ + (((data) & 0x00ff0000) >> 8) | \ + (((data) & 0xff000000) >> 24)) #else -#define DATA_LE64(data) ((data) & 0xffffffffffffffff) +#define DATA_LE32(data) ((data) & 0xffffffff) #endif -#ifdef CONFIG_CPU_BIG_ENDIAN -#define __HEAD_FLAG_BE 1 -#else -#define __HEAD_FLAG_BE 0 -#endif - -#define __HEAD_FLAG_PAGE_SIZE ((PAGE_SHIFT - 10) / 2) - -#define __HEAD_FLAGS ((__HEAD_FLAG_BE << 0) | \ - (__HEAD_FLAG_PAGE_SIZE << 1)) - /* * These will output as part of the Image header, which should be little-endian - * regardless of the endianness of the kernel. While constant values could be - * endian swapped in head.S, all are done here for consistency. + * regardless of the endianness of the kernel. */ #define HEAD_SYMBOLS \ - _kernel_size_le = DATA_LE64(_end - _text); \ - _kernel_offset_le = DATA_LE64(TEXT_OFFSET); \ - _kernel_flags_le = DATA_LE64(__HEAD_FLAGS); + _kernel_size_le = DATA_LE32(_end - _text); #ifdef CONFIG_EFI -- 2.5.0
[toc] | [prev] | [next] | [standalone]
| From | Mark Rutland <mark.rutland@arm.com> |
|---|---|
| Date | 2016-01-13 19:20 +0100 |
| Subject | Re: [PATCH v3 11/21] arm64: avoid R_AARCH64_ABS64 relocations for Image header fields |
| Message-ID | <qQybv-3tX-9@gated-at.bofh.it> |
| In reply to | #1306191 |
On Mon, Jan 11, 2016 at 02:19:04PM +0100, Ard Biesheuvel wrote: > Unfortunately, the current way of using the linker to emit build time > constants into the Image header will no longer work once we switch to > the use of PIE executables. The reason is that such constants are emitted > into the binary using R_AARCH64_ABS64 relocations, which we will resolve > at runtime, not at build time, and the places targeted by those > relocations will contain zeroes before that. > > So move back to assembly time constants or R_AARCH64_ABS32 relocations > (which, interestingly enough, do get resolved at build time) To me it seems very odd that ABS64 and ABS32 are treated differently, and it makes me somewhat uncomfortable becuase it feels like a bug. Do we know whether the inconsistency between ABS64 and ABS32 was deliberate? I couldn't spot anything declaring a difference in the AArch64 ELF spec, and I'm not sure where else to look. Thanks, Mark. > Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> > --- > arch/arm64/include/asm/assembler.h | 15 ++++++++ > arch/arm64/kernel/head.S | 17 +++++++-- > arch/arm64/kernel/image.h | 37 ++++++-------------- > 3 files changed, 40 insertions(+), 29 deletions(-) > > diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h > index d8bfcc1ce923..e211af783a3d 100644 > --- a/arch/arm64/include/asm/assembler.h > +++ b/arch/arm64/include/asm/assembler.h > @@ -222,4 +222,19 @@ lr .req x30 // link register > .size __pi_##x, . - x; \ > ENDPROC(x) > > + .macro le16, val > + .byte \val & 0xff > + .byte (\val >> 8) & 0xff > + .endm > + > + .macro le32, val > + le16 \val > + le16 (\val >> 16) > + .endm > + > + .macro le64, val > + le32 \val > + le32 (\val >> 32) > + .endm > + > #endif /* __ASM_ASSEMBLER_H */ > diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S > index 350515276541..211f75e673f4 100644 > --- a/arch/arm64/kernel/head.S > +++ b/arch/arm64/kernel/head.S > @@ -51,6 +51,17 @@ > #define KERNEL_START _text > #define KERNEL_END _end > > +#ifdef CONFIG_CPU_BIG_ENDIAN > +#define __HEAD_FLAG_BE 1 > +#else > +#define __HEAD_FLAG_BE 0 > +#endif > + > +#define __HEAD_FLAG_PAGE_SIZE ((PAGE_SHIFT - 10) / 2) > + > +#define __HEAD_FLAGS ((__HEAD_FLAG_BE << 0) | \ > + (__HEAD_FLAG_PAGE_SIZE << 1)) > + > /* > * Kernel startup entry point. > * --------------------------- > @@ -83,9 +94,9 @@ efi_head: > b stext // branch to kernel start, magic > .long 0 // reserved > #endif > - .quad _kernel_offset_le // Image load offset from start of RAM, little-endian > - .quad _kernel_size_le // Effective size of kernel image, little-endian > - .quad _kernel_flags_le // Informative flags, little-endian > + le64 TEXT_OFFSET // Image load offset from start of RAM, little-endian > + .long _kernel_size_le, 0 // Effective size of kernel image, little-endian > + le64 __HEAD_FLAGS // Informative flags, little-endian > .quad 0 // reserved > .quad 0 // reserved > .quad 0 // reserved > diff --git a/arch/arm64/kernel/image.h b/arch/arm64/kernel/image.h > index bc2abb8b1599..bb6b0e69d0a4 100644 > --- a/arch/arm64/kernel/image.h > +++ b/arch/arm64/kernel/image.h > @@ -26,41 +26,26 @@ > * There aren't any ELF relocations we can use to endian-swap values known only > * at link time (e.g. the subtraction of two symbol addresses), so we must get > * the linker to endian-swap certain values before emitting them. > + * Note that this will not work for 64-bit values: these are resolved using > + * R_AARCH64_ABS64 relocations, which are fixed up at runtime rather than at > + * build time when building the PIE executable (for KASLR). > */ > #ifdef CONFIG_CPU_BIG_ENDIAN > -#define DATA_LE64(data) \ > - ((((data) & 0x00000000000000ff) << 56) | \ > - (((data) & 0x000000000000ff00) << 40) | \ > - (((data) & 0x0000000000ff0000) << 24) | \ > - (((data) & 0x00000000ff000000) << 8) | \ > - (((data) & 0x000000ff00000000) >> 8) | \ > - (((data) & 0x0000ff0000000000) >> 24) | \ > - (((data) & 0x00ff000000000000) >> 40) | \ > - (((data) & 0xff00000000000000) >> 56)) > +#define DATA_LE32(data) \ > + ((((data) & 0x000000ff) << 24) | \ > + (((data) & 0x0000ff00) << 8) | \ > + (((data) & 0x00ff0000) >> 8) | \ > + (((data) & 0xff000000) >> 24)) > #else > -#define DATA_LE64(data) ((data) & 0xffffffffffffffff) > +#define DATA_LE32(data) ((data) & 0xffffffff) > #endif > > -#ifdef CONFIG_CPU_BIG_ENDIAN > -#define __HEAD_FLAG_BE 1 > -#else > -#define __HEAD_FLAG_BE 0 > -#endif > - > -#define __HEAD_FLAG_PAGE_SIZE ((PAGE_SHIFT - 10) / 2) > - > -#define __HEAD_FLAGS ((__HEAD_FLAG_BE << 0) | \ > - (__HEAD_FLAG_PAGE_SIZE << 1)) > - > /* > * These will output as part of the Image header, which should be little-endian > - * regardless of the endianness of the kernel. While constant values could be > - * endian swapped in head.S, all are done here for consistency. > + * regardless of the endianness of the kernel. > */ > #define HEAD_SYMBOLS \ > - _kernel_size_le = DATA_LE64(_end - _text); \ > - _kernel_offset_le = DATA_LE64(TEXT_OFFSET); \ > - _kernel_flags_le = DATA_LE64(__HEAD_FLAGS); > + _kernel_size_le = DATA_LE32(_end - _text); > > #ifdef CONFIG_EFI > > -- > 2.5.0 >
[toc] | [prev] | [next] | [standalone]
| From | Ard Biesheuvel <ard.biesheuvel@linaro.org> |
|---|---|
| Date | 2016-01-13 19:50 +0100 |
| Subject | Re: [PATCH v3 11/21] arm64: avoid R_AARCH64_ABS64 relocations for Image header fields |
| Message-ID | <qQyEx-3Fw-3@gated-at.bofh.it> |
| In reply to | #1308706 |
On 13 January 2016 at 19:12, Mark Rutland <mark.rutland@arm.com> wrote: > On Mon, Jan 11, 2016 at 02:19:04PM +0100, Ard Biesheuvel wrote: >> Unfortunately, the current way of using the linker to emit build time >> constants into the Image header will no longer work once we switch to >> the use of PIE executables. The reason is that such constants are emitted >> into the binary using R_AARCH64_ABS64 relocations, which we will resolve >> at runtime, not at build time, and the places targeted by those >> relocations will contain zeroes before that. >> >> So move back to assembly time constants or R_AARCH64_ABS32 relocations >> (which, interestingly enough, do get resolved at build time) > > To me it seems very odd that ABS64 and ABS32 are treated differently, > and it makes me somewhat uncomfortable becuase it feels like a bug. > > Do we know whether the inconsistency between ABS64 and ABS32 was > deliberate? > > I couldn't spot anything declaring a difference in the AArch64 ELF > spec, and I'm not sure where else to look. > My assumption is that PIE only defers resolving R_AARCH64_ABS64 relocations since those are the only ones that be used to refer to memory addresses >> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> >> --- >> arch/arm64/include/asm/assembler.h | 15 ++++++++ >> arch/arm64/kernel/head.S | 17 +++++++-- >> arch/arm64/kernel/image.h | 37 ++++++-------------- >> 3 files changed, 40 insertions(+), 29 deletions(-) >> >> diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h >> index d8bfcc1ce923..e211af783a3d 100644 >> --- a/arch/arm64/include/asm/assembler.h >> +++ b/arch/arm64/include/asm/assembler.h >> @@ -222,4 +222,19 @@ lr .req x30 // link register >> .size __pi_##x, . - x; \ >> ENDPROC(x) >> >> + .macro le16, val >> + .byte \val & 0xff >> + .byte (\val >> 8) & 0xff >> + .endm >> + >> + .macro le32, val >> + le16 \val >> + le16 (\val >> 16) >> + .endm >> + >> + .macro le64, val >> + le32 \val >> + le32 (\val >> 32) >> + .endm >> + >> #endif /* __ASM_ASSEMBLER_H */ >> diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S >> index 350515276541..211f75e673f4 100644 >> --- a/arch/arm64/kernel/head.S >> +++ b/arch/arm64/kernel/head.S >> @@ -51,6 +51,17 @@ >> #define KERNEL_START _text >> #define KERNEL_END _end >> >> +#ifdef CONFIG_CPU_BIG_ENDIAN >> +#define __HEAD_FLAG_BE 1 >> +#else >> +#define __HEAD_FLAG_BE 0 >> +#endif >> + >> +#define __HEAD_FLAG_PAGE_SIZE ((PAGE_SHIFT - 10) / 2) >> + >> +#define __HEAD_FLAGS ((__HEAD_FLAG_BE << 0) | \ >> + (__HEAD_FLAG_PAGE_SIZE << 1)) >> + >> /* >> * Kernel startup entry point. >> * --------------------------- >> @@ -83,9 +94,9 @@ efi_head: >> b stext // branch to kernel start, magic >> .long 0 // reserved >> #endif >> - .quad _kernel_offset_le // Image load offset from start of RAM, little-endian >> - .quad _kernel_size_le // Effective size of kernel image, little-endian >> - .quad _kernel_flags_le // Informative flags, little-endian >> + le64 TEXT_OFFSET // Image load offset from start of RAM, little-endian >> + .long _kernel_size_le, 0 // Effective size of kernel image, little-endian >> + le64 __HEAD_FLAGS // Informative flags, little-endian >> .quad 0 // reserved >> .quad 0 // reserved >> .quad 0 // reserved >> diff --git a/arch/arm64/kernel/image.h b/arch/arm64/kernel/image.h >> index bc2abb8b1599..bb6b0e69d0a4 100644 >> --- a/arch/arm64/kernel/image.h >> +++ b/arch/arm64/kernel/image.h >> @@ -26,41 +26,26 @@ >> * There aren't any ELF relocations we can use to endian-swap values known only >> * at link time (e.g. the subtraction of two symbol addresses), so we must get >> * the linker to endian-swap certain values before emitting them. >> + * Note that this will not work for 64-bit values: these are resolved using >> + * R_AARCH64_ABS64 relocations, which are fixed up at runtime rather than at >> + * build time when building the PIE executable (for KASLR). >> */ >> #ifdef CONFIG_CPU_BIG_ENDIAN >> -#define DATA_LE64(data) \ >> - ((((data) & 0x00000000000000ff) << 56) | \ >> - (((data) & 0x000000000000ff00) << 40) | \ >> - (((data) & 0x0000000000ff0000) << 24) | \ >> - (((data) & 0x00000000ff000000) << 8) | \ >> - (((data) & 0x000000ff00000000) >> 8) | \ >> - (((data) & 0x0000ff0000000000) >> 24) | \ >> - (((data) & 0x00ff000000000000) >> 40) | \ >> - (((data) & 0xff00000000000000) >> 56)) >> +#define DATA_LE32(data) \ >> + ((((data) & 0x000000ff) << 24) | \ >> + (((data) & 0x0000ff00) << 8) | \ >> + (((data) & 0x00ff0000) >> 8) | \ >> + (((data) & 0xff000000) >> 24)) >> #else >> -#define DATA_LE64(data) ((data) & 0xffffffffffffffff) >> +#define DATA_LE32(data) ((data) & 0xffffffff) >> #endif >> >> -#ifdef CONFIG_CPU_BIG_ENDIAN >> -#define __HEAD_FLAG_BE 1 >> -#else >> -#define __HEAD_FLAG_BE 0 >> -#endif >> - >> -#define __HEAD_FLAG_PAGE_SIZE ((PAGE_SHIFT - 10) / 2) >> - >> -#define __HEAD_FLAGS ((__HEAD_FLAG_BE << 0) | \ >> - (__HEAD_FLAG_PAGE_SIZE << 1)) >> - >> /* >> * These will output as part of the Image header, which should be little-endian >> - * regardless of the endianness of the kernel. While constant values could be >> - * endian swapped in head.S, all are done here for consistency. >> + * regardless of the endianness of the kernel. >> */ >> #define HEAD_SYMBOLS \ >> - _kernel_size_le = DATA_LE64(_end - _text); \ >> - _kernel_offset_le = DATA_LE64(TEXT_OFFSET); \ >> - _kernel_flags_le = DATA_LE64(__HEAD_FLAGS); >> + _kernel_size_le = DATA_LE32(_end - _text); >> >> #ifdef CONFIG_EFI >> >> -- >> 2.5.0 >>
[toc] | [prev] | [next] | [standalone]
| From | Ard Biesheuvel <ard.biesheuvel@linaro.org> |
|---|---|
| Date | 2016-01-14 10:00 +0100 |
| Subject | Re: [PATCH v3 11/21] arm64: avoid R_AARCH64_ABS64 relocations for Image header fields |
| Message-ID | <qQLV8-4BL-17@gated-at.bofh.it> |
| In reply to | #1308725 |
On 13 January 2016 at 19:48, Ard Biesheuvel <ard.biesheuvel@linaro.org> wrote: > On 13 January 2016 at 19:12, Mark Rutland <mark.rutland@arm.com> wrote: >> On Mon, Jan 11, 2016 at 02:19:04PM +0100, Ard Biesheuvel wrote: >>> Unfortunately, the current way of using the linker to emit build time >>> constants into the Image header will no longer work once we switch to >>> the use of PIE executables. The reason is that such constants are emitted >>> into the binary using R_AARCH64_ABS64 relocations, which we will resolve >>> at runtime, not at build time, and the places targeted by those >>> relocations will contain zeroes before that. >>> >>> So move back to assembly time constants or R_AARCH64_ABS32 relocations >>> (which, interestingly enough, do get resolved at build time) >> >> To me it seems very odd that ABS64 and ABS32 are treated differently, >> and it makes me somewhat uncomfortable becuase it feels like a bug. >> >> Do we know whether the inconsistency between ABS64 and ABS32 was >> deliberate? >> >> I couldn't spot anything declaring a difference in the AArch64 ELF >> spec, and I'm not sure where else to look. >> > > My assumption is that PIE only defers resolving R_AARCH64_ABS64 > relocations since those are the only ones that can be used to refer to > memory addresses > OK, digging into the binutils source code, it turns out that indeed, ABSnn relocations where nn equals the ELFnn memory size are treated differently, but only if they have default visibility. This is simply a result of the fact the code path is shared between shared libraries and PIE executables, since PIE executable are fully linked. It also means that we can simply work around it by emitting the linker symbols as hidden. >>> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> >>> --- >>> arch/arm64/include/asm/assembler.h | 15 ++++++++ >>> arch/arm64/kernel/head.S | 17 +++++++-- >>> arch/arm64/kernel/image.h | 37 ++++++-------------- >>> 3 files changed, 40 insertions(+), 29 deletions(-) >>> >>> diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h >>> index d8bfcc1ce923..e211af783a3d 100644 >>> --- a/arch/arm64/include/asm/assembler.h >>> +++ b/arch/arm64/include/asm/assembler.h >>> @@ -222,4 +222,19 @@ lr .req x30 // link register >>> .size __pi_##x, . - x; \ >>> ENDPROC(x) >>> >>> + .macro le16, val >>> + .byte \val & 0xff >>> + .byte (\val >> 8) & 0xff >>> + .endm >>> + >>> + .macro le32, val >>> + le16 \val >>> + le16 (\val >> 16) >>> + .endm >>> + >>> + .macro le64, val >>> + le32 \val >>> + le32 (\val >> 32) >>> + .endm >>> + >>> #endif /* __ASM_ASSEMBLER_H */ >>> diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S >>> index 350515276541..211f75e673f4 100644 >>> --- a/arch/arm64/kernel/head.S >>> +++ b/arch/arm64/kernel/head.S >>> @@ -51,6 +51,17 @@ >>> #define KERNEL_START _text >>> #define KERNEL_END _end >>> >>> +#ifdef CONFIG_CPU_BIG_ENDIAN >>> +#define __HEAD_FLAG_BE 1 >>> +#else >>> +#define __HEAD_FLAG_BE 0 >>> +#endif >>> + >>> +#define __HEAD_FLAG_PAGE_SIZE ((PAGE_SHIFT - 10) / 2) >>> + >>> +#define __HEAD_FLAGS ((__HEAD_FLAG_BE << 0) | \ >>> + (__HEAD_FLAG_PAGE_SIZE << 1)) >>> + >>> /* >>> * Kernel startup entry point. >>> * --------------------------- >>> @@ -83,9 +94,9 @@ efi_head: >>> b stext // branch to kernel start, magic >>> .long 0 // reserved >>> #endif >>> - .quad _kernel_offset_le // Image load offset from start of RAM, little-endian >>> - .quad _kernel_size_le // Effective size of kernel image, little-endian >>> - .quad _kernel_flags_le // Informative flags, little-endian >>> + le64 TEXT_OFFSET // Image load offset from start of RAM, little-endian >>> + .long _kernel_size_le, 0 // Effective size of kernel image, little-endian >>> + le64 __HEAD_FLAGS // Informative flags, little-endian >>> .quad 0 // reserved >>> .quad 0 // reserved >>> .quad 0 // reserved >>> diff --git a/arch/arm64/kernel/image.h b/arch/arm64/kernel/image.h >>> index bc2abb8b1599..bb6b0e69d0a4 100644 >>> --- a/arch/arm64/kernel/image.h >>> +++ b/arch/arm64/kernel/image.h >>> @@ -26,41 +26,26 @@ >>> * There aren't any ELF relocations we can use to endian-swap values known only >>> * at link time (e.g. the subtraction of two symbol addresses), so we must get >>> * the linker to endian-swap certain values before emitting them. >>> + * Note that this will not work for 64-bit values: these are resolved using >>> + * R_AARCH64_ABS64 relocations, which are fixed up at runtime rather than at >>> + * build time when building the PIE executable (for KASLR). >>> */ >>> #ifdef CONFIG_CPU_BIG_ENDIAN >>> -#define DATA_LE64(data) \ >>> - ((((data) & 0x00000000000000ff) << 56) | \ >>> - (((data) & 0x000000000000ff00) << 40) | \ >>> - (((data) & 0x0000000000ff0000) << 24) | \ >>> - (((data) & 0x00000000ff000000) << 8) | \ >>> - (((data) & 0x000000ff00000000) >> 8) | \ >>> - (((data) & 0x0000ff0000000000) >> 24) | \ >>> - (((data) & 0x00ff000000000000) >> 40) | \ >>> - (((data) & 0xff00000000000000) >> 56)) >>> +#define DATA_LE32(data) \ >>> + ((((data) & 0x000000ff) << 24) | \ >>> + (((data) & 0x0000ff00) << 8) | \ >>> + (((data) & 0x00ff0000) >> 8) | \ >>> + (((data) & 0xff000000) >> 24)) >>> #else >>> -#define DATA_LE64(data) ((data) & 0xffffffffffffffff) >>> +#define DATA_LE32(data) ((data) & 0xffffffff) >>> #endif >>> >>> -#ifdef CONFIG_CPU_BIG_ENDIAN >>> -#define __HEAD_FLAG_BE 1 >>> -#else >>> -#define __HEAD_FLAG_BE 0 >>> -#endif >>> - >>> -#define __HEAD_FLAG_PAGE_SIZE ((PAGE_SHIFT - 10) / 2) >>> - >>> -#define __HEAD_FLAGS ((__HEAD_FLAG_BE << 0) | \ >>> - (__HEAD_FLAG_PAGE_SIZE << 1)) >>> - >>> /* >>> * These will output as part of the Image header, which should be little-endian >>> - * regardless of the endianness of the kernel. While constant values could be >>> - * endian swapped in head.S, all are done here for consistency. >>> + * regardless of the endianness of the kernel. >>> */ >>> #define HEAD_SYMBOLS \ >>> - _kernel_size_le = DATA_LE64(_end - _text); \ >>> - _kernel_offset_le = DATA_LE64(TEXT_OFFSET); \ >>> - _kernel_flags_le = DATA_LE64(__HEAD_FLAGS); >>> + _kernel_size_le = DATA_LE32(_end - _text); >>> >>> #ifdef CONFIG_EFI >>> >>> -- >>> 2.5.0 >>>
[toc] | [prev] | [next] | [standalone]
Page 1 of 2 [1] 2 Next page →
Back to top | Article view | linux.kernel
csiph-web