Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > linux.kernel > #1299398 > unrolled thread
| Started by | Ard Biesheuvel <ard.biesheuvel@linaro.org> |
|---|---|
| First post | 2015-12-30 16:30 +0100 |
| Last post | 2015-12-30 16:40 +0100 |
| Articles | 8 — 1 participant |
Back to article view | Back to linux.kernel
[PATCH v2 00/13] arm64: implement support for KASLR Ard Biesheuvel <ard.biesheuvel@linaro.org> - 2015-12-30 16:30 +0100
[PATCH v2 13/13] arm64: efi: invoke EFI_RNG_PROTOCOL to supply KASLR randomness Ard Biesheuvel <ard.biesheuvel@linaro.org> - 2015-12-30 16:30 +0100
[PATCH v2 10/13] arm64: avoid dynamic relocations in early boot code Ard Biesheuvel <ard.biesheuvel@linaro.org> - 2015-12-30 16:30 +0100
[PATCH v2 11/13] arm64: allow kernel Image to be loaded anywhere in physical memory Ard Biesheuvel <ard.biesheuvel@linaro.org> - 2015-12-30 16:30 +0100
[PATCH v2 06/13] arm64: move kernel image to base of vmalloc area Ard Biesheuvel <ard.biesheuvel@linaro.org> - 2015-12-30 16:30 +0100
[PATCH v2 07/13] arm64: add support for module PLTs Ard Biesheuvel <ard.biesheuvel@linaro.org> - 2015-12-30 16:30 +0100
[PATCH v2 12/13] arm64: add support for relocatable kernel Ard Biesheuvel <ard.biesheuvel@linaro.org> - 2015-12-30 16:30 +0100
[PATCH v2 01/13] of/fdt: make memblock minimum physical address arch configurable Ard Biesheuvel <ard.biesheuvel@linaro.org> - 2015-12-30 16:40 +0100
| From | Ard Biesheuvel <ard.biesheuvel@linaro.org> |
|---|---|
| Date | 2015-12-30 16:30 +0100 |
| Subject | [PATCH v2 00/13] arm64: implement support for KASLR |
| Message-ID | <qLqRk-8m8-9@gated-at.bofh.it> |
This series implements KASLR for arm64, by building the kernel as a PIE
executable that can relocate itself at runtime, and moving it to a random
offset in the vmalloc area. This v2 also implements physical randomization,
i.e., it allows the kernel to deal with being loaded at any physical offset
(modulo the required alignment), and invokes the EFI_RNG_PROTOCOL from the
UEFI stub to obtain random bits and perform the actual randomization of the
physical load address.
Changes since v1/RFC:
- This series now implements fully independent virtual and physical address
randomization at load time. I have recycled some patches from this series:
http://thread.gmane.org/gmane.linux.ports.arm.kernel/455151, and updated the
final UEFI stub patch to randomize the physical address as well.
- Added a patch to deal with the way KVM on arm64 makes assumptions about the
relation between kernel symbols and the linear mapping (on which the HYP
mapping is based), as these assumptions cease to be valid once we move the
kernel Image out of the linear mapping.
- Updated the module PLT patch so it works on BE kernels as well.
- Moved the constant Image header values to head.S, and updated the linker
script to provide the kernel size using R_AARCH64_ABS32 relocation rather
than a R_AARCH64_ABS64 relocation, since those are always resolved at build
time. This allows me to get rid of the post-build perl script to swab header
values on BE kernels.
- Minor style tweaks.
Notes:
- These patches apply on top of Mark Rutland's pagetable rework series:
http://thread.gmane.org/gmane.linux.ports.arm.kernel/462438
- The arm64 Image is uncompressed by default, and the Elf64_Rela format uses
24 bytes per relocation entry. This results in considerable bloat (i.e., a
couple of MBs worth of relocation data in an .init section). However, no
build time postprocessing is required, we rely fully on the toolchain to
produce the image
- We have to rely on the bootloader to supply some randomness in register x1
upon kernel entry. Since we have no decompressor, it is simply not feasible
to collect randomness in the head.S code path before mapping the kernel and
enabling the MMU.
- The EFI_RNG_PROTOCOL that is invoked in patch #13 to supply randomness on
UEFI systems is not universally available. A QEMU/KVM firmware image that
implements a pseudo-random version is available here:
http://people.linaro.org/~ard.biesheuvel/QEMU_EFI.fd.aarch64-rng.bz2
(requires access to PMCCNTR_EL0 and support for AES instructions)
See below for instructions how to run the pseudo-random version on real
hardware.
- Only mildly tested. Help appreciated.
Code can be found here:
git://git.linaro.org/people/ard.biesheuvel/linux-arm.git arm64-kaslr-v2
https://git.linaro.org/people/ard.biesheuvel/linux-arm.git/shortlog/refs/heads/arm64-kaslr-v2
Patch #1 updates the OF code to allow the minimum memblock physical address to
be overridden by the arch.
Patch #2 introduces KIMAGE_VADDR as the base of the kernel virtual region.
Patch #3 memblock_reserve()'s the .bss, swapper_pg_dir and idmap_pg_dir
individually.
Patch #4 rewrites early_fixmap_init() so it does not rely on the linear mapping
(i.e., the use of phys_to_virt() is avoided)
Patch #5 updates KVM on arm64 so it can deal with kernel symbols whose addresses
are not covered by the linear mapping.
Patch #6 moves the kernel virtual mapping to the vmalloc area, along with the
module region which is kept right below it, as before.
Patch #7 adds support for PLTs in modules so that relative branches can be
resolved via a PLT if the target is out of range.
Patch #8 moves to the x86 version of the extable implementation so that it no
longer contains absolute addresses that require fixing up at relocation time,
but uses relative offsets instead.
Patch #9 reverts some changes to the Image header population code so we no
longer depend on the linker to populate the header fields. This is necessary
since the R_AARCH64_ABS relocations that are emitted for these fields are not
resolved at build time for PIE executables.
Patch #10 updates the code in head.S that needs to execute before relocation to
avoid the use of values that are subject to dynamic relocation. These values
will not be populated in PIE executables.
Patch #11 allows the kernel Image to be loaded anywhere in physical memory, by
decoupling PHYS_OFFSET from the base of the kernel image.
Patch #12 implements the core KASLR, by taking randomness supplied in register x1
and using it to move the kernel inside the vmalloc area.
Patch #13 adds an invocation of the EFI_RNG_PROTOCOL to supply randomness to the
kernel proper.
Ard Biesheuvel (13):
of/fdt: make memblock minimum physical address arch configurable
arm64: introduce KIMAGE_VADDR as the virtual base of the kernel region
arm64: use more granular reservations for static page table
allocations
arm64: decouple early fixmap init from linear mapping
arm64: kvm: deal with kernel symbols outside of linear mapping
arm64: move kernel image to base of vmalloc area
arm64: add support for module PLTs
arm64: use relative references in exception tables
arm64: avoid R_AARCH64_ABS64 relocations for Image header fields
arm64: avoid dynamic relocations in early boot code
arm64: allow kernel Image to be loaded anywhere in physical memory
arm64: add support for relocatable kernel
arm64: efi: invoke EFI_RNG_PROTOCOL to supply KASLR randomness
Documentation/arm64/booting.txt | 15 ++-
arch/arm/include/asm/kvm_asm.h | 2 +
arch/arm/include/asm/kvm_mmu.h | 2 +
arch/arm/kvm/arm.c | 9 +-
arch/arm/kvm/mmu.c | 12 +-
arch/arm64/Kconfig | 18 +++
arch/arm64/Makefile | 10 +-
arch/arm64/include/asm/assembler.h | 17 ++-
arch/arm64/include/asm/boot.h | 5 +
arch/arm64/include/asm/compiler.h | 2 +
arch/arm64/include/asm/futex.h | 4 +-
arch/arm64/include/asm/kasan.h | 17 +--
arch/arm64/include/asm/kernel-pgtable.h | 5 +-
arch/arm64/include/asm/kvm_asm.h | 21 +--
arch/arm64/include/asm/kvm_mmu.h | 2 +
arch/arm64/include/asm/memory.h | 37 ++++--
arch/arm64/include/asm/module.h | 11 ++
arch/arm64/include/asm/pgtable.h | 7 -
arch/arm64/include/asm/uaccess.h | 16 +--
arch/arm64/include/asm/virt.h | 4 -
arch/arm64/kernel/Makefile | 1 +
arch/arm64/kernel/armv8_deprecated.c | 4 +-
arch/arm64/kernel/efi-entry.S | 9 +-
arch/arm64/kernel/head.S | 133 ++++++++++++++++---
arch/arm64/kernel/image.h | 37 ++----
arch/arm64/kernel/module-plts.c | 137 ++++++++++++++++++++
arch/arm64/kernel/module.c | 7 +
arch/arm64/kernel/module.lds | 4 +
arch/arm64/kernel/setup.c | 15 ++-
arch/arm64/kernel/vmlinux.lds.S | 29 +++--
arch/arm64/kvm/debug.c | 4 +-
arch/arm64/mm/dump.c | 12 +-
arch/arm64/mm/extable.c | 102 ++++++++++++++-
arch/arm64/mm/init.c | 75 +++++++++--
arch/arm64/mm/mmu.c | 132 +++++++------------
drivers/firmware/efi/libstub/arm-stub.c | 1 -
drivers/firmware/efi/libstub/arm64-stub.c | 134 ++++++++++++++++---
drivers/of/fdt.c | 5 +-
include/linux/efi.h | 5 +-
scripts/sortextable.c | 6 +-
virt/kvm/arm/vgic-v3.c | 2 +-
41 files changed, 813 insertions(+), 257 deletions(-)
create mode 100644 arch/arm64/kernel/module-plts.c
create mode 100644 arch/arm64/kernel/module.lds
EFI_RNG_PROTOCOL on real hardware
=================================
To test whether your UEFI implements the EFI_RNG_PROTOCOL, download the
following executable and run it from the UEFI Shell:
http://people.linaro.org/~ard.biesheuvel/RngTest.efi
FS0:\> rngtest
UEFI RNG Protocol Testing :
----------------------------
-- Locate UEFI RNG Protocol : [Fail - Status = Not Found]
If your UEFI does not implement the EFI_RNG_PROTOCOL, you can download and
install the pseudo-random version that uses the generic timer and PMCCNTR_EL0
values and permutes them using a couple of rounds of AES.
http://people.linaro.org/~ard.biesheuvel/RngDxe.efi
NOTE: not for production!! This is a quick and dirty hack to test the KASLR
code, and is not suitable for anything else.
FS0:\> rngdxe
FS0:\> rngtest
UEFI RNG Protocol Testing :
----------------------------
-- Locate UEFI RNG Protocol : [Pass]
-- Call RNG->GetInfo() interface :
>> Supported RNG Algorithm (Count = 2) :
0) 44F0DE6E-4D8C-4045-A8C7-4DD168856B9E
1) E43176D7-B6E8-4827-B784-7FFDC4B68561
-- Call RNG->GetRNG() interface :
>> RNG with default algorithm : [Pass]
>> RNG with SP800-90-HMAC-256 : [Fail - Status = Unsupported]
>> RNG with SP800-90-Hash-256 : [Fail - Status = Unsupported]
>> RNG with SP800-90-CTR-256 : [Pass]
>> RNG with X9.31-3DES : [Fail - Status = Unsupported]
>> RNG with X9.31-AES : [Fail - Status = Unsupported]
>> RNG with RAW Entropy : [Pass]
-- Random Number Generation Test with default RNG Algorithm (20 Rounds):
01) - 27
02) - 61E8
03) - 496FD8
04) - DDD793BF
05) - B6C37C8E23
06) - 4D183C604A96
07) - 9363311DB61298
08) - 5715A7294F4E436E
09) - F0D4D7BAA0DD52318E
10) - C88C6EBCF4C0474D87C3
11) - B5594602B482A643932172
12) - CA7573F704B2089B726B9CF1
13) - A93E9451CB533DCFBA87B97C33
14) - 45AA7B83DB6044F7BBAB031F0D24
15) - 3DD7A4D61F34ADCB400B5976730DCF
16) - 4DD168D21FAB8F59708330D6A9BEB021
17) - 4BBB225E61C465F174254159467E65939F
18) - 030A156C9616337A20070941E702827DA8E1
19) - AB0FC11C9A4E225011382A9D164D9D55CA2B64
20) - 72B9B4735DC445E5DA6AF88DE965B7E87CB9A23C
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[toc] | [next] | [standalone]
| From | Ard Biesheuvel <ard.biesheuvel@linaro.org> |
|---|---|
| Date | 2015-12-30 16:30 +0100 |
| Subject | [PATCH v2 13/13] arm64: efi: invoke EFI_RNG_PROTOCOL to supply KASLR randomness |
| Message-ID | <qLqRl-8m8-29@gated-at.bofh.it> |
| In reply to | #1299398 |
Since arm64 does not use a decompressor that supplies an execution
environment where it is feasible to some extent to provide a source of
randomness, the arm64 KASLR kernel depends on the bootloader to supply
some random bits in register x1 upon kernel entry.
On UEFI systems, we can use the EFI_RNG_PROTOCOL, if supplied, to obtain
some random bits. At the same time, use it to randomize the offset of the
kernel Image in physical memory.
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
arch/arm64/kernel/efi-entry.S | 7 +-
drivers/firmware/efi/libstub/arm-stub.c | 1 -
drivers/firmware/efi/libstub/arm64-stub.c | 134 +++++++++++++++++---
include/linux/efi.h | 5 +-
4 files changed, 127 insertions(+), 20 deletions(-)
diff --git a/arch/arm64/kernel/efi-entry.S b/arch/arm64/kernel/efi-entry.S
index f82036e02485..f41073dde7e0 100644
--- a/arch/arm64/kernel/efi-entry.S
+++ b/arch/arm64/kernel/efi-entry.S
@@ -110,7 +110,7 @@ ENTRY(entry)
2:
/* Jump to kernel entry point */
mov x0, x20
- mov x1, xzr
+ ldr x1, efi_rnd
mov x2, xzr
mov x3, xzr
br x21
@@ -119,6 +119,9 @@ efi_load_fail:
mov x0, #EFI_LOAD_ERROR
ldp x29, x30, [sp], #32
ret
+ENDPROC(entry)
+
+ENTRY(efi_rnd)
+ .quad 0, 0
entry_end:
-ENDPROC(entry)
diff --git a/drivers/firmware/efi/libstub/arm-stub.c b/drivers/firmware/efi/libstub/arm-stub.c
index 950c87f5d279..f580bcdfae4f 100644
--- a/drivers/firmware/efi/libstub/arm-stub.c
+++ b/drivers/firmware/efi/libstub/arm-stub.c
@@ -145,7 +145,6 @@ void efi_char16_printk(efi_system_table_t *sys_table_arg,
out->output_string(out, str);
}
-
/*
* This function handles the architcture specific differences between arm and
* arm64 regarding where the kernel image must be loaded and any memory that
diff --git a/drivers/firmware/efi/libstub/arm64-stub.c b/drivers/firmware/efi/libstub/arm64-stub.c
index 78dfbd34b6bf..4e5c306346b4 100644
--- a/drivers/firmware/efi/libstub/arm64-stub.c
+++ b/drivers/firmware/efi/libstub/arm64-stub.c
@@ -13,6 +13,68 @@
#include <asm/efi.h>
#include <asm/sections.h>
+struct efi_rng_protocol_t {
+ efi_status_t (*get_info)(struct efi_rng_protocol_t *,
+ unsigned long *,
+ efi_guid_t *);
+ efi_status_t (*get_rng)(struct efi_rng_protocol_t *,
+ efi_guid_t *,
+ unsigned long,
+ u8 *out);
+};
+
+extern struct {
+ u64 virt_seed;
+ u64 phys_seed;
+} efi_rnd;
+
+static int efi_get_random_bytes(efi_system_table_t *sys_table)
+{
+ efi_guid_t rng_proto = EFI_RNG_PROTOCOL_GUID;
+ efi_status_t status;
+ struct efi_rng_protocol_t *rng;
+
+ status = sys_table->boottime->locate_protocol(&rng_proto, NULL,
+ (void **)&rng);
+ if (status == EFI_NOT_FOUND) {
+ pr_efi(sys_table, "EFI_RNG_PROTOCOL unavailable, no randomness supplied\n");
+ return EFI_SUCCESS;
+ }
+
+ if (status != EFI_SUCCESS)
+ return status;
+
+ return rng->get_rng(rng, NULL, sizeof(efi_rnd), (u8 *)&efi_rnd);
+}
+
+static efi_status_t get_dram_top(efi_system_table_t *sys_table_arg, u64 *top)
+{
+ unsigned long map_size, desc_size;
+ efi_memory_desc_t *memory_map;
+ efi_status_t status;
+ int l;
+
+ status = efi_get_memory_map(sys_table_arg, &memory_map, &map_size,
+ &desc_size, NULL, NULL);
+ if (status != EFI_SUCCESS)
+ return status;
+
+ for (l = 0; l < map_size; l += desc_size) {
+ efi_memory_desc_t *md = (void *)memory_map + l;
+
+ if (md->attribute & EFI_MEMORY_WB) {
+ u64 phys_end = md->phys_addr +
+ md->num_pages * EFI_PAGE_SIZE;
+ if (phys_end > *top)
+ *top = phys_end;
+ }
+ }
+
+ efi_call_early(free_pool, memory_map);
+
+ return EFI_SUCCESS;
+}
+
efi_status_t __init handle_kernel_image(efi_system_table_t *sys_table_arg,
unsigned long *image_addr,
unsigned long *image_size,
@@ -27,6 +89,14 @@ efi_status_t __init handle_kernel_image(efi_system_table_t *sys_table_arg,
void *old_image_addr = (void *)*image_addr;
unsigned long preferred_offset;
+ if (IS_ENABLED(CONFIG_ARM64_RELOCATABLE_KERNEL)) {
+ status = efi_get_random_bytes(sys_table_arg);
+ if (status != EFI_SUCCESS) {
+ pr_efi_err(sys_table_arg, "efi_get_random_bytes() failed\n");
+ return status;
+ }
+ }
+
/*
* The preferred offset of the kernel Image is TEXT_OFFSET bytes beyond
* a 2 MB aligned base, which itself may be lower than dram_base, as
@@ -36,13 +106,42 @@ efi_status_t __init handle_kernel_image(efi_system_table_t *sys_table_arg,
if (preferred_offset < dram_base)
preferred_offset += SZ_2M;
- /* Relocate the image, if required. */
kernel_size = _edata - _text;
- if (*image_addr != preferred_offset) {
- kernel_memsize = kernel_size + (_end - _edata);
+ kernel_memsize = kernel_size + (_end - _edata);
+
+ if (IS_ENABLED(CONFIG_ARM64_RELOCATABLE_KERNEL) && efi_rnd.phys_seed) {
+ /*
+ * If KASLR is enabled, and we have some randomness available,
+ * locate the kernel at a randomized offset in physical memory.
+ */
+ u64 dram_top = dram_base;
+
+ status = get_dram_top(sys_table_arg, &dram_top);
+ if (status != EFI_SUCCESS) {
+ pr_efi_err(sys_table_arg, "get_dram_size() failed\n");
+ return status;
+ }
+
+ kernel_memsize += SZ_2M;
+ nr_pages = round_up(kernel_memsize, EFI_ALLOC_ALIGN) /
+ EFI_PAGE_SIZE;
/*
- * First, try a straight allocation at the preferred offset.
+ * Use the random seed to scale the size and add it to the DRAM
+ * base. Note that this may give suboptimal results on systems
+ * with discontiguous DRAM regions with large holes between them.
+ */
+ *reserve_addr = dram_base +
+ ((dram_top - dram_base) >> 16) * (u16)efi_rnd.phys_seed;
+
+ status = efi_call_early(allocate_pages, EFI_ALLOCATE_MAX_ADDRESS,
+ EFI_LOADER_DATA, nr_pages,
+ (efi_physical_addr_t *)reserve_addr);
+
+ *image_addr = round_up(*reserve_addr, SZ_2M) + TEXT_OFFSET;
+ } else {
+ /*
+ * Else, try a straight allocation at the preferred offset.
* This will work around the issue where, if dram_base == 0x0,
* efi_low_alloc() refuses to allocate at 0x0 (to prevent the
* address of the allocation to be mistaken for a FAIL return
@@ -52,27 +151,30 @@ efi_status_t __init handle_kernel_image(efi_system_table_t *sys_table_arg,
* Mustang), we can still place the kernel at the address
* 'dram_base + TEXT_OFFSET'.
*/
+ if (*image_addr == preferred_offset)
+ return EFI_SUCCESS;
+
*image_addr = *reserve_addr = preferred_offset;
nr_pages = round_up(kernel_memsize, EFI_ALLOC_ALIGN) /
EFI_PAGE_SIZE;
status = efi_call_early(allocate_pages, EFI_ALLOCATE_ADDRESS,
EFI_LOADER_DATA, nr_pages,
(efi_physical_addr_t *)reserve_addr);
+ }
+
+ if (status != EFI_SUCCESS) {
+ kernel_memsize += TEXT_OFFSET;
+ status = efi_low_alloc(sys_table_arg, kernel_memsize,
+ SZ_2M, reserve_addr);
+
if (status != EFI_SUCCESS) {
- kernel_memsize += TEXT_OFFSET;
- status = efi_low_alloc(sys_table_arg, kernel_memsize,
- SZ_2M, reserve_addr);
-
- if (status != EFI_SUCCESS) {
- pr_efi_err(sys_table_arg, "Failed to relocate kernel\n");
- return status;
- }
- *image_addr = *reserve_addr + TEXT_OFFSET;
+ pr_efi_err(sys_table_arg, "Failed to relocate kernel\n");
+ return status;
}
- memcpy((void *)*image_addr, old_image_addr, kernel_size);
- *reserve_size = kernel_memsize;
+ *image_addr = *reserve_addr + TEXT_OFFSET;
}
-
+ memcpy((void *)*image_addr, old_image_addr, kernel_size);
+ *reserve_size = kernel_memsize;
return EFI_SUCCESS;
}
diff --git a/include/linux/efi.h b/include/linux/efi.h
index 569b5a866bb1..13783fdc9bdd 100644
--- a/include/linux/efi.h
+++ b/include/linux/efi.h
@@ -299,7 +299,7 @@ typedef struct {
void *open_protocol_information;
void *protocols_per_handle;
void *locate_handle_buffer;
- void *locate_protocol;
+ efi_status_t (*locate_protocol)(efi_guid_t *, void *, void **);
void *install_multiple_protocol_interfaces;
void *uninstall_multiple_protocol_interfaces;
void *calculate_crc32;
@@ -599,6 +599,9 @@ void efi_native_runtime_setup(void);
#define EFI_PROPERTIES_TABLE_GUID \
EFI_GUID( 0x880aaca3, 0x4adc, 0x4a04, 0x90, 0x79, 0xb7, 0x47, 0x34, 0x08, 0x25, 0xe5 )
+#define EFI_RNG_PROTOCOL_GUID \
+ EFI_GUID( 0x3152bca5, 0xeade, 0x433d, 0x86, 0x2e, 0xc0, 0x1c, 0xdc, 0x29, 0x1f, 0x44 )
+
typedef struct {
efi_guid_t guid;
u64 table;
--
2.5.0
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[toc] | [prev] | [next] | [standalone]
| From | Ard Biesheuvel <ard.biesheuvel@linaro.org> |
|---|---|
| Date | 2015-12-30 16:30 +0100 |
| Subject | [PATCH v2 10/13] arm64: avoid dynamic relocations in early boot code |
| Message-ID | <qLqRl-8m8-25@gated-at.bofh.it> |
| In reply to | #1299398 |
Before implementing KASLR for arm64 by building a self-relocating PIE executable, we have to ensure that values we use before the relocation routine is executed are not subject to dynamic relocation themselves. This applies not only to virtual addresses, but also to values that are supplied by the linker at build time and relocated using R_AARCH64_ABS64 relocations. So instead, use assemble time constants, or force the use of static relocations by folding the constants into the instructions. Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org> --- arch/arm64/kernel/efi-entry.S | 2 +- arch/arm64/kernel/head.S | 39 +++++++++++++------- 2 files changed, 27 insertions(+), 14 deletions(-) diff --git a/arch/arm64/kernel/efi-entry.S b/arch/arm64/kernel/efi-entry.S index a773db92908b..f82036e02485 100644 --- a/arch/arm64/kernel/efi-entry.S +++ b/arch/arm64/kernel/efi-entry.S @@ -61,7 +61,7 @@ ENTRY(entry) */ mov x20, x0 // DTB address ldr x0, [sp, #16] // relocated _text address - ldr x21, =stext_offset + movz x21, #:abs_g0:stext_offset add x21, x0, x21 /* diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S index ccbb1bd46026..1230fa93fd8c 100644 --- a/arch/arm64/kernel/head.S +++ b/arch/arm64/kernel/head.S @@ -78,12 +78,11 @@ * in the entry routines. */ __HEAD - +_head: /* * DO NOT MODIFY. Image header expected by Linux boot-loaders. */ #ifdef CONFIG_EFI -efi_head: /* * This add instruction has no meaningful effect except that * its opcode forms the magic "MZ" signature required by UEFI. @@ -105,14 +104,14 @@ efi_head: .byte 0x4d .byte 0x64 #ifdef CONFIG_EFI - .long pe_header - efi_head // Offset to the PE header. + .long pe_header - _head // Offset to the PE header. #else .word 0 // reserved #endif #ifdef CONFIG_EFI .globl __efistub_stext_offset - .set __efistub_stext_offset, stext - efi_head + .set __efistub_stext_offset, stext - _head .align 3 pe_header: .ascii "PE" @@ -135,7 +134,7 @@ optional_header: .long _end - stext // SizeOfCode .long 0 // SizeOfInitializedData .long 0 // SizeOfUninitializedData - .long __efistub_entry - efi_head // AddressOfEntryPoint + .long __efistub_entry - _head // AddressOfEntryPoint .long __efistub_stext_offset // BaseOfCode extra_header_fields: @@ -150,7 +149,7 @@ extra_header_fields: .short 0 // MinorSubsystemVersion .long 0 // Win32VersionValue - .long _end - efi_head // SizeOfImage + .long _end - _head // SizeOfImage // Everything before the kernel image is considered part of the header .long __efistub_stext_offset // SizeOfHeaders @@ -230,11 +229,13 @@ ENTRY(stext) * On return, the CPU will be ready for the MMU to be turned on and * the TCR will have been set. */ - ldr x27, =__mmap_switched // address to jump to after + ldr x27, 0f // address to jump to after // MMU has been enabled adr_l lr, __enable_mmu // return (PIC) address b __cpu_setup // initialise processor ENDPROC(stext) + .align 3 +0: .quad __mmap_switched - (_head - TEXT_OFFSET) + KIMAGE_VADDR /* * Preserve the arguments passed by the bootloader in x0 .. x3 @@ -402,7 +403,8 @@ __create_page_tables: mov x0, x26 // swapper_pg_dir ldr x5, =KIMAGE_VADDR create_pgd_entry x0, x5, x3, x6 - ldr x6, =KERNEL_END // __va(KERNEL_END) + ldr w6, kernel_img_size + add x6, x6, x5 mov x3, x24 // phys offset create_block_map x0, x7, x3, x5, x6 @@ -419,6 +421,9 @@ __create_page_tables: mov lr, x27 ret ENDPROC(__create_page_tables) + +kernel_img_size: + .long _end - (_head - TEXT_OFFSET) .ltorg /* @@ -426,6 +431,10 @@ ENDPROC(__create_page_tables) */ .set initial_sp, init_thread_union + THREAD_START_SP __mmap_switched: + adr_l x8, vectors // load VBAR_EL1 with + msr vbar_el1, x8 // relocated address + isb + adr_l x6, __bss_start adr_l x7, __bss_stop @@ -609,13 +618,19 @@ ENTRY(secondary_startup) adrp x26, swapper_pg_dir bl __cpu_setup // initialise processor - ldr x21, =secondary_data - ldr x27, =__secondary_switched // address to jump to after enabling the MMU + ldr x8, =KIMAGE_VADDR + ldr w9, 0f + sub x27, x8, w9, sxtw // address to jump to after enabling the MMU b __enable_mmu ENDPROC(secondary_startup) +0: .long (_text - TEXT_OFFSET) - __secondary_switched ENTRY(__secondary_switched) - ldr x0, [x21] // get secondary_data.stack + adr_l x5, vectors + msr vbar_el1, x5 + isb + + ldr_l x0, secondary_data // get secondary_data.stack mov sp, x0 mov x29, #0 b secondary_start_kernel @@ -638,8 +653,6 @@ __enable_mmu: ubfx x2, x1, #ID_AA64MMFR0_TGRAN_SHIFT, 4 cmp x2, #ID_AA64MMFR0_TGRAN_SUPPORTED b.ne __no_granule_support - ldr x5, =vectors - msr vbar_el1, x5 msr ttbr0_el1, x25 // load TTBR0 msr ttbr1_el1, x26 // load TTBR1 isb -- 2.5.0 -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
[toc] | [prev] | [next] | [standalone]
| From | Ard Biesheuvel <ard.biesheuvel@linaro.org> |
|---|---|
| Date | 2015-12-30 16:30 +0100 |
| Subject | [PATCH v2 11/13] arm64: allow kernel Image to be loaded anywhere in physical memory |
| Message-ID | <qLqRl-8m8-31@gated-at.bofh.it> |
| In reply to | #1299398 |
This relaxes the kernel Image placement requirements, so that it
may be placed at any 2 MB aligned offset in physical memory.
This is accomplished by ignoring PHYS_OFFSET when installing
memblocks, and accounting for the apparent virtual offset of
the kernel Image. As a result, virtual address references
below PAGE_OFFSET are correctly mapped onto physical references
into the kernel Image regardless of where it sits in memory.
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
Documentation/arm64/booting.txt | 12 ++---
arch/arm64/include/asm/boot.h | 5 ++
arch/arm64/include/asm/kvm_mmu.h | 2 +-
arch/arm64/include/asm/memory.h | 15 +++---
arch/arm64/kernel/head.S | 6 ++-
arch/arm64/mm/init.c | 50 +++++++++++++++++++-
arch/arm64/mm/mmu.c | 12 +++++
7 files changed, 86 insertions(+), 16 deletions(-)
diff --git a/Documentation/arm64/booting.txt b/Documentation/arm64/booting.txt
index 701d39d3171a..03e02ebc1b0c 100644
--- a/Documentation/arm64/booting.txt
+++ b/Documentation/arm64/booting.txt
@@ -117,14 +117,14 @@ Header notes:
depending on selected features, and is effectively unbound.
The Image must be placed text_offset bytes from a 2MB aligned base
-address near the start of usable system RAM and called there. Memory
-below that base address is currently unusable by Linux, and therefore it
-is strongly recommended that this location is the start of system RAM.
-The region between the 2 MB aligned base address and the start of the
-image has no special significance to the kernel, and may be used for
-other purposes.
+address anywhere in usable system RAM and called there. The region
+between the 2 MB aligned base address and the start of the image has no
+special significance to the kernel, and may be used for other purposes.
At least image_size bytes from the start of the image must be free for
use by the kernel.
+NOTE: versions prior to v4.6 cannot make use of memory below the
+physical offset of the Image so it is recommended that the Image be
+placed as close as possible to the start of system RAM.
Any memory described to the kernel (even that below the start of the
image) which is not marked as reserved from the kernel (e.g., with a
diff --git a/arch/arm64/include/asm/boot.h b/arch/arm64/include/asm/boot.h
index 81151b67b26b..984cb0fa61ce 100644
--- a/arch/arm64/include/asm/boot.h
+++ b/arch/arm64/include/asm/boot.h
@@ -11,4 +11,9 @@
#define MIN_FDT_ALIGN 8
#define MAX_FDT_SIZE SZ_2M
+/*
+ * arm64 requires the kernel image to be 2 MB aligned
+ */
+#define MIN_KIMG_ALIGN SZ_2M
+
#endif
diff --git a/arch/arm64/include/asm/kvm_mmu.h b/arch/arm64/include/asm/kvm_mmu.h
index 0899026a2821..7e9516365b76 100644
--- a/arch/arm64/include/asm/kvm_mmu.h
+++ b/arch/arm64/include/asm/kvm_mmu.h
@@ -73,7 +73,7 @@
#define KERN_TO_HYP(kva) ((unsigned long)kva - PAGE_OFFSET + HYP_PAGE_OFFSET)
-#define kvm_ksym_ref(sym) ((void *)&sym - KIMAGE_VADDR + PAGE_OFFSET)
+#define kvm_ksym_ref(sym) phys_to_virt((u64)&sym - kimage_voffset)
/*
* We currently only support a 40bit IPA.
diff --git a/arch/arm64/include/asm/memory.h b/arch/arm64/include/asm/memory.h
index 1dcbf142d36c..557228658666 100644
--- a/arch/arm64/include/asm/memory.h
+++ b/arch/arm64/include/asm/memory.h
@@ -88,10 +88,10 @@
#define __virt_to_phys(x) ({ \
phys_addr_t __x = (phys_addr_t)(x); \
__x >= PAGE_OFFSET ? (__x - PAGE_OFFSET + PHYS_OFFSET) : \
- (__x - KIMAGE_VADDR + PHYS_OFFSET); })
+ (__x - kimage_voffset); })
#define __phys_to_virt(x) ((unsigned long)((x) - PHYS_OFFSET + PAGE_OFFSET))
-#define __phys_to_kimg(x) ((unsigned long)((x) - PHYS_OFFSET + KIMAGE_VADDR))
+#define __phys_to_kimg(x) ((unsigned long)((x) + kimage_voffset))
/*
* Convert a page to/from a physical address
@@ -121,13 +121,14 @@ extern phys_addr_t memstart_addr;
/* PHYS_OFFSET - the physical address of the start of memory. */
#define PHYS_OFFSET ({ memstart_addr; })
+/* the offset between the kernel virtual and physical mappings */
+extern u64 kimage_voffset;
+
/*
- * The maximum physical address that the linear direct mapping
- * of system RAM can cover. (PAGE_OFFSET can be interpreted as
- * a 2's complement signed quantity and negated to derive the
- * maximum size of the linear mapping.)
+ * Allow all memory at the discovery stage. We will clip it later.
*/
-#define MAX_MEMBLOCK_ADDR ({ memstart_addr - PAGE_OFFSET - 1; })
+#define MIN_MEMBLOCK_ADDR 0
+#define MAX_MEMBLOCK_ADDR U64_MAX
/*
* PFNs are used to describe any physical page; this means
diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S
index 1230fa93fd8c..01a33e42ed70 100644
--- a/arch/arm64/kernel/head.S
+++ b/arch/arm64/kernel/head.S
@@ -445,7 +445,11 @@ __mmap_switched:
2:
adr_l sp, initial_sp, x4
str_l x21, __fdt_pointer, x5 // Save FDT pointer
- str_l x24, memstart_addr, x6 // Save PHYS_OFFSET
+
+ ldr x0, =KIMAGE_VADDR // Save the offset between
+ sub x24, x0, x24 // the kernel virtual and
+ str_l x24, kimage_voffset, x0 // physical mappings
+
mov x29, #0
#ifdef CONFIG_KASAN
bl kasan_early_init
diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c
index 2cfc9c54bf51..6aafe15c7754 100644
--- a/arch/arm64/mm/init.c
+++ b/arch/arm64/mm/init.c
@@ -35,6 +35,7 @@
#include <linux/efi.h>
#include <linux/swiotlb.h>
+#include <asm/boot.h>
#include <asm/fixmap.h>
#include <asm/kernel-pgtable.h>
#include <asm/memory.h>
@@ -158,9 +159,56 @@ static int __init early_mem(char *p)
}
early_param("mem", early_mem);
+static void __init enforce_memory_limit(void)
+{
+ const phys_addr_t kbase = round_down(__pa(_text), MIN_KIMG_ALIGN);
+ u64 to_remove = memblock_phys_mem_size() - memory_limit;
+ phys_addr_t max_addr = 0;
+ struct memblock_region *r;
+
+ if (memory_limit == (phys_addr_t)ULLONG_MAX)
+ return;
+
+ /*
+ * The kernel may be high up in physical memory, so try to apply the
+ * limit below the kernel first, and only let the generic handling
+ * take over if it turns out we haven't clipped enough memory yet.
+ */
+ for_each_memblock(memory, r) {
+ if (r->base + r->size > kbase) {
+ u64 rem = min(to_remove, kbase - r->base);
+
+ max_addr = r->base + rem;
+ to_remove -= rem;
+ break;
+ }
+ if (to_remove <= r->size) {
+ max_addr = r->base + to_remove;
+ to_remove = 0;
+ break;
+ }
+ to_remove -= r->size;
+ }
+
+ memblock_remove(0, max_addr);
+
+ if (to_remove)
+ memblock_enforce_memory_limit(memory_limit);
+}
+
void __init arm64_memblock_init(void)
{
- memblock_enforce_memory_limit(memory_limit);
+ /*
+ * Remove the memory that we will not be able to cover
+ * with the linear mapping.
+ */
+ const s64 linear_region_size = -(s64)PAGE_OFFSET;
+
+ memblock_remove(round_down(memblock_start_of_DRAM(),
+ 1 << SWAPPER_TABLE_SHIFT) +
+ linear_region_size, ULLONG_MAX);
+
+ enforce_memory_limit();
/*
* Register the kernel text, kernel data, initrd, and initial
diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c
index 6275d183c005..10067385e40f 100644
--- a/arch/arm64/mm/mmu.c
+++ b/arch/arm64/mm/mmu.c
@@ -44,6 +44,9 @@
u64 idmap_t0sz = TCR_T0SZ(VA_BITS);
+u64 kimage_voffset __read_mostly;
+EXPORT_SYMBOL(kimage_voffset);
+
/*
* Empty_zero_page is a special page that is used for zero-initialized data
* and COW.
@@ -326,6 +329,15 @@ static void __init map_mem(pgd_t *pgd)
{
struct memblock_region *reg;
+ /*
+ * Select a suitable value for the base of physical memory.
+ * This should be equal to or below the lowest usable physical
+ * memory address, and aligned to PUD/PMD size so that we can map
+ * it efficiently.
+ */
+ memstart_addr = round_down(memblock_start_of_DRAM(),
+ 1 << SWAPPER_TABLE_SHIFT);
+
/* map all the memory banks */
for_each_memblock(memory, reg) {
phys_addr_t start = reg->base;
--
2.5.0
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[toc] | [prev] | [next] | [standalone]
| From | Ard Biesheuvel <ard.biesheuvel@linaro.org> |
|---|---|
| Date | 2015-12-30 16:30 +0100 |
| Subject | [PATCH v2 06/13] arm64: move kernel image to base of vmalloc area |
| Message-ID | <qLqRl-8m8-37@gated-at.bofh.it> |
| In reply to | #1299398 |
This moves the module area to right before the vmalloc area, and
moves the kernel image to the base of the vmalloc area. This is
an intermediate step towards implementing kASLR, where the kernel
image can be located anywhere in the vmalloc area.
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
arch/arm64/include/asm/kasan.h | 17 ++++-----
arch/arm64/include/asm/kernel-pgtable.h | 5 +--
arch/arm64/include/asm/memory.h | 17 ++++++---
arch/arm64/include/asm/pgtable.h | 7 ----
arch/arm64/kernel/setup.c | 13 +++++++
arch/arm64/mm/dump.c | 12 +++----
arch/arm64/mm/init.c | 20 +++++------
arch/arm64/mm/mmu.c | 37 ++------------------
8 files changed, 56 insertions(+), 72 deletions(-)
diff --git a/arch/arm64/include/asm/kasan.h b/arch/arm64/include/asm/kasan.h
index 2774fa384c47..476d56e0f04c 100644
--- a/arch/arm64/include/asm/kasan.h
+++ b/arch/arm64/include/asm/kasan.h
@@ -1,19 +1,16 @@
#ifndef __ASM_KASAN_H
#define __ASM_KASAN_H
-#ifndef __ASSEMBLY__
-
#ifdef CONFIG_KASAN
#include <linux/linkage.h>
-#include <asm/memory.h>
/*
* KASAN_SHADOW_START: beginning of the kernel virtual addresses.
* KASAN_SHADOW_END: KASAN_SHADOW_START + 1/8 of kernel virtual addresses.
*/
-#define KASAN_SHADOW_START (VA_START)
-#define KASAN_SHADOW_END (KASAN_SHADOW_START + (1UL << (VA_BITS - 3)))
+#define KASAN_SHADOW_START (VA_START)
+#define KASAN_SHADOW_END (KASAN_SHADOW_START + (_AC(1, UL) << (VA_BITS - 3)))
/*
* This value is used to map an address to the corresponding shadow
@@ -25,14 +22,18 @@
* should satisfy the following equation:
* KASAN_SHADOW_OFFSET = KASAN_SHADOW_END - (1ULL << 61)
*/
-#define KASAN_SHADOW_OFFSET (KASAN_SHADOW_END - (1ULL << (64 - 3)))
+#define KASAN_SHADOW_OFFSET (KASAN_SHADOW_END - (_AC(1, ULL) << (64 - 3)))
+#ifndef __ASSEMBLY__
void kasan_init(void);
asmlinkage void kasan_early_init(void);
+#endif
#else
+
+#ifndef __ASSEMBLY__
static inline void kasan_init(void) { }
#endif
-#endif
-#endif
+#endif /* CONFIG_KASAN */
+#endif /* __ASM_KASAN_H */
diff --git a/arch/arm64/include/asm/kernel-pgtable.h b/arch/arm64/include/asm/kernel-pgtable.h
index a459714ee29e..daa8a7b9917a 100644
--- a/arch/arm64/include/asm/kernel-pgtable.h
+++ b/arch/arm64/include/asm/kernel-pgtable.h
@@ -70,8 +70,9 @@
/*
* Initial memory map attributes.
*/
-#define SWAPPER_PTE_FLAGS (PTE_TYPE_PAGE | PTE_AF | PTE_SHARED)
-#define SWAPPER_PMD_FLAGS (PMD_TYPE_SECT | PMD_SECT_AF | PMD_SECT_S)
+#define SWAPPER_PTE_FLAGS (PTE_TYPE_PAGE | PTE_AF | PTE_SHARED | PTE_UXN)
+#define SWAPPER_PMD_FLAGS (PMD_TYPE_SECT | PMD_SECT_AF | PMD_SECT_S | \
+ PMD_SECT_UXN)
#if ARM64_SWAPPER_USES_SECTION_MAPS
#define SWAPPER_MM_MMUFLAGS (PMD_ATTRINDX(MT_NORMAL) | SWAPPER_PMD_FLAGS)
diff --git a/arch/arm64/include/asm/memory.h b/arch/arm64/include/asm/memory.h
index bea9631b34a8..1dcbf142d36c 100644
--- a/arch/arm64/include/asm/memory.h
+++ b/arch/arm64/include/asm/memory.h
@@ -51,14 +51,23 @@
#define VA_BITS (CONFIG_ARM64_VA_BITS)
#define VA_START (UL(0xffffffffffffffff) << VA_BITS)
#define PAGE_OFFSET (UL(0xffffffffffffffff) << (VA_BITS - 1))
-#define KIMAGE_VADDR (PAGE_OFFSET)
-#define MODULES_END (KIMAGE_VADDR)
-#define MODULES_VADDR (MODULES_END - SZ_64M)
-#define PCI_IO_END (MODULES_VADDR - SZ_2M)
+#define PCI_IO_END (PAGE_OFFSET - SZ_2M)
#define PCI_IO_START (PCI_IO_END - PCI_IO_SIZE)
#define FIXADDR_TOP (PCI_IO_START - SZ_2M)
#define TASK_SIZE_64 (UL(1) << VA_BITS)
+#ifndef CONFIG_KASAN
+#define MODULES_VADDR (VA_START)
+#else
+#include <asm/kasan.h>
+#define MODULES_VADDR (KASAN_SHADOW_END)
+#endif
+
+#define MODULES_END (MODULES_VADDR + SZ_64M)
+
+#define KIMAGE_VADDR (MODULES_END)
+#define VMALLOC_START (MODULES_END)
+
#ifdef CONFIG_COMPAT
#define TASK_SIZE_32 UL(0x100000000)
#define TASK_SIZE (test_thread_flag(TIF_32BIT) ? \
diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgtable.h
index 0664468466fb..93203a6b9574 100644
--- a/arch/arm64/include/asm/pgtable.h
+++ b/arch/arm64/include/asm/pgtable.h
@@ -42,13 +42,6 @@
*/
#define VMEMMAP_SIZE ALIGN((1UL << (VA_BITS - PAGE_SHIFT)) * sizeof(struct page), PUD_SIZE)
-#ifndef CONFIG_KASAN
-#define VMALLOC_START (VA_START)
-#else
-#include <asm/kasan.h>
-#define VMALLOC_START (KASAN_SHADOW_END + SZ_64K)
-#endif
-
#define VMALLOC_END (PAGE_OFFSET - PUD_SIZE - VMEMMAP_SIZE - SZ_64K)
#define vmemmap ((struct page *)(VMALLOC_END + SZ_64K))
diff --git a/arch/arm64/kernel/setup.c b/arch/arm64/kernel/setup.c
index cfed56f0ad26..96177a7c0f05 100644
--- a/arch/arm64/kernel/setup.c
+++ b/arch/arm64/kernel/setup.c
@@ -53,6 +53,7 @@
#include <asm/cpufeature.h>
#include <asm/cpu_ops.h>
#include <asm/kasan.h>
+#include <asm/kernel-pgtable.h>
#include <asm/sections.h>
#include <asm/setup.h>
#include <asm/smp_plat.h>
@@ -291,6 +292,18 @@ u64 __cpu_logical_map[NR_CPUS] = { [0 ... NR_CPUS-1] = INVALID_HWID };
void __init setup_arch(char **cmdline_p)
{
+ static struct vm_struct vmlinux_vm __initdata = {
+ .addr = (void *)KIMAGE_VADDR,
+ .size = 0,
+ .flags = VM_IOREMAP,
+ .caller = setup_arch,
+ };
+
+ vmlinux_vm.size = round_up((unsigned long)_end - KIMAGE_VADDR,
+ 1 << SWAPPER_BLOCK_SHIFT);
+ vmlinux_vm.phys_addr = __pa(KIMAGE_VADDR);
+ vm_area_add_early(&vmlinux_vm);
+
pr_info("Boot CPU: AArch64 Processor [%08x]\n", read_cpuid_id());
sprintf(init_utsname()->machine, ELF_PLATFORM);
diff --git a/arch/arm64/mm/dump.c b/arch/arm64/mm/dump.c
index 5a22a119a74c..e83ffb00560c 100644
--- a/arch/arm64/mm/dump.c
+++ b/arch/arm64/mm/dump.c
@@ -35,7 +35,9 @@ struct addr_marker {
};
enum address_markers_idx {
- VMALLOC_START_NR = 0,
+ MODULES_START_NR = 0,
+ MODULES_END_NR,
+ VMALLOC_START_NR,
VMALLOC_END_NR,
#ifdef CONFIG_SPARSEMEM_VMEMMAP
VMEMMAP_START_NR,
@@ -45,12 +47,12 @@ enum address_markers_idx {
FIXADDR_END_NR,
PCI_START_NR,
PCI_END_NR,
- MODULES_START_NR,
- MODUELS_END_NR,
KERNEL_SPACE_NR,
};
static struct addr_marker address_markers[] = {
+ { MODULES_VADDR, "Modules start" },
+ { MODULES_END, "Modules end" },
{ VMALLOC_START, "vmalloc() Area" },
{ VMALLOC_END, "vmalloc() End" },
#ifdef CONFIG_SPARSEMEM_VMEMMAP
@@ -61,9 +63,7 @@ static struct addr_marker address_markers[] = {
{ FIXADDR_TOP, "Fixmap end" },
{ PCI_IO_START, "PCI I/O start" },
{ PCI_IO_END, "PCI I/O end" },
- { MODULES_VADDR, "Modules start" },
- { MODULES_END, "Modules end" },
- { PAGE_OFFSET, "Kernel Mapping" },
+ { PAGE_OFFSET, "Linear Mapping" },
{ -1, NULL },
};
diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c
index 8e678d05ad84..2cfc9c54bf51 100644
--- a/arch/arm64/mm/init.c
+++ b/arch/arm64/mm/init.c
@@ -305,22 +305,26 @@ void __init mem_init(void)
#ifdef CONFIG_KASAN
" kasan : 0x%16lx - 0x%16lx (%6ld GB)\n"
#endif
+ " modules : 0x%16lx - 0x%16lx (%6ld MB)\n"
" vmalloc : 0x%16lx - 0x%16lx (%6ld GB)\n"
+ " .init : 0x%p" " - 0x%p" " (%6ld KB)\n"
+ " .text : 0x%p" " - 0x%p" " (%6ld KB)\n"
+ " .data : 0x%p" " - 0x%p" " (%6ld KB)\n"
#ifdef CONFIG_SPARSEMEM_VMEMMAP
" vmemmap : 0x%16lx - 0x%16lx (%6ld GB maximum)\n"
" 0x%16lx - 0x%16lx (%6ld MB actual)\n"
#endif
" fixed : 0x%16lx - 0x%16lx (%6ld KB)\n"
" PCI I/O : 0x%16lx - 0x%16lx (%6ld MB)\n"
- " modules : 0x%16lx - 0x%16lx (%6ld MB)\n"
- " memory : 0x%16lx - 0x%16lx (%6ld MB)\n"
- " .init : 0x%p" " - 0x%p" " (%6ld KB)\n"
- " .text : 0x%p" " - 0x%p" " (%6ld KB)\n"
- " .data : 0x%p" " - 0x%p" " (%6ld KB)\n",
+ " memory : 0x%16lx - 0x%16lx (%6ld MB)\n",
#ifdef CONFIG_KASAN
MLG(KASAN_SHADOW_START, KASAN_SHADOW_END),
#endif
+ MLM(MODULES_VADDR, MODULES_END),
MLG(VMALLOC_START, VMALLOC_END),
+ MLK_ROUNDUP(__init_begin, __init_end),
+ MLK_ROUNDUP(_text, _etext),
+ MLK_ROUNDUP(_sdata, _edata),
#ifdef CONFIG_SPARSEMEM_VMEMMAP
MLG((unsigned long)vmemmap,
(unsigned long)vmemmap + VMEMMAP_SIZE),
@@ -329,11 +333,7 @@ void __init mem_init(void)
#endif
MLK(FIXADDR_START, FIXADDR_TOP),
MLM(PCI_IO_START, PCI_IO_END),
- MLM(MODULES_VADDR, MODULES_END),
- MLM(PAGE_OFFSET, (unsigned long)high_memory),
- MLK_ROUNDUP(__init_begin, __init_end),
- MLK_ROUNDUP(_text, _etext),
- MLK_ROUNDUP(_sdata, _edata));
+ MLM(PAGE_OFFSET, (unsigned long)high_memory));
#undef MLK
#undef MLM
diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c
index a78fc5a882da..6275d183c005 100644
--- a/arch/arm64/mm/mmu.c
+++ b/arch/arm64/mm/mmu.c
@@ -322,40 +322,6 @@ static void create_mapping_late(phys_addr_t phys, unsigned long virt,
__create_pgd_mapping(init_mm.pgd, phys, virt, size, prot, late_alloc);
}
-static void __init __map_memblock(pgd_t *pgd, phys_addr_t start, phys_addr_t end)
-{
-
- unsigned long kernel_start = __pa(_stext);
- unsigned long kernel_end = __pa(_end);
-
- /*
- * The kernel itself is mapped at page granularity. Map all other
- * memory, making sure we don't overwrite the existing kernel mappings.
- */
-
- /* No overlap with the kernel. */
- if (end < kernel_start || start >= kernel_end) {
- __create_pgd_mapping(pgd, start, __phys_to_virt(start),
- end - start, PAGE_KERNEL, early_alloc);
- return;
- }
-
- /*
- * This block overlaps the kernel mapping. Map the portion(s) which
- * don't overlap.
- */
- if (start < kernel_start)
- __create_pgd_mapping(pgd, start,
- __phys_to_virt(start),
- kernel_start - start, PAGE_KERNEL,
- early_alloc);
- if (kernel_end < end)
- __create_pgd_mapping(pgd, kernel_end,
- __phys_to_virt(kernel_end),
- end - kernel_end, PAGE_KERNEL,
- early_alloc);
-}
-
static void __init map_mem(pgd_t *pgd)
{
struct memblock_region *reg;
@@ -370,7 +336,8 @@ static void __init map_mem(pgd_t *pgd)
if (memblock_is_nomap(reg))
continue;
- __map_memblock(pgd, start, end);
+ __create_pgd_mapping(pgd, start, __phys_to_virt(start),
+ end - start, PAGE_KERNEL, early_alloc);
}
}
--
2.5.0
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[toc] | [prev] | [next] | [standalone]
| From | Ard Biesheuvel <ard.biesheuvel@linaro.org> |
|---|---|
| Date | 2015-12-30 16:30 +0100 |
| Subject | [PATCH v2 07/13] arm64: add support for module PLTs |
| Message-ID | <qLqRl-8m8-39@gated-at.bofh.it> |
| In reply to | #1299398 |
This adds support for emitting PLTs at module load time for relative
branches that are out of range. This is a prerequisite for KASLR,
which may place the kernel and the modules anywhere in the vmalloc
area, making it likely that branch target offsets exceed the maximum
range of +/- 128 MB.
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
arch/arm64/Kconfig | 4 +
arch/arm64/Makefile | 4 +
arch/arm64/include/asm/module.h | 11 ++
arch/arm64/kernel/Makefile | 1 +
arch/arm64/kernel/module-plts.c | 137 ++++++++++++++++++++
arch/arm64/kernel/module.c | 7 +
arch/arm64/kernel/module.lds | 4 +
7 files changed, 168 insertions(+)
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index 871f21783866..827e78f33944 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -704,6 +704,10 @@ config ARM64_LSE_ATOMICS
endmenu
+config ARM64_MODULE_PLTS
+ bool
+ select HAVE_MOD_ARCH_SPECIFIC
+
endmenu
menu "Boot options"
diff --git a/arch/arm64/Makefile b/arch/arm64/Makefile
index cd822d8454c0..d4654830e536 100644
--- a/arch/arm64/Makefile
+++ b/arch/arm64/Makefile
@@ -45,6 +45,10 @@ ifeq ($(CONFIG_ARM64_ERRATUM_843419), y)
KBUILD_CFLAGS_MODULE += -mcmodel=large
endif
+ifeq ($(CONFIG_ARM64_MODULE_PLTS),y)
+KBUILD_LDFLAGS_MODULE += -T $(srctree)/arch/arm64/kernel/module.lds
+endif
+
# Default value
head-y := arch/arm64/kernel/head.o
diff --git a/arch/arm64/include/asm/module.h b/arch/arm64/include/asm/module.h
index e80e232b730e..7b8cd3dc9d8e 100644
--- a/arch/arm64/include/asm/module.h
+++ b/arch/arm64/include/asm/module.h
@@ -20,4 +20,15 @@
#define MODULE_ARCH_VERMAGIC "aarch64"
+#ifdef CONFIG_ARM64_MODULE_PLTS
+struct mod_arch_specific {
+ struct elf64_shdr *core_plt;
+ struct elf64_shdr *init_plt;
+ int core_plt_count;
+ int init_plt_count;
+};
+#endif
+
+u64 get_module_plt(struct module *mod, void *loc, u64 val);
+
#endif /* __ASM_MODULE_H */
diff --git a/arch/arm64/kernel/Makefile b/arch/arm64/kernel/Makefile
index 474691f8b13a..f42b0fff607f 100644
--- a/arch/arm64/kernel/Makefile
+++ b/arch/arm64/kernel/Makefile
@@ -30,6 +30,7 @@ arm64-obj-$(CONFIG_COMPAT) += sys32.o kuser32.o signal32.o \
../../arm/kernel/opcodes.o
arm64-obj-$(CONFIG_FUNCTION_TRACER) += ftrace.o entry-ftrace.o
arm64-obj-$(CONFIG_MODULES) += arm64ksyms.o module.o
+arm64-obj-$(CONFIG_ARM64_MODULE_PLTS) += module-plts.o
arm64-obj-$(CONFIG_PERF_EVENTS) += perf_regs.o perf_callchain.o
arm64-obj-$(CONFIG_HW_PERF_EVENTS) += perf_event.o
arm64-obj-$(CONFIG_HAVE_HW_BREAKPOINT) += hw_breakpoint.o
diff --git a/arch/arm64/kernel/module-plts.c b/arch/arm64/kernel/module-plts.c
new file mode 100644
index 000000000000..4a8ef9ea01ee
--- /dev/null
+++ b/arch/arm64/kernel/module-plts.c
@@ -0,0 +1,137 @@
+/*
+ * Copyright (C) 2014-2015 Linaro Ltd. <ard.biesheuvel@linaro.org>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#include <linux/elf.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+
+struct plt_entry {
+ __le32 mov0; /* movn x16, #0x.... */
+ __le32 mov1; /* movk x16, #0x...., lsl #16 */
+ __le32 mov2; /* movk x16, #0x...., lsl #32 */
+ __le32 br; /* br x16 */
+} __aligned(8);
+
+static bool in_init(const struct module *mod, void *addr)
+{
+ return (u64)addr - (u64)mod->module_init < mod->init_size;
+}
+
+u64 get_module_plt(struct module *mod, void *loc, u64 val)
+{
+ struct plt_entry entry = {
+ cpu_to_le32(0x92800010 | (((~val ) & 0xffff)) << 5),
+ cpu_to_le32(0xf2a00010 | ((( val >> 16) & 0xffff)) << 5),
+ cpu_to_le32(0xf2c00010 | ((( val >> 32) & 0xffff)) << 5),
+ cpu_to_le32(0xd61f0200)
+ }, *plt;
+ int i, *count;
+
+ if (in_init(mod, loc)) {
+ plt = (struct plt_entry *)mod->arch.init_plt->sh_addr;
+ count = &mod->arch.init_plt_count;
+ } else {
+ plt = (struct plt_entry *)mod->arch.core_plt->sh_addr;
+ count = &mod->arch.core_plt_count;
+ }
+
+ /* Look for an existing entry pointing to 'val' */
+ for (i = 0; i < *count; i++)
+ if (plt[i].mov0 == entry.mov0 &&
+ plt[i].mov1 == entry.mov1 &&
+ plt[i].mov2 == entry.mov2)
+ return (u64)&plt[i];
+
+ i = (*count)++;
+ plt[i] = entry;
+ return (u64)&plt[i];
+}
+
+static int duplicate_rel(Elf64_Addr base, const Elf64_Rela *rela, int num)
+{
+ int i;
+
+ for (i = 0; i < num; i++) {
+ if (rela[i].r_info == rela[num].r_info &&
+ rela[i].r_addend == rela[num].r_addend)
+ return 1;
+ }
+ return 0;
+}
+
+/* Count how many PLT entries we may need */
+static unsigned int count_plts(Elf64_Addr base, const Elf64_Rela *rela, int num)
+{
+ unsigned int ret = 0;
+ int i;
+
+ /*
+ * Sure, this is order(n^2), but it's usually short, and not
+ * time critical
+ */
+ for (i = 0; i < num; i++)
+ switch (ELF64_R_TYPE(rela[i].r_info)) {
+ case R_AARCH64_JUMP26:
+ case R_AARCH64_CALL26:
+ if (!duplicate_rel(base, rela, i))
+ ret++;
+ break;
+ }
+ return ret;
+}
+
+int module_frob_arch_sections(Elf_Ehdr *ehdr, Elf_Shdr *sechdrs,
+ char *secstrings, struct module *mod)
+{
+ unsigned long core_plts = 0, init_plts = 0;
+ Elf64_Shdr *s, *sechdrs_end = sechdrs + ehdr->e_shnum;
+
+ /*
+ * To store the PLTs, we expand the .text section for core module code
+ * and the .init.text section for initialization code.
+ */
+ for (s = sechdrs; s < sechdrs_end; ++s)
+ if (strcmp(".core.plt", secstrings + s->sh_name) == 0)
+ mod->arch.core_plt = s;
+ else if (strcmp(".init.plt", secstrings + s->sh_name) == 0)
+ mod->arch.init_plt = s;
+
+ if (!mod->arch.core_plt || !mod->arch.init_plt) {
+ pr_err("%s: sections missing\n", mod->name);
+ return -ENOEXEC;
+ }
+
+ for (s = sechdrs + 1; s < sechdrs_end; ++s) {
+ const Elf64_Rela *rels = (void *)ehdr + s->sh_offset;
+ int numrels = s->sh_size / sizeof(Elf64_Rela);
+ Elf64_Shdr *dstsec = sechdrs + s->sh_info;
+
+ if (s->sh_type != SHT_RELA)
+ continue;
+
+ if (strstr(secstrings + s->sh_name, ".init"))
+ init_plts += count_plts(dstsec->sh_addr, rels, numrels);
+ else
+ core_plts += count_plts(dstsec->sh_addr, rels, numrels);
+ }
+
+ mod->arch.core_plt->sh_type = SHT_NOBITS;
+ mod->arch.core_plt->sh_flags = SHF_EXECINSTR | SHF_ALLOC;
+ mod->arch.core_plt->sh_addralign = L1_CACHE_BYTES;
+ mod->arch.core_plt->sh_size = core_plts * sizeof(struct plt_entry);
+ mod->arch.core_plt_count = 0;
+
+ mod->arch.init_plt->sh_type = SHT_NOBITS;
+ mod->arch.init_plt->sh_flags = SHF_EXECINSTR | SHF_ALLOC;
+ mod->arch.init_plt->sh_addralign = L1_CACHE_BYTES;
+ mod->arch.init_plt->sh_size = init_plts * sizeof(struct plt_entry);
+ mod->arch.init_plt_count = 0;
+ pr_debug("%s: core.plt=%lld, init.plt=%lld\n", __func__,
+ mod->arch.core_plt->sh_size, mod->arch.init_plt->sh_size);
+ return 0;
+}
diff --git a/arch/arm64/kernel/module.c b/arch/arm64/kernel/module.c
index f4bc779e62e8..14880d77ec6f 100644
--- a/arch/arm64/kernel/module.c
+++ b/arch/arm64/kernel/module.c
@@ -388,6 +388,13 @@ int apply_relocate_add(Elf64_Shdr *sechdrs,
case R_AARCH64_CALL26:
ovf = reloc_insn_imm(RELOC_OP_PREL, loc, val, 2, 26,
AARCH64_INSN_IMM_26);
+
+ if (IS_ENABLED(CONFIG_ARM64_MODULE_PLTS) &&
+ ovf == -ERANGE) {
+ val = get_module_plt(me, loc, val);
+ ovf = reloc_insn_imm(RELOC_OP_PREL, loc, val, 2,
+ 26, AARCH64_INSN_IMM_26);
+ }
break;
default:
diff --git a/arch/arm64/kernel/module.lds b/arch/arm64/kernel/module.lds
new file mode 100644
index 000000000000..3682fa107918
--- /dev/null
+++ b/arch/arm64/kernel/module.lds
@@ -0,0 +1,4 @@
+SECTIONS {
+ .core.plt : { BYTE(0) }
+ .init.plt : { BYTE(0) }
+}
--
2.5.0
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[toc] | [prev] | [next] | [standalone]
| From | Ard Biesheuvel <ard.biesheuvel@linaro.org> |
|---|---|
| Date | 2015-12-30 16:30 +0100 |
| Subject | [PATCH v2 12/13] arm64: add support for relocatable kernel |
| Message-ID | <qLqRl-8m8-41@gated-at.bofh.it> |
| In reply to | #1299398 |
This adds support for runtime relocation of the kernel Image, by
building it as a PIE (ET_DYN) executable and applying the dynamic
relocations in the early boot code.
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
Documentation/arm64/booting.txt | 3 +-
arch/arm64/Kconfig | 13 ++++
arch/arm64/Makefile | 6 +-
arch/arm64/include/asm/memory.h | 3 +
arch/arm64/kernel/head.S | 75 +++++++++++++++++++-
arch/arm64/kernel/setup.c | 22 +++---
arch/arm64/kernel/vmlinux.lds.S | 9 +++
scripts/sortextable.c | 4 +-
8 files changed, 117 insertions(+), 18 deletions(-)
diff --git a/Documentation/arm64/booting.txt b/Documentation/arm64/booting.txt
index 03e02ebc1b0c..b17181eb4a43 100644
--- a/Documentation/arm64/booting.txt
+++ b/Documentation/arm64/booting.txt
@@ -109,7 +109,8 @@ Header notes:
1 - 4K
2 - 16K
3 - 64K
- Bits 3-63: Reserved.
+ Bit 3: Relocatable kernel.
+ Bits 4-63: Reserved.
- When image_size is zero, a bootloader should attempt to keep as much
memory as possible free for use by the kernel immediately after the
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index 54eeab140bca..f458fb9e0dce 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -363,6 +363,7 @@ config ARM64_ERRATUM_843419
bool "Cortex-A53: 843419: A load or store might access an incorrect address"
depends on MODULES
default y
+ select ARM64_MODULE_CMODEL_LARGE
help
This option builds kernel modules using the large memory model in
order to avoid the use of the ADRP instruction, which can cause
@@ -709,6 +710,18 @@ config ARM64_MODULE_PLTS
bool
select HAVE_MOD_ARCH_SPECIFIC
+config ARM64_MODULE_CMODEL_LARGE
+ bool
+
+config ARM64_RELOCATABLE_KERNEL
+ bool "Kernel address space layout randomization (KASLR)"
+ select ARM64_MODULE_PLTS
+ select ARM64_MODULE_CMODEL_LARGE
+ help
+ This feature randomizes the virtual address of the kernel image, to
+ harden against exploits that rely on knowledge about the absolute
+ addresses of certain kernel data structures.
+
endmenu
menu "Boot options"
diff --git a/arch/arm64/Makefile b/arch/arm64/Makefile
index d4654830e536..75dc477d45f5 100644
--- a/arch/arm64/Makefile
+++ b/arch/arm64/Makefile
@@ -15,6 +15,10 @@ CPPFLAGS_vmlinux.lds = -DTEXT_OFFSET=$(TEXT_OFFSET)
OBJCOPYFLAGS :=-O binary -R .note -R .note.gnu.build-id -R .comment -S
GZFLAGS :=-9
+ifneq ($(CONFIG_ARM64_RELOCATABLE_KERNEL),)
+LDFLAGS_vmlinux += -pie
+endif
+
KBUILD_DEFCONFIG := defconfig
# Check for binutils support for specific extensions
@@ -41,7 +45,7 @@ endif
CHECKFLAGS += -D__aarch64__
-ifeq ($(CONFIG_ARM64_ERRATUM_843419), y)
+ifeq ($(CONFIG_ARM64_MODULE_CMODEL_LARGE), y)
KBUILD_CFLAGS_MODULE += -mcmodel=large
endif
diff --git a/arch/arm64/include/asm/memory.h b/arch/arm64/include/asm/memory.h
index 557228658666..afab3e669e19 100644
--- a/arch/arm64/include/asm/memory.h
+++ b/arch/arm64/include/asm/memory.h
@@ -121,6 +121,9 @@ extern phys_addr_t memstart_addr;
/* PHYS_OFFSET - the physical address of the start of memory. */
#define PHYS_OFFSET ({ memstart_addr; })
+/* the virtual base of the kernel image (minus TEXT_OFFSET) */
+extern u64 kimage_vaddr;
+
/* the offset between the kernel virtual and physical mappings */
extern u64 kimage_voffset;
diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S
index 01a33e42ed70..ab582ee58b58 100644
--- a/arch/arm64/kernel/head.S
+++ b/arch/arm64/kernel/head.S
@@ -59,8 +59,15 @@
#define __HEAD_FLAG_PAGE_SIZE ((PAGE_SHIFT - 10) / 2)
+#ifdef CONFIG_ARM64_RELOCATABLE_KERNEL
+#define __HEAD_FLAG_RELOC 1
+#else
+#define __HEAD_FLAG_RELOC 0
+#endif
+
#define __HEAD_FLAGS ((__HEAD_FLAG_BE << 0) | \
- (__HEAD_FLAG_PAGE_SIZE << 1))
+ (__HEAD_FLAG_PAGE_SIZE << 1) | \
+ (__HEAD_FLAG_RELOC << 3))
/*
* Kernel startup entry point.
@@ -231,6 +238,9 @@ ENTRY(stext)
*/
ldr x27, 0f // address to jump to after
// MMU has been enabled
+#ifdef CONFIG_ARM64_RELOCATABLE_KERNEL
+ add x27, x27, x23 // add KASLR displacement
+#endif
adr_l lr, __enable_mmu // return (PIC) address
b __cpu_setup // initialise processor
ENDPROC(stext)
@@ -243,6 +253,16 @@ ENDPROC(stext)
preserve_boot_args:
mov x21, x0 // x21=FDT
+#ifdef CONFIG_ARM64_RELOCATABLE_KERNEL
+ /*
+ * Mask off the bits of the random value supplied in x1 so it can serve
+ * as a KASLR displacement value which will move the kernel image to a
+ * random offset in the lower half of the VMALLOC area.
+ */
+ mov x23, #(1 << (VA_BITS - 2)) - 1
+ and x23, x23, x1, lsl #SWAPPER_BLOCK_SHIFT
+#endif
+
adr_l x0, boot_args // record the contents of
stp x21, x1, [x0] // x0 .. x3 at kernel entry
stp x2, x3, [x0, #16]
@@ -402,6 +422,9 @@ __create_page_tables:
*/
mov x0, x26 // swapper_pg_dir
ldr x5, =KIMAGE_VADDR
+#ifdef CONFIG_ARM64_RELOCATABLE_KERNEL
+ add x5, x5, x23 // add KASLR displacement
+#endif
create_pgd_entry x0, x5, x3, x6
ldr w6, kernel_img_size
add x6, x6, x5
@@ -443,10 +466,52 @@ __mmap_switched:
str xzr, [x6], #8 // Clear BSS
b 1b
2:
+
+#ifdef CONFIG_ARM64_RELOCATABLE_KERNEL
+
+#define R_AARCH64_RELATIVE 0x403
+#define R_AARCH64_ABS64 0x101
+
+ /*
+ * Iterate over each entry in the relocation table, and apply the
+ * relocations in place.
+ */
+ adr_l x8, __dynsym_start // start of symbol table
+ adr_l x9, __reloc_start // start of reloc table
+ adr_l x10, __reloc_end // end of reloc table
+
+0: cmp x9, x10
+ b.hs 2f
+ ldp x11, x12, [x9], #24
+ ldr x13, [x9, #-8]
+ cmp w12, #R_AARCH64_RELATIVE
+ b.ne 1f
+ add x13, x13, x23 // relocate
+ str x13, [x11, x23]
+ b 0b
+
+1: cmp w12, #R_AARCH64_ABS64
+ b.ne 0b
+ add x12, x12, x12, lsl #1 // symtab offset: 24x top word
+ add x12, x8, x12, lsr #(32 - 3) // ... shifted into bottom word
+ ldrsh w14, [x12, #6] // Elf64_Sym::st_shndx
+ ldr x15, [x12, #8] // Elf64_Sym::st_value
+ cmp w14, #-0xf // SHN_ABS (0xfff1) ?
+ add x14, x15, x23 // relocate
+ csel x15, x14, x15, ne
+ add x15, x13, x15
+ str x15, [x11, x23]
+ b 0b
+
+2: adr_l x8, kimage_vaddr // make relocated kimage_vaddr
+ dc cvac, x8 // value visible to secondaries
+ dsb sy // with MMU off
+#endif
+
adr_l sp, initial_sp, x4
str_l x21, __fdt_pointer, x5 // Save FDT pointer
- ldr x0, =KIMAGE_VADDR // Save the offset between
+ ldr_l x0, kimage_vaddr // Save the offset between
sub x24, x0, x24 // the kernel virtual and
str_l x24, kimage_voffset, x0 // physical mappings
@@ -462,6 +527,10 @@ ENDPROC(__mmap_switched)
* hotplug and needs to have the same protections as the text region
*/
.section ".text","ax"
+
+ENTRY(kimage_vaddr)
+ .quad _text - TEXT_OFFSET
+
/*
* If we're fortunate enough to boot at EL2, ensure that the world is
* sane before dropping to EL1.
@@ -622,7 +691,7 @@ ENTRY(secondary_startup)
adrp x26, swapper_pg_dir
bl __cpu_setup // initialise processor
- ldr x8, =KIMAGE_VADDR
+ ldr x8, kimage_vaddr
ldr w9, 0f
sub x27, x8, w9, sxtw // address to jump to after enabling the MMU
b __enable_mmu
diff --git a/arch/arm64/kernel/setup.c b/arch/arm64/kernel/setup.c
index 96177a7c0f05..2faee6042e99 100644
--- a/arch/arm64/kernel/setup.c
+++ b/arch/arm64/kernel/setup.c
@@ -292,16 +292,15 @@ u64 __cpu_logical_map[NR_CPUS] = { [0 ... NR_CPUS-1] = INVALID_HWID };
void __init setup_arch(char **cmdline_p)
{
- static struct vm_struct vmlinux_vm __initdata = {
- .addr = (void *)KIMAGE_VADDR,
- .size = 0,
- .flags = VM_IOREMAP,
- .caller = setup_arch,
- };
-
- vmlinux_vm.size = round_up((unsigned long)_end - KIMAGE_VADDR,
- 1 << SWAPPER_BLOCK_SHIFT);
- vmlinux_vm.phys_addr = __pa(KIMAGE_VADDR);
+ static struct vm_struct vmlinux_vm __initdata;
+
+ vmlinux_vm.addr = (void *)kimage_vaddr;
+ vmlinux_vm.size = round_up((u64)_end - kimage_vaddr,
+ SWAPPER_BLOCK_SIZE);
+ vmlinux_vm.phys_addr = __pa(kimage_vaddr);
+ vmlinux_vm.flags = VM_IOREMAP;
+ vmlinux_vm.caller = setup_arch;
+
vm_area_add_early(&vmlinux_vm);
pr_info("Boot CPU: AArch64 Processor [%08x]\n", read_cpuid_id());
@@ -367,7 +366,8 @@ void __init setup_arch(char **cmdline_p)
conswitchp = &dummy_con;
#endif
#endif
- if (boot_args[1] || boot_args[2] || boot_args[3]) {
+ if ((!IS_ENABLED(CONFIG_ARM64_RELOCATABLE_KERNEL) && boot_args[1]) ||
+ boot_args[2] || boot_args[3]) {
pr_err("WARNING: x1-x3 nonzero in violation of boot protocol:\n"
"\tx1: %016llx\n\tx2: %016llx\n\tx3: %016llx\n"
"This indicates a broken bootloader or old kernel\n",
diff --git a/arch/arm64/kernel/vmlinux.lds.S b/arch/arm64/kernel/vmlinux.lds.S
index f935f082188d..cc1486039338 100644
--- a/arch/arm64/kernel/vmlinux.lds.S
+++ b/arch/arm64/kernel/vmlinux.lds.S
@@ -148,6 +148,15 @@ SECTIONS
.altinstr_replacement : {
*(.altinstr_replacement)
}
+ .rela : ALIGN(8) {
+ __reloc_start = .;
+ *(.rela .rela*)
+ __reloc_end = .;
+ }
+ .dynsym : ALIGN(8) {
+ __dynsym_start = .;
+ *(.dynsym)
+ }
. = ALIGN(PAGE_SIZE);
__init_end = .;
diff --git a/scripts/sortextable.c b/scripts/sortextable.c
index af247c70fb66..5ecbedefdb0f 100644
--- a/scripts/sortextable.c
+++ b/scripts/sortextable.c
@@ -266,9 +266,9 @@ do_file(char const *const fname)
break;
} /* end switch */
if (memcmp(ELFMAG, ehdr->e_ident, SELFMAG) != 0
- || r2(&ehdr->e_type) != ET_EXEC
+ || (r2(&ehdr->e_type) != ET_EXEC && r2(&ehdr->e_type) != ET_DYN)
|| ehdr->e_ident[EI_VERSION] != EV_CURRENT) {
- fprintf(stderr, "unrecognized ET_EXEC file %s\n", fname);
+ fprintf(stderr, "unrecognized ET_EXEC/ET_DYN file %s\n", fname);
fail_file();
}
--
2.5.0
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[toc] | [prev] | [next] | [standalone]
| From | Ard Biesheuvel <ard.biesheuvel@linaro.org> |
|---|---|
| Date | 2015-12-30 16:40 +0100 |
| Subject | [PATCH v2 01/13] of/fdt: make memblock minimum physical address arch configurable |
| Message-ID | <qLr10-8qm-17@gated-at.bofh.it> |
| In reply to | #1299398 |
By default, early_init_dt_add_memory_arch() ignores memory below
the base of the kernel image since it won't be addressable via the
linear mapping. However, this is not appropriate anymore once we
decouple the kernel text mapping from the linear mapping, so archs
may want to drop the low limit entirely. So allow the minimum to be
overridden by setting MIN_MEMBLOCK_ADDR.
Acked-by: Mark Rutland <mark.rutland@arm.com>
Acked-by: Rob Herring <robh@kernel.org>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
drivers/of/fdt.c | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/drivers/of/fdt.c b/drivers/of/fdt.c
index d2430298a309..0455564f8cbc 100644
--- a/drivers/of/fdt.c
+++ b/drivers/of/fdt.c
@@ -971,13 +971,16 @@ int __init early_init_dt_scan_chosen(unsigned long node, const char *uname,
}
#ifdef CONFIG_HAVE_MEMBLOCK
+#ifndef MIN_MEMBLOCK_ADDR
+#define MIN_MEMBLOCK_ADDR __pa(PAGE_OFFSET)
+#endif
#ifndef MAX_MEMBLOCK_ADDR
#define MAX_MEMBLOCK_ADDR ((phys_addr_t)~0)
#endif
void __init __weak early_init_dt_add_memory_arch(u64 base, u64 size)
{
- const u64 phys_offset = __pa(PAGE_OFFSET);
+ const u64 phys_offset = MIN_MEMBLOCK_ADDR;
if (!PAGE_ALIGNED(base)) {
if (size < PAGE_SIZE - (base & ~PAGE_MASK)) {
--
2.5.0
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[toc] | [prev] | [standalone]
Back to top | Article view | linux.kernel
csiph-web