Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1651660 > unrolled thread

Re: [PATCH 2/7] RISC-V: arch/riscv Makefile and Kconfigs

Back to article view | Back to linux.kernel

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: [PATCH 2/7] RISC-V: arch/riscv Makefile and Kconfigs Palmer Dabbelt <palmer@dabbelt.com> - 2017-05-27 03:30 +0200
    Re: [PATCH 2/7] RISC-V: arch/riscv Makefile and Kconfigs Arnd Bergmann <arnd@arndb.de> - 2017-05-29 13:20 +0200
      Re: [PATCH 2/7] RISC-V: arch/riscv Makefile and Kconfigs Palmer Dabbelt <palmer@dabbelt.com> - 2017-06-06 07:00 +0200
        Re: [PATCH 2/7] RISC-V: arch/riscv Makefile and Kconfigs Arnd Bergmann <arnd@arndb.de> - 2017-06-06 11:30 +0200
          Re: [PATCH 2/7] RISC-V: arch/riscv Makefile and Kconfigs Palmer Dabbelt <palmer@dabbelt.com> - 2017-06-06 22:40 +0200

#1651660 — Re: [PATCH 2/7] RISC-V: arch/riscv Makefile and Kconfigs

On Tue, 23 May 2017 04:46:22 PDT (-0700), Arnd Bergmann wrote:
> On Tue, May 23, 2017 at 2:41 AM, Palmer Dabbelt <palmer@dabbelt.com> wrote:
>> ---
>>  arch/riscv/.gitignore                |  35 ++++
>>  arch/riscv/Kconfig                   | 300 +++++++++++++++++++++++++++++++++++
>>  arch/riscv/Makefile                  |  64 ++++++++
>>  arch/riscv/configs/riscv32_spike     |  47 ++++++
>>  arch/riscv/configs/riscv64_freedom-u |  52 ++++++
>>  arch/riscv/configs/riscv64_qemu      |  64 ++++++++
>>  arch/riscv/configs/riscv64_spike     |  45 ++++++
>>  7 files changed, 607 insertions(+)
>>  create mode 100644 arch/riscv/.gitignore
>>  create mode 100644 arch/riscv/Kconfig
>>  create mode 100644 arch/riscv/Makefile
>>  create mode 100644 arch/riscv/configs/riscv32_spike
>>  create mode 100644 arch/riscv/configs/riscv64_freedom-u
>>  create mode 100644 arch/riscv/configs/riscv64_qemu
>>  create mode 100644 arch/riscv/configs/riscv64_spike
>>
>> diff --git a/arch/riscv/.gitignore b/arch/riscv/.gitignore
>> new file mode 100644
>> index 000000000000..376d06eb5d52
>> --- /dev/null
>> +++ b/arch/riscv/.gitignore
>> @@ -0,0 +1,35 @@
>> +# Now un-ignore all files.
>> +!*
>> +
>> +# But then re-ignore the files listed in the Linux .gitignore
>> +# Normal rules
>> +#
>> +.*
>> +*.o
>> +*.o.*
>> +*.a
>
> This doesn't seem to belong here: There is no reason for riscv
> to be different from all other architectures. Is something wrong
> with the top-level .gitignore? If so, we should just fix it there.

Sorry, this snuck in from how we used to track the kernel.  It's been fixed.

>> diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
>> new file mode 100644
>> index 000000000000..510ead1d3343
>> --- /dev/null
>> +++ b/arch/riscv/Kconfig
>> @@ -0,0 +1,300 @@
>> +#
>> +# For a description of the syntax of this configuration file,
>> +# see Documentation/kbuild/kconfig-language.txt.
>> +#
>> +
>> +config RISCV
>> +       def_bool y
>> +       select OF
>> +       select OF_EARLY_FLATTREE
>> +       select OF_IRQ
>> +       select ARCH_HAS_ATOMIC64_DEC_IF_POSITIVE
>> +       select ARCH_WANT_FRAME_POINTERS
>> +       select CLONE_BACKWARDS
>> +       select COMMON_CLK
>> +       select GENERIC_CLOCKEVENTS
>> +       select GENERIC_CPU_DEVICES
>> +       select GENERIC_IRQ_SHOW
>> +       select GENERIC_PCI_IOMAP
>
> You normally don't want GENERIC_PCI_IOMAP, unless your
> inb()/outb() uses other instructions than your readl()/writel()

We don't have any special instructions for inb/outb, but there is a special
fence to ensure ordering.

It looks like most of my candidates for patterning things after use
GENERIC_PCI_IOMAP, but I ended up setting GENERIC_IOMAP and things still at
least build and boot without any PCI

  https://github.com/riscv/riscv-linux/commit/c43c599b0dd9d15886c03a9dd179f8936b0cbb2e

I'll be sure to check if I broke PCI, as I don't actually know what's going on
here.

>> +config MMU
>> +       def_bool y
>
> Just a general question: has there been any interest in a no-MMU
> version?

One of the fun things about RISC-V is that there's at least some interest in
_everything_... :).  There hasn't been any serious interest, and I don't know
of anyone building systems where no-MMU Linux would run (our DDR phy is a lot
bigger than our MMU), but I wouldn't rule it out.

>> +# even on 32-bit, physical (and DMA) addresses are > 32-bits
>> +config ARCH_PHYS_ADDR_T_64BIT
>> +       def_bool y
>> +
>> +config ARCH_DMA_ADDR_T_64BIT
>> +       def_bool y
>
> Are you required to use 64-bit addressing for RAM on 32-bit
> architectures though? Using 32-bit dma_addr_t and phys_addr_t
> when possible makes some code noticeably more efficient.
>
>> +config PGTABLE_LEVELS
>> +       int
>> +       default 3 if 64BIT
>> +       default 2
>
> With 2-level page tables, you usually can't address much more
> than 32-bit physical memory anyway, so I'd guess that most
> 32-bit chips would actually put their RAM under the 4GB boundary.

We can address 34 bits of physical address space on Sv32 (the 32-bit virtual
addressing scheme in RV32).  If this is a meaningful performance constraint
then we could always make this configurable.

>> +config RV_ATOMIC
>> +       bool "Use atomic memory instructions (RV32A or RV64A)"
>> +       default y
>> +
>> +config RV_SYSRISCV_ATOMIC
>> +       bool "Include support for atomic operation syscalls"
>> +       default n
>> +       help
>> +         If atomic memory instructions are present, i.e.,
>> +         CONFIG_RV_ATOMIC, this includes support for the syscall that
>> +         provides atomic accesses.  This is only useful to run
>> +         binaries that require atomic access but were compiled with
>> +         -mno-atomic.
>> +
>> +         If CONFIG_RV_ATOMIC is unset, this option is mandatory.
>
> Just express this in Kconfig terms to prevent misconfiguration:
>
> config RV_SYSRISCV_ATOMIC
>        bool "Include support for atomic operation syscalls" if RV_ATOMIC
>        default !RV_ATOMIC

Done

  https://github.com/riscv/riscv-linux/commit/921703e0a6639d10826e5bbad73f4998977dc244

> I wonder what the cost would be of always providing the syscalls
> for compatibility. This is also something worth putting into a VDSO
> instead of exposing the syscall:
>
> That way, user space that is built with -mno-atomic can call into
> the vdso, which depending on the hardware support will perform
> the atomic operation directly or enter the syscall.

I think the theory is that most Linux-capable systems are going to have the A
extension, and that this is really there for educational/hobbyist use.

The actual implementation isn't that expensive: it's no-SMP-only, so it's just
a disable/enable interrupts that wraps a compare+exchange.  I think putting it
in the VDSO is actually the right thing to do: that way non-A user code will
still have reasonable performance.

I'll add it to my TODO list.

>> +config PCI_DOMAINS
>> +       def_bool PCI
>> +
>> +config PCI_DOMAINS_GENERIC
>> +       def_bool PCI
>> +
>> +config PCI_SYSCALL
>> +       def_bool PCI
>
> I don't think you want PCI_SYSCALL

OK.  I'm a bit worried we need this to run DOOM, but I'll give everything a
shot without it and see what happens

  https://github.com/riscv/riscv-linux/commit/710b5bb81ab99cf1d49a74bb706f6aae179fc111

[toc] | [next] | [standalone]

#1652425

On Sat, May 27, 2017 at 2:57 AM, Palmer Dabbelt <palmer@dabbelt.com> wrote:
> On Tue, 23 May 2017 04:46:22 PDT (-0700), Arnd Bergmann wrote:
>> On Tue, May 23, 2017 at 2:41 AM, Palmer Dabbelt <palmer@dabbelt.com> wrote:
>>> diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
>>> new file mode 100644
>>> index 000000000000..510ead1d3343
>>> --- /dev/null
>>> +++ b/arch/riscv/Kconfig
>>> @@ -0,0 +1,300 @@
>>> +#
>>> +# For a description of the syntax of this configuration file,
>>> +# see Documentation/kbuild/kconfig-language.txt.
>>> +#
>>> +
>>> +config RISCV
>>> +       def_bool y
>>> +       select OF
>>> +       select OF_EARLY_FLATTREE
>>> +       select OF_IRQ
>>> +       select ARCH_HAS_ATOMIC64_DEC_IF_POSITIVE
>>> +       select ARCH_WANT_FRAME_POINTERS
>>> +       select CLONE_BACKWARDS
>>> +       select COMMON_CLK
>>> +       select GENERIC_CLOCKEVENTS
>>> +       select GENERIC_CPU_DEVICES
>>> +       select GENERIC_IRQ_SHOW
>>> +       select GENERIC_PCI_IOMAP
>>
>> You normally don't want GENERIC_PCI_IOMAP, unless your
>> inb()/outb() uses other instructions than your readl()/writel()
>
> We don't have any special instructions for inb/outb, but there is a special
> fence to ensure ordering.

Ah, interesting. What is the exact behavior of that fence? (this is slightly
unrelated but worth looking at anyway)

> It looks like most of my candidates for patterning things after use
> GENERIC_PCI_IOMAP, but I ended up setting GENERIC_IOMAP and things still at
> least build and boot without any PCI
>
>   https://github.com/riscv/riscv-linux/commit/c43c599b0dd9d15886c03a9dd179f8936b0cbb2e
>
> I'll be sure to check if I broke PCI, as I don't actually know what's going on
> here.

Actually I made a mistake: GENERIC_PCI_IOMAP is fine, please use that,
GENERIC_IOMAP is the option you don't want, and you already don't
enable that one.

>>> +# even on 32-bit, physical (and DMA) addresses are > 32-bits
>>> +config ARCH_PHYS_ADDR_T_64BIT
>>> +       def_bool y
>>> +
>>> +config ARCH_DMA_ADDR_T_64BIT
>>> +       def_bool y
>>
>> Are you required to use 64-bit addressing for RAM on 32-bit
>> architectures though? Using 32-bit dma_addr_t and phys_addr_t
>> when possible makes some code noticeably more efficient.
>>
>>> +config PGTABLE_LEVELS
>>> +       int
>>> +       default 3 if 64BIT
>>> +       default 2
>>
>> With 2-level page tables, you usually can't address much more
>> than 32-bit physical memory anyway, so I'd guess that most
>> 32-bit chips would actually put their RAM under the 4GB boundary.
>
> We can address 34 bits of physical address space on Sv32 (the 32-bit virtual
> addressing scheme in RV32).  If this is a meaningful performance constraint
> then we could always make this configurable.

I'd suggest to leave it turned off initially and only use it once someone
actually builds a system that makes use of the high address space.

Of course if there is already hardware that needs it, there has to
be a way to turn it on.

This raises a much more general question about how you want to deal
with SoC implementations in the future. The two most common ways of
doing this are:

- Every major platform gets a Kconfig option in the architecture menu,
  and that selects the essential drivers (irqchip, clocksource, pinctrl,
  clk, ...) that you need for that platform, along with architecture features
  (ISA level and optional features, ...)

- The architecture code knows nothing about the SoC and just keeps
   to the basics (CPU architecture level selection, SMP/MMU/etc enabled,
   selecting drivers that everyone needs) and leaves the rest up to be
   selected in the defconfig file.

On ARM, we have a bit of both, which is not as good as being
consistent one way or another.

>>> +config RV_ATOMIC
>>> +       bool "Use atomic memory instructions (RV32A or RV64A)"
>>> +       default y
>>> +
>>> +config RV_SYSRISCV_ATOMIC
>>> +       bool "Include support for atomic operation syscalls"
>>> +       default n
>>> +       help
>>> +         If atomic memory instructions are present, i.e.,
>>> +         CONFIG_RV_ATOMIC, this includes support for the syscall that
>>> +         provides atomic accesses.  This is only useful to run
>>> +         binaries that require atomic access but were compiled with
>>> +         -mno-atomic.
>>> +
>>> +         If CONFIG_RV_ATOMIC is unset, this option is mandatory.
>>
>> I wonder what the cost would be of always providing the syscalls
>> for compatibility. This is also something worth putting into a VDSO
>> instead of exposing the syscall:
>>
>> That way, user space that is built with -mno-atomic can call into
>> the vdso, which depending on the hardware support will perform
>> the atomic operation directly or enter the syscall.
>
> I think the theory is that most Linux-capable systems are going to have the A
> extension, and that this is really there for educational/hobbyist use.
>
> The actual implementation isn't that expensive: it's no-SMP-only, so it's just
> a disable/enable interrupts that wraps a compare+exchange.  I think putting it
> in the VDSO is actually the right thing to do: that way non-A user code will
> still have reasonable performance.
>
> I'll add it to my TODO list.

This would be another variant of the question above.

>>> +config PCI_DOMAINS
>>> +       def_bool PCI
>>> +
>>> +config PCI_DOMAINS_GENERIC
>>> +       def_bool PCI
>>> +
>>> +config PCI_SYSCALL
>>> +       def_bool PCI
>>
>> I don't think you want PCI_SYSCALL
>
> OK.  I'm a bit worried we need this to run DOOM, but I'll give everything a
> shot without it and see what happens

I'd argue that's a user space bug that we want to fix anyway to make
the code portable to other architectures.

        Arnd

[toc] | [prev] | [next] | [standalone]

#1658342

On Mon, 29 May 2017 04:17:40 PDT (-0700), Arnd Bergmann wrote:
> On Sat, May 27, 2017 at 2:57 AM, Palmer Dabbelt <palmer@dabbelt.com> wrote:
>> On Tue, 23 May 2017 04:46:22 PDT (-0700), Arnd Bergmann wrote:
>>> On Tue, May 23, 2017 at 2:41 AM, Palmer Dabbelt <palmer@dabbelt.com> wrote:
>>>> diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
>>>> new file mode 100644
>>>> index 000000000000..510ead1d3343
>>>> --- /dev/null
>>>> +++ b/arch/riscv/Kconfig
>>>> @@ -0,0 +1,300 @@
>>>> +#
>>>> +# For a description of the syntax of this configuration file,
>>>> +# see Documentation/kbuild/kconfig-language.txt.
>>>> +#
>>>> +
>>>> +config RISCV
>>>> +       def_bool y
>>>> +       select OF
>>>> +       select OF_EARLY_FLATTREE
>>>> +       select OF_IRQ
>>>> +       select ARCH_HAS_ATOMIC64_DEC_IF_POSITIVE
>>>> +       select ARCH_WANT_FRAME_POINTERS
>>>> +       select CLONE_BACKWARDS
>>>> +       select COMMON_CLK
>>>> +       select GENERIC_CLOCKEVENTS
>>>> +       select GENERIC_CPU_DEVICES
>>>> +       select GENERIC_IRQ_SHOW
>>>> +       select GENERIC_PCI_IOMAP
>>>
>>> You normally don't want GENERIC_PCI_IOMAP, unless your
>>> inb()/outb() uses other instructions than your readl()/writel()
>>
>> We don't have any special instructions for inb/outb, but there is a special
>> fence to ensure ordering.
>
> Ah, interesting. What is the exact behavior of that fence? (this is slightly
> unrelated but worth looking at anyway)
>
>> It looks like most of my candidates for patterning things after use
>> GENERIC_PCI_IOMAP, but I ended up setting GENERIC_IOMAP and things still at
>> least build and boot without any PCI
>>
>>   https://github.com/riscv/riscv-linux/commit/c43c599b0dd9d15886c03a9dd179f8936b0cbb2e
>>
>> I'll be sure to check if I broke PCI, as I don't actually know what's going on
>> here.
>
> Actually I made a mistake: GENERIC_PCI_IOMAP is fine, please use that,
> GENERIC_IOMAP is the option you don't want, and you already don't
> enable that one.

Great -- I actually ended up reverting the patch already as I couldn't figure
out how to make it work.  That was the one big thing on my TODO list before a
v2, so with any luck I'll be able to get a new patch set out soon.

>>>> +# even on 32-bit, physical (and DMA) addresses are > 32-bits
>>>> +config ARCH_PHYS_ADDR_T_64BIT
>>>> +       def_bool y
>>>> +
>>>> +config ARCH_DMA_ADDR_T_64BIT
>>>> +       def_bool y
>>>
>>> Are you required to use 64-bit addressing for RAM on 32-bit
>>> architectures though? Using 32-bit dma_addr_t and phys_addr_t
>>> when possible makes some code noticeably more efficient.
>>>
>>>> +config PGTABLE_LEVELS
>>>> +       int
>>>> +       default 3 if 64BIT
>>>> +       default 2
>>>
>>> With 2-level page tables, you usually can't address much more
>>> than 32-bit physical memory anyway, so I'd guess that most
>>> 32-bit chips would actually put their RAM under the 4GB boundary.
>>
>> We can address 34 bits of physical address space on Sv32 (the 32-bit virtual
>> addressing scheme in RV32).  If this is a meaningful performance constraint
>> then we could always make this configurable.
>
> I'd suggest to leave it turned off initially and only use it once someone
> actually builds a system that makes use of the high address space.
>
> Of course if there is already hardware that needs it, there has to
> be a way to turn it on.

There isn't, and I'm find with 32-bit physical addresses on RV32I.  I doubt
anyone will be building big RV32 systems.  I'll make the change for the v2.

> This raises a much more general question about how you want to deal
> with SoC implementations in the future. The two most common ways of
> doing this are:
>
> - Every major platform gets a Kconfig option in the architecture menu,
>   and that selects the essential drivers (irqchip, clocksource, pinctrl,
>   clk, ...) that you need for that platform, along with architecture features
>   (ISA level and optional features, ...)
>
> - The architecture code knows nothing about the SoC and just keeps
>    to the basics (CPU architecture level selection, SMP/MMU/etc enabled,
>    selecting drivers that everyone needs) and leaves the rest up to be
>    selected in the defconfig file.
>
> On ARM, we have a bit of both, which is not as good as being
> consistent one way or another.

This is actually an open question in RISC-V land right now.  We should be
spinning up a platform specification working group this summer to try and work
things out.  While this will have to be ironed out, I believe the plan is to
define a small number of base platforms (maybe one for embedded systems with no
programmable PMAs, and one for larger machines with a bit more
configurability).  I'd anticipate that we'll have a platform Kconfig menu entry
for every platform that gets written down in a specification (just like we have
an entry for our base ISAs) and then defconfig entries for various
implementations that select the relevant platform in addition to the drivers
actually on board.

For now we've got a handful of defconfig entries for the various platforms we
support (the ISA simulator and our FPGA implementation), but there's no silicon
so we're not stuck with what's there.  I don't anticipate we'll add more than a
handful of these until the platform spec work is underway.

>>>> +config RV_ATOMIC
>>>> +       bool "Use atomic memory instructions (RV32A or RV64A)"
>>>> +       default y
>>>> +
>>>> +config RV_SYSRISCV_ATOMIC
>>>> +       bool "Include support for atomic operation syscalls"
>>>> +       default n
>>>> +       help
>>>> +         If atomic memory instructions are present, i.e.,
>>>> +         CONFIG_RV_ATOMIC, this includes support for the syscall that
>>>> +         provides atomic accesses.  This is only useful to run
>>>> +         binaries that require atomic access but were compiled with
>>>> +         -mno-atomic.
>>>> +
>>>> +         If CONFIG_RV_ATOMIC is unset, this option is mandatory.
>>>
>>> I wonder what the cost would be of always providing the syscalls
>>> for compatibility. This is also something worth putting into a VDSO
>>> instead of exposing the syscall:
>>>
>>> That way, user space that is built with -mno-atomic can call into
>>> the vdso, which depending on the hardware support will perform
>>> the atomic operation directly or enter the syscall.
>>
>> I think the theory is that most Linux-capable systems are going to have the A
>> extension, and that this is really there for educational/hobbyist use.
>>
>> The actual implementation isn't that expensive: it's no-SMP-only, so it's just
>> a disable/enable interrupts that wraps a compare+exchange.  I think putting it
>> in the VDSO is actually the right thing to do: that way non-A user code will
>> still have reasonable performance.
>>
>> I'll add it to my TODO list.
>
> This would be another variant of the question above.

After talking with people a bit about this, I think the sane thing to do here
is to just always provide the atomic system call (and VDSO implementation), and
to default to enabling the atomic extensions.  While I expect non-atomic
implementations to be very much the exception, there's a stronger argument for
supporting non-atomic userspace programs with reasonable performance on atomic
systems -- the first round of small Linux-capable embedded systems might not
support atomic (though ours will), but we don't want to wed software to a
syscall in this case.

Before the VDSO was suggested I was leaning towards disabling the atomic system
call by default, but since the VDSO implementation will provide very good
performance for non-atomic userspace code on atomic systems I think it's best
to just enable it everywhere.  It's only a few bytes of binary size.

>>>> +config PCI_DOMAINS
>>>> +       def_bool PCI
>>>> +
>>>> +config PCI_DOMAINS_GENERIC
>>>> +       def_bool PCI
>>>> +
>>>> +config PCI_SYSCALL
>>>> +       def_bool PCI
>>>
>>> I don't think you want PCI_SYSCALL
>>
>> OK.  I'm a bit worried we need this to run DOOM, but I'll give everything a
>> shot without it and see what happens
>
> I'd argue that's a user space bug that we want to fix anyway to make
> the code portable to other architectures.

I buy it.

[toc] | [prev] | [next] | [standalone]

#1658530

On Tue, Jun 6, 2017 at 6:56 AM, Palmer Dabbelt <palmer@dabbelt.com> wrote:
> On Mon, 29 May 2017 04:17:40 PDT (-0700), Arnd Bergmann wrote:
>> On Sat, May 27, 2017 at 2:57 AM, Palmer Dabbelt <palmer@dabbelt.com> wrote:
>>> On Tue, 23 May 2017 04:46:22 PDT (-0700), Arnd Bergmann wrote:
>> This raises a much more general question about how you want to deal
>> with SoC implementations in the future. The two most common ways of
>> doing this are:
>>
>> - Every major platform gets a Kconfig option in the architecture menu,
>>   and that selects the essential drivers (irqchip, clocksource, pinctrl,
>>   clk, ...) that you need for that platform, along with architecture features
>>   (ISA level and optional features, ...)
>>
>> - The architecture code knows nothing about the SoC and just keeps
>>    to the basics (CPU architecture level selection, SMP/MMU/etc enabled,
>>    selecting drivers that everyone needs) and leaves the rest up to be
>>    selected in the defconfig file.
>>
>> On ARM, we have a bit of both, which is not as good as being
>> consistent one way or another.
>
> This is actually an open question in RISC-V land right now.  We should be
> spinning up a platform specification working group this summer to try and work
> things out.  While this will have to be ironed out, I believe the plan is to
> define a small number of base platforms (maybe one for embedded systems with no
> programmable PMAs, and one for larger machines with a bit more
> configurability).  I'd anticipate that we'll have a platform Kconfig menu entry
> for every platform that gets written down in a specification (just like we have
> an entry for our base ISAs) and then defconfig entries for various
> implementations that select the relevant platform in addition to the drivers
> actually on board.
>
> For now we've got a handful of defconfig entries for the various platforms we
> support (the ISA simulator and our FPGA implementation), but there's no silicon
> so we're not stuck with what's there.  I don't anticipate we'll add more than a
> handful of these until the platform spec work is underway.

Ok. Another related point, which may or may not be obvious: when you
come up with platform definitions, they should not be mutually exclusive.

For instance, supporting both MMU/NOMMU, big/little-endian or 32/64-bit
kernels will of course require building separate binaries, but almost every
other configuration option should be backward compatible: An SMP
kernel should run on a uniprocessor machine and vice versa (using only
one CPU), and you should be able to run a kernel with support multiple
instruction set revisions by restricting the build to the smallest subset
of instructions.

On older ARM platforms and most MIPS platforms, we are still
restricted to building a kernel binary that will only run on a particular
SoC family and not even another SoC with the same CPU core.
Fixing this for most ARM platforms required a lot of work that you
should avoid by requiring them all to work with a common kernel
from the start.

Similarly, ARM has an incompatibility that prevents us from running
on older (ARMv4/v5) along with newer (ARMv6 or higher) instruction
set versions with a single kernel. Avoiding this on RISC-V may
become challenging as one of the strengths of the architecture is
its flexibility: Someone may come up with their own architecture
extensions that they really want to support in the kernel but can't
get it to work without making the kernel binary incompatible with
other implementations. Do you already have a policy for how to
deal with this? Usually by the time someone ships hardware, it's
too late and it becomes hard to argue for their kernel port to not
get merged.

>>>>> +config RV_ATOMIC
>>>>> +       bool "Use atomic memory instructions (RV32A or RV64A)"
>>>>> +       default y
>>>>> +
>>>>> +config RV_SYSRISCV_ATOMIC
>>>>> +       bool "Include support for atomic operation syscalls"
>>>>> +       default n
>>>>> +       help
>>>>> +         If atomic memory instructions are present, i.e.,
>>>>> +         CONFIG_RV_ATOMIC, this includes support for the syscall that
>>>>> +         provides atomic accesses.  This is only useful to run
>>>>> +         binaries that require atomic access but were compiled with
>>>>> +         -mno-atomic.
>>>>> +
>>>>> +         If CONFIG_RV_ATOMIC is unset, this option is mandatory.
>>>>
>>>> I wonder what the cost would be of always providing the syscalls
>>>> for compatibility. This is also something worth putting into a VDSO
>>>> instead of exposing the syscall:
>>>>
>>>> That way, user space that is built with -mno-atomic can call into
>>>> the vdso, which depending on the hardware support will perform
>>>> the atomic operation directly or enter the syscall.
>>>
>>> I think the theory is that most Linux-capable systems are going to have the A
>>> extension, and that this is really there for educational/hobbyist use.
>>>
>>> The actual implementation isn't that expensive: it's no-SMP-only, so it's just
>>> a disable/enable interrupts that wraps a compare+exchange.  I think putting it
>>> in the VDSO is actually the right thing to do: that way non-A user code will
>>> still have reasonable performance.
>>>
>>> I'll add it to my TODO list.
>>
>> This would be another variant of the question above.
>
> After talking with people a bit about this, I think the sane thing to do here
> is to just always provide the atomic system call (and VDSO implementation), and
> to default to enabling the atomic extensions.  While I expect non-atomic
> implementations to be very much the exception, there's a stronger argument for
> supporting non-atomic userspace programs with reasonable performance on atomic
> systems -- the first round of small Linux-capable embedded systems might not
> support atomic (though ours will), but we don't want to wed software to a
> syscall in this case.
>
> Before the VDSO was suggested I was leaning towards disabling the atomic system
> call by default, but since the VDSO implementation will provide very good
> performance for non-atomic userspace code on atomic systems I think it's best
> to just enable it everywhere.  It's only a few bytes of binary size.

Ok.

        Arnd

[toc] | [prev] | [next] | [standalone]

#1659184

On Tue, 06 Jun 2017 02:20:50 PDT (-0700), Arnd Bergmann wrote:
> On Tue, Jun 6, 2017 at 6:56 AM, Palmer Dabbelt <palmer@dabbelt.com> wrote:
>> On Mon, 29 May 2017 04:17:40 PDT (-0700), Arnd Bergmann wrote:
>>> On Sat, May 27, 2017 at 2:57 AM, Palmer Dabbelt <palmer@dabbelt.com> wrote:
>>>> On Tue, 23 May 2017 04:46:22 PDT (-0700), Arnd Bergmann wrote:
>>> This raises a much more general question about how you want to deal
>>> with SoC implementations in the future. The two most common ways of
>>> doing this are:
>>>
>>> - Every major platform gets a Kconfig option in the architecture menu,
>>>   and that selects the essential drivers (irqchip, clocksource, pinctrl,
>>>   clk, ...) that you need for that platform, along with architecture features
>>>   (ISA level and optional features, ...)
>>>
>>> - The architecture code knows nothing about the SoC and just keeps
>>>    to the basics (CPU architecture level selection, SMP/MMU/etc enabled,
>>>    selecting drivers that everyone needs) and leaves the rest up to be
>>>    selected in the defconfig file.
>>>
>>> On ARM, we have a bit of both, which is not as good as being
>>> consistent one way or another.
>>
>> This is actually an open question in RISC-V land right now.  We should be
>> spinning up a platform specification working group this summer to try and work
>> things out.  While this will have to be ironed out, I believe the plan is to
>> define a small number of base platforms (maybe one for embedded systems with no
>> programmable PMAs, and one for larger machines with a bit more
>> configurability).  I'd anticipate that we'll have a platform Kconfig menu entry
>> for every platform that gets written down in a specification (just like we have
>> an entry for our base ISAs) and then defconfig entries for various
>> implementations that select the relevant platform in addition to the drivers
>> actually on board.
>>
>> For now we've got a handful of defconfig entries for the various platforms we
>> support (the ISA simulator and our FPGA implementation), but there's no silicon
>> so we're not stuck with what's there.  I don't anticipate we'll add more than a
>> handful of these until the platform spec work is underway.
>
> Ok. Another related point, which may or may not be obvious: when you
> come up with platform definitions, they should not be mutually exclusive.
>
> For instance, supporting both MMU/NOMMU, big/little-endian or 32/64-bit
> kernels will of course require building separate binaries, but almost every
> other configuration option should be backward compatible: An SMP
> kernel should run on a uniprocessor machine and vice versa (using only
> one CPU), and you should be able to run a kernel with support multiple
> instruction set revisions by restricting the build to the smallest subset
> of instructions.

We're actually trying really hard to ensure a single binary can run anywhere:
for example, an XLEN (the RISC-V term for the general purpose register bit
width) agnostic code sequence can be run that determines if you're on a 32-bit
or 64-bit platform without taking any traps, then enumerate the entire set of
user and supervisor extensions available.  In theory one could build a kernel
image that supports both RV32I and RV64I, as well as optionally supporting
paging.  This won't happen for Linux, but our debugger examines the target chip
using this mechanism.

An explicit goal of the supervisor specification is to make it possible for one
Linux image to run on every platform.  Right now the only constraints we have
is that SMP kernels require atomics.  Specifically:

 * non-SMP kernels run on multiprocessor systems, picking an arbitrary hart to
   run on (or hart 0 if there's no atomics).
 * The kernel is always built with floating-point enabled, it just never sees
   the "FPU is on" bit if there's no FPU and therefor doesn't execute any
   F-extension instructions.

[Though of course after writing this I went to check our boot code and found it
using an unguarded AMO...  Luckily it was an easy fix.  Thanks!]

The big hole right now is that the SBI (our version of Alpha's PAL code) isn't
specified because it's part of the upcoming platform spec.  We'll just have to
home this isn't a problem.

> On older ARM platforms and most MIPS platforms, we are still
> restricted to building a kernel binary that will only run on a particular
> SoC family and not even another SoC with the same CPU core.
> Fixing this for most ARM platforms required a lot of work that you
> should avoid by requiring them all to work with a common kernel
> from the start.

We're hoping that everyone is more amenable to play ball here after seeing how
much pain it was in ARM land.  We'll see how well the platform specification
goes.

> Similarly, ARM has an incompatibility that prevents us from running
> on older (ARMv4/v5) along with newer (ARMv6 or higher) instruction
> set versions with a single kernel. Avoiding this on RISC-V may
> become challenging as one of the strengths of the architecture is
> its flexibility: Someone may come up with their own architecture
> extensions that they really want to support in the kernel but can't
> get it to work without making the kernel binary incompatible with
> other implementations. Do you already have a policy for how to
> deal with this? Usually by the time someone ships hardware, it's
> too late and it becomes hard to argue for their kernel port to not
> get merged.

The RISC-V ISA defines extension mechanisms that allows new non-standard
extensions to coexist, so as long as everyone is well behaved we should be
safe.  I believe there is some mechanism to disallow bad behavior via the
RISC-V foundation and a certification process, but we'll see if that actually
has any teeth.

Of course, we're hoping to only need to support standard extensions, but
nothing ever goes that well :).

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web