Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.arch.embedded > #30715

Re: 64-bit embedded computing is here and now

Show all headers | View raw

On 6/10/2021 3:45 AM, Dimiter_Popoff wrote:

[attrs elided]

>>>>> Not needing to assign overlapping logical addresses to anything
>>>>> can make a big difference to how the OS is done.
>>>>
>>>> That depends on what you expect from the OS.  If you are
>>>> comfortable with the possibility of bugs propagating between
>>>> different subsystems, then you can live with a logical address
>>>> space that exactly coincides with a physical address space.
>>>
>>> So how does the linear 64 bt address space get in the way of
>>> any protection you want to implement? Pages are still 4 k and
>>> each has its own protection attributes governed by the OS,
>>> it is like that with 32 bit processors as well (I talk power, I am
>>> not interested in half baked stuff like ARM, risc-v etc., I don't
>>> know if there could be a problem like that with one of these).
>>
>> With a linear address space, you typically have to link EVERYTHING
>> as a single image to place each thing in its own piece of memory
>> (or use segment based addressing).
> 
> Nothing could be further from the truth. What kind of crippled
> environment can make you think that? Code can be position
> independent on processors which are not dead by design nowadays.
> When I started dps some 27 years ago I allowed program modules
> to demand a fixed address on which they would reside. This exists
> to this day and has been used 0 (zero) times. Same about object
> descriptors, program library modules etc., the first system call
> I wrote is called "allocm$", allocate memory. You request a number
> of bytes and you get back an address and the actual number of
> bytes you were given (it comes rounded by the memory cluster
> size, typically 4k (a page). This was the *first* thing I did.
> And yes, all allocation is done using worst fit strategy, sometimes
> enhanced worst fit - things the now popular OS-s have yet to get to,
> they still have to defragment their disks, LOL.

You missed my point -- possibly because this issue was raised
BEFORE pointing out how much DYNAMIC management of the MMU
(typically an OS delegated acticity) "buys you":
     "That depends on what you expect from the OS."

If you can ignore the MMU *completely*, then the OS is greatly
simplified.  YOU (developer) take on the responsibilites of remembering
what is where, etc.  EVERYTHING is visible to EVERYONE and at
EVERYTIME.  The OS doesn't have to get involved in the management
of objects/tasks/etc.  That's YOUR responsibility to ensure
your taskA doesn't go dicking around with taskB's resources.

Welcome to the 8/16b world!

The next step up is to statically deploy the MMU.  You build
a SINGLE logical address space to suit your liking.  Then, map
the underlying physical resources to it as best fits.  And,
this never needs to change -- memory doesn't "move around",
it doesn't change characteristics (readable, writeable,
exeuctable, accessable-by-X, etc.)!

But, you can't then change permissions based on which task is
executing -- unless you want to dick with the MMU dynamically
(or swap between N discrete sets of STATIC page tables that
define the many different ways M tasks can share permissions)

So, you *just* use the MMU as a Memory Protection Unit; you mark
sections of memory that have CODE in them as no-write, you mark
regions with DATA as no-execute, and everything else as no-access.

And that's the way it stays for EVERY task!

This lets you convert RAM to ROM and prevents "fetches" from "DATA"
memory.  It ensures your code is never overwritten and that the
processor never tries to execute out of "data memory" and NOTHING
tries to access address regions that are "empty"!

You've implemented a 1980's vintage protection scheme (this is how
we designed arcade pieces, back then, as you wanted your CODE
and FRAME BUFFER to occupy the same limited range of addresses)

<yawn>

Once you start using the MMU to dynamically *manage* memory (which
includes altering protections and re-mapping), then the cost of the
OS increases -- because these are typically things that are delegated
*to* the OS.

Whether or not you have overlapping address spaces or a single
flat address space is immaterial -- you need to dynamically manage
separate page tables for each task in either scheme.  You can't
argue that the OS doesn't need to dick with the MMU "because it's
a flat address space" -- unless you forfeit those abilities
(that I illustrated in my post).

If you want to compare a less-able OS to one that is more featured,
then its disingenuous to blame that on overlapping address spaces;
the real "blame" lies in the support of more advanced features.

The goal of an OS should be to make writing *correct* code easier
by providing features as enhancements.  It's why the OS typically
reads disk files instead of replicating that file system and driver
code into each task that needs to do so.  Or, why it implements
delays/timers -- so each task doesn't reinvent the wheel (with its
own unique set of bugs).

You can live without an OS.  But, typically only for a trivial
application.  And, you're not likely to use a 64b processor just
to count characters received on a serial port!  Or as an egg timer!

>> I can share code between tasks without conflicting addressing;
>> the "data" for one instance of the app is isolated from other
>> instances while the code is untouched -- the code doesn't even
>> need to know that it is being invoked on different "data"
>> from one timeslice to the next.  In a flat address space,
>> you'd need the equivalent of a "context pointer" that you'd
>> have to pass to the "shared code".  And, have to hope that
>> all of your context could be represented in a single such
>> reference!  (I can rearrange physical pages so they each
>> appear "where expected" to a bit of const CODE).
>>
>> Similarly, the data passed (or shared) from one task (process) to
>> another can "appear" at entirely different logical addresses
>> "at the same time" as befitting the needs of each task WITHOUT
>> CONCERN (or awareness) of the existence of the other task.
>> Again, I don't need to pass a pointer to the data; the address
>> space has been manipulated to make sure it's where it should be.
> 
> So how do you pass the offset from the page beginning if you do
> not pass an address.

YOU pass an object to the OS and let the OS map it where *it*
wants, with possible hints from the targeted task (logical address
space).

I routinely pass multiple-page-sized objects around the system.

"Here's a 20MB telephone recording, memory mapped (to wherever YOU,
its recipient, want it).  Because it is memory mapped and has its
own pager, the actual amount of physical memory that is in use
at any given time can vary -- based on the resource allocation
you've been granted and the current resource availability in the
system.  E.g., there may be as little as one page of physical
data present at any given time -- and that page may "move" to
back a different logical address based on WHERE you are presently
looking!

Go through and sort out when Bob is speaking and when Tom is speaking.
"Return" an object of UNKNOWN length that lists each of these time
intervals along with the speaker assumed to be talking in each.  Tell
me where you (the OS) decided it would best fit into my logical address
space, after consulting the hint I provided (but that you may not have
been able to honor because the result ended up *bigger* than the "hole"
I had imagined it fitting into).  No need to tell me how big it really
is as I will be able to parse it (cuz I know how you will have built that
list) and the OS will track the memory that it uses so all I have to  do
is free() it (it may be built out of 1K pages, 4K pages, 16MB pages)!"

How is this HARDER to do when a single task has an entire 64b address
space instead of when it has to SHARE *a* single address space among
all tasks/objects?

> And how is page manipulation simpler and/or safer than just passing
> an address, sounds like a recipe for quite a mess to me.

The MMU has made that mapping a "permanent" part of THIS task's
address space.  It isn't visible to any other task -- why *should*
it be?  Why does the pointer need to indirectly reflect the fact
that portions of that SINGLE address space are ineligible to
contain said object because of OTHER unrelated (to this task) objects??

> In a 64 bit address space there is nothing stopping you to
> pass addresses or not passing them and allow access to areas
> you want to and disallow it elsewhere.

And I can't do that in N overlapping 64b address spaces?

The only "win" you get is by exposing everything to everyone.
That's not the way software is evolving.  Compartmentalization
(to protect from other actors), opacity (to hide implementation
details), accessors (instead of exposing actual data), etc.

This comes at a cost -- in performance as well as OS design.
But, *seems* to be worth the effort, given how "mainstream"
development is heading.

> Other than that there is nothing to be gained by a 64 bit architecture
> really, on 32 bit machines you do have FPUs, vector units etc.
> doing calculation probably faster than the integer unit of a
> 64 bit processor.
> The *whole point* of a 64 bit core is the 64 bit address space.

No, the whole point of a 64b core is the 64b registers.
You can package a 64b CPU so that only 20! address lines
are bonded out.  This limits the physical address space
to 20b.  What value to making the logical address
space bigger -- so you can leave gaps for expansion
between objects??

>> The needs of a task can be met by resources "harvested" from
>> some other task.  E.g., where is the stack for your TaskA?
>> How large is it?  How much of it is in-use *now*?  How much
>> can it GROW before it bumps into something (because that something
>> occupies space in "its" address space).
> 
> This is the beauty of 64 bit logical address space. You allocate
> enough logical memory and then you allocate physical on demand,
> this is what MMUs are there for. If you want to grow your stack
> indefinitely - the messy C style - you can just allocate it
> a few gigabytes of logical memory and use the first few kilobytes
> of it to no waste of resources. Of course there are much slicker
> ways to deal with memory allocation.

Again, how is this any harder with "overlapping" 64b address spaces?
Or, how is it EASIER with nonoverlap?

>> I start a task (thread) with a single page of stack.  And, a
>> limit on how much it is allowed to consume during its execution.
>> Then, when it pushes something "off the end" of that page,
>> I fault a new page in and map it at the faulting address.
>> This continues as the task's stack needs grow.
> 
> This is called "allocate on demand" and has been around
> for times immemorial, check my former paragraph.

I'm not trying to be "novel".  Rather, showing that these
features come from the MMU -- not a "nonoverlapping"
(or overlapping!) address space.

I.e., the take away from all this is the MMU is the win
AND the cost for the OS.  Without it, the OS gets simpler...
and less capable!

>> When I run out of available pages, I do a GC cycle to
>> reclaim pages from (other?) tasks that are no longer using
>> them.
> 
> This is called "memory swapping", also for times immemorial.
> For the case when there is no physical memory to reclaim, that
> is.
> The first version of dps - some decades ago - ran on a CPU32
> (a 68340). It had no MMU so I implemented "memory blocks",
> a task can declare a piece  a swap-able block and allow/disallow
> its swapping. Those blocks would then be shared or written to disk when
> more memory was needed etc., memory swapping without an MMU.
> Worked fine, must be still working for code I have not
> touched since on my power machines, all those decades later.

There's no disk involved.  The amount of physical memory
is limited to what's on-board (unless I try to move resources
to another node or -- *gack* -- use a scratch table in the RDBMS
as a backing store).

Recovering "no longer in use" portions of stack is "low hanging fruit";
look at the task's stack pointer and you know how much allocated stack
is no longer in use.  Try to recover it (of course, the task
may immediately fault another page back into play but that's
an optimization issue).

If there is no "low hanging fruit", then I ask tasks to voluntarily
relinquish memory.  Some tasks may have requested "extra" memory
in order to precompute results for future requests/activities.
If it was available -- and if the task wanted to "pay" for it -- then
the OS would grant the allocation (knowing that it could eventually
revoke it!)  They could relinquish those resources at the expense of
having to recompute those things at a later date ("on demand" *or* when
memory is again available).

If I can't recover enough resources "voluntarily", then I
*take* memory away from a (selected) task and inform it
(raise an exception that it will handle as soon as it gets
a timeslice) of that "theft".  It will either recover from
the loss (because it was being greedy and didn't elect
to forfeit excess memory that it had allocated when I asked,
earlier) *or* it will crash.  <shrug>  When you run out
of resources, SOMETHING has to give (and the OS is better
suited to determining WHAT than the individual tasks are...
they ALL think *they* are important!)

Again, "what do you expect from your OS?"

>> In this way, I can effectively SHARE a stack (or heap)
>> between multiple tasks -- without having to give any
>> consideration for where, in memory, they (or the stacks!)
>> reside.
> 
> You can do this in a linear address space, too - this is what
> the MMU is for.

Yes, see?  There's nothing special about a flat address space!

>> I can move a page from one task (full of data) to another
>> task at some place that the destination task finds "convenient".
>> I can import a page from another network device or export
>> one *to* another device.
> 
> So instead of simply passing an address you have to switch page
> translation entries, adjust them on each task switch, flush and
> sync whatever it takes - does not sound very efficient to me.

It's not intended to be fast/efficient.  It's intended to ensure
that the recipient -- AND ONLY THE RECIPIENT -- is *now*
granted access to that page's contents.  depending on semantics,
it can create a copy of an object or "move" the object, leaving
a "hole" in the original location.

[I.e., if move semantics, then the original owner shouldn't be
trying to access something that he's "given away"!  Any access,
by him, to that memory region should signal a fatal exception!]

If you don't care who sees what, then you don't need the MMU!
And we're back to my initial paragraph of this reply!  :>

>> Because each task's address space is effectively empty/sparse,
>> mapping a page doesn't require much effort to find a "free"
>> place for it.
> 
> This is the beauty of having the 64 bit address space, you always
> have enough logical memory. The "64 bit address space per task"
> buys you *nothing*.

If "always having enough logical memory" is such a great thing,
isn't having MORE logical memory (because you've moved other
things into OVERLAPPING portions of that memory space) an
EVEN BETTER thing?

Again, what does your flat addressing BUY the OS in terms of
complexity reduction?  (your initial assumption)
    "...a big difference to how the OS is done"

Back to comp.arch.embedded | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

64-bit embedded computing is here and now James Brakefield <jim.brakefield@ieee.org> - 2021-06-07 07:47 -0700
  Re: 64-bit embedded computing is here and now Don Y <blockedofcourse@foo.invalid> - 2021-06-07 14:13 -0700
  Re: 64-bit embedded computing is here and now Paul Rubin <no.email@nospam.invalid> - 2021-06-07 22:31 -0700
    Re: 64-bit embedded computing is here and now David Brown <david.brown@hesbynett.no> - 2021-06-08 07:59 +0200
      Re: 64-bit embedded computing is here and now Don Y <blockedofcourse@foo.invalid> - 2021-06-08 00:39 -0700
        Re: 64-bit embedded computing is here and now David Brown <david.brown@hesbynett.no> - 2021-06-08 13:04 +0200
          Re: 64-bit embedded computing is here and now Don Y <blockedofcourse@foo.invalid> - 2021-06-08 17:30 -0700
            Re: 64-bit embedded computing is here and now David Brown <david.brown@hesbynett.no> - 2021-06-09 09:17 +0200
              Re: 64-bit embedded computing is here and now Don Y <blockedofcourse@foo.invalid> - 2021-06-09 03:12 -0700
                Re: 64-bit embedded computing is here and now George Neuner <gneuner2@comcast.net> - 2021-06-12 12:58 -0400
                Re: 64-bit embedded computing is here and now Don Y <blockedofcourse@foo.invalid> - 2021-06-12 12:40 -0700
              Re: 64-bit embedded computing is here and now Paul Rubin <no.email@nospam.invalid> - 2021-06-09 09:41 -0700
                Re: 64-bit embedded computing is here and now Don Y <blockedofcourse@foo.invalid> - 2021-06-09 10:12 -0700
                Re: 64-bit embedded computing is here and now DJ Delorie <dj@delorie.com> - 2021-06-09 13:16 -0400
                Re: 64-bit embedded computing is here and now Phil Hobbs <pcdhSpamMeSenseless@electrooptical.net> - 2021-06-09 13:44 -0400
                Re: 64-bit embedded computing is here and now Dimiter_Popoff <dp@tgi-sci.com> - 2021-06-09 22:03 +0300
                Re: 64-bit embedded computing is here and now Phil Hobbs <pcdhSpamMeSenseless@electrooptical.net> - 2021-06-09 15:22 -0400
                Re: 64-bit embedded computing is here and now Paul Rubin <no.email@nospam.invalid> - 2021-06-09 12:58 -0700
                Re: 64-bit embedded computing is here and now Don Y <blockedofcourse@foo.invalid> - 2021-06-09 16:26 -0700
                Re: 64-bit embedded computing is here and now Dimiter_Popoff <dp@tgi-sci.com> - 2021-06-09 23:12 +0300
        Re: 64-bit embedded computing is here and now James Brakefield <jim.brakefield@ieee.org> - 2021-06-08 12:38 -0700
          Re: 64-bit embedded computing is here and now David Brown <david.brown@hesbynett.no> - 2021-06-08 22:11 +0200
            Re: 64-bit embedded computing is here and now James Brakefield <jim.brakefield@ieee.org> - 2021-06-08 14:25 -0700
            Re: 64-bit embedded computing is here and now George Neuner <gneuner2@comcast.net> - 2021-06-09 00:16 -0400
              Re: 64-bit embedded computing is here and now David Brown <david.brown@hesbynett.no> - 2021-06-09 10:40 +0200
                Re: 64-bit embedded computing is here and now Hans-Bernhard Bröker <HBBroeker@t-online.de> - 2021-06-09 22:52 +0200
                Re: 64-bit embedded computing is here and now David Brown <david.brown@hesbynett.no> - 2021-06-10 09:37 +0200
          Re: 64-bit embedded computing is here and now Dimiter_Popoff <dp@tgi-sci.com> - 2021-06-09 01:01 +0300
            Re: 64-bit embedded computing is here and now Don Y <blockedofcourse@foo.invalid> - 2021-06-08 18:29 -0700
              Re: 64-bit embedded computing is here and now Dimiter_Popoff <dp@tgi-sci.com> - 2021-06-09 20:56 +0300
                Re: 64-bit embedded computing is here and now Don Y <blockedofcourse@foo.invalid> - 2021-06-09 17:12 -0700
                Re: 64-bit embedded computing is here and now Dimiter_Popoff <dp@tgi-sci.com> - 2021-06-10 13:45 +0300
                Re: 64-bit embedded computing is here and now Don Y <blockedofcourse@foo.invalid> - 2021-06-10 06:55 -0700
                Re: 64-bit embedded computing is here and now Dimiter_Popoff <dp@tgi-sci.com> - 2021-06-10 18:32 +0300
                Re: 64-bit embedded computing is here and now Don Y <blockedofcourse@foo.invalid> - 2021-06-10 14:09 -0700
                Re: 64-bit embedded computing is here and now Dimiter_Popoff <dp@tgi-sci.com> - 2021-06-11 01:13 +0300
                Re: 64-bit embedded computing is here and now Don Y <blockedofcourse@foo.invalid> - 2021-06-10 21:55 -0700
                Re: 64-bit embedded computing is here and now Dimiter_Popoff <dp@tgi-sci.com> - 2021-06-11 14:14 +0300
                Re: 64-bit embedded computing is here and now Don Y <blockedofcourse@foo.invalid> - 2021-06-11 05:10 -0700
                Re: 64-bit embedded computing is here and now Dimiter_Popoff <dp@tgi-sci.com> - 2021-06-11 16:35 +0300
                Re: 64-bit embedded computing is here and now Don Y <blockedofcourse@foo.invalid> - 2021-06-11 16:24 -0700
              Re: 64-bit embedded computing is here and now George Neuner <gneuner2@comcast.net> - 2021-06-14 05:41 -0400
          Re: 64-bit embedded computing is here and now Don Y <blockedofcourse@foo.invalid> - 2021-06-08 16:51 -0700
          Re: 64-bit embedded computing is here and now Paul Rubin <no.email@nospam.invalid> - 2021-06-08 18:20 -0700
            Re: 64-bit embedded computing is here and now Theo <theom+news@chiark.greenend.org.uk> - 2021-06-09 18:07 +0100
              Re: 64-bit embedded computing is here and now Paul Rubin <no.email@nospam.invalid> - 2021-06-09 10:34 -0700
                Re: 64-bit embedded computing is here and now Don Y <blockedofcourse@foo.invalid> - 2021-06-09 16:37 -0700
              Re: 64-bit embedded computing is here and now Don Y <blockedofcourse@foo.invalid> - 2021-06-09 16:43 -0700
      Re: 64-bit embedded computing is here and now Theo <theom+news@chiark.greenend.org.uk> - 2021-06-08 15:46 +0100
        Re: 64-bit embedded computing is here and now David Brown <david.brown@hesbynett.no> - 2021-06-08 22:18 +0200
          Re: 64-bit embedded computing is here and now Dimiter_Popoff <dp@tgi-sci.com> - 2021-06-08 23:39 +0300
            Re: 64-bit embedded computing is here and now Don Y <blockedofcourse@foo.invalid> - 2021-06-08 18:33 -0700
            Re: 64-bit embedded computing is here and now David Brown <david.brown@hesbynett.no> - 2021-06-09 10:59 +0200
              Re: 64-bit embedded computing is here and now Dimiter_Popoff <dp@tgi-sci.com> - 2021-06-09 21:00 +0300
                Re: 64-bit embedded computing is here and now David Brown <david.brown@hesbynett.no> - 2021-06-09 20:55 +0200
                Re: 64-bit embedded computing is here and now Dimiter_Popoff <dp@tgi-sci.com> - 2021-06-09 22:06 +0300
        Re: 64-bit embedded computing is here and now Don Y <blockedofcourse@foo.invalid> - 2021-06-08 16:00 -0700
          Re: 64-bit embedded computing is here and now Theo <theom+news@chiark.greenend.org.uk> - 2021-06-09 13:10 +0100
            Re: 64-bit embedded computing is here and now Don Y <blockedofcourse@foo.invalid> - 2021-06-09 06:19 -0700

csiph-web