Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.arch.embedded > #25466

Re: g++ on Cortex-M with no dynamic memory

Show all headers | View raw

On 11/8/2016 11:26 PM, Dimiter_Popoff wrote:
>>>> If rotating fans and rotating disk can't be used, the only alternative
>>>> is a Flash disk with limited number of write cycles.
>>>
>>> While generally so VM does not necessarily need to swap memory.
>>> If your physical memory is say 64M and your logical space is set to
>>> say 1G _and_ you know the 64M will never be all used up, the benefit
>>> of the VM is that fragmentation will be much less of a problem.
>>
>> It also lets you "write protect" memory, trap on writes to
>> memory that shouldn't be written, trap on writes to memory
>> that *should* be written, provide protected address spaces,
>> share memory with different processes (and in different places),
>> move large blocks of memory in constant time, etc.
>
> All of the protection etc., of course. When I write something new
> to run under dps many of the mistakes I make are captured by the
> "attempted access to non-allocated space, press CR to kill task" :-).

Yes, handy at debug time but also handy for (VERY) long-lived
processes;  "CAN'T HAPPEN" had really better NOT happen!

> There are also inconveniences associated with VM though.
> Your interrupt handlers cannot be in page translated memory without
> killing the system's latency figures (get an IRQ while fixing a
> page fault and hhave the IRQ handled create a fault of its own....
> ooops, here is the reset button :). So - like it is in dps - the
> interrupt handlers must be in BAT (block array translated) memory
> which has to be set upon boot time such that it can fit all
> allocation requests it will get during a boot session.

There are LOTS of "inconveniences"!

But, there are also lots of *opportunities* that are simply
unavailable without that hardware.

E.g., when I release virtual memory, it need not immediately
be made available to satisfy another request -- ANY "free"
page can suffice!  So, *this* free'd page can be scrubbed clean
(so process A doesn't leak information to process B through this
uncontrolled channel).  That "scrubbing" can happen when the
system has time to do it -- instead of immediately on the
release of the page (as long as it is scrubbed prior to
reallocation)

>> Put the stack "someplace special" (in that large, "empty" address
>> space) and let the hardware tell you when you need to allocate
>> another page for the stack -- or, when you've *underrun* it, etc.
>
> Yes, in dps a task gets started with an allocated data section
> and its (user) stack pointer is put on top of it, command line
> data is on its bottom. Needs just to be large enough, this is
> "logical memory" being allocated, physical gets there just after
> page faults. Well and then there is the same - but much shorter - for
> the system stack and then there is the initialized data section etc.

Part of a thread's configuration specifies how much stack MUST
be resident (in which case, it is allocated/mmap'ed when the task
is spawned) and a maximum size to which it will be allowed to
dynamically grow.  Other parameters govern how aggressively
the system tries to reclaim no-longer-in-use stack space.
(you don't want to foster pathological behaviors wherein you reclaim
a page only to see it faulted back in as soon as the task resumes!)

>> [Things that are hard to do with pure software mechanisms]
>>
>>> Then if memory is allocated using worst-fit strategy fragmentation
>>> is not much of an issue even without VM.
>>>
>>> To me (writing under DPS) stack usage and general allocation
>>> are just both in use, stacks are nice because of what you already
>>> stated, generic allocation (not sure how you'd call that, when
>>> you do a system call to allocate that much memory and you get
>>> returned the address and the actually allocated size (the latter
>>> probably somewhat more than requested because of the granularity))
>>> is good because it can be deallocated "out of order" (unlike stack
>>> frames).
>>
>> The problem with most memory allocators is they are used for a
>> hodge-podge of unstructured/unordered requests.  E.g., override
>> new() and you'll have a better chance at a "well behaved" memory
>> subsystem.
>
> In dps I call "allocation" the act when memory is allocated to a
> requestor based on a bitmap - a bit per cluster. The lowest level
> is the system allocation - a cluster being a page, 4k for power; so
> there is the LCAT (logical memory cluster allocation table) and the
> PCAT (physical memory CAT). Then there is the page translation table.
>
> Tasks get memory allocated by requesting some and getting back
> an address and allocated size (size being a multiple of the cluster
> size, i.e. 4k on 32 bit power). The allocated piece can be recorded
> in a "task history record" so it is deallocated by the system upon
> task kill or it can be not put there (the task can still have put a call
> point in its history records which is called upon kill).
>
> On top of that there is a dps object (dps maintains a runtime objects
> system) called memory_pool; it can have a cluster size as small as just
> a few bytes and be of variable size, can be "made" even from within
> a script, can be "removed" from there etc. Then the pools can get
> automatically chained if needed - i.e. if some object requests
> more memory than there is in any of the pools of a chain a new
> pool is "made" etc.

I support multiple "heaps" that run in the logical memory space.
Depending on the policies the caller uses to manage them, they
can end up as classic heaps, buffer pools, FIFO's, etc.  Let
the developer figure out how memory will behave in his application
and let the heap implementation simply be the mechanisms to make
that happen (the developer sets *policy* in how he uses each heap).

The VMM runs beneath all this.

There are also bridge objects that essentially map the heaps
conveniently onto VMM pages.

As damn near everything in my world is IPC/RPC, this lets
me move objects and sets of objects (e.g., parameters)
between processes just by twiddling the underlying VMM
system.  Of course, silly for small objects that can
simply be passed by copy.

And, for asynchronous operations, it provides immutability
without "burdening" the caller with a copy operation.
E.g., I can pass a frame of raw video to a remote process
and not worry about the caller altering the frame *after*
the call (but before it has actually been "transmitted").
If the caller doesn't fiddle with the frame's contents
until AFTER it has been transmitted, then the call was
"free".  OTOH, if the caller attempts to alter it's contents
BEFORE it is actually transmitted, then the VMM system ensures
a clean copy is transmitted without the caller's "corruption".

[The system then knows to "consume" such an extra copy
after it has been "used"]

> Since having all this I have not had any necessity to think "how
> do I allocate this or that", and well, I have never thought of
> stack growth as of "allocation" (though I do write "allocate a
> new stack frame" when I make one :-)).

Most VMM operations happen without the developer's knowledge,
in my case.  The OS takes care of keeping track of all the
magic so the developer sees a "clean" interface free of
hazzards, etc.

Its tricky to provide these hooks without EXPOSING them
(which would require a more sophisticated class of developer!)

Back to comp.arch.embedded | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

g++ on Cortex-M with no dynamic memory Dave Nadler <drn@nadler.com> - 2016-11-05 14:09 -0700
  Re: g++ on Cortex-M with no dynamic memory antispam@math.uni.wroc.pl - 2016-11-07 02:55 +0000
    Re: g++ on Cortex-M with no dynamic memory Tom Gardner <spamjunk@blueyonder.co.uk> - 2016-11-07 07:52 +0000
      Re: g++ on Cortex-M with no dynamic memory Wouter van Ooijen <wouter@voti.nl> - 2016-11-07 09:11 +0100
        Re: g++ on Cortex-M with no dynamic memory Tim Wescott <tim@seemywebsite.com> - 2016-11-07 10:08 -0600
      Re: g++ on Cortex-M with no dynamic memory David Brown <david.brown@hesbynett.no> - 2016-11-07 10:02 +0100
      Re: g++ on Cortex-M with no dynamic memory Dave Nadler <drn@nadler.com> - 2016-11-08 12:01 -0800
    Re: g++ on Cortex-M with no dynamic memory Dave Nadler <drn@nadler.com> - 2016-11-07 04:13 -0800
      Re: g++ on Cortex-M with no dynamic memory Tim Wescott <tim@seemywebsite.com> - 2016-11-07 10:15 -0600
        Re: g++ on Cortex-M with no dynamic memory Wouter van Ooijen <wouter@voti.nl> - 2016-11-07 18:39 +0100
          Re: g++ on Cortex-M with no dynamic memory Tim Wescott <seemywebsite@myfooter.really> - 2016-11-07 11:59 -0600
            Re: g++ on Cortex-M with no dynamic memory Wouter van Ooijen <wouter@voti.nl> - 2016-11-07 19:40 +0100
            Re: g++ on Cortex-M with no dynamic memory Dave Nadler <drn@nadler.com> - 2016-11-08 11:59 -0800
              Re: g++ on Cortex-M with no dynamic memory Wouter van Ooijen <wouter@voti.nl> - 2016-11-08 23:30 +0100
                Re: g++ on Cortex-M with no dynamic memory David Brown <david.brown@hesbynett.no> - 2016-11-09 09:23 +0100
                Re: g++ on Cortex-M with no dynamic memory Wouter van Ooijen <wouter@voti.nl> - 2016-11-10 07:34 +0100
                Re: g++ on Cortex-M with no dynamic memory David Brown <david.brown@hesbynett.no> - 2016-11-10 10:43 +0100
                Re: g++ on Cortex-M with no dynamic memory Dave Nadler <drn@nadler.com> - 2016-11-11 16:53 -0800
                Re: g++ on Cortex-M with no dynamic memory David Brown <david.brown@hesbynett.no> - 2016-11-12 14:21 +0100
                Re: g++ on Cortex-M with no dynamic memory Jack <jack4747@gmail.com> - 2016-11-21 04:08 -0800
          Re: g++ on Cortex-M with no dynamic memory upsidedown@downunder.com - 2016-11-07 23:01 +0200
            Re: g++ on Cortex-M with no dynamic memory Don Y <blockedofcourse@foo.invalid> - 2016-11-07 14:14 -0700
              Re: g++ on Cortex-M with no dynamic memory upsidedown@downunder.com - 2016-11-08 10:57 +0200
                Re: g++ on Cortex-M with no dynamic memory Don Y <blockedofcourse@foo.invalid> - 2016-11-08 02:31 -0700
                Re: g++ on Cortex-M with no dynamic memory upsidedown@downunder.com - 2016-11-08 13:52 +0200
                Re: g++ on Cortex-M with no dynamic memory Dimiter_Popoff <dp@tgi-sci.com> - 2016-11-08 15:11 +0200
                Re: g++ on Cortex-M with no dynamic memory Don Y <blockedofcourse@foo.invalid> - 2016-11-08 08:01 -0700
                Re: g++ on Cortex-M with no dynamic memory Dimiter_Popoff <dp@tgi-sci.com> - 2016-11-09 08:26 +0200
                Re: g++ on Cortex-M with no dynamic memory Don Y <blockedofcourse@foo.invalid> - 2016-11-09 00:55 -0700
                Re: g++ on Cortex-M with no dynamic memory Dimiter_Popoff <dp@tgi-sci.com> - 2016-11-10 10:27 +0200
                Re: g++ on Cortex-M with no dynamic memory Don Y <blockedofcourse@foo.invalid> - 2016-11-10 02:40 -0700
                Re: g++ on Cortex-M with no dynamic memory Don Y <blockedofcourse@foo.invalid> - 2016-11-08 06:58 -0700
            Re: g++ on Cortex-M with no dynamic memory Wouter van Ooijen <wouter@voti.nl> - 2016-11-07 23:01 +0100
              Re: g++ on Cortex-M with no dynamic memory Dave Nadler <drn@nadler.com> - 2016-11-08 11:38 -0800
        Re: g++ on Cortex-M with no dynamic memory Tim Wescott <tim@seemywebsite.com> - 2016-11-07 20:02 -0600
    Re: g++ on Cortex-M with no dynamic memory upsidedown@downunder.com - 2016-11-07 22:56 +0200
      Re: g++ on Cortex-M with no dynamic memory Don Y <blockedofcourse@foo.invalid> - 2016-11-07 14:11 -0700
        Re: g++ on Cortex-M with no dynamic memory David Brown <david.brown@hesbynett.no> - 2016-11-08 10:37 +0100
          Re: g++ on Cortex-M with no dynamic memory upsidedown@downunder.com - 2016-11-08 13:38 +0200
            Re: g++ on Cortex-M with no dynamic memory David Brown <david.brown@hesbynett.no> - 2016-11-08 13:32 +0100
            Re: g++ on Cortex-M with no dynamic memory Don Y <blockedofcourse@foo.invalid> - 2016-11-08 07:09 -0700
  Re: g++ on Cortex-M with no dynamic memory Wouter van Ooijen <wouter@voti.nl> - 2016-11-07 08:10 +0100

csiph-web