Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.arch.embedded > #18717 > unrolled thread

x86 real mode

Back to article view | Back to comp.arch.embedded

Contents

  x86 real mode Don Y <this@is.not.me.com> - 2014-10-17 07:23 -0700
    Re: x86 real mode Arlet Ottens <usenet+5@c-scape.nl> - 2014-10-17 16:31 +0200
      Re: x86 real mode Don Y <this@is.not.me.com> - 2014-10-17 07:45 -0700
        Re: x86 real mode Arlet Ottens <usenet+5@c-scape.nl> - 2014-10-17 16:53 +0200
          Re: x86 real mode Don Y <this@is.not.me.com> - 2014-10-17 08:28 -0700
            Re: x86 real mode Tauno Voipio <tauno.voipio@notused.fi.invalid> - 2014-10-17 18:35 +0300
            Re: x86 real mode Arlet Ottens <usenet+5@c-scape.nl> - 2014-10-17 17:36 +0200
              Re: x86 real mode Don Y <this@is.not.me.com> - 2014-10-17 13:54 -0700
                Re: x86 real mode Grant Edwards <invalid@invalid.invalid> - 2014-10-17 21:02 +0000
                  Re: x86 real mode Don Y <this@is.not.me.com> - 2014-10-17 14:54 -0700
                Re: x86 real mode upsidedown@downunder.com - 2014-10-18 10:09 +0300
            Re: x86 real mode Grant Edwards <invalid@invalid.invalid> - 2014-10-17 16:11 +0000
        Re: x86 real mode Stefan Reuther <stefan.news@arcor.de> - 2014-10-17 18:37 +0200
        Re: x86 real mode Les Cargill <lcargill99@comcast.com> - 2014-10-17 12:58 -0500
    Re: x86 real mode David Brown <david.brown@hesbynett.no> - 2014-10-17 16:52 +0200
      Re: x86 real mode glen herrmannsfeldt <gah@ugcs.caltech.edu> - 2014-10-17 16:45 +0000
    Re: x86 real mode Robert Wessel <robertwessel2@yahoo.com> - 2014-10-17 10:03 -0500
    Re: x86 real mode Grant Edwards <invalid@invalid.invalid> - 2014-10-17 16:07 +0000
    Re: x86 real mode upsidedown@downunder.com - 2014-10-17 19:57 +0300
    Re: x86 real mode Les Cargill <lcargill99@comcast.com> - 2014-10-17 12:57 -0500
    Re: x86 real mode Tauno Voipio <tauno.voipio@notused.fi.invalid> - 2014-10-17 21:18 +0300
      Re: x86 real mode glen herrmannsfeldt <gah@ugcs.caltech.edu> - 2014-10-17 21:48 +0000
        Re: x86 real mode Don Y <this@is.not.me.com> - 2014-10-17 15:06 -0700
          Re: x86 real mode glen herrmannsfeldt <gah@ugcs.caltech.edu> - 2014-10-17 22:25 +0000
            Re: x86 real mode Don Y <this@is.not.me.com> - 2014-10-17 15:39 -0700
              Re: x86 real mode George Neuner <gneuner2@comcast.net> - 2014-10-17 22:47 -0400
                Re: x86 real mode Don Y <this@is.not.me.com> - 2014-10-17 21:10 -0700
                  Re: x86 real mode upsidedown@downunder.com - 2014-10-18 11:32 +0300
              Re: x86 real mode upsidedown@downunder.com - 2014-10-18 11:10 +0300
                Re: x86 real mode Don Y <this@is.not.me.com> - 2014-10-18 08:20 -0700
                  Re: x86 real mode upsidedown@downunder.com - 2014-10-18 20:34 +0300
                    Re: x86 real mode Don Y <this@is.not.me.com> - 2014-10-18 12:53 -0700
            Re: x86 real mode upsidedown@downunder.com - 2014-10-18 10:30 +0300
            Re: x86 real mode David Brown <david.brown@hesbynett.no> - 2014-10-18 20:22 +0200
        Re: x86 real mode upsidedown@downunder.com - 2014-10-18 10:22 +0300
          Re: x86 real mode glen herrmannsfeldt <gah@ugcs.caltech.edu> - 2014-10-18 09:06 +0000
            Re: x86 real mode Robert Wessel <robertwessel2@yahoo.com> - 2014-10-18 08:43 -0500
            Re: x86 real mode Gerd Brix <gbrix@gmx.eu> - 2014-10-18 16:39 +0200
            Re: x86 real mode Don Y <this@is.not.me.com> - 2014-10-18 17:52 -0700
              Re: x86 real mode George Neuner <gneuner2@comcast.net> - 2014-10-20 01:35 -0400
                Re: x86 real mode upsidedown@downunder.com - 2014-10-20 10:14 +0300
                  Re: x86 real mode George Neuner <gneuner2@comcast.net> - 2014-10-20 14:12 -0400
                Re: x86 real mode Don Y <this@is.not.me.com> - 2014-10-20 08:15 -0700
      Re: x86 real mode Les Cargill <lcargill99@comcast.com> - 2014-10-17 18:39 -0500
        Re: x86 real mode Tauno Voipio <tauno.voipio@notused.fi.invalid> - 2014-10-18 10:46 +0300
    Re: x86 real mode George Neuner <gneuner2@comcast.net> - 2014-10-17 15:29 -0400
      Re: x86 real mode Don Y <this@is.not.me.com> - 2014-10-17 14:32 -0700
        Re: x86 real mode Hans-Bernhard Bröker <HBBroeker@t-online.de> - 2014-10-18 00:09 +0200
          Re: x86 real mode glen herrmannsfeldt <gah@ugcs.caltech.edu> - 2014-10-17 22:31 +0000
            Re: x86 real mode upsidedown@downunder.com - 2014-10-18 12:04 +0300
              Re: x86 real mode glen herrmannsfeldt <gah@ugcs.caltech.edu> - 2014-10-18 23:22 +0000
              Re: x86 real mode Don Y <this@is.not.me.com> - 2014-10-18 20:59 -0700
        Re: x86 real mode upsidedown@downunder.com - 2014-10-18 11:53 +0300
          Re: x86 real mode Don Y <this@is.not.me.com> - 2014-10-18 09:34 -0700
      Re: x86 real mode glen herrmannsfeldt <gah@ugcs.caltech.edu> - 2014-10-17 22:00 +0000
        Re: x86 real mode Hans-Bernhard Bröker <HBBroeker@t-online.de> - 2014-10-18 00:26 +0200
          Re: x86 real mode glen herrmannsfeldt <gah@ugcs.caltech.edu> - 2014-10-17 22:36 +0000
        Re: x86 real mode George Neuner <gneuner2@comcast.net> - 2014-10-17 21:45 -0400
          Re: x86 real mode glen herrmannsfeldt <gah@ugcs.caltech.edu> - 2014-10-18 03:26 +0000
            Re: x86 real mode Robert Wessel <robertwessel2@yahoo.com> - 2014-10-17 23:26 -0500
              Re: x86 real mode glen herrmannsfeldt <gah@ugcs.caltech.edu> - 2014-10-18 05:42 +0000
            Re: x86 real mode Tauno Voipio <tauno.voipio@notused.fi.invalid> - 2014-10-18 10:53 +0300
              Re: x86 real mode glen herrmannsfeldt <gah@ugcs.caltech.edu> - 2014-10-18 09:12 +0000
          Re: x86 real mode Robert Wessel <robertwessel2@yahoo.com> - 2014-10-17 22:40 -0500
        Re: x86 real mode George Neuner <gneuner2@comcast.net> - 2014-10-17 22:07 -0400

Page 3 of 4 — ← Prev page 1 2 [3] 4  Next page →

#18842

On Mon, 20 Oct 2014 01:35:03 -0400, George Neuner
<gneuner2@comcast.net> wrote:

>On Sat, 18 Oct 2014 17:52:08 -0700, Don Y <this@is.not.me.com> wrote:
>
>
>>Effectively, a "program" only executes in two rings -- a
>>"user" ring ("4") and the "system" ring ("0").
>>
>>As with many technical issues, coming up with the "right number"
>>is hard.  Too few and you lose the value they present.  Too
>>many and a "program" spends too much time crossing protection domains
>>("for little value")
>
>I think there's a clear need for at least 4 general protection zones:
>kernel, device driver, OS services and user applications.

That should be quite sufficient. 

The interrupt/device driver  environment should have an own "ring"
especially, if there are a lot of third party device drivers, which
quality and compatibility with other kernel mode services might be
questionable.

Running OS services such as file systems (e.g. RMS in VAX in executive
mode) in some intermediate ring between kernel and user mode makes
sense, since that only need to address the user address space in
addition own ring, while kernel/driver code will access the OS service
ring as well.
  
>Beyond that you're getting into user politics.

There are usually private user mode address spaces for each process,
so a single user mode ring would be sufficient. If there are needs to
share memory between two user processes, memory mapped files etc. with
process specific (RW/RO/EXEC) access should be sufficient.

>IIRC NT 3.x and OS/2 1.x/2.x did use rings 0, 1 and 3.  NT 4 retreated
>to using only rings 0 and 3, and I *think* OS/2 Warp versions did
>also.  

NT3.x had the GUI routines in user mode. Since this was a quite low
level interface, the frequents switching between user and kernel mode
caused a lot of overhead.

With NT 4.0, those routines were moved to kernel mode. Apparently they
did not check the codes at all, since in NT4 SP0, you could crash the
whole system, by mistakenly sending a NULL pointer, when a valid
pointer was required. The access in kernel mode to the unmapped 0 page
of caused an unhandled page fault and crash. 

The first thing a kernel mode API should do is to check, if the
requested memory access would have been allowed from the calling (here
user mode) and immediately reject such request, before even attempting
such memory access. With NT4 SP1, most of these checks were added and
that is the first more or less stable NT4 version.

>And x86 now has dropped segmentation altogether in 64-bit mode
>so the rings aren't even available.

Early 386+ processors only had some protection bits in the segment
registers. If I understand later 32 and new 64 bit processors have
extra protection bits somewhere in the page table hierarchy. 

On a decent virtual memory hardware, these bits should be on each page
table entry.
 

[toc] | [prev] | [next] | [standalone]

#18854

On Mon, 20 Oct 2014 10:14:35 +0300, upsidedown@downunder.com wrote:

>On Mon, 20 Oct 2014 01:35:03 -0400, George Neuner
><gneuner2@comcast.net> wrote:
>
>>And x86 now has dropped segmentation altogether in 64-bit mode
>>so the rings aren't even available.
>
>Early 386+ processors only had some protection bits in the segment
>registers. If I understand later 32 and new 64 bit processors have
>extra protection bits somewhere in the page table hierarchy. 

x86-32 have both segmentation and paging and the protections overlap
somewhat, but the rings are part of the segment.

The only new thing (and not so new) is NX - No eXecute - bits on the
pages.


>On a decent virtual memory hardware, these bits should be on each page
>table entry.
 
There are important use cases - JIT compilation and GC prominently
among them - that involve rapidly changing permissions on large blocks
of addresses.  It is inefficient to do this at page granularity and
that is one reason for the introduction of "large" VMM pages into most
OSes.  However, large pages present their own set of problems such as
protection granularity, lengthy page I/O, address space fragmentation
and pollution[*], interoperability issues dealing with differently
sized pages (allowed by most OSes) including frame allocation
fragmentation, partial pages, etc.

Segments done properly (which Don and I have discussed in the past)
solve a lot of protection domain problems very nicely.  They are _not_
a good solution for virtualizing memory, but they should be part of
any general protection solution.

Intel's attempt to layer protection on top of the existing real mode
base registers was not well thought out ... they should have left the
existing base:offset addressing alone (or removed it entirely) and
introduced a new separate protection API.  Unfortunately, Intel is the
only experience of protected segments that most programmers have and
so they are, IMO, too willing to dispense with something that can be
very useful simply because they haven't seen it done well.

George

[*] even in the immense 56-bit address space of today's 64-bit
processors, overly large VMM pages can be wasteful.  Some systems now
allow pages to be up to 1GB and most permit different processes to use
different page sizes (some even permit a single process to use
different page sizes).  Physical memories, while large, still are tiny
relative to the address space and though "allocated" addresses consume
no physical memory, "mapped" addresses do and touching a page maps all
the addresses it contains (and possibly also maps additional space in
backing store).

[toc] | [prev] | [next] | [standalone]

#18850

Hi George,

On 10/19/2014 10:35 PM, George Neuner wrote:
> On Sat, 18 Oct 2014 17:52:08 -0700, Don Y <this@is.not.me.com> wrote:
>
>> Effectively, a "program" only executes in two rings -- a
>> "user" ring ("4") and the "system" ring ("0").
>>
>> As with many technical issues, coming up with the "right number"
>> is hard.  Too few and you lose the value they present.  Too
>> many and a "program" spends too much time crossing protection domains
>> ("for little value")
>
> I think there's a clear need for at least 4 general protection zones:
> kernel, device driver, OS services and user applications.  Beyond that
> you're getting into user politics.
>
> IIRC NT 3.x and OS/2 1.x/2.x did use rings 0, 1 and 3.  NT 4 retreated
> to using only rings 0 and 3, and I *think* OS/2 Warp versions did
> also.  And x86 now has dropped segmentation altogether in 64-bit mode
> so the rings aren't even available.

I think the multics folks looked at this with an attitude of
"if these features were free, what would be the 'right/best' way
of designing a complex system".

I.e., a *user program* could benefit from the same schemes that
benefit the "system".  So, let's extend the ring concept into
user-land!

Of course, crossing protection boundaries is not free.  And, designing
with that sort of strict structure takes considerably more planning than
the ad hoc approaches so common!  :-/

I'm sure I've got a copy of Organick here, someplace...  but, I'd wager
few (if any!) user-land programs took advantage of any of this!  [A
possible exception *may* have been The Consistent System]

As we've discussed before, I really think just two execution levels
are adequate for most systems **if** you have fine-grained control
of access to peripherals (whether they are memory- or I/O-mapped).

[toc] | [prev] | [next] | [standalone]

#18767

Tauno Voipio wrote:
> On 17.10.14 17:23, Don Y wrote:
>> Hi,
>>
>> [What a screwed up architecture!]
>>
>> Thx,
>> --don
>
> Actually, Intel did not intend to use the 8086 real mode
> as such, only as bootstrap mode for 80286. The success
> of 8088 and original PC came as a surprise, and it influenced
> the design of 80386 with the virtual 8086 mode.
>
> There were no PC's at the design time of 8086 and 80286,
> and the aim was for well protected real-time multitasking
> applications.

For an 8086? I'm pretty sure it was all 100% "real mode".

> So it was natural that 80286 does not voluntarily
> return back to real mode from protected mode. This led to
> a massive kludge in PC/AT, where the keyboard controller
> can reset the main processor and the RTC CMOS RAM will
> contain a code explaining why the reset was this time.
>
-- 
Les Cargill

[toc] | [prev] | [next] | [standalone]

#18781

On 18.10.14 02:39, Les Cargill wrote:
> Tauno Voipio wrote:
>> On 17.10.14 17:23, Don Y wrote:
>>> Hi,
>>>
>>> [What a screwed up architecture!]
>>>
>>> Thx,
>>> --don
>>
>> Actually, Intel did not intend to use the 8086 real mode
>> as such, only as bootstrap mode for 80286. The success
>> of 8088 and original PC came as a surprise, and it influenced
>> the design of 80386 with the virtual 8086 mode.
>>
>> There were no PC's at the design time of 8086 and 80286,
>> and the aim was for well protected real-time multitasking
>> applications.
>
> For an 8086? I'm pretty sure it was all 100% "real mode".

No - it was intentionally made similar to the boot-up mode
of a 80286. There would not be any segment registers in
any sensible 16+ bit processor. A bank switcher would have
been much simpler.

A segmented processor architecture was well known decades
before the 80(2)86, so the registers were originally
intended to point to segment descriptors a la -286+.

I was at an invitational Intel meeting in 1980 (PC is from -81)
and had some interesting discussions with the designers. They
already had some of -386 planned, so that -286 would not
become incompatible with it.

-- 

-TV

[toc] | [prev] | [next] | [standalone]

#18745

Hi Don,

On Fri, 17 Oct 2014 07:23:40 -0700, Don Y <this@is.not.me.com> wrote:

>What are the most conservative, *practical* expectations I can
>make living in x86 real mode:  TEXT of 64K and DATA of (disjoint)
>64K?  BSS in it's own segment?  Or, shared with DATA?

If you use the tiny (64K code+data) or small (64K code, 64K data)
model, .bss and stack share space with your data.  If you choose a
model with multiple data segments, .bss and stack can be separate (but
tool dependent you may have to ask for it explicitly).


>And, how seemlessly will the compiler let me *implicitly* move
>those segments around as well as between them?  (e.g., practical
>limitations on code/data sizes).  "PC" handled 640K so should
>I expect that to be the size of my playground?

The linker determines where your segments get placed (assuming the OS
permits the requested placement).


>[Presumably, any "object" is constrained to fit within a single
>segment]

In the "huge" model arrays and structs can straddle segments. However,
you should take care that no individual element of an array or struct
lies across a segment boundary ... I have found that many compilers
don't renormalize intermediate pointers when indexing from a huge
pointer, so you can get into trouble when the offset is close to or
exceeds 64K.  I've been bitten by that more times than I care to
admit.
[The solution is to deliberately construct a new huge pointer.  When
you store the new pointer it will be normalized and thus safe to use.]

Arithmetic on "far" pointers affects only the 16-bit offset and wraps
at the segment boundary.  This means that two far pointers having
different segments can't reasonably be compared.  If you are using
multiple data segments and you need to compare pointers, you have to
use normalized "huge" pointers.  Do be aware that huge pointer
arithmetic can be quite a bit slower due to (re)normalization of the
results.

You really only need to choose the huge model if you expect a single
array or struct to exceed 64K.  If you need huge pointers purely for
comparison, they can be mixed with far pointers in any of the multiple
data segment models.  Because far pointer arithmetic is faster, they
should be preferred wherever you can live with their limitations.


>I imagine this will all be accomplished in the linkage editor
>(not visible to the source code).

Depends on the tool - some compilers use pragmas to control placement
of objects into particular segments.  Placement of the segments
themselves is controlled by the linker (and/or OS).


>[What a screwed up architecture!]

It isn't bad until you grow beyond 64K data and are forced into
explicit use of far or huge pointers.  There's a model that allows
>64K code and <=64K data where the compiler transparently handles code
pointers so you don't have to worry about them.


>Thx,
>--don

George

[toc] | [prev] | [next] | [standalone]

#18753

Hi George,

On 10/17/2014 12:29 PM, George Neuner wrote:
> On Fri, 17 Oct 2014 07:23:40 -0700, Don Y <this@is.not.me.com> wrote:
>
>> What are the most conservative, *practical* expectations I can
>> make living in x86 real mode:  TEXT of 64K and DATA of (disjoint)
>> 64K?  BSS in it's own segment?  Or, shared with DATA?
>
> If you use the tiny (64K code+data) or small (64K code, 64K data)
> model, .bss and stack share space with your data.  If you choose a
> model with multiple data segments, .bss and stack can be separate (but
> tool dependent you may have to ask for it explicitly).

It seems the "safest" (most conservative) assumption is to figure
a single 64KB address space.  If, instead, the x86 ALWAYS prepared
a separate data space (etc), then I could assume a larger model.

Or, if the tiny model was IMPRACTICAL for any use (and was just
included as an homage to the 8085).

Or, if ints were always 32b, etc.

I.e., it seems safe to assume the tiny model was intended to be
*usable* and not just "an engineering/marketing exercise".

>> And, how seemlessly will the compiler let me *implicitly* move
>> those segments around as well as between them?  (e.g., practical
>> limitations on code/data sizes).  "PC" handled 640K so should
>> I expect that to be the size of my playground?
>
> The linker determines where your segments get placed (assuming the OS
> permits the requested placement).

OK.  Point of my question being the oddities of the x86 architecture
aren't (necessarily) exposed in the software.  I need to come up
with a solution that is readily portable to non-x86 architectures.

>> [Presumably, any "object" is constrained to fit within a single
>> segment]
>
> In the "huge" model arrays and structs can straddle segments. However,
> you should take care that no individual element of an array or struct
> lies across a segment boundary ... I have found that many compilers
> don't renormalize intermediate pointers when indexing from a huge
> pointer, so you can get into trouble when the offset is close to or
> exceeds 64K.  I've been bitten by that more times than I care to
> admit.
> [The solution is to deliberately construct a new huge pointer.  When
> you store the new pointer it will be normalized and thus safe to use.]

Understood.  I recall the 4KB object limitations that the Z180 presented.
This shouldn't be hard to accommodate (if considered AHEAD OF TIME and
not shoehorned into the design after-the-fact).

(actually, I think I can probably leverage some of my more ancient codebases
if I can locate them  :-/  I hadn't considered that possibility!)

Thanks!
--don

[toc] | [prev] | [next] | [standalone]

#18759

Am 17.10.2014 um 23:32 schrieb Don Y:

> It seems the "safest" (most conservative) assumption is to figure
> a single 64KB address space.

Huh?  How do you figure that the possibility that causes your own code 
to make _the_most_ assumptions about the behaviour of unknown, 
surrounding code, to be the _safest_?  When did it become safe to make 
unwarranted assumptions?

> If, instead, the x86 ALWAYS prepared
> a separate data space (etc), then I could assume a larger model.

Well, yes, it essentially always does just that.  All code addressing is 
relative to CS, all data addressing is relative to DS or SS, at least by 
default.  And since all those are allowed to be different, the only 
a-priori _safe_ assumption obviously is that they will be.

> Or, if the tiny model was IMPRACTICAL for any use

It's practical if the program is small enough to allow it, and the 
program loader being used supports it.

The usual one at the time did: tiny model is what MS-DOS *.com format 
programs use.  They're copied verbatim from disk to RAM (beginning 0x100 
bytes into a 64 KB page) , then all three primary segment registers are 
set to the start of that RAM region (i.e. CS=DS=SS=constant) and the 
loader just jumps to CS::0x0100.  So code and pre-initialized data are 
in one big block of RAM.  The remainder of those 64K are assumed to be 
used for .bss and stack.

The advantage was that the program loader doesn't have to make any code 
modifications to handle relocation of the code ... all addresses in the 
code are just offsets relative to the load base, which itself doesn't 
ever appear in the code, because it's kept in the segment registers.

The disadvantage was that the loader _didn't_ perform relocations, so 
there was no way to load bigger programs.  So along came the EXE program 
file format, which is a whole different kettle of fish.

> I.e., it seems safe to assume the tiny model was intended to be
> *usable* and not just "an engineering/marketing exercise".

It stopped being truly usable when programs outgrew 64 KB for code and 
all(!) data combined.

[toc] | [prev] | [next] | [standalone]

#18762

Hans-Bernhard Bröker <HBBroeker@t-online.de> wrote:
> Am 17.10.2014 um 23:32 schrieb Don Y:
 
(snip)
>> If, instead, the x86 ALWAYS prepared
>> a separate data space (etc), then I could assume a larger model.
 
> Well, yes, it essentially always does just that.  All code addressing is 
> relative to CS, all data addressing is relative to DS or SS, at least by 
> default.  And since all those are allowed to be different, the only 
> a-priori _safe_ assumption obviously is that they will be.

Not so unusual to keep DS and SS together.

Also, even for the 8080 you could have separate address space
for stack and data. There was a way to decode which data references
were to the stack.  I don't know anyone ever did that, though.
 
>> Or, if the tiny model was IMPRACTICAL for any use
 
> It's practical if the program is small enough to allow it, and the 
> program loader being used supports it.

Well, pretty much they did it the same way as CP/M, which allows
for easy porting.
 
> The usual one at the time did: tiny model is what MS-DOS *.com format 
> programs use.  They're copied verbatim from disk to RAM (beginning 0x100 
> bytes into a 64 KB page) , then all three primary segment registers are 
> set to the start of that RAM region (i.e. CS=DS=SS=constant) and the 
> loader just jumps to CS::0x0100.  So code and pre-initialized data are 
> in one big block of RAM.  The remainder of those 64K are assumed to be 
> used for .bss and stack.

> The advantage was that the program loader doesn't have to make any code 
> modifications to handle relocation of the code ... all addresses in the 
> code are just offsets relative to the load base, which itself doesn't 
> ever appear in the code, because it's kept in the segment registers.
 
> The disadvantage was that the loader _didn't_ perform relocations, so 
> there was no way to load bigger programs.  So along came the EXE program 
> file format, which is a whole different kettle of fish.
 
The descriptions I remember for COM didn't say that you were stuck
with 64K, but that it was the program's responsibility to keep
track of everything. You had all the rest of memory to use,
COMMAND.COM would be reloaded from disk if you wrote over it.
 
>> I.e., it seems safe to assume the tiny model was intended to be
>> *usable* and not just "an engineering/marketing exercise".
 
> It stopped being truly usable when programs outgrew 64 KB 
> for code and all(!) data combined.

-- glen

[toc] | [prev] | [next] | [standalone]

#18787

On Fri, 17 Oct 2014 22:31:44 +0000 (UTC), glen herrmannsfeldt
<gah@ugcs.caltech.edu> wrote:

>Hans-Bernhard Bröker <HBBroeker@t-online.de> wrote:
>> Am 17.10.2014 um 23:32 schrieb Don Y:
> 
>(snip)
>>> If, instead, the x86 ALWAYS prepared
>>> a separate data space (etc), then I could assume a larger model.
> 
>> Well, yes, it essentially always does just that.  All code addressing is 
>> relative to CS, all data addressing is relative to DS or SS, at least by 
>> default.  And since all those are allowed to be different, the only 
>> a-priori _safe_ assumption obviously is that they will be.
>
>Not so unusual to keep DS and SS together.
>
>Also, even for the 8080 you could have separate address space
>for stack and data. There was a way to decode which data references
>were to the stack.  I don't know anyone ever did that, though.

Some of the 8080 support chip had some exotic outputs that you could
determine what type of access was intended, but did also include
data/stack access  and not just instruction/RWdata access ?

[toc] | [prev] | [next] | [standalone]

#18813

upsidedown@downunder.com wrote:

(snip, I wrote)

>>Not so unusual to keep DS and SS together.

>>Also, even for the 8080 you could have separate address space
>>for stack and data. There was a way to decode which data references
>>were to the stack.  I don't know anyone ever did that, though.
 
> Some of the 8080 support chip had some exotic outputs that you could
> determine what type of access was intended, but did also include
> data/stack access  and not just instruction/RWdata access ?

I believe all three. You can have separate address spaces for
code, stack, and data. 

Among others, Intel claimed a 64K entry call stack, though as
it is two bytes each it should only be 32K. Maybe with extra logic
you can make it 64K. But only if it is a separate address space
from code and data.

-- glen

[toc] | [prev] | [next] | [standalone]

#18816

On 10/18/2014 2:04 AM, upsidedown@downunder.com wrote:
>> Also, even for the 8080 you could have separate address space
>> for stack and data. There was a way to decode which data references
>> were to the stack.  I don't know anyone ever did that, though.
>
> Some of the 8080 support chip had some exotic outputs that you could
> determine what type of access was intended, but did also include
> data/stack access  and not just instruction/RWdata access ?

On the 8080, you could "snoop" on the internal actions of the processor.
On the 8085, much of this "status" was covered up and replaced by a
terser "read/write:data/code" summary.
On the Z80, even less information was *easily* accessible (aside from
"fetch/notfetch".

OTOH, you could often get a glimpse of the internal *operations* by
monitoring pins at "undocumented" times.  E.g., IIRC, it was easy to
see 16 bit increments (opcodes) "on the pins".

And, the encodings of many opcodes made it easy to snoop on "select"
operations with a minimal of external hardware.  E.g., you could
use things like the RST's as an economical means of syncing/commanding
external hardware without elaborate decoding logic.

[And, the fact that you could jam ANY instruction opcode onto the bus
during an INTA cycle allowed for some really clever implementations
effectively making chip internals accessible to external hardware
"for low cost"]

The Z80 (and later 180 family parts) tended to be more *practical* in
the signals that they presented to the pin drivers.  E.g., extending the
I/O space to 64K was a significant win (esp because you didn't have
to *use* it that way!)

It would have been interesting to see what Zilog *could* have
done with the Z80 had they not been so dysfunctional at that time.

Talk about throwing away your market...  :-/

[toc] | [prev] | [next] | [standalone]

#18786

On Fri, 17 Oct 2014 14:32:20 -0700, Don Y <this@is.not.me.com> wrote:

>Hi George,
>
>On 10/17/2014 12:29 PM, George Neuner wrote:
>> On Fri, 17 Oct 2014 07:23:40 -0700, Don Y <this@is.not.me.com> wrote:
>>
>>> What are the most conservative, *practical* expectations I can
>>> make living in x86 real mode:  TEXT of 64K and DATA of (disjoint)
>>> 64K?  BSS in it's own segment?  Or, shared with DATA?
>>
>> If you use the tiny (64K code+data) or small (64K code, 64K data)
>> model, .bss and stack share space with your data.  If you choose a
>> model with multiple data segments, .bss and stack can be separate (but
>> tool dependent you may have to ask for it explicitly).
>
>It seems the "safest" (most conservative) assumption is to figure
>a single 64KB address space.  If, instead, the x86 ALWAYS prepared
>a separate data space (etc), then I could assume a larger model.
>
>Or, if the tiny model was IMPRACTICAL for any use (and was just
>included as an homage to the 8085).
>
>Or, if ints were always 32b, etc.
>
>I.e., it seems safe to assume the tiny model was intended to be
>*usable* and not just "an engineering/marketing exercise".

What is the problem with 64 KiB program address space with much larger
total system memory ?

In a previous company, my department used machines with 64 KiB or 128
KiB (I+D) and we never had problems with the program address space
limits. Just split the functionality into a sufficient number of
tasks.

Of course if you need to access a huge data base, this was
problematic. 

[toc] | [prev] | [next] | [standalone]

#18797

On 10/18/2014 1:53 AM, upsidedown@downunder.com wrote:
> On Fri, 17 Oct 2014 14:32:20 -0700, Don Y <this@is.not.me.com> wrote:
>
>> Hi George,
>>
>> On 10/17/2014 12:29 PM, George Neuner wrote:
>>> On Fri, 17 Oct 2014 07:23:40 -0700, Don Y <this@is.not.me.com> wrote:
>>>
>>>> What are the most conservative, *practical* expectations I can
>>>> make living in x86 real mode:  TEXT of 64K and DATA of (disjoint)
>>>> 64K?  BSS in it's own segment?  Or, shared with DATA?
>>>
>>> If you use the tiny (64K code+data) or small (64K code, 64K data)
>>> model, .bss and stack share space with your data.  If you choose a
>>> model with multiple data segments, .bss and stack can be separate (but
>>> tool dependent you may have to ask for it explicitly).
>>
>> It seems the "safest" (most conservative) assumption is to figure
>> a single 64KB address space.  If, instead, the x86 ALWAYS prepared
>> a separate data space (etc), then I could assume a larger model.
>>
>> Or, if the tiny model was IMPRACTICAL for any use (and was just
>> included as an homage to the 8085).
>>
>> Or, if ints were always 32b, etc.
>>
>> I.e., it seems safe to assume the tiny model was intended to be
>> *usable* and not just "an engineering/marketing exercise".
>
> What is the problem with 64 KiB program address space with much larger
> total system memory ?

Given a CLEAN SLATE, there's no problem with *anything*!  :>
The "problem" lies in living within existing constraints AND
writing portable code.  E.g., no "near"/"far" nonsense.  Take the
machine the way it existed when your code was invoked, "do your
thing" -- and leave the machine in the state that you found it
(or, explore ALL of the consequences of ANY changes you may
have introduced to its state)

> In a previous company, my department used machines with 64 KiB or 128
> KiB (I+D) and we never had problems with the program address space
> limits. Just split the functionality into a sufficient number of
> tasks.

Sure, and I've written UNIX-like OS's to run on Z80's (bankswitching
instead of swapping).  No reason the terminal handler needs to
co-reside in the same address space as the OS; no reason for the
OS to share an address space with the application, either!

But, that sort of code is anything BUT "portable".  The next
host architecture may have an entirely different set of
constraints imposed by its hardware.

> Of course if you need to access a huge data base, this was
> problematic.

[toc] | [prev] | [next] | [standalone]

#18757

George Neuner <gneuner2@comcast.net> wrote:

(snip)

> In the "huge" model arrays and structs can straddle segments. However,
> you should take care that no individual element of an array or struct
> lies across a segment boundary ... I have found that many compilers
> don't renormalize intermediate pointers when indexing from a huge
> pointer, so you can get into trouble when the offset is close to or
> exceeds 64K.  I've been bitten by that more times than I care to
> admit.

Seems like a compiler bug to me.  If you assign to a far pointer,
then it is your fault.

> [The solution is to deliberately construct a new huge pointer.  When
> you store the new pointer it will be normalized and thus safe to use.]
 
> Arithmetic on "far" pointers affects only the 16-bit offset and wraps
> at the segment boundary.  This means that two far pointers having
> different segments can't reasonably be compared.  If you are using
> multiple data segments and you need to compare pointers, you have to
> use normalized "huge" pointers.  Do be aware that huge pointer
> arithmetic can be quite a bit slower due to (re)normalization of the
> results.

You don't say what language you use, but C only allows comparison
of pointers to different objects as equal or not equal. With large
model, objects (arrays and structs) can't get larger than 64K.
(For C, they can't even equal 64K.)  A C compiler should generate
properly comparable (for equal or not equal) pointers in large
model.
 
> You really only need to choose the huge model if you expect a single
> array or struct to exceed 64K.  If you need huge pointers purely for
> comparison, they can be mixed with far pointers in any of the multiple
> data segment models.  Because far pointer arithmetic is faster, they
> should be preferred wherever you can live with their limitations.

In many cases, I have used 2D arrays, actual C arrays of pointers
to arrays.  As long as the array of pointers and pointed-to arrays
fit in 64K, you can get plenty big enough. 

I believe in the OS/2 1.x days, I had 16M on my system before
converting to OS/2 2.x and appropriate compilers.

I could generate arrays to at least 1 million elements of double,
so about 1000x1000, which easily fits. 

>>I imagine this will all be accomplished in the linkage editor
>>(not visible to the source code).
 
> Depends on the tool - some compilers use pragmas to control placement
> of objects into particular segments.  Placement of the segments
> themselves is controlled by the linker (and/or OS).
 
>>[What a screwed up architecture!]
 
> It isn't bad until you grow beyond 64K data and are forced into
> explicit use of far or huge pointers.  

As long as you stay within small or large, it isn't all that hard.
It is mixed models (using a few far pointers in small model) that
gets confusing. Go completely to large, never use the far keyword,
and everything works fine.

> There's a model that allows
>>64K code and <=64K data where the compiler transparently handles code
> pointers so you don't have to worry about them.

There is medium and compact, and I forget which is which.

I usually generate libraries for small and large, and only used those.
(When you install the MS C compiler, you select which models you
plan to use, and it generates the appropriate libraries.)

-- glen

[toc] | [prev] | [next] | [standalone]

#18761

Am 18.10.2014 um 00:00 schrieb glen herrmannsfeldt:
> Seems like a compiler bug to me.  If you assign to a far pointer,
> then it is your fault.

Since there was no formal definition other than the individual 
compiler's documentation of what a "far" pointer did and did not to in 
various situations, the only way it could be a bug would have been if it 
didn't match its own documentation.

> You don't say what language you use, but C only allows comparison
> of pointers to different objects as equal or not equal.

Since neither "far" nor "huge" pointers even exist in the language 
standard, none of its rules apply to them unless the compiler authors 
decide to say so.

They decided not to.  I actually had to grow some data structures' sizes 
to the next power-of-two so a huge pointer could correctly walk a >64KiB 
array of it.  The horror!

[toc] | [prev] | [next] | [standalone]

#18763

Hans-Bernhard Bröker <HBBroeker@t-online.de> wrote:
> Am 18.10.2014 um 00:00 schrieb glen herrmannsfeldt:
>> Seems like a compiler bug to me.  If you assign to a far pointer,
>> then it is your fault.
 
> Since there was no formal definition other than the individual 
> compiler's documentation of what a "far" pointer did and did not to in 
> various situations, the only way it could be a bug would have been 
> if it didn't match its own documentation.

Well, it has to match the standard that the compiler claims.

If you aren't using mixed models, the compiler has to consistently
support the model in use. (That is, no near or far keyword.)

But most probably didn't want to go all the way to huge, as it
would slow down (and increase code size) for all pointers. 

Mixed model, it is your responsibility to get it right.

>> You don't say what language you use, but C only allows comparison
>> of pointers to different objects as equal or not equal.
 
> Since neither "far" nor "huge" pointers even exist in the language 
> standard, none of its rules apply to them unless the compiler authors 
> decide to say so.

The C standard allows for a variety of pointer models, including
those of large and huge model. It doesn't allow for the "far"
and "huge" keyword.
 
> They decided not to.  I actually had to grow some data structures' sizes 
> to the next power-of-two so a huge pointer could correctly walk a >64KiB 
> array of it.  The horror!

-- glen

[toc] | [prev] | [next] | [standalone]

#18768

On Fri, 17 Oct 2014 22:00:01 +0000 (UTC), glen herrmannsfeldt
<gah@ugcs.caltech.edu> wrote:

>George Neuner <gneuner2@comcast.net> wrote:
> 
>> Arithmetic on "far" pointers affects only the 16-bit offset and wraps
>> at the segment boundary.  This means that two far pointers having
>> different segments can't reasonably be compared.  If you are using
>> multiple data segments and you need to compare pointers, you have to
>> use normalized "huge" pointers.  Do be aware that huge pointer
>> arithmetic can be quite a bit slower due to (re)normalization of the
>> results.
>
>You don't say what language you use, but C only allows comparison
>of pointers to different objects as equal or not equal. With large
>model, objects (arrays and structs) can't get larger than 64K.
>(For C, they can't even equal 64K.)  A C compiler should generate
>properly comparable (for equal or not equal) pointers in large
>model.

No, actually it won't - at least not in the "large" model with Intel
or Micro$oft compilers.

Far pointers are constructed by left shifting the segment value by 4
bits and adding a 16 bit offset - i.e. they are a 20 bit address
expressed as 4:16.  In real mode there are no reserved segment values
so there are 4096 possible far pointers for *every* location in memory
(with wraparound arithmetic there is a possible segment base every
16-bytes within -32K..+32K of the location).

Huge pointers are 16:4, the 20 bit address normalized to a 16 bit
segment and a 4 bit offset.  This ensures that every distinct huge
pointer value references a unique location in the 1MB space.

The problem is that normalization is costly (a few extra instructions)
and compilers don't do it unless you explicitly ask for it by using
the huge model or by using huge pointers in other models.

 
>In many cases, I have used 2D arrays, actual C arrays of pointers
>to arrays.  As long as the array of pointers and pointed-to arrays
>fit in 64K, you can get plenty big enough. 
>
>I believe in the OS/2 1.x days, I had 16M on my system before
>converting to OS/2 2.x and appropriate compilers.

OS/2 1.x operated in 80286 protected mode, which has other issues that
are not relevant to Don's question.

That said, you are absolutely correct that there are ways to get
around the limitations of 16-bit segments.


>There is medium and compact, and I forget which is which.

Yeah, I don't recall which is which just now, but one is 64K code +
1MB data and the other is the reverse.  

The "large" model allows 1MB code+data using unnormalized pointers.
The "huge" mode is the same but using normalized pointers.

George

[toc] | [prev] | [next] | [standalone]

#18771

George Neuner <gneuner2@comcast.net> wrote:

(snip, someone wrote)

>>> different segments can't reasonably be compared.  If you are using
>>> multiple data segments and you need to compare pointers, you have to
>>> use normalized "huge" pointers.  Do be aware that huge pointer
>>> arithmetic can be quite a bit slower due to (re)normalization of the
>>> results.

(then I wrote)
>>You don't say what language you use, but C only allows comparison
>>of pointers to different objects as equal or not equal. With large
>>model, objects (arrays and structs) can't get larger than 64K.
>>(For C, they can't even equal 64K.)  A C compiler should generate
>>properly comparable (for equal or not equal) pointers in large
>>model.
 
> No, actually it won't - at least not in the "large" model with Intel
> or Micro$oft compilers.
 
> Far pointers are constructed by left shifting the segment value by 4
> bits and adding a 16 bit offset - i.e. they are a 20 bit address
> expressed as 4:16.  In real mode there are no reserved segment values
> so there are 4096 possible far pointers for *every* location in memory
> (with wraparound arithmetic there is a possible segment base every
> 16-bytes within -32K..+32K of the location).

Yes, but the compiler should make sure that only one of those
is ever used. That is, unless you cheat and change them yourself.
 
> Huge pointers are 16:4, the 20 bit address normalized to a 16 bit
> segment and a 4 bit offset.  This ensures that every distinct huge
> pointer value references a unique location in the 1MB space.

I tried to avoid huge, but yes. 

Actually, no. I am pretty sure that at least MSC 6.x, which is what
I used at the time, generated segment selectors 4K apart and
16 bit offsets.  The other way doesn't work right with OS/2. 
You don't want to allocates a segment selector for every 16 bytes.

The compilers ran under either DOS or OS/2, and generated the
same code in either case. I even used to generate bound executables
which run under either DOS or OS/2.

But in any case, it is the responsibility of the compiler to
make it work, if you follow the C standard. Normalizing them
yourself isn't standard C.  Casting a huge pointer to a far
pointer isn't, either. 

I don't think I ever tried arrays of struct of unusual size. 
It might have padded them out.  I did use large model pretty much
all the time, and never had any problems. 
 
> The problem is that normalization is costly (a few extra instructions)
> and compilers don't do it unless you explicitly ask for it by using
> the huge model or by using huge pointers in other models.

And I pretty much never used it. But a 4K entry array of pointers
to 8K doubles allows for 32MB which was big for a 286.
 
>>In many cases, I have used 2D arrays, actual C arrays of pointers
>>to arrays.  As long as the array of pointers and pointed-to arrays
>>fit in 64K, you can get plenty big enough. 

>>I believe in the OS/2 1.x days, I had 16M on my system before
>>converting to OS/2 2.x and appropriate compilers.
 
> OS/2 1.x operated in 80286 protected mode, which has other issues that
> are not relevant to Don's question.
 
> That said, you are absolutely correct that there are ways to get
> around the limitations of 16-bit segments.
 
>>There is medium and compact, and I forget which is which.
 
> Yeah, I don't recall which is which just now, but one is 64K code +
> 1MB data and the other is the reverse.  
 
> The "large" model allows 1MB code+data using unnormalized pointers.
> The "huge" mode is the same but using normalized pointers.

Actually, large allows for a lot more than 1MB. And OS/2 will swap
whole segments to/from disk, so you can do really large arrays.

OS/2 1.x allowed for 8K segment selectors for a user process,
each could be up to 64K, so 512MB.  With DOS, only 640K, or if
you use the right display board, you can go up a litle more.

I started OS/2 1.0 in 1990, and tried not to use DOS after that.

-- glen

[toc] | [prev] | [next] | [standalone]

#18775

On Sat, 18 Oct 2014 03:26:45 +0000 (UTC), glen herrmannsfeldt
<gah@ugcs.caltech.edu> wrote:

>George Neuner <gneuner2@comcast.net> wrote:
>
>(snip, someone wrote)
>
>>>> different segments can't reasonably be compared.  If you are using
>>>> multiple data segments and you need to compare pointers, you have to
>>>> use normalized "huge" pointers.  Do be aware that huge pointer
>>>> arithmetic can be quite a bit slower due to (re)normalization of the
>>>> results.
>
>(then I wrote)
>>>You don't say what language you use, but C only allows comparison
>>>of pointers to different objects as equal or not equal. With large
>>>model, objects (arrays and structs) can't get larger than 64K.
>>>(For C, they can't even equal 64K.)  A C compiler should generate
>>>properly comparable (for equal or not equal) pointers in large
>>>model.
> 
>> No, actually it won't - at least not in the "large" model with Intel
>> or Micro$oft compilers.
> 
>> Far pointers are constructed by left shifting the segment value by 4
>> bits and adding a 16 bit offset - i.e. they are a 20 bit address
>> expressed as 4:16.  In real mode there are no reserved segment values
>> so there are 4096 possible far pointers for *every* location in memory
>> (with wraparound arithmetic there is a possible segment base every
>> 16-bytes within -32K..+32K of the location).
>
>Yes, but the compiler should make sure that only one of those
>is ever used. That is, unless you cheat and change them yourself.
> 
>> Huge pointers are 16:4, the 20 bit address normalized to a 16 bit
>> segment and a 4 bit offset.  This ensures that every distinct huge
>> pointer value references a unique location in the 1MB space.
>
>I tried to avoid huge, but yes. 
>
>Actually, no. I am pretty sure that at least MSC 6.x, which is what
>I used at the time, generated segment selectors 4K apart and
>16 bit offsets.  The other way doesn't work right with OS/2. 
>You don't want to allocates a segment selector for every 16 bytes.


IIRC, the usual scheme for huge mode support was to always use 64KB
segments, and then tweak the segment arithmetic to match the tiling.
So for real mode, consecutive segments for a huge object were 4096
apart.  In PM, the OS allocated consecutive selectors, which were
numerically 8 apart.  So an 80KB huge object might be in segments
0x1234 and 0x2234 in real mode, and in selectors 0x4567 and 0x456F in
protected mode.  The environment provided some run-time readable
variables to define the increment (4096 vs. 8) and corresponding shift
counts (12 and 3).

[toc] | [prev] | [next] | [standalone]

Page 3 of 4 — ← Prev page 1 2 [3] 4  Next page →

Back to top | Article view | comp.arch.embedded

csiph-web