Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > alt.os.development > #8347

Re: 64-bit detection

Show all headers | View raw

"Rod Pemberton" <boo@fasdfrewar.cdm> wrote in message 
news:op.x1m0bkpvyfako5@localhost...
> On Fri, 10 Jul 2015 08:41:43 -0400, James Harris
> <james.harris.1@gmail.com> wrote:
>
>> You may be thinking of canonical addressing wherein the top N bits of 
>> an  address have to be all the same, either all zeroes or all ones.
>
> How do they become set (ones) or cleared (zeroes)?
>
> Is that from sign-extension or done by user/assembler?

I don't think that the upper bits normally get *explicitly* set or 
cleared. It is just that the values given out by the memory allocator 
would be canonically valid. For example, these statements might appear 
in an app.

  xp = &x;
  y = malloc(100);

The first of those would get a canonical address because x will have 
been placed at a suitable memory address when the app was loaded.

The second would get a canonical address because malloc would have been 
set up to work with valid addresses. For example, if you wanted to, you 
could choose to have malloc use low addresses and a kmalloc to give out 
high addresses for kernel memory. Or you could have both malloc and 
kmalloc give out addresses from the low part of memory (where all 
addresses have leading zeroes) as you would if you were running with 
identity paging.

>> You may be thinking of canonical addressing wherein the top N bits of 
>> an  address have to be all the same, either all zeroes or all ones. 
>> AIUI  these are the bits of an address prior to translation via the 
>> page  tables, i.e. the rule applies to virtual addresses rather than 
>> physical  ones.
>
> How many bits are in the address for long mode? 64? 70? 80? 36? ...

Virtual addresses have 64 bits, AIUI.

> What value is N?

According to Wikipedia https://en.wikipedia.org/wiki/X86-64 on AMD's 
initial processors N would be 16 (because they require the top 16 bits 
to match) but the number for a given CPU will depend on that CPU.

> I.e., at this point in time, ISTM, that the canonical addressing
> would limit size of the total address range, if some bits must
> always be set or cleared.  E.g., if N=32, and physical addresses
> are 64-bits, the actual virtual and physical address range would
> only be 32-bits.

It does limit the range but not by much. There can be many address 
spaces and each address space will always have far more addresses than 
the machine has physical RAM.

If you want an example it seems that a number of processors allow 48 
bits of virtual address and 40 bits of physical address. They would 
thereby have 256 times as much address space as the maximum possible 
amount of RAM that the machine could have, and probably many more times 
the amount of actual RAM.

...

>> As we have discussed before, your source is far too dependent on 
>> particular compilers, IMO, but that's your choice.
>>
>> [...]
>>
>> Of course, you would need to migrate to a compiler which could 
>> produce  Long Mode object code. Just one more reason to get away from 
>> compiler  dependence in your source.
>
> I love how you always mention that in regards to _my_ code.

Sorry ... no persecution intended. :-)

Actally, when I mentioned it I was thinking that, like myself, you have 
considered getting parts of the same OS source code to compile to 32-bit 
and 64-bit x86 so I thought it may be of interest.

> My OS wasn't intended to be an OS or to be independent of those
> compilers.  An early goal was to execute executables from both
> compilers.

OK.

> So, yes, while there is a tremendous amount of
> independence, there is also much dependence too.  If I decide
> to no longer use either compiler, then a portion of the design
> of my OS will need to be reworked.

OK.

> E.g., if I decided on a Linux
> compiler, part of the OS would need to be reworked to fit Linux'
> syscall interface.  Or, if I decide to convert to my own toolchain,
> I might be free of such constraints.

I wonder if the code could be adjusted so that is not dependent on 
either DPMI or Linux and did not need to be converted to any particular 
toolchain. Perhaps such code needs an OS-specific linker and loader but 
not an OS-specific compiler. Maybe even the linker could be a generic 
one.

> Hosted C compilers emit code for a specific host OS, specific
> processor mode, specific memory set up such as paging or
> segmented or flat, interrupt method, position independent code
> or not, static or dynamic linked code, C run-time, etc.

You raise some very interesting points. I haven't, until recently, had 
to consider such things and I have been deciding on a few application 
memory layouts (and checking that there are ways to get an *existing* 
build chain to use them). That has delayed my responses to you in the 
"How to bind HLLs to OS calls" thread. I have two of your posts there to 
get back to once I have worked something out. I think I am nearly there. 
I can, at least, make a comment or two here on the points you mentioned 
above.

RP: Hosted C compilers emitting code for:

RP: specific host OS.

If you mean the hosting of the generated code then IMO the compiler 
should not generate *any* OS-dependent code. It should be able to do 
that as long as the source code doesn't make any calls specific to a 
certain OS, shouldn't it?

RP: specific processor mode.

If you mean 16-bit or 64-bit etc then that's OK. Emitting code for a 
particular CPU mode is what the compiler *should* do.

RP: a specific memory setup such as paging or segmented or flat.

So far I intend all apps just to use a flat memory. Perhaps it is odd 
that anything else was ever considered. Exposing the underlying 
different segmentation models so directly to apps was always a bad idea, 
IMO. I can explain that further if you wish but don't want to bore you. 
Suffice to say that when all apps use flat memory (or think they are 
using flat memory) the issues don't arise.

RP: interrupt method.

I am not sure what you mean there but you may be referring to writing 
interrupt-handling code in C. I do that but use assembly to set up a 
suitable environment first. (To be clear, the assembly you saw before 
had two parts: 1. Receive the interrupt and save CPU state which was the 
bulk of the code, 2. set up the environment ready for HLL calls, which 
was small.)

RP: position-independent code or not.

This is a hot topic in that it is something I have been looking at in 
the past couple of days. I have looked at how compilers emit code for 
x86 and ARM for both stand alone apps (not PIC) and shared libraries 
(PIC). I am not sure whether or not I like the existing approach which 
makes PIC slow and bulky but I can at least see why it is done that way: 
shared libraries need to access their static and global data, if they 
have any. I may have an answer to this that is not compiler-depdendent 
but it would make this post overlong so I won't include it just now.

RP: statically or dynamically linked code.

I would want to allow programmers to use the OS as they wish but my 
preference is to have apps distributed as folders of object modules. As 
such, the OS itself would link them so all such linking would appear to 
be dynamic but could use static mechanisms. I am not sure whether that 
covers what you meant, though.

RP: C run time.

Possibly a particular compiler's C run time can be avoided by 1) not 
having a C main function and 2) not invoking any features from the C 
run-time code...?

> IIRC, you're using multiple 16-bit and 32-bit C compilers too
> which aren't part of your own toolchain.  So, I think you're
> going to run into a bunch of dependencies too.  I don't see
> how you can avoid it entirely while using a hosted C compiler.

Do the above points cover the things you had in mind? I should say that 
I am not convinced yet. I am just feeling my way forward on this.

> AIUI, both the OS and it's C compiler must be bootstrapped
> and working together to not have such issues.  That, of course,
> comes after the fact, unless you decide to use C compilers from
> another host as-is for your OS and after your OS is complete,
> like I was doing with DOS compilers.
>
> hosted C compiler -> OS in C -> C compiler ported/written for new OS

ISTM OK for a compiler to generate code for a particular CPU mode. That 
is a fundamental job of a compiler. But I am not sure that a compiler 
should be tied to a particular OS. Maybe it should. Maybe it shouldn't. 
I can see that many are. I am just not sure yet that that is a good or 
necessary approach.

In the other thread we spoke about OS calls. The mechanism used in that 
to make OS calls should work on any compiler and *not* use any compiler 
dependencies so maybe that is an example of avoiding the OS-dependencies 
built in to a compiler.

> For my original design, the initial and final C compilers above are
> the same.  I don't have to bootstrap, port, or code a C compiler
> for my OS.  I use the same C compiler I used to write the OS.

OK.

> For
> your design, they're likely going to be different.  You need a
> hosted C compiler that can produce host independent code, and once
> the initial OS is complete, you're going to need another C compiler
> designed to specifically work with your OS, i.e., hosted, that must
> be bootstrapped, ported, or written from scratch.  So, the need for
> compiler independence for your design is greater, IMO IIUC.

What would it take for a compiler to be independent of the hosting OS? 
Some suggestions:

* The compiler, when compiling code, should not make calls that are 
dependent on a particular OS. It should handle IO and the like by having 
a library to interface the compiler's calls to those of the underlying 
OS.

* The compiler should not generate code which is dependent on a 
particular OS. Perhaps it should handle this by having the compiler do 
only what it is told to do...!

James

Back to alt.os.development | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

64-bit detection "Benjamin David Lunt" <zfysz@fysnet.net> - 2015-07-09 16:47 -0700
  Re: 64-bit detection "Rod Pemberton" <boo@fasdfrewar.cdm> - 2015-07-09 23:41 -0400
    Re: 64-bit detection "James Harris" <james.harris.1@gmail.com> - 2015-07-10 08:30 +0100
      Re: 64-bit detection "Rod Pemberton" <boo@fasdfrewar.cdm> - 2015-07-10 05:17 -0400
        Re: 64-bit detection "James Harris" <james.harris.1@gmail.com> - 2015-07-10 13:41 +0100
          Re: 64-bit detection "James Harris" <james.harris.1@gmail.com> - 2015-07-12 19:04 +0100
            Re: 64-bit detection "Rod Pemberton" <boo@fasdfrewar.cdm> - 2015-07-13 02:30 -0400
              Re: 64-bit detection "Alexei A. Frounze" <alexfrunews@gmail.com> - 2015-07-13 02:07 -0700
                Re: 64-bit detection "Rod Pemberton" <boo@fasdfrewar.cdm> - 2015-07-13 23:10 -0400
                Re: 64-bit detection "Alexei A. Frounze" <alexfrunews@gmail.com> - 2015-07-14 01:23 -0700
              Re: 64-bit detection "James Harris" <james.harris.1@gmail.com> - 2015-07-13 12:34 +0100
                Re: 64-bit detection "Rod Pemberton" <boo@fasdfrewar.cdm> - 2015-07-13 22:50 -0400
                Re: 64-bit detection "James Harris" <james.harris.1@gmail.com> - 2015-07-14 08:37 +0100
                Re: 64-bit detection "Alexei A. Frounze" <alexfrunews@gmail.com> - 2015-07-14 01:41 -0700
                Re: 64-bit detection "James Harris" <james.harris.1@gmail.com> - 2015-07-14 10:04 +0100
      Re: 64-bit detection "Benjamin David Lunt" <zfysz@fysnet.net> - 2015-07-10 08:44 -0700
      Re: 64-bit detection "Benjamin David Lunt" <zfysz@fysnet.net> - 2015-07-10 10:52 -0700
    Re: 64-bit detection "Benjamin David Lunt" <zfysz@fysnet.net> - 2015-07-10 08:38 -0700
      Re: 64-bit detection "Benjamin David Lunt" <zfysz@fysnet.net> - 2015-07-10 10:58 -0700
  Re: 64-bit detection CN <qmbmnp3799@pacbell.net> - 2015-07-09 23:00 -0700
    Re: 64-bit detection "Benjamin David Lunt" <zfysz@fysnet.net> - 2015-07-10 08:40 -0700
      Re: 64-bit detection "James Harris" <james.harris.1@gmail.com> - 2015-07-10 21:29 +0100

csiph-web