Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.os.msdos.programmer > #1224

Re: Smaller C compiler

Show all headers | View raw

On Wednesday, December 11, 2013 7:47:36 AM UTC-8, Rod Pemberton wrote:
> On Wed, 11 Dec 2013 06:06:12 -0500, Alexei A. Frounze  <...@gmail.com> 
> wrote:

[snip]

> Yeah, I've never seen much written about how Turbo C generated
> decent code - not excellent - so fast.  I'm still curious.
>
> > It's not trivial to implement most of the basic C programming language
> > features in a small compiler and have it compile itself into 64K code
> > + 64K data and at the same time do more than trivial optimizations. For
> > more features and for better optimization I need more memory than that
> > (I'm close to the limit at the moment and I still haven't implemented
> > struct!).
>
> Today, I use files instead of memory for most of my compiler projects.
> You can use fseek() and/or ftell(), or fgetpos() and/or fsetpos(), to
> move around within the file(s).  A cleanly formatted fprintf() to write
> the data, and fscanf() to (re)read the data.  I happen to like my
> programs to output to stdout by default or a file when specified.
> Obviously, you can't rewrite earlier data if the file pointer is stdout.
> Of course, there are file size limits set by the operating system
> as well as representation limits for the seek functions.
>
> Memory is far faster, if you keep your linked-lists, etc, formats
> simple with minimal memory movement, clearing, and/or compaction etc.
> Although, since there are no file I/O like functions which work on
> memory, the coding for memory is more work upfront.  Obviously,
> the quantity of memory is limited which can be a problem with larger
> files, or the limited capacity of the memory allocator.  For memory
> based programs, I usually just warn and exit, if the file is too large.
>
> About five years ago, I did create a set of memory functions which
> treat memory as a file.  Mainly, I saw it as a defect of C to not
> have them.  I called them m_fopen, m_fwrite, m_fread, m_fclose,
> m_rewind, but, you mwrite, mread, mclose, mopen, mrewind would be
> more appropriate.  Basically, this required creating file I/O like
> in stdio.h, but simplified somewhat for memory.  Unfortunately, they
> are only tested for one time usage, single memory file, not multiple
> memory files.  So, five functions, a handful of helper functions,
> an MFILE struct, and a struct for a link to link MFILE structs into
> a linked-list to keep track.  In the end, I decided that file I/O
> using tmpfiles(), which generally use memory for speed, was a better
> choice, since the file I/O for files was fully tested.
>
> > To use more memory I'll need to write and rewrite lots of code
> > to make it still work in DOS in 16-bit mode, which is one of the goals.
> ...
> > I may need to split the compiler into smaller parts and run them one by
> > one or use many more segments of memory (via far pointers) or do
> > something else (use files as memory). At the same time I don't want the
> > compiler to be tied to DOS or have lots of DOS-specific code in its
> > core. And that's a more important goal. It's quite bad when you can't
> > recompile code because it's not portable enough.
>
> That could be a difficult situation.
>
> You're always going to be dealing with segmented memory for 16-bit DOS,
> e.g., far near huge FP_SEG FP_OFF MK_FP.  You might also need to use
> DPMI calls for 32-bit DOS.  That's all environment specific code ...,
> i.e., "non-portable".  The pragma's for packing is different for each
> compiler too.  Include files will be different too.  Buffering of
> streams, especially to the screen, will be different also.  Personally,
> I have no problem using macro's provided by the compiler for each
> compiler and for each processor mode to separate code.  E.g., every
> program for DJGPP and OpenWatcom I have has some code like this:

[snip]

> There will be some at the top for specific compiler includes, and
> usually some in the code for DPMI/MK_FP, or different functions.
> The more you deviate from ANSI C and move into DOS specific code,
> the more different and compiler specific the C code becomes.

I've just spent a couple of days fooling around the ideas of multiple 
segments, far pointers and the huge memory model (as Borland or whoever 
coined it).

The result, I've got Smaller C to compile itself into 16/32-bit real-mode 
code, still a regular DOS MZ .EXE (~200KB). Ints and pointers are 32-bit. 
Pointers are physical addresses (the top 12 bits naturally get lost during 
conversion to segment:offset pairs). Neither of FP_SEG(), FP_OFF() and 
MK_FP() is needed. 64KB segment limit(ation)s are significantly relaxed. The 
stack is still limited to 64K, which is OK, and individual functions are 
limited to 32K in size, which shouldn't be much of a problem either. Other 
than that, unless you want to write inline assembly code (*), things look 
nice and clean. You can use ((unsigned char*)0xB8000)[0] to access the 
character at the top left corner of the screen in text mode.

Of course, the cheap and conservative adjustments in the code generator 
result in significantly worse performance of code compiled using this huge 
memory model, and the code gets bigger as well. But it's OK. It's still a 
toyish compiler, and while it's at this level, it should not be considered 
for serious/important stuff. But for relatively small things (the DOS world 
has now shrunk to small and few things anyway), including some hobby OS dev 
projects, it can be used.

Also, this should allow creation of a usable standard library now that we 
can have 32-bit integers in real-mode code (fseek() and ftell() with their 
at least 32-bit-long file offsets are not a problem anymore).

This new huge stuff is still raw, but I'm expecting to commit it to github 
no later than January 2014.

(*) The code generator generates relocation tables for the huge memory 
model. They are used at startup. Inline assembly code will need to either 
avoid referencing global variables and functions by their names or it will 
need to contribute a record or a few to the relocation tables, which is not 
a big deal, but isn't very nice looking.

Alex

Back to comp.os.msdos.programmer | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Smaller C compiler "Alexei A. Frounze" <alexfrunews@gmail.com> - 2013-12-01 08:15 -0800
  Re: Smaller C compiler "Auric__" <not.my.real@email.address> - 2013-12-01 18:42 +0000
    Re: Smaller C compiler "Alexei A. Frounze" <alexfrunews@gmail.com> - 2013-12-01 19:14 -0800
      Re: Smaller C compiler "Auric__" <not.my.real@email.address> - 2013-12-02 11:13 +0000
        Re: Smaller C compiler "Alexei A. Frounze" <alexfrunews@gmail.com> - 2013-12-02 03:21 -0800
          Re: Smaller C compiler "Auric__" <not.my.real@email.address> - 2013-12-03 06:21 +0000
            Re: Smaller C compiler "Alexei A. Frounze" <alexfrunews@gmail.com> - 2013-12-02 23:17 -0800
              Re: Smaller C compiler "Auric__" <not.my.real@email.address> - 2013-12-03 13:31 +0000
                Re: Smaller C compiler "Alexei A. Frounze" <alexfrunews@gmail.com> - 2013-12-03 06:05 -0800
                Re: Smaller C compiler "Rod Pemberton" <dont_use_email@xnohavenotit.cnm> - 2013-12-03 10:47 -0500
                Re: Smaller C compiler "Alexei A. Frounze" <alexfrunews@gmail.com> - 2013-12-03 21:18 -0800
  Re: Smaller C compiler "Rod Pemberton" <dont_use_email@xnohavenotit.cnm> - 2013-12-03 12:23 -0500
    Re: Smaller C compiler "Rod Pemberton" <dont_use_email@xnohavenotit.cnm> - 2013-12-03 16:26 -0500
      Re: Smaller C compiler "Alexei A. Frounze" <alexfrunews@gmail.com> - 2013-12-03 23:58 -0800
        Re: Smaller C compiler "Rod Pemberton" <dont_use_email@xnohavenotit.cnm> - 2013-12-04 04:31 -0500
          Re: Smaller C compiler "Rod Pemberton" <dont_use_email@xnohavenotit.cnm> - 2013-12-04 05:01 -0500
            Re: Smaller C compiler "Alexei A. Frounze" <alexfrunews@gmail.com> - 2013-12-04 02:20 -0800
              Re: Smaller C compiler "Rod Pemberton" <dont_use_email@xnohavenotit.cnm> - 2013-12-05 06:33 -0500
                Re: Smaller C compiler "Alexei A. Frounze" <alexfrunews@gmail.com> - 2013-12-05 21:37 -0800
                Re: Smaller C compiler "Rod Pemberton" <dont_use_email@xnohavenotit.cnm> - 2013-12-06 12:19 -0500
                Re: Smaller C compiler "Alexei A. Frounze" <alexfrunews@gmail.com> - 2013-12-06 22:23 -0800
                Re: Smaller C compiler "Rod Pemberton" <dont_use_email@xnohavenotit.cnm> - 2013-12-07 13:48 -0500
                Re: Smaller C compiler "Alexei A. Frounze" <alexfrunews@gmail.com> - 2013-12-07 16:34 -0800
                Re: Smaller C compiler "Rod Pemberton" <dont_use_email@xnohavenotit.cnm> - 2013-12-08 02:12 -0500
                Re: Smaller C compiler "Alexei A. Frounze" <alexfrunews@gmail.com> - 2013-12-13 03:32 -0800
                Re: Smaller C compiler "Rod Pemberton" <dont_use_email@xnohavenotit.cnm> - 2013-12-15 04:47 -0500
                Re: Smaller C compiler "Alexei A. Frounze" <alexfrunews@gmail.com> - 2013-12-15 02:14 -0800
                Re: Smaller C compiler "Rod Pemberton" <dont_use_email@xnohavenotit.cnm> - 2013-12-15 13:21 -0500
          Re: Smaller C compiler "Alexei A. Frounze" <alexfrunews@gmail.com> - 2013-12-04 23:39 -0800
            Re: Smaller C compiler "Rod Pemberton" <dont_use_email@xnohavenotit.cnm> - 2013-12-05 06:38 -0500
              Re: Smaller C compiler "Alexei A. Frounze" <alexfrunews@gmail.com> - 2013-12-05 21:40 -0800
              Re: Smaller C compiler "Alexei A. Frounze" <alexfrunews@gmail.com> - 2013-12-06 04:19 -0800
                Re: Smaller C compiler "Rod Pemberton" <dont_use_email@xnohavenotit.cnm> - 2013-12-06 13:40 -0500
                Re: Smaller C compiler "Alexei A. Frounze" <alexfrunews@gmail.com> - 2013-12-14 21:07 -0800
    Re: Smaller C compiler "Alexei A. Frounze" <alexfrunews@gmail.com> - 2013-12-03 22:22 -0800
      Re: Smaller C compiler "Rod Pemberton" <dont_use_email@xnohavenotit.cnm> - 2013-12-04 04:30 -0500
        Re: Smaller C compiler "Alexei A. Frounze" <alexfrunews@gmail.com> - 2013-12-04 23:21 -0800
          Re: Smaller C compiler "Rod Pemberton" <dont_use_email@xnohavenotit.cnm> - 2013-12-05 04:23 -0500
            Re: Smaller C compiler "Alexei A. Frounze" <alexfrunews@gmail.com> - 2013-12-05 21:31 -0800
  Re: Smaller C compiler "Auric__" <not.my.real@email.address> - 2013-12-10 04:59 +0000
    Re: Smaller C compiler "Auric__" <not.my.real@email.address> - 2013-12-10 06:17 +0000
      Re: Smaller C compiler "Alexei A. Frounze" <alexfrunews@gmail.com> - 2013-12-11 02:36 -0800
        Re: Smaller C compiler "Auric__" <not.my.real@email.address> - 2013-12-11 13:44 +0000
  Re: Smaller C compiler Harry Potter <rose.joseph12@yahoo.com> - 2013-12-10 10:41 -0800
    Re: Smaller C compiler "Alexei A. Frounze" <alexfrunews@gmail.com> - 2013-12-11 03:06 -0800
      Re: Smaller C compiler "Rod Pemberton" <dont_use_email@xnohavenotit.cnm> - 2013-12-11 10:47 -0500
        Re: Smaller C compiler Harry Potter <rose.joseph12@yahoo.com> - 2013-12-11 08:01 -0800
          Re: Smaller C compiler "Rod Pemberton" <dont_use_email@xnohavenotit.cnm> - 2013-12-11 13:40 -0500
        Re: Smaller C compiler "Alexei A. Frounze" <alexfrunews@gmail.com> - 2013-12-11 22:07 -0800
        Re: Smaller C compiler "Alexei A. Frounze" <alexfrunews@gmail.com> - 2013-12-25 02:33 -0800
          Re: Smaller C compiler "Alexei A. Frounze" <alexfrunews@gmail.com> - 2013-12-29 03:24 -0800
  Re: Smaller C compiler "Alexei A. Frounze" <alexfrunews@gmail.com> - 2013-12-21 15:55 -0800
  Re: Smaller C compiler "Auric__" <not.my.real@email.address> - 2013-12-29 17:16 +0000
    Re: Smaller C compiler "Rod Pemberton" <dont_use_email@xnohavenotit.cnm> - 2013-12-29 21:54 -0500
      Re: Smaller C compiler "Alexei A. Frounze" <alexfrunews@gmail.com> - 2013-12-29 19:38 -0800
    Re: Smaller C compiler "Alexei A. Frounze" <alexfrunews@gmail.com> - 2013-12-29 19:36 -0800
      Re: Smaller C compiler "Auric__" <not.my.real@email.address> - 2013-12-30 05:07 +0000
        Re: Smaller C compiler "Alexei A. Frounze" <alexfrunews@gmail.com> - 2013-12-29 21:58 -0800
        Re: Smaller C compiler "Alexei A. Frounze" <alexfrunews@gmail.com> - 2014-01-05 19:43 -0800
          Re: Smaller C compiler "Auric__" <not.my.real@email.address> - 2014-01-06 15:27 +0000
            Re: Smaller C compiler "Rod Pemberton" <dont_use_email@xnohavenotit.cnm> - 2014-01-06 19:51 -0500
              Re: Smaller C compiler "Auric__" <not.my.real@email.address> - 2014-01-07 04:27 +0000
                Re: Smaller C compiler "Alexei A. Frounze" <alexfrunews@gmail.com> - 2014-01-06 21:52 -0800
          Re: Smaller C compiler "Alexei A. Frounze" <alexfrunews@gmail.com> - 2014-02-16 22:57 -0800
            Re: Smaller C compiler "Alexei A. Frounze" <alexfrunews@gmail.com> - 2014-02-25 00:17 -0800
              Re: Smaller C compiler "Alexei A. Frounze" <alexfrunews@gmail.com> - 2014-03-01 21:31 -0800
                Re: Smaller C compiler "Alexei A. Frounze" <alexfrunews@gmail.com> - 2014-03-10 01:46 -0700
                Re: Smaller C compiler "Alexei A. Frounze" <alexfrunews@gmail.com> - 2014-04-20 20:19 -0700
                Re: Smaller C compiler Harry Potter <rose.joseph12@yahoo.com> - 2014-04-23 11:20 -0700
                Re: Smaller C compiler "Alexei A. Frounze" <alexfrunews@gmail.com> - 2014-04-23 11:41 -0700
                Re: Smaller C compiler Harry Potter <rose.joseph12@yahoo.com> - 2014-04-25 06:08 -0700
                Re: Smaller C compiler "Alexei A. Frounze" <alexfrunews@gmail.com> - 2014-09-14 03:06 -0700
                Re: Smaller C compiler "Alexei A. Frounze" <alexfrunews@gmail.com> - 2014-11-09 03:26 -0800
                Re: Smaller C compiler "Alexei A. Frounze" <alexfrunews@gmail.com> - 2014-11-28 04:06 -0800
                Re: Smaller C compiler "Alexei A. Frounze" <alexfrunews@gmail.com> - 2014-12-21 01:56 -0800
                Re: Smaller C compiler "Alexei A. Frounze" <alexfrunews@gmail.com> - 2015-01-10 10:37 -0800
                Re: Smaller C compiler "Alexei A. Frounze" <alexfrunews@gmail.com> - 2015-04-19 23:55 -0700
                Re: Smaller C compiler "Auric__" <not.my.real@email.address> - 2015-04-22 06:13 +0000
                Re: Smaller C compiler "Alexei A. Frounze" <alexfrunews@gmail.com> - 2015-04-22 00:12 -0700
                Re: Smaller C compiler "Alexei A. Frounze" <alexfrunews@gmail.com> - 2015-04-25 21:11 -0700
                Re: Smaller C compiler "Alexei A. Frounze" <alexfrunews@gmail.com> - 2015-05-16 17:49 -0700
                Re: Smaller C compiler "Bill Buckels" <bbuckels@mts.net> - 2015-05-19 20:01 -0500
                Re: Smaller C compiler "Alexei A. Frounze" <alexfrunews@gmail.com> - 2015-05-19 18:18 -0700
                Re: Smaller C compiler "Alexei A. Frounze" <alexfrunews@gmail.com> - 2015-08-15 02:13 -0700
                Re: Smaller C compiler "Alexei A. Frounze" <alexfrunews@gmail.com> - 2015-09-06 15:55 -0700

csiph-web