Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.forth > #56940

Re: LLVM and Forth

Show all headers | View raw

On Sat, 8 Apr 2017 22:45:51 -0700 (PDT)
Joel Rees <joel.rees@gmail.com> wrote:

> On Friday, April 7, 2017 at 4:32:11 PM UTC+9, Rod Pemberton wrote:
> > On Thu, 06 Apr 2017 12:33:44 GMT
> > Spiros Bousbouras <spibou@(something).(something)> wrote:

> > > Example : the expression  array[i]  where i is beyond the end of
> > > the array.  
> > 
> > FYI, C explicitly allows you to access one element past the end of
> > the array.  This is to allow you to access the nul character when
> > comparing strings to check for string termination.  
> 
> Now we know who has been writing all that vulnerable code. (Just
> kidding. Sort of.)

This is used within the C libraries, e.g., string and memory functions,
as well as by programmers.

> I have heard teachers say that setting a pointer to NULL and then
> dereferencing it is safe. 

Setting a pointer to any explicit value is undefined behavior in C.
This is implementation defined behavior.  Most OSes and resulting C
compilers are reasonable in that they allow the programmer to access
memory mapped devices.  Hence, they allow you to assign values to
pointers.

> (There are, in fact, some embedded environments where it is safe, but
> those are special cases. Still not what I call clean coding practice.)

In many OS environments, you'll end up with a page fault.  The OS will
not map the zero'th page in order to detect NULL pointer dereferences.

> C array syntax allows you to access whatever a [pointer] points at,
> and pointer parameters can be named as if they were indefinitely
> allocated arrays. Moreover, the grammar specifically allows arbitrary
> array references even when the l-value is an array with a specific
> declared size.

I'm not sure I quite follow here.

The C language has array declarations, but doesn't have arrays as a
fundamental type as part of the language.  C uses the [] subscript
operator with a pointer and offset to effect array usage.  The name of
an array reduces to a pointer to the zero'th element of the array.  This
allows the array name to be used with the subscript operator.

> In the past, certain compiler vendors allocated extra space around
> arrays to "help protect" the end users from programmers who did not
> understand that C does not bounds check arrays, and therefore assumed
> that no breakage this time meant their loops or other array
> processing was correct. This would give the appearance of allowing
> dereferencing one past the end of an array.

C explicitly allows dereferencing one past the end of an array.  As
stated previously, this requirement is in ALL of the C specifications.

> It also allowed programmers to get thoroughly confused when they
> started working with multi-dimension arrays, because the compiler and
> libraries can't cheat like that for you on multi-dimension arrays.
...

> If you read, for instance, the library documentation for string
> processing functions in the standard library (fgets(), for instance),
> you will note that the buffer size passed in must include room for a
> trailing NUL if you want to be sure the trailing NUL gets stored.
> 
> This kind of thing should help you understand that something doesn't
> match what you just said.

You're confusing strings and arrays.  C doesn't have strings as a
fundamental type, but uses arrays, which are a partial type.  Part of
the parsing process of C requires that an extra character for nul be
added onto the end of the string.  This is done automatically by the
compiler unless a specific size is declared for the string.  In the case
of fgets(), the programmer must account for the extra character needed
to terminate strings.

> The standard does specify that the run-time has to allow a pointer to
> point one past the end of an allocated area, as long as the pointer
> is not dereferenced in that state. This allows a loop that has a
> post-increment in the test to complete safely. 
...

> But that means, no, you must not actually access one beyond. That way
> lies one-off pointer errors which become vulnerabilities which allow
> servers and workstations to be pwned by malicious access.

Accessing one beyond is required to check for a nul character.  E.g.,

while(*str++); /* find end-of-string *//* This is not UB */

> >  If you go beyond that though,
> > it's undefined ...  However, many C implementations use the ability
> > to access outside a C object behind the scenes without any issues,
> > e.g., to access a memory header prior to malloc'd memory blocks.  
> 
> When you talk about memory allocation headers, you imply two levels
> (at least) of allocation -- a mass block allocation, and a fine-grain
> allocation.

It doesn't imply that.  The memory block and header can be allocated at
once.

> If you try to access beyond either end of a mass block allocation,
> you should expect segment faults.

This assumes that paging is enabled and memory isn't contiguously
mapped.  The later is an implicit requirement of C.  Every C object is
required to be contiguous.  This is usually effected by making memory
contiguously mapped.

> Within a mass block, the library maintains an allocation list of some
> sort, and that list might use headers that sit either edge of the
> fine-grained allocation block. The grammar and syntax do allow you to
> take a look at those headers, but you really should not, because
> other library architects don't want to risk allocation headers being
> overwritten by stray pointers. So the allocation lists are elsewhere
> in those runt-times. 
...

> The grammar and syntax allow the accesses, and it is not just one
> past. A million past is also allowed by the grammar and syntax. The
> run-time, however, is only required to complete an operation safely
> if it is within the declared bounds. 

This is not true.  For C, you can legally only access data within C
objects or malloc'd memory.  Any other access is undefined behavior.  C
makes an exception for strings to allow accessing one past the
end-of-string to check for nul characters.  Assigning a value directly
to a pointer is also undefined behavior for C.  Many environments allow
this to access memory mapped devices.

> > > Thinking in terms of assembly : "array[i] will be translated into
> > > an assembly instruction which calculates an address of the form
> > > base + i * constant .  
> > 
> > This is a requirement of the subscript operator [] for K&R, ANSI,
> > and ISO C.   
> 
> Not really required, just the most natural interpretation of a
> one-dimensional array reference.
> 
> > It's in all the standards.  They define "E1[E2]" to be
> > equivalent to "*((E1)+(E2))".  The subscript operator takes a
> > pointer to a type and also an index.  The index is scaled by the
> > size of the data type pointed to.  The pointer which cannot be a
> > pointer-to-void 'void *' since 'void' has no data type, i.e., no
> > size, pointer only. The pointer used in the subscript operator can
> > be any pointer to any valid type, not just pointer for array
> > types.   
> 
> Up to here you are just talking about arbitrary math.

No, I'm talking about how the subscript operator [] in C works.  Most
people think it's for array indexes (or indices), but it works with
pointers too.  C doesn't have arrays within the language itself, but
has array declarations.  These are reduced to pointers by the compiler,
as mentioned elsewhere.

> > The arguments to the
> > subscript operator can be either order, e.g., "0123456789"[i] is
> > equivalent to i["0123456789"] .  
> 
> This does not make sense. I know what "string"[index] means, but I
> and the compiler are both extremely puzzled by index["string"]. This
> is not perl. "string" may resolve to a pointer, but pointers and
> indexes have two different sorts of existence, which you refer to
> when you mention void pointers. You can't use a pointer as an index
> very meaningfully.

This is valid C.  The compiler knows the type of both parameters for
the subscript operator [] . The compiler knows that the string is a
pointer to char and it knows the index is one of the integer types.

> > Technically, you should think of the
> > subscript operator as just the left-brace, then you'll understand
> > it's like any other operator in C, e.g., arithmetic +, -> component
> > selection, or whatever.  
> 
> Syntactically similar, semantically significantly different.
> 
...

> > > Your second reference has
> > >         char *index;
> > >         ....
> > >             *index++ = toupper(*index);
> > > 
> > > .He doesn't explain why he decided to write code like this as
> > > opposed to *index = toupper(*index);
> > >         index++ ;  
> > 
> > I'm guessing that this is most likely just pre-ANSI C.  Most C style
> > guides etc are older and are for K&R C.  E.g., most will recommend
> > that you use #defines and mask with logical-and instead of using
> > bit-fields, since many earlier C compilers were broken.   
> 
> The implementation of toupper() is not really relevant to the
> semantics of 
> 
>     *index = toupper( *index++ );
> 
> demonstrates the need of sequence points. The assignment is a
> sequence point there is the assignment operator, "=".
> 
> ++index is equivalent to index = index + 1 in several ways, but the
> increment does not contain a sequence point. 
>

It indicates the author didn't follow ANSI C coding rules which require
that pre- and post- increment and pre- and post-decrement be removed
from surrounding code.  This is because they introduced sequence point
concept with ANSI C and broke the expected order of operations.

> > They also
> > recommend avoiding enums for similar reasons.  They also recommend
> > placing numbers in comparisons as "if(0==value)" instead of
> > "if(value==0)" to detect incorrect assignments such as
> > "if(value=0)". etc.  They'll also say not to use the address of a
> > struct by itself, i.e., as a pointer, since an early, widespread C
> > compiler, PCC, didn't keep track of the struct's name or somesuch
> > issue ...  ANSI C added other problems, e.g., obsoleted implicit
> > int's since they caused a syntax conflict with typedef's, messed up
> > pre- and post- increment and decrement operators, etc.  Of course,
> > C has some required behavior, which is not part of some of the
> > standards, such as the C struct hack and Duff's device, which must
> > work correctly.  
>  
> I'm finding the above too hard to parse. Try again?
>  

The point was there are a variety of things recommended by C style
guidelines due problems with C compilers.  Also, some required C
behavior is not required by the specifications.


Rod Pemberton
-- 
All it takes for humanity to conquer the world's toughest problems
is to hope, to believe, to share, and to do, in cooperation.

Back to comp.lang.forth | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

LLVM and Forth krishna.myneni@ccreweb.org - 2017-04-01 06:36 -0700
  Re: LLVM and Forth anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2017-04-01 14:57 +0000
    Re: LLVM and Forth krishna.myneni@ccreweb.org - 2017-04-01 11:55 -0700
      Re: LLVM and Forth anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2017-04-02 15:24 +0000
  Re: LLVM and Forth rickman <gnuarm@gmail.com> - 2017-04-01 11:21 -0400
    Re: LLVM and Forth krishna.myneni@ccreweb.org - 2017-04-01 11:57 -0700
      Re: LLVM and Forth Andrew Haley <andrew29@littlepinkcloud.invalid> - 2017-04-02 04:02 -0500
        Re: LLVM and Forth krishna.myneni@ccreweb.org - 2017-04-02 06:35 -0700
          Re: LLVM and Forth anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2017-04-02 14:41 +0000
          Re: LLVM and Forth minforth@arcor.de - 2017-04-03 01:09 -0700
            Re: LLVM and Forth krishna.myneni@ccreweb.org - 2017-04-03 05:32 -0700
              Re: LLVM and Forth anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2017-04-03 13:52 +0000
                Re: LLVM and Forth minforth@arcor.de - 2017-04-03 07:59 -0700
                Re: LLVM and Forth anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2017-04-03 16:08 +0000
                Re: LLVM and Forth Andrew Haley <andrew29@littlepinkcloud.invalid> - 2017-04-03 12:22 -0500
                Re: LLVM and Forth Paul Rubin <no.email@nospam.invalid> - 2017-04-03 20:10 -0700
                Re: LLVM and Forth minforth@arcor.de - 2017-04-03 22:19 -0700
                Re: LLVM and Forth albert@cherry.spenarnc.xs4all.nl (Albert van der Horst) - 2017-04-04 16:12 +0200
                Re: LLVM and Forth Walter Banks <walter@bytecraft.com> - 2017-04-05 17:27 -0400
            Re: LLVM and Forth anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2017-04-03 14:29 +0000
          Re: LLVM and Forth Andrew Haley <andrew29@littlepinkcloud.invalid> - 2017-04-03 03:17 -0500
            Re: LLVM and Forth minforth@arcor.de - 2017-04-03 05:04 -0700
            Re: LLVM and Forth anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2017-04-03 12:55 +0000
              Re: LLVM and Forth Andrew Haley <andrew29@littlepinkcloud.invalid> - 2017-04-03 10:08 -0500
                Re: LLVM and Forth anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2017-04-03 17:05 +0000
                Re: LLVM and Forth Andrew Haley <andrew29@littlepinkcloud.invalid> - 2017-04-04 11:29 -0500
                Re: LLVM and Forth albert@cherry.spenarnc.xs4all.nl (Albert van der Horst) - 2017-04-04 19:11 +0200
                Re: LLVM and Forth Paul Rubin <no.email@nospam.invalid> - 2017-04-04 21:29 -0700
                Re: LLVM and Forth anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2017-04-05 15:01 +0000
                Re: LLVM and Forth Alex <alex@rivadpm.com> - 2017-04-05 18:42 +0100
                Re: LLVM and Forth Andrew Haley <andrew29@littlepinkcloud.invalid> - 2017-04-06 04:01 -0500
                Re: LLVM and Forth Spiros Bousbouras <spibou@gmail.com> - 2017-04-06 18:48 +0000
                Re: LLVM and Forth Joel Rees <joel.rees@gmail.com> - 2017-04-06 18:42 -0700
                Re: LLVM and Forth Andrew Haley <andrew29@littlepinkcloud.invalid> - 2017-04-07 04:02 -0500
                Re: LLVM and Forth Joel Rees <joel.rees@gmail.com> - 2017-04-08 21:18 -0700
                Re: LLVM and Forth Spiros Bousbouras <spibou@gmail.com> - 2017-04-08 22:43 +0000
                Re: LLVM and Forth Joel Rees <joel.rees@gmail.com> - 2017-04-08 20:54 -0700
                Re: LLVM and Forth anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2017-04-09 14:53 +0000
                Re: LLVM and Forth Spiros Bousbouras <spibou@gmail.com> - 2017-04-10 00:38 +0000
                Re: LLVM and Forth James Kuyper <jameskuyper@verizon.net> - 2017-04-09 22:46 -0400
                Re: LLVM and Forth Robert Wessel <robertwessel2@yahoo.com> - 2017-04-09 22:27 -0500
                Re: LLVM and Forth Spiros Bousbouras <spibou@gmail.com> - 2017-04-10 12:36 +0000
                Re: LLVM and Forth Ben Bacarisse <ben.usenet@bsb.me.uk> - 2017-04-10 14:09 +0100
                Re: LLVM and Forth Spiros Bousbouras <spibou@gmail.com> - 2017-04-10 13:07 +0000
                Re: LLVM and Forth Ben Bacarisse <ben.usenet@bsb.me.uk> - 2017-04-10 16:18 +0100
                Re: LLVM and Forth "James R. Kuyper" <jameskuyper@verizon.net> - 2017-04-10 11:51 -0400
                Re: LLVM and Forth Spiros Bousbouras <spibou@gmail.com> - 2017-04-12 21:32 +0000
                Re: LLVM and Forth "James R. Kuyper" <jameskuyper@verizon.net> - 2017-04-12 18:29 -0400
                Re: LLVM and Forth hughaguilar96@gmail.com - 2017-04-12 15:29 -0700
                Re: LLVM and Forth albert@cherry.spenarnc.xs4all.nl (Albert van der Horst) - 2017-04-13 02:55 +0200
                Re: LLVM and Forth Joel Rees <joel.rees@gmail.com> - 2017-04-10 18:38 -0700
                Re: LLVM and Forth Andrew Haley <andrew29@littlepinkcloud.invalid> - 2017-04-10 11:54 -0500
                Re: LLVM and Forth Paul Rubin <no.email@nospam.invalid> - 2017-04-10 10:17 -0700
              Re: LLVM and Forth Alex <alex@rivadpm.com> - 2017-04-03 17:02 +0100
              Re: LLVM and Forth hughaguilar96@gmail.com - 2017-04-03 10:03 -0700
            Re: LLVM and Forth krishna.myneni@ccreweb.org - 2017-04-04 03:25 -0700
              Re: LLVM and Forth Spiros Bousbouras <spibou@gmail.com> - 2017-04-04 11:24 +0000
                Re: LLVM and Forth krishna.myneni@ccreweb.org - 2017-04-04 04:55 -0700
                Re: LLVM and Forth Spiros Bousbouras <spibou@gmail.com> - 2017-04-04 12:28 +0000
                Re: LLVM and Forth anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2017-04-04 16:16 +0000
              Re: LLVM and Forth albert@cherry.spenarnc.xs4all.nl (Albert van der Horst) - 2017-04-04 15:38 +0200
                Re: LLVM and Forth albert@cherry.spenarnc.xs4all.nl (Albert van der Horst) - 2017-04-04 16:06 +0200
                Re: LLVM and Forth krishna.myneni@ccreweb.org - 2017-04-04 17:57 -0700
                Re: LLVM and Forth Rod Pemberton <NeedNotReplyHere@xrsevnneqk.cem> - 2017-04-05 00:44 -0400
                Re: LLVM and Forth minforth@arcor.de - 2017-04-05 00:00 -0700
                Re: LLVM and Forth krishna.myneni@ccreweb.org - 2017-04-05 08:47 -0700
                Re: LLVM and Forth Andrew Haley <andrew29@littlepinkcloud.invalid> - 2017-04-05 11:04 -0500
                Re: LLVM and Forth john <anon@example.com> - 2017-04-05 18:04 +0100
                Re: LLVM and Forth hughaguilar96@gmail.com - 2017-04-05 13:04 -0700
                Re: LLVM and Forth Paul Rubin <no.email@nospam.invalid> - 2017-04-05 18:20 -0700
                Re: LLVM and Forth krishna.myneni@ccreweb.org - 2017-04-05 20:14 -0700
                Re: LLVM and Forth Spiros Bousbouras <spibou@gmail.com> - 2017-04-06 12:33 +0000
                Re: LLVM and Forth krishna.myneni@ccreweb.org - 2017-04-06 10:49 -0700
                Re: LLVM and Forth Alex <alex@rivadpm.com> - 2017-04-06 18:55 +0100
                Re: LLVM and Forth Spiros Bousbouras <spibou@gmail.com> - 2017-04-06 20:41 +0000
                Re: LLVM and Forth Alex <alex@rivadpm.com> - 2017-04-06 21:47 +0100
                Re: LLVM and Forth Alex <alex@rivadpm.com> - 2017-04-06 21:56 +0100
                Re: LLVM and Forth Rod Pemberton <NeedNotReplyHere@xrsevnneqk.cem> - 2017-04-06 23:59 -0400
                Re: LLVM and Forth Joel Rees <joel.rees@gmail.com> - 2017-04-06 17:10 -0700
                Re: LLVM and Forth Rod Pemberton <NeedNotReplyHere@xrsevnneqk.cem> - 2017-04-07 03:33 -0400
                Re: LLVM and Forth Spiros Bousbouras <spibou@gmail.com> - 2017-04-08 22:48 +0000
                Re: LLVM and Forth Rod Pemberton <NeedNotReplyHere@xrsevnneqk.cem> - 2017-04-09 15:06 -0400
                Re: LLVM and Forth Joel Rees <joel.rees@gmail.com> - 2017-04-08 22:45 -0700
                Re: LLVM and Forth anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2017-04-09 14:27 +0000
                Re: LLVM and Forth Joel Rees <joel.rees@gmail.com> - 2017-04-09 21:26 -0700
                Re: LLVM and Forth Rod Pemberton <NeedNotReplyHere@xrsevnneqk.cem> - 2017-04-09 13:35 -0400
                Re: LLVM and Forth Paul Rubin <no.email@nospam.invalid> - 2017-04-09 14:07 -0700
                Re: LLVM and Forth minforth@arcor.de - 2017-04-05 12:16 -0700
                Re: LLVM and Forth krishna.myneni@ccreweb.org - 2017-04-05 20:27 -0700
                Re: LLVM and Forth Rod Pemberton <NeedNotReplyHere@xrsevnneqk.cem> - 2017-04-06 05:24 -0400
                Re: LLVM and Forth krishna.myneni@ccreweb.org - 2017-04-06 03:26 -0700
                Re: LLVM and Forth minforth@arcor.de - 2017-04-06 04:05 -0700
                Re: LLVM and Forth Andrew Haley <andrew29@littlepinkcloud.invalid> - 2017-04-06 06:50 -0500
                Re: LLVM and Forth Rod Pemberton <NeedNotReplyHere@xrsevnneqk.cem> - 2017-04-06 05:23 -0400
                Re: LLVM and Forth Andrew Haley <andrew29@littlepinkcloud.invalid> - 2017-04-06 04:44 -0500
                Re: LLVM and Forth Alex <alex@rivadpm.com> - 2017-04-06 10:57 +0100
                Re: LLVM and Forth peter.m.falth@gmail.com - 2017-04-05 05:29 -0700
                Re: LLVM and Forth "Rudy Velthuis" <newsgroups@rvelthuis.de> - 2017-04-14 02:04 +0000
                Re: LLVM and Forth albert@cherry.spenarnc.xs4all.nl (Albert van der Horst) - 2017-04-15 14:03 +0200
                Re: LLVM and Forth "Rudy Velthuis" <newsgroups@rvelthuis.de> - 2017-04-16 00:44 +0000
                Re: LLVM and Forth alex@rivadpm.com - 2017-04-16 05:53 -0700
                Re: LLVM and Forth anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2017-04-16 15:43 +0000
                Re: LLVM and Forth "Rudy Velthuis" <newsgroups@rvelthuis.de> - 2017-04-16 22:27 +0000
                Re: LLVM and Forth albert@cherry.spenarnc.xs4all.nl (Albert van der Horst) - 2017-04-18 02:45 +0200
                Re: LLVM and Forth Alex <alex@rivadpm.com> - 2017-04-18 18:26 +0100
                Re: LLVM and Forth albert@cherry.spenarnc.xs4all.nl (Albert van der Horst) - 2017-04-20 19:51 +0200
                Re: LLVM and Forth Alex <alex@rivadpm.com> - 2017-04-20 19:08 +0100
                Re: LLVM and Forth albert@cherry.spenarnc.xs4all.nl (Albert van der Horst) - 2017-04-20 20:42 +0200
                Re: LLVM and Forth Alex <alex@rivadpm.com> - 2017-04-20 20:29 +0100
                Re: LLVM and Forth Alex <alex@rivadpm.com> - 2017-04-20 20:48 +0100
                Re: LLVM and Forth "Rudy Velthuis" <newsgroups@rvelthuis.de> - 2017-04-21 11:49 +0000
          Re: LLVM and Forth Rod Pemberton <NeedNotReplyHere@xrsevnneqk.cem> - 2017-04-03 20:39 -0400
  Re: LLVM and Forth Alex <alex@rivadpm.com> - 2017-04-02 21:15 +0100
    Re: LLVM and Forth hughaguilar96@gmail.com - 2017-04-02 15:18 -0700
      Re: LLVM and Forth Alex <alex@rivadpm.com> - 2017-04-03 13:07 +0100
        Re: LLVM and Forth hughaguilar96@gmail.com - 2017-04-03 09:45 -0700
          Re: LLVM and Forth Alex <alex@rivadpm.com> - 2017-04-03 22:52 +0100
            Re: LLVM and Forth hughaguilar96@gmail.com - 2017-04-03 15:25 -0700
              Re: LLVM and Forth Alex <alex@rivadpm.com> - 2017-04-06 09:35 +0100
      Re: LLVM and Forth antispam@math.uni.wroc.pl - 2017-04-06 01:36 +0000
        Re: LLVM and Forth minforth@arcor.de - 2017-04-05 22:00 -0700
          Re: LLVM and Forth Alex <alex@rivadpm.com> - 2017-04-06 09:52 +0100
        Re: LLVM and Forth hughaguilar96@gmail.com - 2017-04-06 20:15 -0700
    Re: LLVM and Forth krishna.myneni@ccreweb.org - 2017-04-03 05:20 -0700
      Re: LLVM and Forth Alex <alex@rivadpm.com> - 2017-04-03 17:11 +0100
  Re: LLVM and Forth Alex <alex@rivadpm.com> - 2017-04-06 17:43 +0100
    Re: LLVM and Forth krishna.myneni@ccreweb.org - 2017-04-06 10:53 -0700

csiph-web