Path: csiph.com!aioe.org!.POSTED!not-for-mail From: Rod Pemberton Newsgroups: comp.lang.forth Subject: Re: LLVM and Forth Date: Sun, 9 Apr 2017 13:35:15 -0400 Organization: Aioe.org NNTP Server Lines: 244 Message-ID: <20170409133515.2b871c2c@_> References: <21153661-abf5-4958-82c7-e3c442b45cb9@googlegroups.com> <065e774c-c5fb-4e68-9489-00b5357142d6@googlegroups.com> <72a319fa-8035-4a17-a5a2-2b2884a57b15@googlegroups.com> <20170405004435.4706b792@_> <20170407033309.1b6469f0@_> NNTP-Posting-Host: ZTD62cW/Uw8bymX5ok+0NA.user.gioia.aioe.org Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Complaints-To: abuse@aioe.org X-Notice: Filtered by postfilter v. 0.8.2 Xref: csiph.com comp.lang.forth:56940 On Sat, 8 Apr 2017 22:45:51 -0700 (PDT) Joel Rees wrote: > On Friday, April 7, 2017 at 4:32:11 PM UTC+9, Rod Pemberton wrote: > > On Thu, 06 Apr 2017 12:33:44 GMT > > Spiros Bousbouras wrote: > > > Example : the expression array[i] where i is beyond the end of > > > the array. > > > > FYI, C explicitly allows you to access one element past the end of > > the array. This is to allow you to access the nul character when > > comparing strings to check for string termination. > > Now we know who has been writing all that vulnerable code. (Just > kidding. Sort of.) This is used within the C libraries, e.g., string and memory functions, as well as by programmers. > I have heard teachers say that setting a pointer to NULL and then > dereferencing it is safe. Setting a pointer to any explicit value is undefined behavior in C. This is implementation defined behavior. Most OSes and resulting C compilers are reasonable in that they allow the programmer to access memory mapped devices. Hence, they allow you to assign values to pointers. > (There are, in fact, some embedded environments where it is safe, but > those are special cases. Still not what I call clean coding practice.) In many OS environments, you'll end up with a page fault. The OS will not map the zero'th page in order to detect NULL pointer dereferences. > C array syntax allows you to access whatever a [pointer] points at, > and pointer parameters can be named as if they were indefinitely > allocated arrays. Moreover, the grammar specifically allows arbitrary > array references even when the l-value is an array with a specific > declared size. I'm not sure I quite follow here. The C language has array declarations, but doesn't have arrays as a fundamental type as part of the language. C uses the [] subscript operator with a pointer and offset to effect array usage. The name of an array reduces to a pointer to the zero'th element of the array. This allows the array name to be used with the subscript operator. > In the past, certain compiler vendors allocated extra space around > arrays to "help protect" the end users from programmers who did not > understand that C does not bounds check arrays, and therefore assumed > that no breakage this time meant their loops or other array > processing was correct. This would give the appearance of allowing > dereferencing one past the end of an array. C explicitly allows dereferencing one past the end of an array. As stated previously, this requirement is in ALL of the C specifications. > It also allowed programmers to get thoroughly confused when they > started working with multi-dimension arrays, because the compiler and > libraries can't cheat like that for you on multi-dimension arrays. ... > If you read, for instance, the library documentation for string > processing functions in the standard library (fgets(), for instance), > you will note that the buffer size passed in must include room for a > trailing NUL if you want to be sure the trailing NUL gets stored. > > This kind of thing should help you understand that something doesn't > match what you just said. You're confusing strings and arrays. C doesn't have strings as a fundamental type, but uses arrays, which are a partial type. Part of the parsing process of C requires that an extra character for nul be added onto the end of the string. This is done automatically by the compiler unless a specific size is declared for the string. In the case of fgets(), the programmer must account for the extra character needed to terminate strings. > The standard does specify that the run-time has to allow a pointer to > point one past the end of an allocated area, as long as the pointer > is not dereferenced in that state. This allows a loop that has a > post-increment in the test to complete safely. ... > But that means, no, you must not actually access one beyond. That way > lies one-off pointer errors which become vulnerabilities which allow > servers and workstations to be pwned by malicious access. Accessing one beyond is required to check for a nul character. E.g., while(*str++); /* find end-of-string *//* This is not UB */ > > If you go beyond that though, > > it's undefined ... However, many C implementations use the ability > > to access outside a C object behind the scenes without any issues, > > e.g., to access a memory header prior to malloc'd memory blocks. > > When you talk about memory allocation headers, you imply two levels > (at least) of allocation -- a mass block allocation, and a fine-grain > allocation. It doesn't imply that. The memory block and header can be allocated at once. > If you try to access beyond either end of a mass block allocation, > you should expect segment faults. This assumes that paging is enabled and memory isn't contiguously mapped. The later is an implicit requirement of C. Every C object is required to be contiguous. This is usually effected by making memory contiguously mapped. > Within a mass block, the library maintains an allocation list of some > sort, and that list might use headers that sit either edge of the > fine-grained allocation block. The grammar and syntax do allow you to > take a look at those headers, but you really should not, because > other library architects don't want to risk allocation headers being > overwritten by stray pointers. So the allocation lists are elsewhere > in those runt-times. ... > The grammar and syntax allow the accesses, and it is not just one > past. A million past is also allowed by the grammar and syntax. The > run-time, however, is only required to complete an operation safely > if it is within the declared bounds. This is not true. For C, you can legally only access data within C objects or malloc'd memory. Any other access is undefined behavior. C makes an exception for strings to allow accessing one past the end-of-string to check for nul characters. Assigning a value directly to a pointer is also undefined behavior for C. Many environments allow this to access memory mapped devices. > > > Thinking in terms of assembly : "array[i] will be translated into > > > an assembly instruction which calculates an address of the form > > > base + i * constant . > > > > This is a requirement of the subscript operator [] for K&R, ANSI, > > and ISO C. > > Not really required, just the most natural interpretation of a > one-dimensional array reference. > > > It's in all the standards. They define "E1[E2]" to be > > equivalent to "*((E1)+(E2))". The subscript operator takes a > > pointer to a type and also an index. The index is scaled by the > > size of the data type pointed to. The pointer which cannot be a > > pointer-to-void 'void *' since 'void' has no data type, i.e., no > > size, pointer only. The pointer used in the subscript operator can > > be any pointer to any valid type, not just pointer for array > > types. > > Up to here you are just talking about arbitrary math. No, I'm talking about how the subscript operator [] in C works. Most people think it's for array indexes (or indices), but it works with pointers too. C doesn't have arrays within the language itself, but has array declarations. These are reduced to pointers by the compiler, as mentioned elsewhere. > > The arguments to the > > subscript operator can be either order, e.g., "0123456789"[i] is > > equivalent to i["0123456789"] . > > This does not make sense. I know what "string"[index] means, but I > and the compiler are both extremely puzzled by index["string"]. This > is not perl. "string" may resolve to a pointer, but pointers and > indexes have two different sorts of existence, which you refer to > when you mention void pointers. You can't use a pointer as an index > very meaningfully. This is valid C. The compiler knows the type of both parameters for the subscript operator [] . The compiler knows that the string is a pointer to char and it knows the index is one of the integer types. > > Technically, you should think of the > > subscript operator as just the left-brace, then you'll understand > > it's like any other operator in C, e.g., arithmetic +, -> component > > selection, or whatever. > > Syntactically similar, semantically significantly different. > ... > > > Your second reference has > > > char *index; > > > .... > > > *index++ = toupper(*index); > > > > > > .He doesn't explain why he decided to write code like this as > > > opposed to *index = toupper(*index); > > > index++ ; > > > > I'm guessing that this is most likely just pre-ANSI C. Most C style > > guides etc are older and are for K&R C. E.g., most will recommend > > that you use #defines and mask with logical-and instead of using > > bit-fields, since many earlier C compilers were broken. > > The implementation of toupper() is not really relevant to the > semantics of > > *index = toupper( *index++ ); > > demonstrates the need of sequence points. The assignment is a > sequence point there is the assignment operator, "=". > > ++index is equivalent to index = index + 1 in several ways, but the > increment does not contain a sequence point. > It indicates the author didn't follow ANSI C coding rules which require that pre- and post- increment and pre- and post-decrement be removed from surrounding code. This is because they introduced sequence point concept with ANSI C and broke the expected order of operations. > > They also > > recommend avoiding enums for similar reasons. They also recommend > > placing numbers in comparisons as "if(0==value)" instead of > > "if(value==0)" to detect incorrect assignments such as > > "if(value=0)". etc. They'll also say not to use the address of a > > struct by itself, i.e., as a pointer, since an early, widespread C > > compiler, PCC, didn't keep track of the struct's name or somesuch > > issue ... ANSI C added other problems, e.g., obsoleted implicit > > int's since they caused a syntax conflict with typedef's, messed up > > pre- and post- increment and decrement operators, etc. Of course, > > C has some required behavior, which is not part of some of the > > standards, such as the C struct hack and Duff's device, which must > > work correctly. > > I'm finding the above too hard to parse. Try again? > The point was there are a variety of things recommended by C style guidelines due problems with C compilers. Also, some required C behavior is not required by the specifications. Rod Pemberton -- All it takes for humanity to conquer the world's toughest problems is to hope, to believe, to share, and to do, in cooperation.