Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.c > #169082 > unrolled thread

[RFC] _Optional: a type qualifier to indicate pointer nullability

Back to article view | Back to comp.lang.c

Contents

  [RFC] _Optional: a type qualifier to indicate pointer nullability Christopher Bazley <cs99cjb@gmail.com> - 2023-01-28 08:15 -0800
    Re: [RFC] _Optional: a type qualifier to indicate pointer nullability Richard Damon <Richard@Damon-Family.org> - 2023-01-28 11:31 -0500
      Re: [RFC] _Optional: a type qualifier to indicate pointer nullability Christopher Bazley <cs99cjb@gmail.com> - 2023-01-29 01:02 -0800
        Re: [RFC] _Optional: a type qualifier to indicate pointer nullability Richard Damon <Richard@Damon-Family.org> - 2023-01-29 13:00 -0500
          Re: [RFC] _Optional: a type qualifier to indicate pointer nullability Christopher Bazley <cs99cjb@gmail.com> - 2023-01-29 12:48 -0800
            Re: [RFC] _Optional: a type qualifier to indicate pointer nullability Richard Damon <Richard@Damon-Family.org> - 2023-01-29 16:37 -0500
              Re: [RFC] _Optional: a type qualifier to indicate pointer nullability Christopher Bazley <cs99cjb@gmail.com> - 2023-01-29 14:18 -0800
                Re: [RFC] _Optional: a type qualifier to indicate pointer nullability Richard Damon <Richard@Damon-Family.org> - 2023-01-29 18:00 -0500
                Re: [RFC] _Optional: a type qualifier to indicate pointer nullability Ben Bacarisse <ben.usenet@bsb.me.uk> - 2023-01-30 00:00 +0000
                  Re: [RFC] _Optional: a type qualifier to indicate pointer nullability Christopher Bazley <cs99cjb@gmail.com> - 2023-01-30 14:57 -0800
                    Re: [RFC] _Optional: a type qualifier to indicate pointer nullability Kaz Kylheku <864-117-4973@kylheku.com> - 2023-01-30 23:28 +0000
                      Re: [RFC] _Optional: a type qualifier to indicate pointer nullability Christopher Bazley <cs99cjb@gmail.com> - 2023-01-30 23:39 -0800
                        Re: [RFC] _Optional: a type qualifier to indicate pointer nullability James Kuyper <jameskuyper@alumni.caltech.edu> - 2023-01-31 12:19 -0500
                          Re: [RFC] _Optional: a type qualifier to indicate pointer nullability Christopher Bazley <cs99cjb@gmail.com> - 2023-01-31 09:39 -0800
                        Re: [RFC] _Optional: a type qualifier to indicate pointer nullability Kaz Kylheku <864-117-4973@kylheku.com> - 2023-02-01 01:59 +0000
                    Re: [RFC] _Optional: a type qualifier to indicate pointer nullability Richard Damon <Richard@Damon-Family.org> - 2023-01-30 21:05 -0500
                      Re: [RFC] _Optional: a type qualifier to indicate pointer nullability Christopher Bazley <cs99cjb@gmail.com> - 2023-01-31 00:05 -0800
                Re: [RFC] _Optional: a type qualifier to indicate pointer nullability James Kuyper <jameskuyper@alumni.caltech.edu> - 2023-01-30 02:13 -0500
            Re: [RFC] _Optional: a type qualifier to indicate pointer nullability Andrew Smallshaw <andrews@sdf.org> - 2023-01-29 21:52 +0000
              Re: [RFC] _Optional: a type qualifier to indicate pointer nullability Christopher Bazley <cs99cjb@gmail.com> - 2023-01-29 14:44 -0800
          Re: [RFC] _Optional: a type qualifier to indicate pointer nullability Christopher Bazley <cs99cjb@gmail.com> - 2023-01-29 13:17 -0800
            Re: [RFC] _Optional: a type qualifier to indicate pointer nullability Richard Damon <Richard@Damon-Family.org> - 2023-01-29 16:48 -0500
              Re: [RFC] _Optional: a type qualifier to indicate pointer nullability Christopher Bazley <cs99cjb@gmail.com> - 2023-01-29 14:36 -0800
                Re: [RFC] _Optional: a type qualifier to indicate pointer nullability Richard Damon <Richard@Damon-Family.org> - 2023-01-29 18:10 -0500
                  Re: [RFC] _Optional: a type qualifier to indicate pointer nullability Ben Bacarisse <ben.usenet@bsb.me.uk> - 2023-01-29 23:33 +0000
                    Re: [RFC] _Optional: a type qualifier to indicate pointer nullability Richard Damon <Richard@Damon-Family.org> - 2023-01-29 18:50 -0500
                      Re: [RFC] _Optional: a type qualifier to indicate pointer nullability Ben Bacarisse <ben.usenet@bsb.me.uk> - 2023-01-30 01:05 +0000
                        Re: [RFC] _Optional: a type qualifier to indicate pointer nullability Richard Damon <Richard@Damon-Family.org> - 2023-01-29 21:02 -0500
                          Re: [RFC] _Optional: a type qualifier to indicate pointer nullability Ben Bacarisse <ben.usenet@bsb.me.uk> - 2023-01-30 14:11 +0000
                            Re: [RFC] _Optional: a type qualifier to indicate pointer nullability Christopher Bazley <cs99cjb@gmail.com> - 2023-01-30 15:22 -0800
        Re: [RFC] _Optional: a type qualifier to indicate pointer nullability James Kuyper <jameskuyper@alumni.caltech.edu> - 2023-01-30 02:19 -0500
          Re: [RFC] _Optional: a type qualifier to indicate pointer nullability Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2023-01-30 03:16 -0800
            Re: [RFC] _Optional: a type qualifier to indicate pointer nullability Richard Harnden <richard.nospam@gmail.com> - 2023-01-30 12:48 +0000
              Re: [RFC] _Optional: a type qualifier to indicate pointer nullability Ben Bacarisse <ben.usenet@bsb.me.uk> - 2023-01-30 14:19 +0000
                Re: [RFC] _Optional: a type qualifier to indicate pointer nullability Richard Harnden <richard.nospam@gmail.com> - 2023-01-30 18:10 +0000
          Re: [RFC] _Optional: a type qualifier to indicate pointer nullability Christopher Bazley <cs99cjb@gmail.com> - 2023-01-30 15:05 -0800
            Re: [RFC] _Optional: a type qualifier to indicate pointer nullability James Kuyper <jameskuyper@alumni.caltech.edu> - 2023-01-31 01:15 -0500
    Re: [RFC] _Optional: a type qualifier to indicate pointer nullability Ben Bacarisse <ben.usenet@bsb.me.uk> - 2023-01-29 02:24 +0000
      Re: [RFC] _Optional: a type qualifier to indicate pointer nullability Christopher Bazley <cs99cjb@gmail.com> - 2023-01-29 01:21 -0800
        Re: [RFC] _Optional: a type qualifier to indicate pointer nullability Ben Bacarisse <ben.usenet@bsb.me.uk> - 2023-01-29 13:35 +0000
          Re: [RFC] _Optional: a type qualifier to indicate pointer nullability Christopher Bazley <cs99cjb@gmail.com> - 2023-01-29 07:21 -0800
        Re: [RFC] _Optional: a type qualifier to indicate pointer nullability Ben Bacarisse <ben.usenet@bsb.me.uk> - 2023-01-29 13:46 +0000
          Re: [RFC] _Optional: a type qualifier to indicate pointer nullability Christopher Bazley <cs99cjb@gmail.com> - 2023-01-29 06:54 -0800
    Re: [RFC] _Optional: a type qualifier to indicate pointer nullability Oğuz <oguzismailuysal@gmail.com> - 2023-01-29 12:24 +0300
      Re: [RFC] _Optional: a type qualifier to indicate pointer nullability Christopher Bazley <cs99cjb@gmail.com> - 2023-01-29 07:55 -0800
      Re: [RFC] _Optional: a type qualifier to indicate pointer nullability Christopher Bazley <cs99cjb@gmail.com> - 2023-01-29 08:03 -0800
        Re: [RFC] _Optional: a type qualifier to indicate pointer nullability Oğuz <oguzismailuysal@gmail.com> - 2023-01-29 20:43 +0300
        Re: [RFC] _Optional: a type qualifier to indicate pointer nullability Keith Thompson <Keith.S.Thompson+u@gmail.com> - 2023-01-29 19:03 -0800
          Re: [RFC] _Optional: a type qualifier to indicate pointer nullability Richard Damon <Richard@Damon-Family.org> - 2023-01-29 22:23 -0500
    Re: [RFC] _Optional: a type qualifier to indicate pointer nullability Thiago Adams <thiago.adams@gmail.com> - 2023-01-29 12:53 -0800
      Re: [RFC] _Optional: a type qualifier to indicate pointer nullability Christopher Bazley <cs99cjb@gmail.com> - 2023-01-29 13:31 -0800
        Re: [RFC] _Optional: a type qualifier to indicate pointer nullability Thiago Adams <thiago.adams@gmail.com> - 2023-01-29 13:42 -0800
          Re: [RFC] _Optional: a type qualifier to indicate pointer nullability Christopher Bazley <cs99cjb@gmail.com> - 2023-01-29 14:27 -0800
            Re: [RFC] _Optional: a type qualifier to indicate pointer nullability Thiago Adams <thiago.adams@gmail.com> - 2023-01-31 04:56 -0800
        Re: [RFC] _Optional: a type qualifier to indicate pointer nullability Thiago Adams <thiago.adams@gmail.com> - 2023-01-29 13:46 -0800
        Re: [RFC] _Optional: a type qualifier to indicate pointer nullability Richard Damon <Richard@Damon-Family.org> - 2023-01-29 16:54 -0500
    Re: [RFC] _Optional: a type qualifier to indicate pointer nullability Tim Rentsch <tr.17687@z991.linuxsc.com> - 2023-01-31 07:54 -0800
      Re: [RFC] _Optional: a type qualifier to indicate pointer nullability Christopher Bazley <cs99cjb@gmail.com> - 2023-01-31 09:25 -0800
        Re: [RFC] _Optional: a type qualifier to indicate pointer nullability bart c <bart4858@gmail.com> - 2023-01-31 14:44 -0800
          Re: [RFC] _Optional: a type qualifier to indicate pointer nullability David Brown <david.brown@hesbynett.no> - 2023-02-01 09:45 +0100
        Re: [RFC] _Optional: a type qualifier to indicate pointer nullability James Kuyper <jameskuyper@alumni.caltech.edu> - 2023-01-31 18:00 -0500
          Re: [RFC] _Optional: a type qualifier to indicate pointer nullability Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2023-01-31 15:52 -0800
            Re: [RFC] _Optional: a type qualifier to indicate pointer nullability Ben Bacarisse <ben.usenet@bsb.me.uk> - 2023-02-01 00:04 +0000
              Re: [RFC] _Optional: a type qualifier to indicate pointer nullability David Brown <david.brown@hesbynett.no> - 2023-02-01 09:29 +0100
                Re: [RFC] _Optional: a type qualifier to indicate pointer nullability Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2023-02-01 02:31 -0800
    Re: [RFC] _Optional: a type qualifier to indicate pointer nullability Siri Cruise <chine.bleu@yahoo.com> - 2023-01-31 18:07 -0800

Page 3 of 4 — ← Prev page 1 2 [3] 4  Next page →

#169098

On Sunday, 29 January 2023 at 13:35:26 UTC, Ben Bacarisse wrote:
> Christopher Bazley <cs9...@gmail.com> writes: 
> 
> > On Sunday, 29 January 2023 at 02:24:27 UTC, Ben Bacarisse wrote: 
> >> Moreover, many previously simple expressions became 
> >> unreadable: 
> >> 
> >> &(&*s)[index] (instead of &s[index]) 
> >> &(&*s)->member (instead of &s->member)
> &s[i] includes a * operation by definition so &s[i] would (if the 
> qualifier were removed by *) do the job of &(&*s)[i].

In theory, you are (of course) correct. In reality, that's not how
the Clang front-end implements those operators. I don't know
what other compilers might do under the hood.

> Maybe there are 
> cases where one does not want &s[i] to be a pointer to an unqualified 
> type. 

Having spend many hours updating a large existing codebase over
my Christmas 'holidays' (to the mild exasperation of my family), my feeling
is that even if one did not want an _Optional qualifier to be implicitly
removed from the result of &s[i] or s->i, enforcing that rule would make
use of _Optional-qualified types too onerous for most programmers.

I've tried to come up with what I think is a good compromise between
maintaining C's expressiveness and allowing non-path-analysing
compilers (including Clang as a stand-alone tool) to do rudimentary
validation of code that uses _Optional-qualified types.

In cases where those two goals came into conflict, I leaned towards good
ergonomics. As I see it, there's no point defining an extension that requires
a lot of extra typing (perhaps deliberately to discourage frivolous usage,
like C++'s casts) because nobody would use it.

> And although p->m is not /defined/ to mean (*p).m I was including it. I 
> was wondering why it was & rather than either explicit or implicit 
> indirection that removed the qualifier.

For the vast majority of code, it won't make any discernable difference,
given that _Optional-qualified values resulting from a pointer dereference
are treated exactly like unqualified values. Note that *, [] and -> don't
remove a 'const' or 'volatile' qualifier either.

For me, the fact that the operand of & is *already* treated specially
swung the argument in favour of changing the semantics of &.
Keeping all of the interesting behaviour in the & operator also
reinforces the idea that the _Optional qualifier is all about
addresses. (The address of any object is not null, by definition.)

Cheers,
Christopher

[toc] | [prev] | [next] | [standalone]

#169096

Christopher Bazley <cs99cjb@gmail.com> writes:

> So far I only shared the PDF of my paper with colleagues and
> the committee. Sorry that the style of the Medium article
> doesn't work for you.

The committee has shared it for you:

https://www.open-std.org/jtc1/sc22/wg14/www/docs/n3089.pdf

though it has the same structure as the blog post with the proposed
changes not appearing until page 15 of 17.

> I suggest referring to my LLVM forum post instead, which is very
> like the submitted paper but with some additional details at the end
> about how I prototyped the new feature in Clang:
>  https://discourse.llvm.org/t/rfc-optional-a-type-qualifier-to-indicate-pointer-nullability/

That has the same structure as well.

I hope you don't think I'm being negative.  I think the idea as merit
but you are in danger of irritating busy experts (who would likely get
it in a few sentences) by building up to a "ta-dah!"  revelation rather
than saying

  "I propose this..."
  "It's new because..."
  "It's better because..."
  "Here's how I prototyped it..."
  "Discussion of merits and demerits..."

-- 
Ben.

[toc] | [prev] | [next] | [standalone]

#169097

On Sunday, 29 January 2023 at 13:46:28 UTC, Ben Bacarisse wrote:
> Christopher Bazley <cs9...@gmail.com> writes: 
> The committee has shared it for you: 
> 
> https://www.open-std.org/jtc1/sc22/wg14/www/docs/n3089.pdf 
> 
> though it has the same structure as the blog post with the proposed 
> changes not appearing until page 15 of 17.

Good, that's one less thing for me to do.

> > I suggest referring to my LLVM forum post instead, which is very 
> > like the submitted paper but with some additional details at the end 
> > about how I prototyped the new feature in Clang: 
> > https://discourse.llvm.org/t/rfc-optional-a-type-qualifier-to-indicate-pointer-nullability/
> That has the same structure as well. 

Unfortunately I spent several months gradually writing the Medium
article and hacking Clang. By the time I finally had a working prototype,
I was so burned out that I couldn't face doing a significant amount of
rewriting or restructuring of the text. Having a new baby and a toddler
hasn't helped (not that I'm expecting or should get sympathy votes for
what is ultimately a rather dry technical proposal).

One thing that really surprised me was the shortcomings of the
existing nullability checker in Clang and the lack of people willing
to defend it. It was hard to get a sufficient understanding of that
codebase in order to improve it. (Not because it's particularly badly
written, but because of sleep-deprivation, interruptions, and lack of
anyone to help.)

Cheers
Christopher

[toc] | [prev] | [next] | [standalone]

#169094

On 1/28/23 7:15 PM, Christopher Bazley wrote:
> I wrote a Medium article with much the same content as my paper, which is published here: https://medium.com/itnext/why-c-needs-a-new-type-qualifier-61ad553cbe71
 >26 min read
Is there a tl;dr?

[toc] | [prev] | [next] | [standalone]

#169099

On Sunday, 29 January 2023 at 09:24:42 UTC, oguzism...@gmail.com wrote:
> On 1/28/23 7:15 PM, Christopher Bazley wrote: 
> > I wrote a Medium article with much the same content as my paper, which is published here: https://medium.com/itnext/why-c-needs-a-new-type-qualifier-61ad553cbe71
> 26 min read 

I'm thinking of removing the sections on prototyping to reduce the
length of the article, but I'll ask the publisher first.

> Is there a tl;dr?

I guess one 'tl;dr' is the abstract, which is in my original post.

Here's another:

Pointer-to-const may have undefined behaviour on write access;
pointer-to-_Optional may have undefined behaviour on read or write
access.

Postulate: improved null safety does not require path-sensitive analysis.

Proposed extension:

 * A new type qualifier, _Optional, indicates that a pointer to a
   so-qualified type may be null. This does not preclude any other
   pointer type from being null.

 * Types other than those of a pointed-to object or pointed-to
   incomplete type shall not be _Optional-qualified in a declaration.

 * The semantics of the unary & operator are modified so that if
   its operand has type “type” then its result has type “pointer to
   type”, with the omission of any _Optional qualifier of the
   pointed-to type.

 * If an operand is a pointer to an _Optional-qualified type and its
   value cannot be statically proven never to be null, then
   implementations may generate a warning of any undefined behaviour
   that would occur if the value were null.

 * A specification of a function type that includes type qualifiers
   no longer has undefined behaviour. Qualifiers that are not
   applicable are ignored (as in C++).

The _Optional qualifier is treated like existing qualifiers when
determining compatibility between types, and when determining whether
a pointer may be implicitly converted to a pointer to a differently-
qualified type.

[toc] | [prev] | [next] | [standalone]

#169100

On Sunday, 29 January 2023 at 09:24:42 UTC, oguzism...@gmail.com wrote:
> On 1/28/23 7:15 PM, Christopher Bazley wrote: 
> > I wrote a Medium article with much the same content as my paper, which is published here: https://medium.com/itnext/why-c-needs-a-new-type-qualifier-61ad553cbe71
> 26 min read 
> Is there a tl;dr?

Maybe you prefer a code example:

void foo(int *);

void bar(_Optional int *i)
{
    *i = 10; // optional warning of unguarded dereference

    if (i) {
        *i = 5; // okay
    }

    int *j = i; // warning: initializing discards qualifiers
    j = i; // warning: assignment discards qualifiers
    foo(i); // warning: passing parameter discards qualifiers
}

[toc] | [prev] | [next] | [standalone]

#169101

On 1/29/23 7:03 PM, Christopher Bazley wrote:
> void foo(int *);
> 
> void bar(_Optional int *i)
> {
>      *i = 10; // optional warning of unguarded dereference
> 
>      if (i) {
>          *i = 5; // okay
>      }
> 
>      int *j = i; // warning: initializing discards qualifiers
>      j = i; // warning: assignment discards qualifiers
>      foo(i); // warning: passing parameter discards qualifiers
> }
Okay, thanks. Good luck

[toc] | [prev] | [next] | [standalone]

#169128

Christopher Bazley <cs99cjb@gmail.com> writes:
> On Sunday, 29 January 2023 at 09:24:42 UTC, oguzism...@gmail.com wrote:
>> On 1/28/23 7:15 PM, Christopher Bazley wrote: 
>> > I wrote a Medium article with much the same content as my paper, which is published here: https://medium.com/itnext/why-c-needs-a-new-type-qualifier-61ad553cbe71
>> 26 min read 
>> Is there a tl;dr?
>
> Maybe you prefer a code example:
>
> void foo(int *);
>
> void bar(_Optional int *i)
> {
>     *i = 10; // optional warning of unguarded dereference

There's already an optional warning, for this case and for everything
else.  If you drop the _Optional, a conforming C compiler can already
warn that `*i` might dereference a null pointer.  (Few if any compilers
do so.)  The role of _Optional (based on this example; I haven't read
the full proposal) seems to be to encourage such a warning.

(Personally, I'd find the example easier to read if the pointer were
called p rather than i; it's difficult not to assume that something
named i or j is an integer.)

>     if (i) {
>         *i = 5; // okay
>     }
>
>     int *j = i; // warning: initializing discards qualifiers
>     j = i; // warning: assignment discards qualifiers
>     foo(i); // warning: passing parameter discards qualifiers
> }

As someone else pointed out, the C standard has no concept of
"warnings", optional or otherwise.  Violations of syntax rules and
constraints require a diagnostic, which may or may not be fatal.
(The common wisdom is that any program that violates a syntax rule
or constraint, *if* it's not rejected, has undefined behavior;
the standard doesn't say so explicitly.)

I suspect your proposal would fit the language better if it were
combined with a substantial redesign of how the language specifies
diagnostics.

-- 
Keith Thompson (The_Other_Keith) Keith.S.Thompson+u@gmail.com
Working, but not speaking, for XCOM Labs
void Void(void) { Void(); } /* The recursive call of the void */

[toc] | [prev] | [next] | [standalone]

#169129

On 1/29/23 10:03 PM, Keith Thompson wrote:
> Christopher Bazley <cs99cjb@gmail.com> writes:
>> On Sunday, 29 January 2023 at 09:24:42 UTC, oguzism...@gmail.com wrote:
>>> On 1/28/23 7:15 PM, Christopher Bazley wrote:
>>>> I wrote a Medium article with much the same content as my paper, which is published here: https://medium.com/itnext/why-c-needs-a-new-type-qualifier-61ad553cbe71
>>> 26 min read
>>> Is there a tl;dr?
>>
>> Maybe you prefer a code example:
>>
>> void foo(int *);
>>
>> void bar(_Optional int *i)
>> {
>>      *i = 10; // optional warning of unguarded dereference
> 
> There's already an optional warning, for this case and for everything
> else.  If you drop the _Optional, a conforming C compiler can already
> warn that `*i` might dereference a null pointer.  (Few if any compilers
> do so.)  The role of _Optional (based on this example; I haven't read
> the full proposal) seems to be to encourage such a warning.
> 

Yes it COULD, but without the _Optional, without looking at the callers 
of bar it would likely generate so may false alarms, it would not be a 
default or commonly enabled warning, as LOTS of code (including the 
standard library) has assumptions that pointers passed in are non-hull.

> (Personally, I'd find the example easier to read if the pointer were
> called p rather than i; it's difficult not to assume that something
> named i or j is an integer.)
> 
>>      if (i) {
>>          *i = 5; // okay
>>      }
>>
>>      int *j = i; // warning: initializing discards qualifiers
>>      j = i; // warning: assignment discards qualifiers
>>      foo(i); // warning: passing parameter discards qualifiers
>> }
> 
> As someone else pointed out, the C standard has no concept of
> "warnings", optional or otherwise.  Violations of syntax rules and
> constraints require a diagnostic, which may or may not be fatal.
> (The common wisdom is that any program that violates a syntax rule
> or constraint, *if* it's not rejected, has undefined behavior;
> the standard doesn't say so explicitly.)
> 
> I suspect your proposal would fit the language better if it were
> combined with a substantial redesign of how the language specifies
> diagnostics.
> 

Personnaly, I see it as a good option for an attribute, which by 
definition doesn't affect the validity of a program, but might affect, 
in an implementaiton dependent or unspeccifed manner optimizations or 
warning.

Putting it purely as something that only compilers/linters that need to 
look at, makes it much more likely to be actually usable.

It also gives a good platform to show what gain can be gotten with just 
partial instrumentation of code to show a usability during "transition".

Until the Standard Committee is willing to establish a minimum level of 
analysis of a program, it is unlikely going to reach a point where such 
markings can generata a "manditory diagnostic" that causes the program 
to be rejected.

That sort of change of baseline expections of the compiler would likely 
also be needed to add "warnings" to the standard in any really manner.

[toc] | [prev] | [next] | [standalone]

#169104

On Saturday, January 28, 2023 at 1:16:07 PM UTC-3, cs9...@gmail.com wrote:
> Hi all, 
> 
> I've just submitted my paper n3089, “_Optional: a type qualifier to indicate pointer nullability” to the WG14 committee. 
> 
> This is either the most important thing I've ever written or a waste of a few months of research, design, prototyping and testing. Obviously, I'm hoping that it isn't the latter, but I'm fully prepared for that outcome! :) If you have time and interest in how to improve the C programming language, then please read on. 
> 
> Abstract: This paper proposes a new type qualifier for the purpose of adding pointer nullability information to C programs. Its goal is to provide value not only for static analysis and documentation, but also for compilers which report errors based only on existing type-compatibility rules. The syntax and semantics are designed to be as familiar (to C programmers) and ergonomic as possible. In contrast, existing solutions are incompatible, confusing, error-prone, and intrusive. 
> 
> I wrote a Medium article with much the same content as my paper, which is published here: https://medium.com/itnext/why-c-needs-a-new-type-qualifier-61ad553cbe71 
> 
> I created a fully working prototype of my change in Clang (both simple type-compatibility rules, and static analyzer) . The stack of my commits (many fixing latent bugs in the existing attempt at path-sensitive analysis of null pointers) is here: https://reviews.llvm.org/D142744 
> 
> I also started a discussion thread on the LLVM forums here: https://discourse.llvm.org/t/rfc-optional-a-type-qualifier-to-indicate-pointer-nullability/68004 
> My initial forum post discusses the necessary changes to Clang in a lot more detail than the WG14 paper does. 
> 
> Thanks for reading!

>" improved null safety does not require path-sensitive analysis."

But it is very incomplete without it!

I am working with static analysis (hobby project) but I first
focused in ownership.

But some ideas also applies for null check.

See Extension - [[destroy, free]] attributes
http://thradams.com/cake/manual.html

also  (select free attribute)
http://thradams.com/cake/playground.html

The basic and general idea is to have something
I called "imaginary attributes." for declarators

In the context of null checks the imaginary attributes would be
path-sensitive.

for instance

if (p != NULL) {
   here p has the imaginary attribute NOT NULL
}

"imaginary attributes" are not part of the type. 
Instead its is a compile time attribute that may change.

To say a pointer cannot be null we can say

void f(int * [[notnull]] p) 

But this is not part of the type system. Instead it is a tip
for static analysis.

How do you imagine ignore/override the flow analysis? with casts?

In the "imaginary attributes" I also have a way of override the the flow analysis
setting or clearing the imaginary flags. Doing this the programmer is always
on the control.
I also can check in compile time these imaginary flags.

Applying the ideas on null check I would have something like

[[may return null]] void * malloc();
int *p = malloc(N);

now p have the "imaginary attribute" "I may be null"
but if we check (p != null) { } the imaginary attributes goes to "not null" until
end of if scope.

If I understood right you want to make it part of the type system
not depending on flow analysis, then you have some benefits
but also requires lots of casts because they are not interfered..

[toc] | [prev] | [next] | [standalone]

#169107

On Sunday, 29 January 2023 at 20:53:23 UTC, Thiago Adams wrote:
> On Saturday, January 28, 2023 at 1:16:07 PM UTC-3, cs9...@gmail.com wrote: 
> >" improved null safety does not require path-sensitive analysis."
> But it is very incomplete without it! 

Based on my experimentation on a large existing codebase
over Christmas, not nearly so incomplete as to provide no value.

A member of the C standard committee made clear to me months ago
that no extension to the language that *requires* (compiler) implementors
to do path-sensitive analysis stands a chance of being accepted. I also
think that is the right decision for the committee to make.

Consequently, I've tried to specify a compromise that enables static
analysis whilst still providing value for compilers that do only simple
validation of type compatibility.

> I am working with static analysis (hobby project) but I first 
> focused in ownership. 

Good luck with that :)

> The basic and general idea is to have something 
> I called "imaginary attributes." for declarators 
> 
> In the context of null checks the imaginary attributes would be 
> path-sensitive. 
> 
> for instance 
> 
> if (p != NULL) { 
> here p has the imaginary attribute NOT NULL 
> } 
> 
> "imaginary attributes" are not part of the type. 
> Instead its is a compile time attribute that may change. 

This sounds like what Clang's static analyser already
implements, except that it calls them 'constraints' of
the 'program state'.

> How do you imagine ignore/override the flow analysis? with casts? 

No. I hate casts. :) That's one of the reasons I like C more than C++.
Instead, I modified the semantics of the & operator.

I don't want to be rude, but please do read my paper for full details:
https://www.open-std.org/jtc1/sc22/wg14/www/docs/n3089.pdf

Or my blog post:
https://medium.com/itnext/why-c-needs-a-new-type-qualifier-61ad553cbe71

[toc] | [prev] | [next] | [standalone]

#169109

On Sunday, January 29, 2023 at 6:32:02 PM UTC-3, cs9...@gmail.com wrote:
...
> > How do you imagine ignore/override the flow analysis? with casts?
> No. I hate casts. :) That's one of the reasons I like C more than C++. 
> Instead, I modified the semantics of the & operator. 
> 
> I don't want to be rude, but please do read my paper for full details: 
> https://www.open-std.org/jtc1/sc22/wg14/www/docs/n3089.pdf 

This part? 

"All compilers implicitly remove the _Optional qualifier from the type of the pointed-to
object in the result of the expressions &*s1 and &*s2"

if (p != NULL) {
  func(&*p);
}

(then this is just a shortcut for the cast)

[toc] | [prev] | [next] | [standalone]

#169117

On Sunday, 29 January 2023 at 21:42:12 UTC, Thiago Adams wrote:
> On Sunday, January 29, 2023 at 6:32:02 PM UTC-3, cs9...@gmail.com wrote: 
> ...
> > > How do you imagine ignore/override the flow analysis? with casts? 
> > No. I hate casts. :) That's one of the reasons I like C more than C++. 
> > Instead, I modified the semantics of the & operator. 
> > 
> > I don't want to be rude, but please do read my paper for full details: 
> > https://www.open-std.org/jtc1/sc22/wg14/www/docs/n3089.pdf
> This part? 
> 
> "All compilers implicitly remove the _Optional qualifier from the type of the pointed-to 
> object in the result of the expressions &*s1 and &*s2" 
> 
> if (p != NULL) { 
> func(&*p); 
> } 
> 
> (then this is just a shortcut for the cast)

No, it's not a shortcut for a cast. The defined semantics of the &
operator avoids the need to cast. Just like in this example of
strictly-compliant and idiomatic C code:
void *x;
get_ptr(index, &x);
int *num = x;

instead of this (commonplace but horrible) code:
int *num;
get_ptr(index, (void **)&num);

Casting is ugly and throws away all type-safety. Even if the static analyzer were
somehow aware of which casts are valid and which are not, that would
actually remove useful functionality because there might be some cases where
a programmer actually *does* want to override the type system for legitimate
reasons. Standard usage is not one of those situations.

Also, if I had to type this:

if (p != NULL) { 
  func((int *)p); 
} 

Or this:

if (p != NULL) { 
  func(optional_cast<int *>(p)); 
}

Then I would honestly never have finished testing my prototype, nor have
any desire to productise it.

Cheers,
Christopher

[toc] | [prev] | [next] | [standalone]

#169150

On Sunday, January 29, 2023 at 7:27:44 PM UTC-3, cs9...@gmail.com wrote:
> On Sunday, 29 January 2023 at 21:42:12 UTC, Thiago Adams wrote: 
> > On Sunday, January 29, 2023 at 6:32:02 PM UTC-3, cs9...@gmail.com wrote: 
> > ... 
> > > > How do you imagine ignore/override the flow analysis? with casts? 
> > > No. I hate casts. :) That's one of the reasons I like C more than C++. 
> > > Instead, I modified the semantics of the & operator. 
> > > 
> > > I don't want to be rude, but please do read my paper for full details: 
> > > https://www.open-std.org/jtc1/sc22/wg14/www/docs/n3089.pdf 
> > This part? 
> > 
> > "All compilers implicitly remove the _Optional qualifier from the type of the pointed-to 
> > object in the result of the expressions &*s1 and &*s2" 
> > 
> > if (p != NULL) { 
> > func(&*p); 
> > } 
> > 
> > (then this is just a shortcut for the cast)
> No, it's not a shortcut for a cast. The defined semantics of the & 
> operator avoids the need to cast. Just like in this example of 
> strictly-compliant and idiomatic C code: 
> void *x; 
> get_ptr(index, &x); 
> int *num = x; 
> 
> instead of this (commonplace but horrible) code: 
> int *num; 
> get_ptr(index, (void **)&num); 
> 
> Casting is ugly and throws away all type-safety. Even if the static analyzer were 
> somehow aware of which casts are valid and which are not, that would 
> actually remove useful functionality because there might be some cases where 
> a programmer actually *does* want to override the type system for legitimate 
> reasons. Standard usage is not one of those situations. 
> 
> Also, if I had to type this: 
> 
> if (p != NULL) { 
> func((int *)p); 
> } 
> 
> Or this: 
> 
> if (p != NULL) { 
> func(optional_cast<int *>(p)); 
> } 
> 
> Then I would honestly never have finished testing my prototype, nor have 
> any desire to productise it. 
> 
> Cheers, 
> Christopher

In some ways a pointer that cannot be null is similar of C++ references.
I have used C++ for many years. We have a guideline is to use references
as function arguments even when we know the object is always allocated on 
the heap.

For instance:

void F(X &x) {}

int main() {
  X *pX = new X();
  if (pX) {
     F(*pX);
  } 
}

The similarity with your proposal is that it part of the type and not dependent 
on flow analysis.

My view on the subject is that we need flow analysis. The compiler
must infer what is going on, just like programmers do in code reviews.

If the inference is too complicated then the code should be refactored
or give the programmer a way to override the inference.

One difference between programmers and the compiler, is that programmers
can read comments and documentation. So what we need is a way of providing
the "documentation" for the compiler. C23 attributes are a step on this direction, 
promoting standardization.
For instance, programmers will read the documentation for malloc and see that it can
return null.

We could have attributes in one header file like this.

[[may be null]] void* malloc()

then the compiler is aware. No need to repeat this information on the type.

struct X *pX = malloc(sizeof * pX);

null checks with flow analysis is not something new.
It is already implemented in C# and typescript for instance. 

So I don't understand why a similar proposal (with implementation)
would be hard for C? This functionality does not need to be implemented
for all compilers, they could just say the checks and attributes are parsed
but ignored.

[toc] | [prev] | [next] | [standalone]

#169110

...
> I don't want to be rude, but please do read my paper for full details: 
> https://www.open-std.org/jtc1/sc22/wg14/www/docs/n3089.pdf 
Something we agree.
I think the default must be "non null"
As a C user, I would be happy in break compatibility.

For instance, instead of f(int a[static 5]), f(int a[5]) would be OK.
No need for static. 

I also think f(int a[]) or by default should be non null.
then f(0) should not compile.

[toc] | [prev] | [next] | [standalone]

#169115

On 1/29/23 4:31 PM, Christopher Bazley wrote:
> On Sunday, 29 January 2023 at 20:53:23 UTC, Thiago Adams wrote:
>> On Saturday, January 28, 2023 at 1:16:07 PM UTC-3, cs9...@gmail.com wrote:
>>> " improved null safety does not require path-sensitive analysis."
>> But it is very incomplete without it!
> 
> Based on my experimentation on a large existing codebase
> over Christmas, not nearly so incomplete as to provide no value.
> 
> A member of the C standard committee made clear to me months ago
> that no extension to the language that *requires* (compiler) implementors
> to do path-sensitive analysis stands a chance of being accepted. I also
> think that is the right decision for the committee to make.
> 
> Consequently, I've tried to specify a compromise that enables static
> analysis whilst still providing value for compilers that do only simple
> validation of type compatibility.

And this is where I think you have a problem. For a base implementation 
that doesn't do that analysis, you can't require a diagnostic, and a 
modification to the type system that doesn't affect code generation or 
required diagnostics isn't really appropriate as a core language feature.

It really sounds like what you want is a standard form for something 
defined to be strictly a linting control, NOT an actual part of the type 
system. That is the domain of "static analysis"

> 
>> I am working with static analysis (hobby project) but I first
>> focused in ownership.
> 
> Good luck with that :)
> 
>> The basic and general idea is to have something
>> I called "imaginary attributes." for declarators
>>
>> In the context of null checks the imaginary attributes would be
>> path-sensitive.
>>
>> for instance
>>
>> if (p != NULL) {
>> here p has the imaginary attribute NOT NULL
>> }
>>
>> "imaginary attributes" are not part of the type.
>> Instead its is a compile time attribute that may change.
> 
> This sounds like what Clang's static analyser already
> implements, except that it calls them 'constraints' of
> the 'program state'.
> 
>> How do you imagine ignore/override the flow analysis? with casts?
> 
> No. I hate casts. :) That's one of the reasons I like C more than C++.
> Instead, I modified the semantics of the & operator.
> 
> I don't want to be rude, but please do read my paper for full details:
> https://www.open-std.org/jtc1/sc22/wg14/www/docs/n3089.pdf
> 
> Or my blog post:
> https://medium.com/itnext/why-c-needs-a-new-type-qualifier-61ad553cbe71
> 

[toc] | [prev] | [next] | [standalone]

#169151

Christopher Bazley <cs99cjb@gmail.com> writes:

> Hi all,
>
> I've just submitted my paper n3089, "_Optional: a type qualifier to
> indicate pointer nullability"  to the WG14 committee.  [...]
>
> I wrote a Medium article with much the same content as my paper,
> which is published here:
> https://medium.com/itnext/why-c-needs-a-new-type-qualifier-61ad553cbe71
>
> [...]

I looked at the article on Medium and also n3089.  I've read through
most of the postings in the thread here, not necessarily thoroughly
but sometimes only skimming.

What you're proposing is fundamentally broken.  There are two
reasons for this:

One, how the _Optional qualifier works is radically different from
how other type qualifiers work.  That by itself is enough to make
the new feature a non-starter.

Two, I'm not sure I follow what the rules are for how compilers have
to treat _Optional pointers, but it seems one of two things is true.
Either the new feature breaks a lot of existing code, or it doesn't
affect the semantics of the program, in the sense that adding or
taking away _Optional has no effect on whether the behavior is
defined, or what the behavior is, or whether the program may be
rejected.  Breaking existing code is an absolute deal breaker.  But
the other alternative is useless, because there are no consequences;
any warnings produced can be safely ignored, and that's what people
will do, and compilers will have an option to suppress the warnings.

These issues are not minor points that can be patched up.  To be
successful the question needs to be completely re-thought.

By the way, regarding the written proposal (and article), considered
separately from what is being proposed, I'm sorry to be blunt but
it's awful.  I'm thinking of writing a short essay about how to
write these things but I haven't done that yet.  Hopefully soon.

[toc] | [prev] | [next] | [standalone]

#169153

On Tuesday, 31 January 2023 at 15:54:44 UTC, Tim Rentsch wrote:
> rejected. Breaking existing code is an absolute deal breaker. But 
> the other alternative is useless, because there are no consequences; 
> any warnings produced can be safely ignored, and that's what people 
> will do, and compilers will have an option to suppress the warnings. 

Nobody ignores warnings at my place of employment because the
compiler (either gcc or clang) is invoked with -Werror. I don't think that
is particularly unusual either. I've never seen anyone (until you) advocating
ignoring warnings, so if that's the basis of your argument then I'm not
inclined to give the rest of it much credence. Presumably you also
dismiss the value of any static analysis tools, since their use is not
mandated by the C standard. I can't imagine how awful the code that
would result from such an attitude might be.

[toc] | [prev] | [next] | [standalone]

#169155

On Tuesday, 31 January 2023 at 17:25:32 UTC, cs9 wrote:
> On Tuesday, 31 January 2023 at 15:54:44 UTC, Tim Rentsch wrote: 
> > rejected. Breaking existing code is an absolute deal breaker. But 
> > the other alternative is useless, because there are no consequences; 
> > any warnings produced can be safely ignored, and that's what people 
> > will do, and compilers will have an option to suppress the warnings.
> Nobody ignores warnings at my place of employment because the 
> compiler (either gcc or clang) is invoked with -Werror. I don't think that 
> is particularly unusual either. I've never seen anyone (until you) advocating 
> ignoring warnings, so if that's the basis of your argument then I'm not 
> inclined to give the rest of it much credence. Presumably you also 
> dismiss the value of any static analysis tools, since their use is not 
> mandated by the C standard. I can't imagine how awful the code that 
> would result from such an attitude might be.

He didn't advocate ignoring warnings; he said that people will ignore them.

And why not? Surely if an issue was serious, a compiler would make it an error. The only reason to give a warning is to allow it to be ignored. (Although personally I'm surprised that many things I consider hard errors, generate mere warnings.)

Also, whether something is a warning at all, or warning reported as an error, is the choice of whoever invokes the compiler, made via its options. So in C, the user gets to decide how strictly the language is treated, whether the program is correct, has failed or is passable.

[toc] | [prev] | [next] | [standalone]

#169162

On 31/01/2023 23:44, bart c wrote:
> On Tuesday, 31 January 2023 at 17:25:32 UTC, cs9 wrote:
>> On Tuesday, 31 January 2023 at 15:54:44 UTC, Tim Rentsch wrote:
>>> rejected. Breaking existing code is an absolute deal breaker.
>>> But the other alternative is useless, because there are no
>>> consequences; any warnings produced can be safely ignored, and
>>> that's what people will do, and compilers will have an option to
>>> suppress the warnings.
>> Nobody ignores warnings at my place of employment because the 
>> compiler (either gcc or clang) is invoked with -Werror. I don't
>> think that is particularly unusual either. I've never seen anyone
>> (until you) advocating ignoring warnings, so if that's the basis of
>> your argument then I'm not inclined to give the rest of it much
>> credence. Presumably you also dismiss the value of any static
>> analysis tools, since their use is not mandated by the C standard.
>> I can't imagine how awful the code that would result from such an
>> attitude might be.
> 
> He didn't advocate ignoring warnings; he said that people will ignore
> them.
> 

That is unfortunate, but true.  There are programmers who treat warnings 
as "friendly advice" rather than an indication that there could be a 
problem.

> And why not? Surely if an issue was serious, a compiler would make it
> an error. The only reason to give a warning is to allow it to be
> ignored. (Although personally I'm surprised that many things I
> consider hard errors, generate mere warnings.)
> 

Also unfortunate, but true - backwards compatibility affects compilers 
as much as the language, and means most useful warnings on most 
compilers are opt-in, rather than enabled by default, and generated as 
non-fatal warnings rather than hard errors.  This means that existing 
poor-quality but working code continues to build as before, but it also 
means that programmers who don't use their tools well, don't get the 
same aids to spotting problems in their code.

> Also, whether something is a warning at all, or warning reported as
> an error, is the choice of whoever invokes the compiler, made via its
> options. So in C, the user gets to decide how strictly the language
> is treated, whether the program is correct, has failed or is
> passable.

In professional work, it's common that individual programmers don't have 
such freedom to ignore warnings - it is common practice that code has to 
compile warning-free with (at least) the company's standard set of 
warning flags before it can be considered ready to move up the system 
(to code reviews, testing, merging, or whatever).  In such situations, 
it is a project manager or senior developer who decides, rather than 
each individual programmer.

C is used for such a huge variety of types of code that there really is 
no way to pick a single set of warnings that work for everyone.  There 
are commonly-used subsets (thus "-Wall" in gcc and clang), but clang's 
"-Weverything" flag is for compiler testing only, not for real-world coding.

[toc] | [prev] | [next] | [standalone]

Page 3 of 4 — ← Prev page 1 2 [3] 4  Next page →

Back to top | Article view | comp.lang.c

csiph-web