Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.c > #380026 > unrolled thread

Which tools are available for catching UB?

Back to article view | Back to comp.lang.c

Contents

  Which tools are available for catching UB? Anthony Cuozzo <anthony@cuozzo.us> - 2024-01-10 23:15 -0500
    Re: Which tools are available for catching UB? David Brown <david.brown@hesbynett.no> - 2024-01-11 13:43 +0100
      Re: Which tools are available for catching UB? Anthony Cuozzo <anthony@cuozzo.us> - 2024-01-11 18:15 -0500
        Re: Which tools are available for catching UB? Keith Thompson <Keith.S.Thompson+u@gmail.com> - 2024-01-11 16:09 -0800
          Re: Which tools are available for catching UB? Tim Rentsch <tr.17687@z991.linuxsc.com> - 2024-01-14 09:44 -0800
        Re: Which tools are available for catching UB? David Brown <david.brown@hesbynett.no> - 2024-01-12 14:50 +0100
    Re: Which tools are available for catching UB? Richard Kettlewell <invalid@invalid.invalid> - 2024-01-12 08:51 +0000
    Re: Which tools are available for catching UB? Malcolm McLean <malcolm.arthur.mclean@gmail.com> - 2024-01-18 18:17 +0000
    Re: Which tools are available for catching UB? Lew Pitcher <lew.pitcher@digitalfreehold.ca> - 2024-01-18 19:08 +0000
      Re: Which tools are available for catching UB? James Kuyper <jameskuyper@alumni.caltech.edu> - 2024-01-18 14:42 -0500
        Re: Which tools are available for catching UB? Lew Pitcher <lew.pitcher@digitalfreehold.ca> - 2024-01-18 20:24 +0000
      Re: Which tools are available for catching UB? Tim Rentsch <tr.17687@z991.linuxsc.com> - 2024-01-25 19:57 -0800
        Re: Which tools are available for catching UB? Kaz Kylheku <433-929-6894@kylheku.com> - 2024-01-26 04:52 +0000
          Re: Which tools are available for catching UB? Tim Rentsch <tr.17687@z991.linuxsc.com> - 2024-02-10 02:06 -0800
    Re: Which tools are available for catching UB? Kaz Kylheku <433-929-6894@kylheku.com> - 2024-01-18 19:41 +0000
      Re: Which tools are available for catching UB? "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> - 2024-01-18 13:18 -0800
      Re: Which tools are available for catching UB? gazelle@shell.xmission.com (Kenny McCormack) - 2024-01-19 03:08 +0000

#380026 — Which tools are available for catching UB?

The only tool I use regularly for identifying instances of undefined 
behavior is the semantics compiler "kcc" from RV-Match.

Are there any other tools out there besides what ships with e.g., GCC & 
Clang?

Thanks,
--Anthony Cuozzo

[toc] | [next] | [standalone]

#380033

On 11/01/2024 05:15, Anthony Cuozzo wrote:
> The only tool I use regularly for identifying instances of undefined 
> behavior is the semantics compiler "kcc" from RV-Match.
> 
> Are there any other tools out there besides what ships with e.g., GCC & 
> Clang?
> 

Both gcc and clang have "sanitizers".  You compile the code with the 
appropriate options, and the code is augmented with checks for different 
kinds of UB, detected at run-time.  gcc and clang have many of these in 
common, and some that are only implemented in one of them.  Some 
sanitizers can have significant impact on code speed, others do not. 
You will want to try things with different flags to see what works best 
for you.

<https://gcc.gnu.org/onlinedocs/gcc/Instrumentation-Options.html#index-fsanitize_003dundefined>

<https://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html>


Both gcc and clang can also do a great deal of static error checking 
which can find some kinds of UB before running the code.  And there are 
other tools such as clang-tidy, and third-party linters and checkers, 
that can help.  (Some are quite expensive.)

[toc] | [prev] | [next] | [standalone]

#380049

On 1/11/24 07:43, David Brown wrote:
> On 11/01/2024 05:15, Anthony Cuozzo wrote:
>> The only tool I use regularly for identifying instances of undefined 
>> behavior is the semantics compiler "kcc" from RV-Match.
>>
>> Are there any other tools out there besides what ships with e.g., GCC 
>> & Clang?
>>
> 
> Both gcc and clang have "sanitizers".  You compile the code with the 
> appropriate options, and the code is augmented with checks for different 
> kinds of UB, detected at run-time.  gcc and clang have many of these in 
> common, and some that are only implemented in one of them.  Some 
> sanitizers can have significant impact on code speed, others do not. You 
> will want to try things with different flags to see what works best for 
> you.
> 
> <https://gcc.gnu.org/onlinedocs/gcc/Instrumentation-Options.html#index-fsanitize_003dundefined>
> 
> <https://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html>
> 
> 
> Both gcc and clang can also do a great deal of static error checking 
> which can find some kinds of UB before running the code.  And there are 
> other tools such as clang-tidy, and third-party linters and checkers, 
> that can help.  (Some are quite expensive.)
> 
> 

I suppose I was/am looking for static analysis tools which focus on UB, 
but now that I've given it more thought I realize that only a subset of 
UB can be detected at compile time.

Semi-related: Do you know if there's a resource which breaks down UB per 
standard? I'd like to see how things have changed over time.

Thanks,
--Anthony

[toc] | [prev] | [next] | [standalone]

#380053

Anthony Cuozzo <anthony@cuozzo.us> writes:
[...]
> I suppose I was/am looking for static analysis tools which focus on
> UB, but now that I've given it more thought I realize that only a
> subset of UB can be detected at compile time.

Which, in many or most cases, is exactly why it's UB.

Ideally, something's behavior is left undefined because it's impractical
to detect the problem.  In some cases, behavior has been left undefined
(or unspecified, or implementation-defined) because existing
implementations behave differently.

> Semi-related: Do you know if there's a resource which breaks down UB
> per standard? I'd like to see how things have changed over time.

Each edition of the standard has an annex (Annex J in the case of C11)
that summarizes unspecified, undefined, and implementation-defined
behaviors.  The standards themselves cost money, but drafts are freely
available.

Some instances of undefined behavior are specified explicitly.  Others
are undefined just because the standard provides no definition.  Both
kinds are equivalent, and can in principle result in the same kinds of
Bad Things Happening.

-- 
Keith Thompson (The_Other_Keith) Keith.S.Thompson+u@gmail.com
Working, but not speaking, for Medtronic
void Void(void) { Void(); } /* The recursive call of the void */

[toc] | [prev] | [next] | [standalone]

#380142

Keith Thompson <Keith.S.Thompson+u@gmail.com> writes:

> Anthony Cuozzo <anthony@cuozzo.us> writes:
> [...]
>
>> [looking for constructs that are undefined behavior]

>> Semi-related:  Do you know if there's a resource which breaks down UB
>> per standard?  I'd like to see how things have changed over time.
>
> Each edition of the standard has an annex (Annex J in the case of C11)
> that summarizes unspecified, undefined, and implementation-defined
> behaviors.  The standards themselves cost money, but drafts are freely
> available.

Annex J (which is Annex G in C90) is an excellent resource.  A caution
is in order:  not every case of undefined behavior (and probably also
unspecified behavior and implementation-defined behavior) is listed in
Annex J.  Most are, but not all are.  Also, sometimes a statement of UB
in Annex J is not completely accurate, but only an approximation.  It
is still the case that Annex J is an excellent resource, but don't take
it as gospel.

> Some instances of undefined behavior are specified explicitly.  Others
> are undefined just because the standard provides no definition.  Both
> kinds are equivalent, and can in principle result in the same kinds of
> Bad Things Happening.

I think a fine point should be noted here.  Some kinds of undefined
behavior, in addition to being undefined behavior, also require a
diagnostic be issued.  So different kinds of undefined behavior may
not be exactly equivalent - some require some sort of message out of
the compiler, whereas others may get no indication when compiling.
None of this is meant to contradict Keith's statement, just to augment
it with a clarification.

[toc] | [prev] | [next] | [standalone]

#380061

On 12/01/2024 00:15, Anthony Cuozzo wrote:
> On 1/11/24 07:43, David Brown wrote:
>> On 11/01/2024 05:15, Anthony Cuozzo wrote:
>>> The only tool I use regularly for identifying instances of undefined 
>>> behavior is the semantics compiler "kcc" from RV-Match.
>>>
>>> Are there any other tools out there besides what ships with e.g., GCC 
>>> & Clang?
>>>
>>
>> Both gcc and clang have "sanitizers".  You compile the code with the 
>> appropriate options, and the code is augmented with checks for 
>> different kinds of UB, detected at run-time.  gcc and clang have many 
>> of these in common, and some that are only implemented in one of 
>> them.  Some sanitizers can have significant impact on code speed, 
>> others do not. You will want to try things with different flags to see 
>> what works best for you.
>>
>> <https://gcc.gnu.org/onlinedocs/gcc/Instrumentation-Options.html#index-fsanitize_003dundefined>
>>
>> <https://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html>
>>
>>
>> Both gcc and clang can also do a great deal of static error checking 
>> which can find some kinds of UB before running the code.  And there 
>> are other tools such as clang-tidy, and third-party linters and 
>> checkers, that can help.  (Some are quite expensive.)
>>
>>
> 
> I suppose I was/am looking for static analysis tools which focus on UB, 
> but now that I've given it more thought I realize that only a subset of 
> UB can be detected at compile time.

That is absolutely correct.  In fact, most UB can only be detected at 
run time.  Static analysis (in a compiler, or dedicated tools) can 
usually only see some kinds of /potential/ UB.  For example, if you 
write "int foo(void) { return 1 / 0; }", that is not UB in itself - it 
is only UB if your program calls "foo".  And usually the compiler isn't 
able to determine what code will actually be called when you run the 
program, unless it can trace the execution unconditionally from main().

But it is, IMHO, a good idea to find as many of your codes bugs as 
possible using static checking - it's the easiest and cheapest time to 
do it.  gcc and clang both have quite sophisticated warnings and static 
analysis features (with steadily more for each new compiler release), 
and clang also has some stand-alone tools for the job.  There are also 
dedicated tools for particular use-cases (such as tools for checking 
Linux kernel code for certain kinds of problems).  And there are quite a 
number of commercial tools that do very sophisticated static error 
checking, if your budget stretches to buying them.

> 
> Semi-related: Do you know if there's a resource which breaks down UB per 
> standard? I'd like to see how things have changed over time.
> 

Each C standard version has an Annex that lists the explicit UB 
described in the standard - but remember that things that have no 
standards-defined behaviour are also UB in C (though a compiler may 
choose to define them).

[toc] | [prev] | [next] | [standalone]

#380058

Anthony Cuozzo <anthony@cuozzo.us> writes:
> The only tool I use regularly for identifying instances of undefined
> behavior is the semantics compiler "kcc" from RV-Match.
>
> Are there any other tools out there besides what ships with e.g., GCC
> & Clang?

Dynamic analysis:

* https://gcc.gnu.org/onlinedocs/gcc/Instrumentation-Options.html,
  search for ‘sanitize’. Instruments executable to detect various issues
  at runtime.

* https://www.gnu.org/software/libc/manual/html_node/Source-Fortification.html
  Additional bounds checking.

* https://clang.llvm.org/docs/index.html, search for ‘sanitize’. Ditto.

* https://valgrind.org/. Detects various issues in unmodified
  executables.

Static analysis:

* https://clang-analyzer.llvm.org/. Quite limited and struggles with
  false positives IME.

* https://www.synopsys.com/software-integrity/static-analysis-tools-sast/coverity.html
  Extensive checking and does find many real issues but also produces a
  lot of false positives. Pricey.

* https://scan.coverity.com/. Free version of the above for open source
  projects.

-- 
https://www.greenend.org.uk/rjk/

[toc] | [prev] | [next] | [standalone]

#380417

On 11/01/2024 04:15, Anthony Cuozzo wrote:
> The only tool I use regularly for identifying instances of undefined 
> behavior is the semantics compiler "kcc" from RV-Match.
> 
> Are there any other tools out there besides what ships with e.g., GCC & 
> Clang?
> 
> Thanks,
> --Anthony Cuozzo
 >
Almost by definition you can't catch all undefined behaviour, since it 
is "undefined".
Out of bounds array accesses can be caught by sanitizers or valgrind.
C is notorious for this bug, since dynamic arrays have no way of 
obtaining the size by querying the pointer, so size and array have to be 
passed in separate variables, and the potential for them getting out of 
synch is high.

But undefined behaviour like a shift which is out of range is harder to 
catch. Whilst it is undefined in C, it often compiles to valid and 
perfectly well-behaved machine code.

[toc] | [prev] | [next] | [standalone]

#380420

On Wed, 10 Jan 2024 23:15:52 -0500, Anthony Cuozzo wrote:

> The only tool I use regularly for identifying instances of undefined 
> behavior is the semantics compiler "kcc" from RV-Match.
> 
> Are there any other tools out there besides what ships with e.g., GCC & 
> Clang?


By definition (for instance, C11 Section 3.4.3: "undefined behavior") undefined
behaviour is "behavior, upon use of a nonportable or erroneous program construct
or of erroneous data,for which this International Standard imposes no requirements".

Outside of the "erroneous" constructs and data, this also means that "nonportable"
program constructs, for which the International Standard imposes no requirements,
invoke "undefined behaviour", as far as the ISO C standard is concerned.

This means that a single call to a function not defined by your program source
code or by the ISO C standard will invoke "undefined behaviour". So, a program
that calls CopyFile() (a Microsoft Windows API) or open() (a POSIX API) invokes
"undefined behaviour".

While it is certainly possible to write C programs that adhere entirely to the
ISO C standard, many C programs (dare I say, most C programs?) invoke /some/
amount of "undefined behaviour" wrt the C standard, even when the behaviour
/is/ defined by other standards and sources.

So, does "kcc" from RV-Match catch these forms of "undefined behaviour"?"

-- 
Lew Pitcher
"In Skills We Trust"

[toc] | [prev] | [next] | [standalone]

#380425

On 1/18/24 14:08, Lew Pitcher wrote:
...
> By definition (for instance, C11 Section 3.4.3: "undefined behavior")
> undefined
> behaviour is "behavior, upon use of a nonportable or erroneous program
> construct
> or of erroneous data,for which this International Standard imposes no
> requirements".
>
> Outside of the "erroneous" constructs and data, this also means that
> "nonportable"
> program constructs, for which the International Standard imposes no
> requirements,
> invoke "undefined behaviour", as far as the ISO C standard is concerned.
>
> This means that a single call to a function not defined by your
> program source
> code or by the ISO C standard will invoke "undefined behaviour". So, a
> program
> that calls CopyFile() (a Microsoft Windows API) or open() (a POSIX
> API) invokes
> "undefined behaviour".
>
> While it is certainly possible to write C programs that adhere
> entirely to the
> ISO C standard, many C programs (dare I say, most C programs?) invoke
> /some/
> amount of "undefined behaviour" wrt the C standard, even when the
> behaviour
> /is/ defined by other standards and sources.

Keep in mind that "undefined behavior" in C means ONLY that "this
international standard" imposes no requirements. If requirements are
imposed by some other document, such as the documentation for the
library that you're using, those requirements can be sufficient to make
your program useful. If that library's documentation describes the
behavior of the particular function you're calling, that's sufficient
for that function call. If it also claims compatibility with a given
version of the C standard, that implies that when compiling and linking
with that version of the C standard, all requirements that the C
standard would impose on all of your code except that function call also
apply - not because the C standard says so, but because the library's
documentation says so.

[toc] | [prev] | [next] | [standalone]

#380428

On Thu, 18 Jan 2024 14:42:16 -0500, James Kuyper wrote:

> On 1/18/24 14:08, Lew Pitcher wrote:
> ...
>> By definition (for instance, C11 Section 3.4.3: "undefined behavior")
>> undefined
>> behaviour is "behavior, upon use of a nonportable or erroneous program
>> construct
>> or of erroneous data,for which this International Standard imposes no
>> requirements".
>>
>> Outside of the "erroneous" constructs and data, this also means that
>> "nonportable"
>> program constructs, for which the International Standard imposes no
>> requirements,
>> invoke "undefined behaviour", as far as the ISO C standard is concerned.
>>
>> This means that a single call to a function not defined by your
>> program source
>> code or by the ISO C standard will invoke "undefined behaviour". So, a
>> program
>> that calls CopyFile() (a Microsoft Windows API) or open() (a POSIX
>> API) invokes
>> "undefined behaviour".
>>
>> While it is certainly possible to write C programs that adhere
>> entirely to the
>> ISO C standard, many C programs (dare I say, most C programs?) invoke
>> /some/
>> amount of "undefined behaviour" wrt the C standard, even when the
>> behaviour
>> /is/ defined by other standards and sources.
> 
> Keep in mind that "undefined behavior" in C means ONLY that "this
> international standard" imposes no requirements.

My point, exactly.

My question to the OP was, in effect, is the tool that the OP uses
"strict" in it's detection of UB (i.e. calling a program that uses
POSIX apis as exhibiting "undefined behaviour") or does it allow a
looser interpretation?

[snip]

-- 
Lew Pitcher
"In Skills We Trust"

[toc] | [prev] | [next] | [standalone]

#380968

Lew Pitcher <lew.pitcher@digitalfreehold.ca> writes:

> [A] single call to a function not defined by your program source
> code or by the ISO C standard will invoke "undefined behaviour".

That isn't right.  The C standard allows previously translated
translation units "[to] be preserved individually or in libraries."
Those translation units don't have to be your own code or even
necessarily stored, or translated, on the same machine.  In
translation phase 8, "[l]ibrary components are linked to satisfy
external references to functions and objects not defined in the
current translation."  The C standard doesn't specify how the
libraries are located, or even require that you be able to inspect
them, but clearly does require that libraries be consulted to satisfy
external references.  We don't know what code in the libraries will
do, but there is a requirement /on the implementation/ that they be
linked against in phase 8.  The presence of that requirement means
that linking to, or calling, such an external reference is not ipso
facto undefined behavior.  (Obviously it could be undefined behavior
for other reasons, but not just by virtue of there being a call.)

Not knowing what something will do is not the same as undefined
behavior.  The question is Does the C standard give a requirement
about what implementations have to do?  In this case it does.  An
implementation is not free to do whatever it wants just because a
library was previously translated on a different machine.  Code in
a library might (emphasis _might_) provoke undefined behavior if it
is called, but that depends on what the library code is, and is not
something an implementation can just arbitrarily chose to do on its
own.  It's important to understand the difference.

[toc] | [prev] | [next] | [standalone]

#380970

On 2024-01-26, Tim Rentsch <tr.17687@z991.linuxsc.com> wrote:
> Lew Pitcher <lew.pitcher@digitalfreehold.ca> writes:
>
>> [A] single call to a function not defined by your program source
>> code or by the ISO C standard will invoke "undefined behaviour".
>
> That isn't right.  The C standard allows previously translated
> translation units "[to] be preserved individually or in libraries."
> Those translation units don't have to be your own code or even
> necessarily stored, or translated, on the same machine.

This is a strawman interpretation of what Lew is almost certainly
saying, which is the salient point that using a function that is not
somewhere in your program (any translation unit from your sources or any
translated units you brought to the table yourself), and not in the
standard, is undefined behavior.

He can't be literally saying that calling a function foo is undefined
behavior, even if it's found in a libfoo.a that is brought from
another machine, and which has no compatibility issues (like wrong
architecture, unsupported object format, wrong ABI), and is being linked
to the program, and used correctly according to its documentation.

That would be silly, and uncharacteristic of Lew's level of experience,
so it can't be the right interpretation.

-- 
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal
Mastodon: @Kazinator@mstdn.ca

[toc] | [prev] | [next] | [standalone]

#382262

Kaz Kylheku <433-929-6894@kylheku.com> writes:

> On 2024-01-26, Tim Rentsch <tr.17687@z991.linuxsc.com> wrote:
>
>> Lew Pitcher <lew.pitcher@digitalfreehold.ca> writes:
>>
>>> [A] single call to a function not defined by your program source
>>> code or by the ISO C standard will invoke "undefined behaviour".
>>
>> That isn't right.  The C standard allows previously translated
>> translation units "[to] be preserved individually or in libraries."
>> Those translation units don't have to be your own code or even
>> necessarily stored, or translated, on the same machine.
>
> This is a strawman interpretation of what Lew is almost certainly
> saying,

No, it isn't.  You misunderstood my statement.

> which is the salient point that using a function that is not
> somewhere in your program (any translation unit from your sources
> or any translated units you brought to the table yourself), and
> not in the standard, is undefined behavior.

No, it isn't.  Whether a library, for example, was something you put
on the machine yourself, or was put there by a hacker without your
knowledge, doesn't affect the presence or absence of undefined
behavior.  All that matters is what's in the library.  It's
perfectly possible for a library installed by a hacker to perform
only well-defined operations, be well-formed and ABI-compatible,
etc.  Just because you don't know what is in the library doesn't
make it undefined behavior.

[toc] | [prev] | [next] | [standalone]

#380424

On 2024-01-11, Anthony Cuozzo <anthony@cuozzo.us> wrote:
> Are there any other tools out there besides what ships with e.g., GCC & 
> Clang?

All the tools that waste their time hanging out on comp.lang.c are
pretty good for catching UB.

-- 
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal
Mastodon: @Kazinator@mstdn.ca
NOTE: If you use Google Groups, I don't see you, unless you're whitelisted.

[toc] | [prev] | [next] | [standalone]

#380431

On 1/18/2024 11:41 AM, Kaz Kylheku wrote:
> On 2024-01-11, Anthony Cuozzo <anthony@cuozzo.us> wrote:
>> Are there any other tools out there besides what ships with e.g., GCC &
>> Clang?
> 
> All the tools that waste their time hanging out on comp.lang.c are
> pretty good for catching UB.
> 

A little harsh? :^)

[toc] | [prev] | [next] | [standalone]

#380453

In article <20240118114117.651@kylheku.com>,
Kaz Kylheku  <433-929-6894@kylheku.com> wrote:
>On 2024-01-11, Anthony Cuozzo <anthony@cuozzo.us> wrote:
>> Are there any other tools out there besides what ships with e.g., GCC & 
>> Clang?
>
>All the tools that waste their time hanging out on comp.lang.c are
>pretty good for catching UB.

Well done, sir!

(nice play on the word "tool")

-- 
Note that Oprah actually is all the things that The Donald only wishes he were.
For one thing, she actually *is* a billionaire.  She's also actually self-made,
came from nothing, knows how to run businesses, never went bankrupt, is smart
and is mentally stable.

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.c

csiph-web