Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.programming > #799 > unrolled thread

How to implement diagnostic messages?

Back to article view | Back to comp.programming

Contents

  How to implement diagnostic messages? Rui Maciel <rui.maciel@gmail.com> - 2011-09-08 14:58 +0100
    Re: How to implement diagnostic messages? BGB <cr88192@hotmail.com> - 2011-09-08 10:57 -0700
      Re: How to implement diagnostic messages? Ben Bacarisse <ben.usenet@bsb.me.uk> - 2011-09-08 20:53 +0100
        Re: How to implement diagnostic messages? BGB <cr88192@hotmail.com> - 2011-09-08 14:14 -0700
          Re: How to implement diagnostic messages? "Dmitry A. Kazakov" <mailbox@dmitry-kazakov.de> - 2011-09-09 09:30 +0200
      Re: How to implement diagnostic messages? Rui Maciel <rui.maciel@gmail.com> - 2011-09-11 18:27 +0100
        Re: How to implement diagnostic messages? BGB <cr88192@hotmail.com> - 2011-09-11 11:44 -0700
    Re: How to implement diagnostic messages? Ian Collins <ian-news@hotmail.com> - 2011-09-09 10:33 +1200
      Re: How to implement diagnostic messages? Rui Maciel <rui.maciel@gmail.com> - 2011-09-11 18:33 +0100
    Re: How to implement diagnostic messages? James Waldby <not@valid.invalid> - 2011-09-10 00:17 +0000
      Re: How to implement diagnostic messages? Rui Maciel <rui.maciel@gmail.com> - 2011-09-11 18:41 +0100
        Re: How to implement diagnostic messages? BGB <cr88192@hotmail.com> - 2011-09-11 11:48 -0700
          Re: How to implement diagnostic messages? Ian Collins <ian-news@hotmail.com> - 2011-09-12 08:47 +1200
            Re: How to implement diagnostic messages? BGB <cr88192@hotmail.com> - 2011-09-11 14:56 -0700
              Re: How to implement diagnostic messages? Ian Collins <ian-news@hotmail.com> - 2011-09-12 15:00 +1200

#799 — How to implement diagnostic messages?

In some applications it's useful to have, at least in the development 
stages, a way to output diagnostic messages in order to help developers 
check what's happening under the hood.  Yet, although this can be useful and 
in some times even invaluable, there isn't much information on what are 
considered to be good practices regarding the implementation of diagnostic 
messages. 

With this in mind, what are your views on how to implement diagnostic 
messages?  Are simple outputs to stderr enough or are there better ways to 
handle this?


Rui Maciel

[toc] | [next] | [standalone]

#800

On 9/8/2011 6:58 AM, Rui Maciel wrote:
> In some applications it's useful to have, at least in the development
> stages, a way to output diagnostic messages in order to help developers
> check what's happening under the hood.  Yet, although this can be useful and
> in some times even invaluable, there isn't much information on what are
> considered to be good practices regarding the implementation of diagnostic
> messages.
>
> With this in mind, what are your views on how to implement diagnostic
> messages?  Are simple outputs to stderr enough or are there better ways to
> handle this?
>

the great problems with stderr:
the messages are not recorded anywhere (can't look over them in more 
detail later);
in graphical and/or GUI applications, stdout and stderr tend to go 
nowhere, so anything sent there is not visible.

in my own apps, I tended to just use printf a lot.
eventually, I made a macro which hooked printf to one of my own 
functions, and mostly ended up just redirecting its output to a log file 
and to an in-program console (in addition to stdout).

this way, for an app, one can typically see messages in-program, or if 
there is too much output, go and look over the recorded log-file.


later, I also added a "dbgprintf" function, which also accepts an 
integer parameter giving its "debug level" (IIRC, number+flags), but 
have not ended up widely using it.

the idea here would be that dbgprintf would also redirect to the 
console, and to the log file, but that the debug-level could be used to 
cull lower-priority messages.

IIRC, level 0 was equivalent to a printf, and 1..N (limit not set, maybe 
7 or 15) indicate progressively lower-priority messages.

a message with a greater debug-level than the present debug-level will 
simply be discarded (neither shown in console, nor logged).

an unaddressed issue is "where" the messages are from, which often 
matters a lot more than some global "debug level" (one tends to care a 
lot more about messages from the components they are debugging, without 
having to necessarily see a mountain of messages for whatever else is 
going on, like other components loading/saving files, messages from the 
sound mixer, ...).


the current practice, for the most part, is a bit less organized, and 
mostly involves manually adding/commenting/uncommenting printf calls.

or such...

[toc] | [prev] | [next] | [standalone]

#801

BGB <cr88192@hotmail.com> writes:

> On 9/8/2011 6:58 AM, Rui Maciel wrote:
>> In some applications it's useful to have, at least in the development
>> stages, a way to output diagnostic messages in order to help developers
>> check what's happening under the hood.  Yet, although this can be useful and
>> in some times even invaluable, there isn't much information on what are
>> considered to be good practices regarding the implementation of diagnostic
>> messages.
>>
>> With this in mind, what are your views on how to implement diagnostic
>> messages?  Are simple outputs to stderr enough or are there better ways to
>> handle this?
>>
>
> the great problems with stderr:
> the messages are not recorded anywhere (can't look over them in more
> detail later);
> in graphical and/or GUI applications, stdout and stderr tend to go
> nowhere, so anything sent there is not visible.

These are problems with you platform, not with stderr per se.  I can
view output from GUI programs just fine and I can save the output for
later study.

<snip>
-- 
Ben.

[toc] | [prev] | [next] | [standalone]

#802

On 9/8/2011 12:53 PM, Ben Bacarisse wrote:
> BGB<cr88192@hotmail.com>  writes:
>
>> On 9/8/2011 6:58 AM, Rui Maciel wrote:
>>> In some applications it's useful to have, at least in the development
>>> stages, a way to output diagnostic messages in order to help developers
>>> check what's happening under the hood.  Yet, although this can be useful and
>>> in some times even invaluable, there isn't much information on what are
>>> considered to be good practices regarding the implementation of diagnostic
>>> messages.
>>>
>>> With this in mind, what are your views on how to implement diagnostic
>>> messages?  Are simple outputs to stderr enough or are there better ways to
>>> handle this?
>>>
>>
>> the great problems with stderr:
>> the messages are not recorded anywhere (can't look over them in more
>> detail later);
>> in graphical and/or GUI applications, stdout and stderr tend to go
>> nowhere, so anything sent there is not visible.
>
> These are problems with you platform, not with stderr per se.  I can
> view output from GUI programs just fine and I can save the output for
> later study.
>
> <snip>

I develop primarily on Windows, and using MSVC on this target, which I 
would guess is a fairly common use-case.

the issue is, however, that GUI apps seem to discard both stdout and 
stderr, making them not useful on this target (and hence log files are 
far more preferable).

even in console apps, log files may still be preferable, given the 
console only has so many lines of text.


on Linux, unless one runs a GUI app from the console, they will still 
not see the console output, hence log-files are still useful here (and 
don't require mucking around with command-line options or other things 
to save the output).

...

[toc] | [prev] | [next] | [standalone]

#805

On Thu, 08 Sep 2011 14:14:19 -0700, BGB wrote:

> even in console apps, log files may still be preferable, given the 
> console only has so many lines of text.

Rather that GUI implementations are broken per design in these days.

In the field we have trace files of multiple gigabytes size. If put into a
GUI widget that would cause the latter to cache the whole contents into the
memory. Another broken design (of Linux OS, for example) is that an
application may get more memory than the system physically has (RAM +
swap). In the end you likely would need to reboot. But in any case waiting
for several hours in order to see log is not good.

A file is always better, though duplicated onto the standard output.
Another reason for having a file is that the system may crash, so you would
like to have a persistent copy of log.

As for properties, better than "log level" is "tracing category", a
combinations of flags. E.g. error, warning, info, memory management, low
level I/O trace, packet level I/O trace, application level I/O trace etc,
not necessarily ordered as levels are.

It is also useful to have a trace source as a separate "column", thinking
of it in a relational way. If you have a large system you would like to
look after some components, ignoring trace messages from others.

In relational view it would be: time stamp; text; categories flags; source.
If RDBMS were usable for such purposes, one could even trace directly into
a DB, but they aren't. (:-))

-- 
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

[toc] | [prev] | [next] | [standalone]

#808

BGB wrote:

> the great problems with stderr:
> the messages are not recorded anywhere (can't look over them in more
> detail later);
> in graphical and/or GUI applications, stdout and stderr tend to go
> nowhere, so anything sent there is not visible.

Sounds like a good tradeoff.  Considering this scenario, I can understand 
why plain old writes to stdout/stderr are good enough to a lot of people.


> in my own apps, I tended to just use printf a lot.
> eventually, I made a macro which hooked printf to one of my own
> functions, and mostly ended up just redirecting its output to a log file
> and to an in-program console (in addition to stdout).
> 
> this way, for an app, one can typically see messages in-program, or if
> there is too much output, go and look over the recorded log-file.

Sounds good.


> later, I also added a "dbgprintf" function, which also accepts an
> integer parameter giving its "debug level" (IIRC, number+flags), but
> have not ended up widely using it.
> 
> the idea here would be that dbgprintf would also redirect to the
> console, and to the log file, but that the debug-level could be used to
> cull lower-priority messages.
> 
> IIRC, level 0 was equivalent to a printf, and 1..N (limit not set, maybe
> 7 or 15) indicate progressively lower-priority messages.
> 
> a message with a greater debug-level than the present debug-level will
> simply be discarded (neither shown in console, nor logged).

Sounds like a good idea, although it could be beneficial to employ those 
integer parameters not as a priority indicator but as some sort of channel 
descriptor.  This would offer the programmer with the ability to organize 
log messages based on criteria such as which component generated them.  With 
this, it would be possible to add code to provide the end-user with the 
ability to configure what messages would be displayed, and even how they 
were displayed (i.e., write to stdout/stderr, write to a log file or both).


> an unaddressed issue is "where" the messages are from, which often
> matters a lot more than some global "debug level" (one tends to care a
> lot more about messages from the components they are debugging, without
> having to necessarily see a mountain of messages for whatever else is
> going on, like other components loading/saving files, messages from the
> sound mixer, ...).

Probably this could be dealt with if the "integer parameters as channel 
descriptors" approach was followed.  A channel could be registered with the 
logger, where the programmer could also specify a custom prefix to be used 
in those messages.


> the current practice, for the most part, is a bit less organized, and
> mostly involves manually adding/commenting/uncommenting printf calls.

Unfortunately I also tend to do that.  It's a great big ugly hack which gets 
uglier with time.


Rui Maciel

[toc] | [prev] | [next] | [standalone]

#811

On 9/11/2011 10:27 AM, Rui Maciel wrote:
> BGB wrote:
>
>> the great problems with stderr:
>> the messages are not recorded anywhere (can't look over them in more
>> detail later);
>> in graphical and/or GUI applications, stdout and stderr tend to go
>> nowhere, so anything sent there is not visible.
>
> Sounds like a good tradeoff.  Considering this scenario, I can understand
> why plain old writes to stdout/stderr are good enough to a lot of people.
>
>
>> in my own apps, I tended to just use printf a lot.
>> eventually, I made a macro which hooked printf to one of my own
>> functions, and mostly ended up just redirecting its output to a log file
>> and to an in-program console (in addition to stdout).
>>
>> this way, for an app, one can typically see messages in-program, or if
>> there is too much output, go and look over the recorded log-file.
>
> Sounds good.
>
>
>> later, I also added a "dbgprintf" function, which also accepts an
>> integer parameter giving its "debug level" (IIRC, number+flags), but
>> have not ended up widely using it.
>>
>> the idea here would be that dbgprintf would also redirect to the
>> console, and to the log file, but that the debug-level could be used to
>> cull lower-priority messages.
>>
>> IIRC, level 0 was equivalent to a printf, and 1..N (limit not set, maybe
>> 7 or 15) indicate progressively lower-priority messages.
>>
>> a message with a greater debug-level than the present debug-level will
>> simply be discarded (neither shown in console, nor logged).
>
> Sounds like a good idea, although it could be beneficial to employ those
> integer parameters not as a priority indicator but as some sort of channel
> descriptor.  This would offer the programmer with the ability to organize
> log messages based on criteria such as which component generated them.  With
> this, it would be possible to add code to provide the end-user with the
> ability to configure what messages would be displayed, and even how they
> were displayed (i.e., write to stdout/stderr, write to a log file or both).
>

yeah, or both:
the channel is or'ed with the debug-level and any flags.

a difficulty is in making this look nice:
dbgprintf(DBG_LOG_ABORT|mylib_dbgchan|DBG_PRIO_HIGH,
     "something is messed up pretty bad here\n");

well, is not very pretty looking.

I guess in worse cases a person could "#define" a shorthand and use this.


possibly channel numbers/prefixes could be allocated, as 
globally-reserved constants or similar is not likely to scale well.


>
>> an unaddressed issue is "where" the messages are from, which often
>> matters a lot more than some global "debug level" (one tends to care a
>> lot more about messages from the components they are debugging, without
>> having to necessarily see a mountain of messages for whatever else is
>> going on, like other components loading/saving files, messages from the
>> sound mixer, ...).
>
> Probably this could be dealt with if the "integer parameters as channel
> descriptors" approach was followed.  A channel could be registered with the
> logger, where the programmer could also specify a custom prefix to be used
> in those messages.
>

probably...


>
>> the current practice, for the most part, is a bit less organized, and
>> mostly involves manually adding/commenting/uncommenting printf calls.
>
> Unfortunately I also tend to do that.  It's a great big ugly hack which gets
> uglier with time.
>

yeah.

>
> Rui Maciel

[toc] | [prev] | [next] | [standalone]

#804

On 09/ 9/11 01:58 AM, Rui Maciel wrote:
> In some applications it's useful to have, at least in the development
> stages, a way to output diagnostic messages in order to help developers
> check what's happening under the hood.  Yet, although this can be useful and
> in some times even invaluable, there isn't much information on what are
> considered to be good practices regarding the implementation of diagnostic
> messages.

That's because everyone has their favourite!

> With this in mind, what are your views on how to implement diagnostic
> messages?  Are simple outputs to stderr enough or are there better ways to
> handle this?

You typically have three choices in a hosted environment:

1) standard output/error
2) a log file
3) system log

All have their good and bad points and I've seen them all (especially 2 
and 3) used by well regarded applications.

-- 
Ian Collins

[toc] | [prev] | [next] | [standalone]

#809

Ian Collins wrote:

> On 09/ 9/11 01:58 AM, Rui Maciel wrote:
>> In some applications it's useful to have, at least in the development
>> stages, a way to output diagnostic messages in order to help developers
>> check what's happening under the hood.  Yet, although this can be useful
>> and in some times even invaluable, there isn't much information on what
>> are considered to be good practices regarding the implementation of
>> diagnostic messages.
> 
> That's because everyone has their favourite!

Maybe we, as a community, would benefit if there was a standard way (or 
quasi-standard) to approach this problem.  Reinventing the wheel may be a 
good source of fun, but even those who dedicate themselves to reinvent 
wheels may benefit from having a more elaborate starting point.



Rui Maciel

[toc] | [prev] | [next] | [standalone]

#807

On Thu, 08 Sep 2011 14:58:05 +0100, Rui Maciel wrote:

> In some applications it's useful to have, at least in the development
> stages, a way to output diagnostic messages in order to help developers
> check what's happening under the hood.  Yet, although this can be useful
> and in some times even invaluable, there isn't much information on what
> are considered to be good practices regarding the implementation of
> diagnostic messages.
> 
> With this in mind, what are your views on how to implement diagnostic
> messages?  Are simple outputs to stderr enough or are there better ways
> to handle this?

During development of complex code in C, I add (and later remove) 
many ephemeral printf's.  If it seems probable that I might want to
reinstate a printf later, I might comment it out, or preface it by 
if(0), or save a versioned copy.  

On those rare occasions where I'm trying to find the point in the
program where a seg-fault or similar problem is occurring, I add the
letter z at the fronts of several lines, where z invokes a macro 
defined as follows:
   #define z fprintf(stderr,"%s@%3d \n",__FILE__,__LINE__);
Then a few program runs, with z's added or removed as necessary,
will serve to pinpoint the error.  It sometimes is necessary to
add if() conditions before the fprintf(), or add critical variables
to the fprintf() to make sense of what is happening.  

-- 
jiw

[toc] | [prev] | [next] | [standalone]

#810

James Waldby wrote:

> During development of complex code in C, I add (and later remove)
> many ephemeral printf's.  If it seems probable that I might want to
> reinstate a printf later, I might comment it out, or preface it by
> if(0), or save a versioned copy.
> 
> On those rare occasions where I'm trying to find the point in the
> program where a seg-fault or similar problem is occurring, I add the
> letter z at the fronts of several lines, where z invokes a macro
> defined as follows:
> #define z fprintf(stderr,"%s@%3d \n",__FILE__,__LINE__);
> Then a few program runs, with z's added or removed as necessary,
> will serve to pinpoint the error.  It sometimes is necessary to
> add if() conditions before the fprintf(), or add critical variables
> to the fprintf() to make sense of what is happening.

Nice one.  If you don' mind, I'm going to add that to my bag of tricks.  Not 
that I ever write code that segfaults... *cough*


Rui Maciel

[toc] | [prev] | [next] | [standalone]

#812

On 9/11/2011 10:41 AM, Rui Maciel wrote:
> James Waldby wrote:
>
>> During development of complex code in C, I add (and later remove)
>> many ephemeral printf's.  If it seems probable that I might want to
>> reinstate a printf later, I might comment it out, or preface it by
>> if(0), or save a versioned copy.
>>
>> On those rare occasions where I'm trying to find the point in the
>> program where a seg-fault or similar problem is occurring, I add the
>> letter z at the fronts of several lines, where z invokes a macro
>> defined as follows:
>> #define z fprintf(stderr,"%s@%3d \n",__FILE__,__LINE__);
>> Then a few program runs, with z's added or removed as necessary,
>> will serve to pinpoint the error.  It sometimes is necessary to
>> add if() conditions before the fprintf(), or add critical variables
>> to the fprintf() to make sense of what is happening.
>
> Nice one.  If you don' mind, I'm going to add that to my bag of tricks.  Not
> that I ever write code that segfaults... *cough*
>

on the up-side, segfaults are often on the easier end of things to debug.

on the downside, some things are far worse...


more fun is trying to track down where some code, somewhere, writes to a 
bad address, causing problems in potentially an entirely different part 
of the program.

[toc] | [prev] | [next] | [standalone]

#813

On 09/12/11 06:48 AM, BGB wrote:
>
> more fun is trying to track down where some code, somewhere, writes to a
> bad address, causing problems in potentially an entirely different part
> of the program.

That's why we have tools like valgrind and dbx...

-- 
Ian Collins

[toc] | [prev] | [next] | [standalone]

#814

On 9/11/2011 1:47 PM, Ian Collins wrote:
> On 09/12/11 06:48 AM, BGB wrote:
>>
>> more fun is trying to track down where some code, somewhere, writes to a
>> bad address, causing problems in potentially an entirely different part
>> of the program.
>
> That's why we have tools like valgrind and dbx...
>

possibly, except Valgrind doesn't work on Windows AFAIK.

one thing I had done before was to create a number of "tripwires" in the 
GC's heap, and if the MM/GC detects that one of these tripwires has been 
hit (its stored magic value was altered) it uses heuristics to try to 
help make a guess as to which other object was responsible (can help 
narrow down the likely offending region of code).

I had a more capable version in the past, but it had a lot of overhead.
currently I am using a simpler/lighter system, which only really tells 
object types (sadly, unlike its predecessor, it can't tell things like 
the file/line where each object was created, ...).


if one could effectively trap memory assignments, then a nifty strategy 
would be to throw an exception if/whenever one of these tripwires was 
hit. sadly, this would require either a valgrind-like strategy, or OS 
support, namely the ability to trap individual memory writes into a 
given memory region (a very costly way to do this, potentially, would be 
to use double-mapping, partial instruction emulation or single-stepping, 
and exceptions/signals).

other options include OS debugging interfaces, ...

with OS support, one could possible mark the page as Ring-2, so that 
whenever Ring-3 (userspace) code hits it, it signals a page-fault, which 
the kernel then notes is due to a "supervisor" page, passes relevant 
information to a "supervisor" handler (running in Ring-2), which handles 
the access, and then returns to the OS, which returns to Ring-3 
(execution continues). granted, this would still be a bit costly.

alternatively, the OS could implement full-on write-barriers (and a 
write-barrier handler could be responsible to detect and cause an 
exception in the case of a tripwire).

at this point, then if one is running in a debugger, they get control 
back right at the source line which stepped on the tripwire.


or such...

[toc] | [prev] | [next] | [standalone]

#815

On 09/12/11 09:56 AM, BGB wrote:
> On 9/11/2011 1:47 PM, Ian Collins wrote:
>> On 09/12/11 06:48 AM, BGB wrote:
>>>
>>> more fun is trying to track down where some code, somewhere, writes to a
>>> bad address, causing problems in potentially an entirely different part
>>> of the program.
>>
>> That's why we have tools like valgrind and dbx...
>>
>
> possibly, except Valgrind doesn't work on Windows AFAIK.

<snip lots of messy stuff I've never had to do>

I prefer to develop on a platform that gives me the tools I need :)

-- 
Ian Collins

[toc] | [prev] | [standalone]

Back to top | Article view | comp.programming

csiph-web