Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > alt.os.development > #8371 > unrolled thread

System log and format of log entries

Back to article view | Back to alt.os.development

Contents

  System log and format of log entries "James Harris" <james.harris.1@gmail.com> - 2015-07-14 11:42 +0100
    Re: System log and format of log entries "James Harris" <james.harris.1@gmail.com> - 2015-07-14 11:59 +0100
    Re: System log and format of log entries "Rod Pemberton" <boo@fasdfrewar.cdm> - 2015-07-15 03:23 -0400
      Re: System log and format of log entries "Rod Pemberton" <boo@fasdfrewar.cdm> - 2015-07-15 22:46 -0400
    Re: System log and format of log entries "Rod Pemberton" <boo@fasdfrewar.cdm> - 2015-07-15 03:33 -0400
      Re: System log and format of log entries Peter Cheung <mcheung63@gmail.com> - 2015-07-15 00:47 -0700
      Re: System log and format of log entries "Rod Pemberton" <boo@fasdfrewar.cdm> - 2015-07-15 22:42 -0400
        Re: System log and format of log entries "Benjamin David Lunt" <zfysz@fysnet.net> - 2015-07-16 12:18 -0700
          Re: System log and format of log entries "Mike Gonta" <mikegonta@gmail.com> - 2015-07-16 15:50 -0400
            Re: System log and format of log entries "Benjamin David Lunt" <zfysz@fysnet.net> - 2015-07-16 13:24 -0700
              Re: System log and format of log entries "Rod Pemberton" <boo@fasdfrewar.cdm> - 2015-07-17 06:39 -0400
                Re: System log and format of log entries "Benjamin David Lunt" <zfysz@fysnet.net> - 2015-07-17 10:27 -0700
                  Re: System log and format of log entries "Benjamin David Lunt" <zfysz@fysnet.net> - 2015-07-21 16:38 -0700
    Re: System log and format of log entries "James Harris" <james.harris.1@gmail.com> - 2015-07-15 15:15 +0100

#8371 — System log and format of log entries

Anyone have a good approach to laying out entries in an OS system log?

Windows uses some binary internal format (and puts entries in there only 
when it feels like it and when the wind is in the west). Unix uses a 
pure-text format which begins with the date and time.

I am thinking to include two independent sources of time in each log 
entry:

1. The normal date and time in a collatable format such as
      yyyymmdd_hhmmss.sss[timezone]

2. A timestamp from an upcounter such as the TSC or similar

The reason for inclusion of the latter is the weaknesses of the former, 
i.e.

* datetimes may be unknown
* datetimes may be incorrect
* datetimes may only update infrequently such as every second
* datetimes may be changed

All of those weaknesses can make it difficult or impossible usefully to 
determine how long it was between one entry and another. A fine-grained 
upcounter (an unsigned number that increases with each tick of some 
clock) adds the extra detail.

So basically, each log entry is to have

  1. the timestamp
  2. the datetime
  3. the text of the log entry

How does that seem to you? Any suggestions to do something different to 
that?

Of course, there can be the need to alter one or other of the sources of 
those times. Someone may set a new datetime and it is possible to reset 
the upcounter. I am thinking that just one source would be altered at a 
time in the log so that it would always be possible to see and account 
for the changes - but that's a detail.

Another detail is how to represent a log entry but I'll post about that 
separately.

Any thoughts on the principles of having OS log entries include the 
fields as described above?

James

[toc] | [next] | [standalone]

#8372

"James Harris" <james.harris.1@gmail.com> wrote in message 
news:mo2p03$kme$1@dont-email.me...

...

> So basically, each log entry is to have
>
>  1. the timestamp
>  2. the datetime
>  3. the text of the log entry

While the above looks simple there are issues to deal with surrounding 
availability and reliability of the two types of time.

There may not be a suitable source for a timestamp on some machines. 
Even machines which have a suitable upcounter may not make it available 
until the machine has gone some way into its boot process. So there 
needs to be a way to indicate whether the timestamp is present or valid 
etc.

Similarly, a datetime (i.e. yyyy-mm-dd hh:mm:ss etc) may not be 
available but it also has other issues. There may be such a value 
available but it might not be correct or it might not be possible to 
identify how close it is to the real time.

For the datetime I think I need to encode two things: where the value 
came from and how long it has been since it was last corrected. So I 
have come up with a two-hex-digit code to indicate the source and the 
age of last verification:

datetime source:
  0 completely unknown
  2 believed to be fake
  4 unverified local hardware (e.g. CMOS/RTC)
  6 set by a person
  8 derived from external time source hardware
  a derived from apparently public NTP

nominal age of last datetime verification:
  0 longer than a year
  2 less than a year
  4 less than four weeks
  6 less than a day
  8 less than an hour

For example, a two-digit code 68 would mean that the datetime was set by 
a person within the last hour.

The idea is that when a person looks at the log or when a program parses 
the log he or it will be able to see what to make of the datetime value.

I'll stop there and post separately about how to represent the above.

Any thoughts on the approach so far?

James

[toc] | [prev] | [next] | [standalone]

#8380

On Tue, 14 Jul 2015 06:42:34 -0400, James Harris  
<james.harris.1@gmail.com> wrote:

> Anyone have a good approach to laying out entries in an OS system log?
>

No.

All I know is I don't like Linux' syslog file or dmesg output,
and Windows logs were always cryptic.

> Windows uses some binary internal format (and puts entries in there only  
> when it feels like it and when the wind is in the west). Unix uses a  
> pure-text format which begins with the date and time.

Pure text is nice.  It's readable and accessable by humans via a text
editor and text display commands.  It doesn't require an additional
program.  If it's space or tab or comma delimited, you can load it
into Excel etc.  However, the representation isn't as compact which
could require more memory in your OS or kernel, or more flushing of
data to disk.

As long as a text editor or text display commands skipped displaying
control characters, you could use control characters for extra info
only available via a special program.

> I am thinking to include two independent sources of time in each log  
> entry:
>
> 1. The normal date and time in a collatable format such as
>       yyyymmdd_hhmmss.sss[timezone]
>
> 2. A timestamp from an upcounter such as the TSC or similar

You'll need the timezone if you're using daylight saving time.
I.e., times across a time change will be off by an hour.

Personally, I'd probably do year, month day as text, timezone as text,
and the time as an integer emitted as hex, e.g., time_t from time()
or seconds from GMT etc.  The hex and timezone would allow a program
to easily parse the time, should that be needed.  The timezone would
allow for correct times if the file is large enough to maintain times
across a daylight saving time change.  If there is a rapid amount of
data being written to the file, then you'll need hours, minutes, seconds,
fractions of a second, ...  From your comments (snipped), you may
even need a raw processor clock emitted.  It seems like you're expecting
large volumes of data.

> So basically, each log entry is to have
>
>   1. the timestamp
>   2. the datetime
>   3. the text of the log entry
>
> How does that seem to you? Any suggestions to do something different to  
> that?

It's all optional, but I'd consider these:

1) counter as text or raw clock, if rapid volume, for unique message ID
2) date and time, for the admin or user
3) timezone, used with 4)
4) raw time as text, for use by a programs for correct time
5) text message, for the admin or programs

You can break the text messages into components too:

1) program or sub-system (or department)
2) error number
3) error message
...

Oh, and if you're sending it to a pager or twitter, keep the total
line lengths in mind.  E.g., one company I worked for sent 150
character messages, but only managers had 150 character pagers.
Everyone else had 75 or somesuch ...  So, the important part of
the message was never seen by most of their employees, including
their technical staff.  Eventually, about a year and half later,
this problem was discoverd.  They determined that an account had
freaked out at the cost of the pager service and changed the pager
order on his own without notifying anyone in the company.  At first,
they blamed the networking guy who filled out the purchase order ...

> Any thoughts on the principles of having OS log entries include the  
> fields as described above?

What do you mean?  Do you mean without having a C library available?


Rod Pemberton

[toc] | [prev] | [next] | [standalone]

#8389

On Wed, 15 Jul 2015 03:23:45 -0400, Rod Pemberton <boo@fasdfrewar.cdm>  
wrote:

> Oh, and if you're sending it to a pager or twitter, keep the total
> line lengths in mind.  E.g., one company I worked for sent 150
> character messages, but only managers had 150 character pagers.
> Everyone else had 75 or somesuch ...  So, the important part of
> the message was never seen by most of their employees, including
> their technical staff.  Eventually, about a year and half later,
> this problem was discoverd.  They determined that an account had
> freaked out at the cost of the pager service and changed the pager
> order on his own without notifying anyone in the company.  At first,
> they blamed the networking guy who filled out the purchase order ...
>

s/discoverd/discovered
s/account/accountant

RP

-- 
Scientist now say we'll experience a mini-ice in 2030.
So, I guess we need more global warming today ...

[toc] | [prev] | [next] | [standalone]

#8381

On Tue, 14 Jul 2015 12:38:35 -0400, Benjamin David Lunt <zfysz@fysnet.net>  
wrote:

> On a side note, and to go along with the CBI post I posted recently,
> I have now added the code to boot from a USB floppy and then once
> I have loaded my USB drivers, I can then write to the floppy.
> You see, when you boot from the USB floppy, the BIOS emulates it
> as a FDC, therefore you must use INT 13h calls to read and write
> from it.

That's a good thing.  We have access to it.

> However, once you move to pmode, v86 aside, you can no
> longer read or write to the disk in this manner.  Now you must load
> a USB driver to control the USB floppy drive.

That's a bad thing.  I think that's why I can't boot Linux from
an external USB drive.

> If my OS found that there was no floppy disk after boot from the
> USB emulated disk,

Huh?  Why would a "USB emulated disk" use a "floppy disk' at all?

> If my OS found that there was no floppy disk after boot from the
> USB emulated disk, it would, well for a better word, crash.
> "I booted from a floppy disk, where did it go?"

Ok.  You're saying that if after you booted, the OS couldn't find
the boot image, it would crash/halt/hang/panic or somesuch.

Can't you load the boot image into a ramdisk?  Is it only 1.44MB?


Rod Pemberton

[toc] | [prev] | [next] | [standalone]

#8382

I use bochs instrumentation to log every register and 32 bytes of stack for every jump/call. It has timestamp and cpu tick.
Peter.

[toc] | [prev] | [next] | [standalone]

#8388

On Wed, 15 Jul 2015 12:34:22 -0400, Benjamin David Lunt <zfysz@fysnet.net>  
wrote:

> "Rod Pemberton" <boo@fasdfrewar.cdm> wrote in message
> news:op.x1s0lnu1yfako5@localhost...
>> On Tue, 14 Jul 2015 12:38:35 -0400, Benjamin David Lunt  
>> <zfysz@fysnet.net>
>> wrote:

>>> If my OS found that there was no floppy disk after boot from the
>>> USB emulated disk, it would, well for a better word, crash.
>>> "I booted from a floppy disk, where did it go?"
>>
>> Ok.  You're saying that if after you booted, the OS couldn't find
>> the boot image, it would crash/halt/hang/panic or somesuch.
>>
>> Can't you load the boot image into a ramdisk?  Is it only 1.44MB?
>
> The case here is that on boot up, while still in (un)real mode
> and using the BIOS, my loader thinks that it is booting from a
> standard FDC floppy drive using INT 13h services.

Is it safe to use the BIOS from unreal mode?

> As soon as I move to pmode, these services are no longer available,
> so I have two (or more) options.  The first, find all actual FDC
> controllers and add their attached drives to the files system
> module.  Compare any disk found to the one we booted from, and
> you have found your booted from disk.
>
> However, what if this disk was a USB floppy drive?  You haven't
> found the booted from disk.  As soon as my OS tried to access
> the booted from disk to load a driver, write a log file, or
> what ever, it couldn't find the drive and crashed.
>
> This was something that I had never anticipated before, nor
> did I have any reason to fix it until recently.
>
> By now, the USB drivers have been loaded so I enumerate the
> various buses.  Once I have found a floppy drive device, I
> can then compare the inserted disk to the boot signature
> disk I booted from.  Problem solved.
>
> Before, I had yet to add the "oops, you didn't find the
> booted from disk" fix.  I always assumed I could find
> it with the transition from (un)real mode to pmode.
>
> I hadn't added the USB Floppy Disk code yet.
>
> Did that answer your question?

No, not really ...

That explains how you solved your issue, and why you kept
asking us about methods of identifying the boot drive over
the past few years.

Instead of tracking down the boot device by serial number,
I think you could possibly do one of two things to *not* need
to locate and use actual "missing" boot drive:

1) load the boot image into RAM and use RAM as the boot drive
2) don't disable USB floppy emulation for the USB device


Rod Pemberton

-- 
Scientist now say we'll experience a mini-ice in 2030.
So, I guess we need more global warming today ...

[toc] | [prev] | [next] | [standalone]

#8403

"Rod Pemberton" <boo@fasdfrewar.cdm> wrote in message 
news:op.x1uhssosyfako5@localhost...
> On Wed, 15 Jul 2015 12:34:22 -0400, Benjamin David Lunt <zfysz@fysnet.net> 
> wrote:
>
>> The case here is that on boot up, while still in (un)real mode
>> and using the BIOS, my loader thinks that it is booting from a
>> standard FDC floppy drive using INT 13h services.
>
> Is it safe to use the BIOS from unreal mode?

Sure.  The segment registers are still set to values used
by real mode, their limits are just extended.  As long as you
don't use the high 16-bits of a register, when calling BIOS
services, the BIOS knows no difference.

As mentioned earlier, I use a 4meg+ stack located above the
1Meg mark for my loader, which requires unreal mode.  However,
when I need to call a BIOS service, I have to call the wrapper
which will setup a stack within the real mode address space,
use this stack, then copy its contents back to the unreal mode
stack before returning to the caller.

>> Did that answer your question?
>
> No, not really ...
>
> That explains how you solved your issue, and why you kept
> asking us about methods of identifying the boot drive over
> the past few years.
>
> Instead of tracking down the boot device by serial number,
> I think you could possibly do one of two things to *not* need
> to locate and use actual "missing" boot drive:
>
> 1) load the boot image into RAM and use RAM as the boot drive

This is a good idea as long as the boot drive is a floppy or
a fairly small different form of media device.  However, what
if I am booting from an external USB hard drive that is a 160gig
drive?

> 2) don't disable USB floppy emulation for the USB device

If I don't disable the USB floppy emulation, then that particular
USB device is unusable by my OS.

However, once you move from (un)real mode to pmode, the BIOS
is no longer available with disables the USB floppy emulation
anyway.

Also, when enumerating the USB devices, how do you know which
USB device is being "emulated" by the BIOS?  What if I have
an external USB Floppy drive and two thumb drives already
attached?  How do I know which one of these three devices
was booted from and which one the BIOS was emulating as a
FDC or HDC image?

The problem I am running in to, which happens occasionally, is
that once my kernel has loaded, it needs to know which device it
booted from to continue to load drivers from that device.
If more than one bootable image is installed, one of these
bootable images could have different versions or even completely
different Operating Systems all together.

Let's say that I have been working on the USB code to read
a USB thumb drive, and lets say that I call this version
of the USB module version 1.23.45.  Now let's say that I
insert an external USB floppy drive and start working on
the USB module code to use it. Then, I have worked on this code
and got the code to work use an external USB floppy drive
and now have the code to version 5.43.21.

So, let's boot the system from the USB floppy drive.  Once
my kernel has been loaded, what if it found the USB thumb
drive first, found that it is a valid FYSOS image and continued
to load the drivers.  Since the USB module code on the thumb
drive is version 1.23.45, it now cannot find the USB floppy
drive due to the fact that it is an old driver.

Please put aside the fact that all I have to do is update
the thumb drive with the new driver.  I am trying to make
a point here that I need to know which device I booted from
to retrieve the correct and most recent OS files.

For example, let's say that I am building version
1.33.33 and happen to have version 1.22.22 on a
different partition or a complete different device attached
to the system.  What if the kernel happens to find that
1.22.22 version and loads drivers from that device instead
of the booted from 1.33.33 version device.

I know it is a rare case and something that I shouldn't
worry too much about, but why not?  :-)

Ben

-- 
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Forever Young Software
http://www.fysnet.net/index.htm
http://www.fysnet.net/osdesign_book_series.htm
To reply by email, please remove the zzzzzz's

Batteries not included, some Assembly required. 

[toc] | [prev] | [next] | [standalone]

#8404

"Benjamin David Lunt" wrote:

> However, once you move from (un)real mode to pmode, the BIOS
> is no longer available

The BIOS is always available, that's why it's in firmware, it's
just not accessible from pmode.

> with disables the USB floppy emulation anyway.

Are you saying that the mode change alone disables the USB floppy
drive emulation? Changing modes does not disable the USB flash
drive emulation.


Mike Gonta
look and see - many look but few see

http://mikegonta.com

[toc] | [prev] | [next] | [standalone]

#8405

"Mike Gonta" <mikegonta@gmail.com> wrote in message 
news:mo91u2$g2m$1@speranza.aioe.org...
> "Benjamin David Lunt" wrote:
>
>> However, once you move from (un)real mode to pmode, the BIOS
>> is no longer available
>
> The BIOS is always available, that's why it's in firmware, it's
> just not accessible from pmode.
>
>> with disables the USB floppy emulation anyway.
>
> Are you saying that the mode change alone disables the USB floppy
> drive emulation? Changing modes does not disable the USB flash
> drive emulation.

Hi Mike,

From a pmode OS point of view, it is disabled, or for a better
term and as you put it, not accessible.

As you say, the BIOS is still there and as far as it is concerned,
it is still emulating an FDC or HDC.  However, v86 tasks aside,
once you are in pmode, the BIOS and that emulation is no longer
accessible.

That is what I meant.

For the boot code and my loader code, that 1.44meg disk it is
booting from is a FDC type floppy disk drive, and knows no
better.  However, as soon as I move to pmode and the kernel
takes control, there is no guarantee that that floppy disk
is actually on a FDC type drive.  It could be a USB floppy
drive, it could be a USB thumb drive with a 1.44meg image, or
it could be a CDROM with a 1.44meg image.

This is my point.  How do you find out which form of media
you actually booted from?

So far, the only thing I have is to store a few parameters
of the device.  Things like count of sectors, starting sector,
media descriptor, type of file system used, etc. to get
a reasonably close match when scanning all found media devices.

Just as a side note, as soon as I move to pmode and my
kernel takes control, except for the video information and mode
switching, the BIOS might as well be gone.  I don't use, nor do
I rely upon any of the BIOS, either code or data, once my kernel
takes control.

Of course except v86 tasks and video mode switching.  I don't have
a video driver written, since there are so many different types
you would have to write for.

Ben

[toc] | [prev] | [next] | [standalone]

#8410

On Thu, 16 Jul 2015 16:24:59 -0400, Benjamin David Lunt <zfysz@fysnet.net> wrote:

> [Ben's USB boot issues in James log thread]

Ok.

So, in the past few posts, I think you've actually brought up
two issues with booting from USB:

1) boot image not available due to removable media
2) boot image unknown due to inability to map BIOS
emulated device to the USB boot device

> How do you find out which form of media you actually booted from?

1) serial number in boot code closest to booting the OS,
e.g., PBR or VBR
2) serial number for disk plus booted partition id
3) hardware serial number from device itself  (?)
4) user select
5) special file or flag written to booted filesystem
6) don't do that - i.e., warn if multiple bootable devices
...

> So far, the only thing I have is to store a few parameters
> of the device.  Things like count of sectors, starting sector,
> media descriptor, type of file system used, etc. to get
> a reasonably close match when scanning all found media devices.

How close can you get to creating an actual screw up?
e.g., either it's very close or it's definately not going to work.

If you can get really close, but are unable to create an actual
foul up, then it might not be possible to do so ...

> Just as a side note, as soon as I move to pmode and my
> kernel takes control, except for the video information and mode
> switching, the BIOS might as well be gone.  I don't use, nor do
> I rely upon any of the BIOS, either code or data, once my kernel
> takes control.

Me either.  Although, that may need to change, if I ever continue
with it.  See my reply to James in "OS startup goals and steps."


Rod Pemberton

-- 
Scientist now say we'll experience a mini-ice in 2030.
So, I guess we need more global warming today ...

[toc] | [prev] | [next] | [standalone]

#8415

"Rod Pemberton" <boo@fasdfrewar.cdm> wrote in message 
news:op.x1wyj5eqyfako5@localhost...
> On Thu, 16 Jul 2015 16:24:59 -0400, Benjamin David Lunt <zfysz@fysnet.net> 
> wrote:
>
>> [Ben's USB boot issues in James log thread]
>
> Ok.
>
> So, in the past few posts, I think you've actually brought up
> two issues with booting from USB:
>
> 1) boot image not available due to removable media

Yes and no.  Due to removable media due to the fact that
media can be removed.  However, more to the fact that the
drive the media is contained in can be removed.

> 2) boot image unknown due to inability to map BIOS
> emulated device to the USB boot device

Exactly.

>> How do you find out which form of media you actually booted from?
>
> 1) serial number in boot code closest to booting the OS,
> e.g., PBR or VBR
> 2) serial number for disk plus booted partition id
> 3) hardware serial number from device itself  (?)
> 4) user select
> 5) special file or flag written to booted filesystem
> 6) don't do that - i.e., warn if multiple bootable devices

I currently use 1), 2), sometimes 3), and mostly 5).

>> So far, the only thing I have is to store a few parameters
>> of the device.  Things like count of sectors, starting sector,
>> media descriptor, type of file system used, etc. to get
>> a reasonably close match when scanning all found media devices.
>
> How close can you get to creating an actual screw up?
> e.g., either it's very close or it's definately not going to work.
>
> If you can get really close, but are unable to create an actual
> foul up, then it might not be possible to do so ...

No, just the fact that it can happen.  I think I have spent more
time writing these posts about the issue that actually doing anything
about it. :-)  I just got side tracked from the CBI/UFI post,
toward this booting issue.

>> Just as a side note, as soon as I move to pmode and my
>> kernel takes control, except for the video information and mode
>> switching, the BIOS might as well be gone.  I don't use, nor do
>> I rely upon any of the BIOS, either code or data, once my kernel
>> takes control.
>
> Me either.  Although, that may need to change, if I ever continue
> with it.  See my reply to James in "OS startup goals and steps."

Thanks Rod.

Ben

[toc] | [prev] | [next] | [standalone]

#8427

"Rod Pemberton" <boo@fasdfrewar.cdm> wrote in message 
news:op.x14uanxtyfako5@localhost...
> On Fri, 17 Jul 2015 13:27:52 -0400, Benjamin David Lunt <zfysz@fysnet.net> 
> wrote:
>> "Rod Pemberton" <boo@fasdfrewar.cdm> wrote in message
>> news:op.x1wyj5eqyfako5@localhost...
>>> On Thu, 16 Jul 2015 16:24:59 -0400, Benjamin David Lunt 
>>> <zfysz@fysnet.net>
>>> wrote:
>
>>>> How do you find out which form of media you actually booted from?
>>>
>>> 1) serial number in boot code closest to booting the OS,
>>> e.g., PBR or VBR
>>> 2) serial number for disk plus booted partition id
>>> 3) hardware serial number from device itself  (?)
>>> 4) user select
>>> 5) special file or flag written to booted filesystem
>>> 6) don't do that - i.e., warn if multiple bootable devices
>>
>> I currently use 1), 2), sometimes 3), and mostly 5).
>>
>
> EDD3.0 seems to claim it has the answer to your question:

I saw that too, however at the time I was working on this subject,
none of my machines had EDD3.0.  All but one is fairly dated.

<snip EDD3 stuff>

I also used a function, don't remember which, to get the
Port I/O address of the booted from drive.  This helped,
but found out that not all BIOS' supported that function.

The solution that I have come up with, is one of somewhat
a simple solution that should work most if not all of the
time as long as boot sector signatures are used.
(MS puts a signature value just before the partition table)

My boot sector code, once loaded to 0x07C00, should not modify
itself within the 0x07C00 to 0x07DFF range.  Then once I have
enumerated all of the media devices, I simply have to compare
the first sector of the volume (partition/disk/etc.) with the
512 bytes at 0x07C00 and the one that matches should be the
one that I booted from.

At the moment, the only problem I have is that my USB enumeration
code is *after* the check for the booted from drive.  Therefore,
right now, if the booted from drive is a USB device, my code
doesn't find it.

This should be a simple fix, though the thread that finishes the
boot, finishes before the thread that enumerates the USB bus. :-)

The thread that finishes the boot expects to see the booted from
drive when it is finished, yet the USB thread has not completed
yet, if there are numerous USB devices attached.

I will have to decide what I want to do to remedy this.

BTW, my OS is a multi-threading kernel (one task, multiple threads),
with a multi-tasking user environment, each user task is a separate
task capable of multiple threads, with a separate virtual memory
range, IDT, LDT, etc. (I mention this just to clear up the paragraph
above).

Thanks for the information though, I may come back to the EDD3
stuff again.

Ben

-- 
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Forever Young Software
http://www.fysnet.net/index.htm
http://www.fysnet.net/osdesign_book_series.htm
To reply by email, please remove the zzzzzz's

Batteries not included, some Assembly required. 

[toc] | [prev] | [next] | [standalone]

#8384

"Benjamin David Lunt" <zfysz@fysnet.net> wrote in message 
news:mo3eab$tca$1@speranza.aioe.org...
> "James Harris" <james.harris.1@gmail.com> wrote in message 
> news:mo2p03$kme$1@dont-email.me...

>> Anyone have a good approach to laying out entries in an OS system 
>> log?

...

>  tick: module: function: information
>
> Here is an example line of my SATA(AHCI) module output.
>
> 010187780423: SATA: sata_write_sectors:
>    lba = 1012097, count = 2, buf address = 0x031AE800

Thanks, Ben. That's useful to see. I have compared that with other 
sources: Peter's post and Rod's post here, Unix logs, Windows logs, and 
the following RFC about syslog.

  https://www.ietf.org/rfc/rfc3164.txt

As a consequence I have come up with the following contents for each log 
entry.

  64-bit   upcounter
  four octets   severity, classification, flags, and indicators
  text   datetime
  text   path (how the message got here, blank for local machine)
  text   source (program name, instance, sequence number)
  text   name of the userid under which the program was running
  text   entry text

It is a sort of superset as they all have things that I think are 
sensible to include but I have merged some elements in an effort to 
reduce the number of fields and and also added a few small elements. 
While each entry is quite wide not all fields would need to be printed 
when rendering a log to a display such as the screen, but they would be 
stored for examination.

Explanations of the fields are below for anyone who is interested.

* Upcounter. An unsigned upwards counter. Increasing at a constant 
real-time rate if possible such as a modern TSC or HPET, but otherwise 
increasing at a non-constant rate, such as an early TSC.

* Severity. One octet. Higher nibble value set to 1 more than the Unix 
"level" (e.g. Unix level ERR for error conditions has value 4 meaning 
the severity upper octet would be 5) partly so that there is no leading 
zero. Lower nibble a way to qualify the severity in a more fine-grained 
way. Default for the lower nibble 0x8 bit it can be higher or lower.

* Classification. One octet. Used for grouping reports by type of 
sender. No values assigned yet. Subsumes the Windows "Category" and the 
Unix "facility" indication.

* Flags. One octet. Various flags such as whether the upcounter value is 
fake or its source has changed.

* Indicators. One octet. Indicates the datetime's source and how long 
since it was last verified.

* Datetime. A human-readable but also program-parseable representation 
of the current date and time, with the timezone.

* Path. Blank if the log entry is from the local machine. Else, this 
field would be available to hold the path the log report took to get 
here. The origin machine adds its identification first. Each subsequent 
relaying machine can append its id. So, if present, the first part is 
always the originating machine.

* Source. The name of the program which sent the log entry and optional 
instance id and sequence number. The program name can have codes 
appended to indicate the version of the program and the type of the 
report - something that may be relevant only to that build of the 
program.

* Username. The textual name for the userid under which the program was 
running.

* Entry text. The log message (which could include the name of the 
function which issued the message and the sending machine's datetime etc 
as well as any specific message text).

I may change it but that's what I have so far.

James

[toc] | [prev] | [standalone]

Back to top | Article view | alt.os.development

csiph-web