Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.arch > #118802 > unrolled thread

Re: Interrupts in OoO (was: Microarchitectural support for counting)

Back to article view | Back to comp.arch

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: Interrupts in OoO (was: Microarchitectural support for counting) quadibloc@invalid.com (John Savard) - 2026-10-23 23:57 +0000
    Re: Interrupts in OoO Robert Finch <robfi680@gmail.com> - 2026-10-23 21:12 -0400
    Re: Interrupts in OoO (was: Microarchitectural support for counting) MitchAlsup <user5857@newsgrouper.org.invalid> - 2026-10-24 19:20 +0000
      Re: Interrupts in OoO (was: Microarchitectural support for counting) scott@slp53.sl.home (Scott Lurndal) - 2026-10-25 15:53 +0000
        Re: Interrupts in OoO EricP <ThatWouldBeTelling@thevillage.com> - 2026-10-26 11:06 -0400
          Re: Interrupts in OoO Robert Finch <robfi680@gmail.com> - 2026-10-26 13:17 -0400
            Re: Interrupts in OoO EricP <ThatWouldBeTelling@thevillage.com> - 2026-10-26 17:03 -0400
              Re: Interrupts in OoO "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> - 2026-10-26 14:14 -0700
                Re: Interrupts in OoO EricP <ThatWouldBeTelling@thevillage.com> - 2026-10-28 14:39 -0400
                  Re: Interrupts in OoO "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> - 2026-10-28 15:10 -0700
                    Re: Interrupts in OoO EricP <ThatWouldBeTelling@thevillage.com> - 2026-10-29 14:50 -0400
              Re: Interrupts in OoO MitchAlsup <user5857@newsgrouper.org.invalid> - 2026-10-27 16:48 +0000
          Re: Interrupts in OoO MitchAlsup <user5857@newsgrouper.org.invalid> - 2026-10-27 16:35 +0000
            Re: Interrupts in OoO scott@slp53.sl.home (Scott Lurndal) - 2026-10-27 18:33 +0000
              Re: Interrupts in OoO MitchAlsup <user5857@newsgrouper.org.invalid> - 2026-10-27 21:07 +0000
                Re: Interrupts in OoO scott@slp53.sl.home (Scott Lurndal) - 2026-10-27 21:17 +0000
                  Re: Interrupts in OoO MitchAlsup <user5857@newsgrouper.org.invalid> - 2026-10-28 15:35 +0000
                    Re: Interrupts in OoO George Neuner <gneuner2@comcast.net> - 2026-10-28 14:34 -0400
                      Re: Interrupts in OoO scott@slp53.sl.home (Scott Lurndal) - 2026-10-28 20:16 +0000
                      Re: Interrupts in OoO MitchAlsup <user5857@newsgrouper.org.invalid> - 2026-10-29 02:17 +0000
                        Re: Interrupts in OoO "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> - 2026-10-28 21:21 -0700
                        Re: Interrupts in OoO George Neuner <gneuner2@comcast.net> - 2026-10-29 21:49 -0400
                Re: Interrupts in OoO EricP <ThatWouldBeTelling@thevillage.com> - 2026-10-28 14:57 -0400
                  Re: Interrupts in OoO MitchAlsup <user5857@newsgrouper.org.invalid> - 2026-10-29 02:27 +0000
            Re: Interrupts in OoO EricP <ThatWouldBeTelling@thevillage.com> - 2026-10-28 14:19 -0400
              Re: Interrupts in OoO MitchAlsup <user5857@newsgrouper.org.invalid> - 2026-10-28 19:02 +0000
    Re: Interrupts in OoO (was: Microarchitectural support for counting) anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2026-10-26 17:45 +0000
      Re: Interrupts in OoO (was: Microarchitectural support for counting) MitchAlsup <user5857@newsgrouper.org.invalid> - 2026-10-27 16:39 +0000

Page 1 of 2  [1] 2  Next page →

#118802 — Re: Interrupts in OoO (was: Microarchitectural support for counting)

On Sat, 05 Oct 2024 17:57:12 GMT, anton@mips.complang.tuwien.ac.at
(Anton Ertl) wrote:

>Ok, it's more efficient to insert an interrupt call into the
>instruction stream in the decoder: all the in-flight instructions will
>be completed instead of going to waste.  However, the interrupt will
>usually be serviced later, and, as you point out, if the instruction
>stream is redirected for some other reason, you may have to replay the
>interrupt.

A _trap_ can be a problem on an out-of-order machine, since traps
result from exceptions, and so they have to be tied to the instruction
stream.
An interrupt, on the other hand, just needs to get serviced. The
process being interrupted just needs not to be disturbed by it.
That leaves several alternatives.
You could somehow make the machine act as if it has a second core just
to serve interrupts. In fact, some early architectures anticipated SMT
by having a second set of registers for privileged operation.
Or, since interrupt routines _are_ at kernel level... when there's an
interrupt, save the whole internal machine state, including the entire
pipeline. After all, since interrupt routines are privileged, they can
use a special return instruction that sets up the machine's internals
instead of just restoring registers and branching somewhere.
So, while interrupts _can_ be an issue for an OoO machine, they don't
really have to be.

John Savard

[toc] | [next] | [standalone]

#118808 — Re: Interrupts in OoO

On 2026-10-23 7:57 p.m., John Savard wrote:
> On Sat, 05 Oct 2024 17:57:12 GMT, anton@mips.complang.tuwien.ac.at
> (Anton Ertl) wrote:
> 
>> Ok, it's more efficient to insert an interrupt call into the
>> instruction stream in the decoder: all the in-flight instructions will
>> be completed instead of going to waste.  However, the interrupt will
>> usually be serviced later, and, as you point out, if the instruction
>> stream is redirected for some other reason, you may have to replay the
>> interrupt.
> 
For Q+ one build option is similar to this. A poll-for-interrupt 
instruction is periodically inserted into the instruction stream 
(approximately every 100 instructions) when micro-ops are translated and 
queued (could be the beginning of the decode stage). The poll operation 
is a special form of branch and then handled by the branch logic.

The default Q+ build just has a hardware-interrupt flag on the pipeline 
registers, set at the fetch stage. When the pipeline reaches commit the 
flag is checked and interrupt processing takes place if the flag was 
set. Because it is a separate flag the processor knows to honor it even 
if the instructions are no longer valid.

> A _trap_ can be a problem on an out-of-order machine, since traps
> result from exceptions, and so they have to be tied to the instruction
> stream.
> An interrupt, on the other hand, just needs to get serviced. The
> process being interrupted just needs not to be disturbed by it.
> That leaves several alternatives.
> You could somehow make the machine act as if it has a second core just
> to serve interrupts. In fact, some early architectures anticipated SMT
> by having a second set of registers for privileged operation.
> Or, since interrupt routines _are_ at kernel level... when there's an
> interrupt, save the whole internal machine state, including the entire
> pipeline. After all, since interrupt routines are privileged, they can
> use a special return instruction that sets up the machine's internals
> instead of just restoring registers and branching somewhere.
> So, while interrupts _can_ be an issue for an OoO machine, they don't
> really have to be.
> 
> John Savard

[toc] | [prev] | [next] | [standalone]

#118810

quadibloc@invalid.com (John Savard) posted:

> On Sat, 05 Oct 2024 17:57:12 GMT, anton@mips.complang.tuwien.ac.at
> (Anton Ertl) wrote:
> 
> >Ok, it's more efficient to insert an interrupt call into the
> >instruction stream in the decoder: all the in-flight instructions will
> >be completed instead of going to waste.  However, the interrupt will
> >usually be serviced later, and, as you point out, if the instruction
> >stream is redirected for some other reason, you may have to replay the
> >interrupt.
> 
> A _trap_ can be a problem on an out-of-order machine, since traps
> result from exceptions, and so they have to be tied to the instruction
> stream.

We have {Traps, Exceptions, Interrupts}. 

A trap is synchronous to the executing code and we want IP pointing at the
subsequent instruction. A trap may send a small amount of data to the service
at the other end. Not all registers have to be preserved. Registers R0..R15
are considered temporaries at the caller->callee boundary. Those registers
either carry a result or are cleared <HW> at the callee->caller boundary::
in My 66000 SBI.

An Exception is synchronous to the executing code and we want IP pointing
at the offending instruction. What kind of exception is the message to the
service routine. All Thread-State has to be preserved. A Thread should
be able to handle its own exceptions should it communicate same with
Guest OS.

An interrupt is asynchronous to the executing code and we can allow IP
to point at any instruction so long as a) all instructions prior to
*IP appear to have executed, and b) no instructions after *IP appear to
have executed. All Thread-State has to be preserved.

> An interrupt, on the other hand, just needs to get serviced. The
> process being interrupted just needs not to be disturbed by it.

Except via the time dimension...

> That leaves several alternatives.
> You could somehow make the machine act as if it has a second core just
> to serve interrupts. In fact, some early architectures anticipated SMT
> by having a second set of registers for privileged operation.

> Or, since interrupt routines _are_ at kernel level... when there's an
> interrupt, save the whole internal machine state, including the entire
> pipeline. After all, since interrupt routines are privileged, they can
> use a special return instruction that sets up the machine's internals
> instead of just restoring registers and branching somewhere.

One now has to be careful, here. An interrupt can be taken in {HyperVisor,
Host OS, Guest OS, or even Application} although most architectures disallow
the Application to field its own interrupts or exceptions and most definitely
its own SYSCALLs.

Guest OS may receive a Page-Fault from Application and Guest OS has
arranged that its ISR pages are "not paged" and thus it is not setup
to deal with Guest OS Page-Faults in its own ISRs. Host OS can page 
Guest OS pages out at its whim, and take Guest OS Page-Faults without 
Guest OS ever knowing about those paging activities. So, virtualization
has added monkey-wrenches into the verbology ...

> So, while interrupts _can_ be an issue for an OoO machine, they don't
> really have to be.

Interrupts are easier than exceptions, mostly because one does not have to
back up the execution window.
> 
> John Savard

[toc] | [prev] | [next] | [standalone]

#118821

MitchAlsup <user5857@newsgrouper.org.invalid> writes:
>quadibloc@invalid.com (John Savard) posted:
>
>> On Sat, 05 Oct 2024 17:57:12 GMT, anton@mips.complang.tuwien.ac.at
>> (Anton Ertl) wrote:
>> 
>> >Ok, it's more efficient to insert an interrupt call into the
>> >instruction stream in the decoder: all the in-flight instructions will
>> >be completed instead of going to waste.  However, the interrupt will
>> >usually be serviced later, and, as you point out, if the instruction
>> >stream is redirected for some other reason, you may have to replay the
>> >interrupt.
>> 
>> A _trap_ can be a problem on an out-of-order machine, since traps
>> result from exceptions, and so they have to be tied to the instruction
>> stream.
>
>We have {Traps, Exceptions, Interrupts}. 
>
>A trap is synchronous to the executing code and we want IP pointing at the
>subsequent instruction. A trap may send a small amount of data to the service
>at the other end. Not all registers have to be preserved. Registers R0..R15
>are considered temporaries at the caller->callee boundary. Those registers
>either carry a result or are cleared <HW> at the callee->caller boundary::
>in My 66000 SBI.
>
>An Exception is synchronous to the executing code and we want IP pointing
>at the offending instruction. What kind of exception is the message to the
>service routine. All Thread-State has to be preserved. A Thread should
>be able to handle its own exceptions should it communicate same with
>Guest OS.

Exceptions are not alway precise.  For example, an exception
on a posted store (that's spent time in a store buffer, for
example), may have an exception asynchronously.  

[toc] | [prev] | [next] | [standalone]

#118839 — Re: Interrupts in OoO

On 2026-Oct-25 11:53, Scott Lurndal wrote:
> MitchAlsup <user5857@newsgrouper.org.invalid> writes:
>> quadibloc@invalid.com (John Savard) posted:
>>
>>> On Sat, 05 Oct 2024 17:57:12 GMT, anton@mips.complang.tuwien.ac.at
>>> (Anton Ertl) wrote:
>>>
>>>> Ok, it's more efficient to insert an interrupt call into the
>>>> instruction stream in the decoder: all the in-flight instructions will
>>>> be completed instead of going to waste.  However, the interrupt will
>>>> usually be serviced later, and, as you point out, if the instruction
>>>> stream is redirected for some other reason, you may have to replay the
>>>> interrupt.
>>>
>>> A _trap_ can be a problem on an out-of-order machine, since traps
>>> result from exceptions, and so they have to be tied to the instruction
>>> stream.
>>
>> We have {Traps, Exceptions, Interrupts}.
>>
>> A trap is synchronous to the executing code and we want IP pointing at the
>> subsequent instruction. A trap may send a small amount of data to the service
>> at the other end. Not all registers have to be preserved. Registers R0..R15
>> are considered temporaries at the caller->callee boundary. Those registers
>> either carry a result or are cleared <HW> at the callee->caller boundary::
>> in My 66000 SBI.
>>
>> An Exception is synchronous to the executing code and we want IP pointing
>> at the offending instruction. What kind of exception is the message to the
>> service routine. All Thread-State has to be preserved. A Thread should
>> be able to handle its own exceptions should it communicate same with
>> Guest OS.
> 
> Exceptions are not alway precise.  For example, an exception
> on a posted store (that's spent time in a store buffer, for
> example), may have an exception asynchronously.

This is a terminology problem that comes from processors putting
all the "alert" notifications through the same delivery mechanism.
People often use various the terms as interchangeable when really
they are not.

In my terminology I have Exceptions, Interrupts, and Errors.
I take all the well defined stuff and put it in the Exception and
Interrupt buckets, and take all the weird stuff and call it Errors.

Exceptions are defined by the ISA, are precise, and synchronous
to the excepting instruction. There are two kinds, Faults and Traps.
Most exceptions notifications cannot be deferred or "masked".
Floating point exceptions are potentially maskable however if
they are then any auxiliary information about the precise source of
the FP exceptions will be synchronous with instruction that unmasks
a pending floating point exception.

Interrupts are defined as precise and asynchronous.
Their number and priority mechanisms is defined as model specific.
Interrupts are potentially maskable and may be deferred.
The only ISA defined part of interrupts are the master interrupt
mask control instructions INTEN enable and INTDI disable.

Exceptions and Interrupts are defined as precise and restartable.

The mechanism for delivering Exception and Interrupt notifications
is defined as model specific. In practice this just means that
operating systems should modularize this layer of their software
(sometimes called a HAL or Hardware Abstraction Layer).

Errors are model and error specific, may be synchronous or asynchronous,
may be precise (Faults or Traps) or imprecise (Aborts).
Errors may or may not be deferred (maskable).
Errors may not be restartable.
The mechanism for delivering Error notifications is defined as
model and error specific.

[toc] | [prev] | [next] | [standalone]

#118840 — Re: Interrupts in OoO

On 2026-10-26 11:06 a.m., EricP wrote:
> On 2026-Oct-25 11:53, Scott Lurndal wrote:
>> MitchAlsup <user5857@newsgrouper.org.invalid> writes:
>>> quadibloc@invalid.com (John Savard) posted:
>>>
>>>> On Sat, 05 Oct 2024 17:57:12 GMT, anton@mips.complang.tuwien.ac.at
>>>> (Anton Ertl) wrote:
>>>>
>>>>> Ok, it's more efficient to insert an interrupt call into the
>>>>> instruction stream in the decoder: all the in-flight instructions will
>>>>> be completed instead of going to waste.  However, the interrupt will
>>>>> usually be serviced later, and, as you point out, if the instruction
>>>>> stream is redirected for some other reason, you may have to replay the
>>>>> interrupt.
>>>>
>>>> A _trap_ can be a problem on an out-of-order machine, since traps
>>>> result from exceptions, and so they have to be tied to the instruction
>>>> stream.
>>>
>>> We have {Traps, Exceptions, Interrupts}.
>>>
>>> A trap is synchronous to the executing code and we want IP pointing 
>>> at the
>>> subsequent instruction. A trap may send a small amount of data to the 
>>> service
>>> at the other end. Not all registers have to be preserved. Registers 
>>> R0..R15
>>> are considered temporaries at the caller->callee boundary. Those 
>>> registers
>>> either carry a result or are cleared <HW> at the callee->caller 
>>> boundary::
>>> in My 66000 SBI.
>>>
>>> An Exception is synchronous to the executing code and we want IP 
>>> pointing
>>> at the offending instruction. What kind of exception is the message 
>>> to the
>>> service routine. All Thread-State has to be preserved. A Thread should
>>> be able to handle its own exceptions should it communicate same with
>>> Guest OS.
>>
>> Exceptions are not alway precise.  For example, an exception
>> on a posted store (that's spent time in a store buffer, for
>> example), may have an exception asynchronously.
> 
> This is a terminology problem that comes from processors putting
> all the "alert" notifications through the same delivery mechanism.
> People often use various the terms as interchangeable when really
> they are not.
> 
> In my terminology I have Exceptions, Interrupts, and Errors.
> I take all the well defined stuff and put it in the Exception and
> Interrupt buckets, and take all the weird stuff and call it Errors.
> 
> Exceptions are defined by the ISA, are precise, and synchronous
> to the excepting instruction. There are two kinds, Faults and Traps.
> Most exceptions notifications cannot be deferred or "masked".
> Floating point exceptions are potentially maskable however if
> they are then any auxiliary information about the precise source of
> the FP exceptions will be synchronous with instruction that unmasks
> a pending floating point exception.
> 
> Interrupts are defined as precise and asynchronous.
> Their number and priority mechanisms is defined as model specific.
> Interrupts are potentially maskable and may be deferred.
> The only ISA defined part of interrupts are the master interrupt
> mask control instructions INTEN enable and INTDI disable.
> 
> Exceptions and Interrupts are defined as precise and restartable.
> 
> The mechanism for delivering Exception and Interrupt notifications
> is defined as model specific. In practice this just means that
> operating systems should modularize this layer of their software
> (sometimes called a HAL or Hardware Abstraction Layer).
> 
> Errors are model and error specific, may be synchronous or asynchronous,
> may be precise (Faults or Traps) or imprecise (Aborts).
> Errors may or may not be deferred (maskable).
> Errors may not be restartable.
> The mechanism for delivering Error notifications is defined as
> model and error specific.
> 
I have thought that interrupts, traps and others are subsets of 
exceptions, which I think is the way m68k docs describes them. So, I 
would have errors as a subset of exceptions.

Interrupts, traps, errors and exceptions are all often handled by the 
same table which different vendors seem to give different names to.

I think interrupts may only be precise to a point. An interrupt is 
asynchronous, so how precise is it? In q+ the hardware may defer it to a 
short time later.

[toc] | [prev] | [next] | [standalone]

#118850 — Re: Interrupts in OoO

On 2026-Oct-26 13:17, Robert Finch wrote:
> On 2026-10-26 11:06 a.m., EricP wrote:
>> On 2026-Oct-25 11:53, Scott Lurndal wrote:
>>> MitchAlsup <user5857@newsgrouper.org.invalid> writes:
>>>> quadibloc@invalid.com (John Savard) posted:
>>>>
>>>>> On Sat, 05 Oct 2024 17:57:12 GMT, anton@mips.complang.tuwien.ac.at
>>>>> (Anton Ertl) wrote:
>>>>>
>>>>>> Ok, it's more efficient to insert an interrupt call into the
>>>>>> instruction stream in the decoder: all the in-flight instructions will
>>>>>> be completed instead of going to waste.  However, the interrupt will
>>>>>> usually be serviced later, and, as you point out, if the instruction
>>>>>> stream is redirected for some other reason, you may have to replay the
>>>>>> interrupt.
>>>>>
>>>>> A _trap_ can be a problem on an out-of-order machine, since traps
>>>>> result from exceptions, and so they have to be tied to the instruction
>>>>> stream.
>>>>
>>>> We have {Traps, Exceptions, Interrupts}.
>>>>
>>>> A trap is synchronous to the executing code and we want IP pointing at the
>>>> subsequent instruction. A trap may send a small amount of data to the service
>>>> at the other end. Not all registers have to be preserved. Registers R0..R15
>>>> are considered temporaries at the caller->callee boundary. Those registers
>>>> either carry a result or are cleared <HW> at the callee->caller boundary::
>>>> in My 66000 SBI.
>>>>
>>>> An Exception is synchronous to the executing code and we want IP pointing
>>>> at the offending instruction. What kind of exception is the message to the
>>>> service routine. All Thread-State has to be preserved. A Thread should
>>>> be able to handle its own exceptions should it communicate same with
>>>> Guest OS.
>>>
>>> Exceptions are not alway precise.  For example, an exception
>>> on a posted store (that's spent time in a store buffer, for
>>> example), may have an exception asynchronously.
>>
>> This is a terminology problem that comes from processors putting
>> all the "alert" notifications through the same delivery mechanism.
>> People often use various the terms as interchangeable when really
>> they are not.
>>
>> In my terminology I have Exceptions, Interrupts, and Errors.
>> I take all the well defined stuff and put it in the Exception and
>> Interrupt buckets, and take all the weird stuff and call it Errors.
>>
>> Exceptions are defined by the ISA, are precise, and synchronous
>> to the excepting instruction. There are two kinds, Faults and Traps.
>> Most exceptions notifications cannot be deferred or "masked".
>> Floating point exceptions are potentially maskable however if
>> they are then any auxiliary information about the precise source of
>> the FP exceptions will be synchronous with instruction that unmasks
>> a pending floating point exception.
>>
>> Interrupts are defined as precise and asynchronous.
>> Their number and priority mechanisms is defined as model specific.
>> Interrupts are potentially maskable and may be deferred.
>> The only ISA defined part of interrupts are the master interrupt
>> mask control instructions INTEN enable and INTDI disable.
>>
>> Exceptions and Interrupts are defined as precise and restartable.
>>
>> The mechanism for delivering Exception and Interrupt notifications
>> is defined as model specific. In practice this just means that
>> operating systems should modularize this layer of their software
>> (sometimes called a HAL or Hardware Abstraction Layer).
>>
>> Errors are model and error specific, may be synchronous or asynchronous,
>> may be precise (Faults or Traps) or imprecise (Aborts).
>> Errors may or may not be deferred (maskable).
>> Errors may not be restartable.
>> The mechanism for delivering Error notifications is defined as
>> model and error specific.
>>
> I have thought that interrupts, traps and others are subsets of exceptions, which I think is the way m68k docs describes them. So, I would have errors as a subset of exceptions.
> 
> Interrupts, traps, errors and exceptions are all often handled by the same table which different vendors seem to give different names to.
> 
> I think interrupts may only be precise to a point. An interrupt is asynchronous, so how precise is it? In q+ the hardware may defer it to a short time later.

Precise means the processor state corresponds with a sequential model
of program execution where one instruction completes before the next begin.

Synchronous means that the saved state corresponds with the particular
instruction that triggered the state or the instruction after it.
For a Fault the state correspond to the triggering instruction.
For a Trap the state corresponds to the instruction after the
triggering instruction.
All exceptions, whether faults or traps, are synchronous.

Asynchronous means that the state does not correspond with any
particular instruction. All Interrupts are asynchronous.

For an asynchronous Interrupt or synchronous Exception to be
restartable it must be precise.

Each of Exceptions and Interrupts have well defined behaviour.
Exceptions only have one delivery mechanism whereas Interrupts might
have a variety of Interrupt controllers each with its own behavior.

They can also be quite frequent and I didn't want to burden their
dispatching mechanisms with the extra overhead of also dealing with
potential errors that might infrequently occur.
In both cases I would use a variation on branch-and-link to a hardware
state save and generated RIP address so there are no memory references
during the internal state change. This means no double- or triple-faults
and potentially a 1 clock state change.

The problem with errors is that their behaviour is all over the place.
A store operation to memory or device marked as write-through would be
synchronous with the ST instruction and any errors detected could be
associated back to the ST instruction.
A store operation to memory marked as write-back is asynchronous and
can be performed long after the ST instruction has retired and
been removed from the core.

An ECC memory error can be detected and maybe corrected by the
error scavenger sweep and not associated with any instruction at all.

Some devices change state when read. For example a UART receive
buffer can be FIFO queue and reading it consumes the first entry.
If there was a bus parity error after a successful read of the
FIFO then one may or may not want to retry the operation.

By putting infrequent Errors off in their own bucket they can be
dealt with on a case by case basis without burdening the frequent
occurance delivery mechanisms for Exceptions and Interrupts.

[toc] | [prev] | [next] | [standalone]

#118851 — Re: Interrupts in OoO

On 10/26/2026 2:03 PM, EricP wrote:
> On 2026-Oct-26 13:17, Robert Finch wrote:
>> On 2026-10-26 11:06 a.m., EricP wrote:
>>> On 2026-Oct-25 11:53, Scott Lurndal wrote:
>>>> MitchAlsup <user5857@newsgrouper.org.invalid> writes:
>>>>> quadibloc@invalid.com (John Savard) posted:
>>>>>
>>>>>> On Sat, 05 Oct 2024 17:57:12 GMT, anton@mips.complang.tuwien.ac.at
>>>>>> (Anton Ertl) wrote:
>>>>>>
>>>>>>> Ok, it's more efficient to insert an interrupt call into the
>>>>>>> instruction stream in the decoder: all the in-flight instructions 
>>>>>>> will
>>>>>>> be completed instead of going to waste.  However, the interrupt will
>>>>>>> usually be serviced later, and, as you point out, if the instruction
>>>>>>> stream is redirected for some other reason, you may have to 
>>>>>>> replay the
>>>>>>> interrupt.
>>>>>>
>>>>>> A _trap_ can be a problem on an out-of-order machine, since traps
>>>>>> result from exceptions, and so they have to be tied to the 
>>>>>> instruction
>>>>>> stream.
>>>>>
>>>>> We have {Traps, Exceptions, Interrupts}.
>>>>>
>>>>> A trap is synchronous to the executing code and we want IP pointing 
>>>>> at the
>>>>> subsequent instruction. A trap may send a small amount of data to 
>>>>> the service
>>>>> at the other end. Not all registers have to be preserved. Registers 
>>>>> R0..R15
>>>>> are considered temporaries at the caller->callee boundary. Those 
>>>>> registers
>>>>> either carry a result or are cleared <HW> at the callee->caller 
>>>>> boundary::
>>>>> in My 66000 SBI.
>>>>>
>>>>> An Exception is synchronous to the executing code and we want IP 
>>>>> pointing
>>>>> at the offending instruction. What kind of exception is the message 
>>>>> to the
>>>>> service routine. All Thread-State has to be preserved. A Thread should
>>>>> be able to handle its own exceptions should it communicate same with
>>>>> Guest OS.
>>>>
>>>> Exceptions are not alway precise.  For example, an exception
>>>> on a posted store (that's spent time in a store buffer, for
>>>> example), may have an exception asynchronously.
>>>
>>> This is a terminology problem that comes from processors putting
>>> all the "alert" notifications through the same delivery mechanism.
>>> People often use various the terms as interchangeable when really
>>> they are not.
>>>
>>> In my terminology I have Exceptions, Interrupts, and Errors.
>>> I take all the well defined stuff and put it in the Exception and
>>> Interrupt buckets, and take all the weird stuff and call it Errors.
>>>
>>> Exceptions are defined by the ISA, are precise, and synchronous
>>> to the excepting instruction. There are two kinds, Faults and Traps.
>>> Most exceptions notifications cannot be deferred or "masked".
>>> Floating point exceptions are potentially maskable however if
>>> they are then any auxiliary information about the precise source of
>>> the FP exceptions will be synchronous with instruction that unmasks
>>> a pending floating point exception.
>>>
>>> Interrupts are defined as precise and asynchronous.
>>> Their number and priority mechanisms is defined as model specific.
>>> Interrupts are potentially maskable and may be deferred.
>>> The only ISA defined part of interrupts are the master interrupt
>>> mask control instructions INTEN enable and INTDI disable.
>>>
>>> Exceptions and Interrupts are defined as precise and restartable.
>>>
>>> The mechanism for delivering Exception and Interrupt notifications
>>> is defined as model specific. In practice this just means that
>>> operating systems should modularize this layer of their software
>>> (sometimes called a HAL or Hardware Abstraction Layer).
>>>
>>> Errors are model and error specific, may be synchronous or asynchronous,
>>> may be precise (Faults or Traps) or imprecise (Aborts).
>>> Errors may or may not be deferred (maskable).
>>> Errors may not be restartable.
>>> The mechanism for delivering Error notifications is defined as
>>> model and error specific.
>>>
>> I have thought that interrupts, traps and others are subsets of 
>> exceptions, which I think is the way m68k docs describes them. So, I 
>> would have errors as a subset of exceptions.
>>
>> Interrupts, traps, errors and exceptions are all often handled by the 
>> same table which different vendors seem to give different names to.
>>
>> I think interrupts may only be precise to a point. An interrupt is 
>> asynchronous, so how precise is it? In q+ the hardware may defer it to 
>> a short time later.
> 
> Precise means the processor state corresponds with a sequential model
> of program execution where one instruction completes before the next begin.
> 
> Synchronous means that the saved state corresponds with the particular
> instruction that triggered the state or the instruction after it.
> For a Fault the state correspond to the triggering instruction.
> For a Trap the state corresponds to the instruction after the
> triggering instruction.
> All exceptions, whether faults or traps, are synchronous.
> 
> Asynchronous means that the state does not correspond with any
> particular instruction. All Interrupts are asynchronous.
> 
> For an asynchronous Interrupt or synchronous Exception to be
> restartable it must be precise.
> 
> Each of Exceptions and Interrupts have well defined behaviour.
> Exceptions only have one delivery mechanism whereas Interrupts might
> have a variety of Interrupt controllers each with its own behavior.
> 
> They can also be quite frequent and I didn't want to burden their
> dispatching mechanisms with the extra overhead of also dealing with
> potential errors that might infrequently occur.
> In both cases I would use a variation on branch-and-link to a hardware
> state save and generated RIP address so there are no memory references
> during the internal state change. This means no double- or triple-faults
> and potentially a 1 clock state change.
> 
> The problem with errors is that their behaviour is all over the place.
> A store operation to memory or device marked as write-through would be
> synchronous with the ST instruction and any errors detected could be
> associated back to the ST instruction.
> A store operation to memory marked as write-back is asynchronous and
> can be performed long after the ST instruction has retired and
> been removed from the core.
> 
> An ECC memory error can be detected and maybe corrected by the
> error scavenger sweep and not associated with any instruction at all.
> 
> Some devices change state when read. For example a UART receive
> buffer can be FIFO queue and reading it consumes the first entry.
> If there was a bus parity error after a successful read of the
> FIFO then one may or may not want to retry the operation.
> 
> By putting infrequent Errors off in their own bucket they can be
> dealt with on a case by case basis without burdening the frequent
> occurance delivery mechanisms for Exceptions and Interrupts.
> 

Have you ever messed around with a Pthread async cancel?

PTHREAD_CANCEL_ASYNCHRONOUS

Iirc, a pthread emulation over on windows had to use a damn kernel 
driver to do it. I say, avoid at all costs!

[toc] | [prev] | [next] | [standalone]

#118923 — Re: Interrupts in OoO

On 2026-Oct-26 17:14, Chris M. Thomasson wrote:
> On 10/26/2026 2:03 PM, EricP wrote:
>>
> 
> Have you ever messed around with a Pthread async cancel?
> 
> PTHREAD_CANCEL_ASYNCHRONOUS
> 
> Iirc, a pthread emulation over on windows had to use a damn kernel driver to do it. I say, avoid at all costs!

No, I have never used Pthreads and I have no idea
how it is implemented on Windows.

It sounds like it calls TerminateThread which does exactly that,
without giving an application any chance to clean up nicely.
But in the Unix manuals I also see warnings about the use of
async cancel in general on any implementation.

VMS got this one right - the ability to asynchronously force a
process/thread to do a clean exit.
A call to SYS$FORCEEX posts a user mode AST to the process/thread,
which *interrupts* and raises a FORCE_EXIT exception in the process/thread,
allowing stack exception catch handlers or thread exit rundown handlers
a chance to clean up properly (release mutexes, finish a heap
allocation without corrupting the heap, etc).

WNT doesn't have asynchronous (interrupting) user mode APC's,
just synchronous (polled) ones at alertable waits.
So you can't always force another thread to do your bidding like stopping.
(It is actually possible to do this using GetThreadContext and SetThreadContext.
But as this would be non-standard the surrounding code for things like
mutex holders and heap allocators would not expect it and would have no
supporting exit handler code for it.
Also Get/SetThreadContext requires debug privilege.)

[toc] | [prev] | [next] | [standalone]

#118930 — Re: Interrupts in OoO

On 10/28/2026 11:39 AM, EricP wrote:
> On 2026-Oct-26 17:14, Chris M. Thomasson wrote:
>> On 10/26/2026 2:03 PM, EricP wrote:
>>>
>>
>> Have you ever messed around with a Pthread async cancel?
>>
>> PTHREAD_CANCEL_ASYNCHRONOUS
>>
>> Iirc, a pthread emulation over on windows had to use a damn kernel 
>> driver to do it. I say, avoid at all costs!
> 
> No, I have never used Pthreads and I have no idea
> how it is implemented on Windows.

Okay.

> 
> It sounds like it calls TerminateThread which does exactly that,
> without giving an application any chance to clean up nicely.
> But in the Unix manuals I also see warnings about the use of
> async cancel in general on any implementation.

Its odd. It needs an actual kernel driver to implement 
PTHREAD_CANCEL_ASYNCHRONOUS via, iirc, kernel mode APC.






> 
> VMS got this one right - the ability to asynchronously force a
> process/thread to do a clean exit.
> A call to SYS$FORCEEX posts a user mode AST to the process/thread,
> which *interrupts* and raises a FORCE_EXIT exception in the process/thread,
> allowing stack exception catch handlers or thread exit rundown handlers
> a chance to clean up properly (release mutexes, finish a heap
> allocation without corrupting the heap, etc).
> 
> WNT doesn't have asynchronous (interrupting) user mode APC's,
> just synchronous (polled) ones at alertable waits.
> So you can't always force another thread to do your bidding like stopping.
> (It is actually possible to do this using GetThreadContext and 
> SetThreadContext.
> But as this would be non-standard the surrounding code for things like
> mutex holders and heap allocators would not expect it and would have no
> supporting exit handler code for it.
> Also Get/SetThreadContext requires debug privilege.)
> 
> 

[toc] | [prev] | [next] | [standalone]

#118948 — Re: Interrupts in OoO

On 2026-Oct-28 18:10, Chris M. Thomasson wrote:
> On 10/28/2026 11:39 AM, EricP wrote:
>> On 2026-Oct-26 17:14, Chris M. Thomasson wrote:
>>> On 10/26/2026 2:03 PM, EricP wrote:
>>>>
>>>
>>> Have you ever messed around with a Pthread async cancel?
>>>
>>> PTHREAD_CANCEL_ASYNCHRONOUS
>>>
>>> Iirc, a pthread emulation over on windows had to use a damn kernel driver to do it. I say, avoid at all costs!
>>
>> No, I have never used Pthreads and I have no idea
>> how it is implemented on Windows.
> 
> Okay.
> 
>>
>> It sounds like it calls TerminateThread which does exactly that,
>> without giving an application any chance to clean up nicely.
>> But in the Unix manuals I also see warnings about the use of
>> async cancel in general on any implementation.
> 
> Its odd. It needs an actual kernel driver to implement PTHREAD_CANCEL_ASYNCHRONOUS via, iirc, kernel mode APC.

People often use drivers to work around functions requiring
elevated privileges.

>>
>> VMS got this one right - the ability to asynchronously force a
>> process/thread to do a clean exit.
>> A call to SYS$FORCEEX posts a user mode AST to the process/thread,
>> which *interrupts* and raises a FORCE_EXIT exception in the process/thread,
>> allowing stack exception catch handlers or thread exit rundown handlers
>> a chance to clean up properly (release mutexes, finish a heap
>> allocation without corrupting the heap, etc).
>>
>> WNT doesn't have asynchronous (interrupting) user mode APC's,
>> just synchronous (polled) ones at alertable waits.
>> So you can't always force another thread to do your bidding like stopping.
>> (It is actually possible to do this using GetThreadContext and SetThreadContext.
>> But as this would be non-standard the surrounding code for things like
>> mutex holders and heap allocators would not expect it and would have no
>> supporting exit handler code for it.
>> Also Get/SetThreadContext requires debug privilege.)
>>
>>
> 

It appears what I speculated is exactly what they did.

https://sourceforge.net/projects/pthreads4w/files/

This refers to them writing their own function QueueUserAPCEx
to force a really asynchronous (interrupting) subroutine call
onto a thread.

I didn't want to download their files and search them but
found some documentation elsewhere:

https://locklessinc.com/articles/pthreads_on_windows/

You can see below they are doing exactly what I said,
using GetThreadContext and SetThreadContext to "edit" the thread
and jam their own routine call into it.
It sounds like they packaged this into a DLL and invoke it with a
driver to get around the need for giving Debug privilege to everyone.

This won't work for the reasons I said above:
the software above it in the call stack must have been written
*knowing* that this "force exit" can happen and have established
exit handlers for the stack unwind. The OS must define this
force-exit mechanism right from the start so all cleanup code uses it.
VMS did that, WNT did not.

static int pthread_cancel(pthread_t t)
{
	if (t->p_state & PTHREAD_CANCEL_ASYNCHRONOUS)
	{
		/* Dangerous asynchronous cancelling */
		CONTEXT ctxt;
		
		/* Already done? */
		if (t->cancelled) return ESRCH;
		
		ctxt.ContextFlags = CONTEXT_CONTROL;
		
		SuspendThread(t->h);
		GetThreadContext(t->h, &ctxt);
#ifdef _M_X64
		ctxt.Rip = (uintptr_t) _pthread_invoke_cancel;
#else
		ctxt.Eip = (uintptr_t) _pthread_invoke_cancel;
#endif
		SetThreadContext(t->h, &ctxt);
		
		/* Also try deferred Cancelling */
		t->cancelled = 1;
	
		/* Notify everyone to look */
		_InterlockedIncrement(&_pthread_cancelling);
		
		ResumeThread(t->h);
	}

[toc] | [prev] | [next] | [standalone]

#118872 — Re: Interrupts in OoO

EricP <ThatWouldBeTelling@thevillage.com> posted:

> On 2026-Oct-26 13:17, Robert Finch wrote:
> > On 2026-10-26 11:06 a.m., EricP wrote:
> >> On 2026-Oct-25 11:53, Scott Lurndal wrote:
> >>> MitchAlsup <user5857@newsgrouper.org.invalid> writes:
----------snip-----------

> Precise means the processor state corresponds with a sequential model
> of program execution where one instruction completes before the next begin.
> 
> Synchronous means that the saved state corresponds with the particular
> instruction that triggered the state or the instruction after it.
> For a Fault the state correspond to the triggering instruction.
> For a Trap the state corresponds to the instruction after the
> triggering instruction.
> All exceptions, whether faults or traps, are synchronous.
> 
> Asynchronous means that the state does not correspond with any
> particular instruction. All Interrupts are asynchronous.
> 
> For an asynchronous Interrupt or synchronous Exception to be
> restartable it must be precise.

Some architectures defined "continue" model of returning from exceptions
{68000} where enough state was saved so the ESR could finish the failed
instruction and then continue onward with the next instruction.

Both seem "workable" while precise has won overall.
 
> Each of Exceptions and Interrupts have well defined behaviour.
> Exceptions only have one delivery mechanism whereas Interrupts might
> have a variety of Interrupt controllers each with its own behavior.
> 
> They can also be quite frequent and I didn't want to burden their
> dispatching mechanisms with the extra overhead of also dealing with
> potential errors that might infrequently occur.
> In both cases I would use a variation on branch-and-link to a hardware
> state save and generated RIP address so there are no memory references
> during the internal state change. This means no double- or triple-faults
> and potentially a 1 clock state change.
> 
> The problem with errors is that their behaviour is all over the place.
> A store operation to memory or device marked as write-through would be
> synchronous with the ST instruction and any errors detected could be
> associated back to the ST instruction.
> A store operation to memory marked as write-back is asynchronous and
> can be performed long after the ST instruction has retired and
> been removed from the core.
> 
> An ECC memory error can be detected and maybe corrected by the
> error scavenger sweep and not associated with any instruction at all.
> 
> Some devices change state when read. For example a UART receive
> buffer can be FIFO queue and reading it consumes the first entry.
> If there was a bus parity error after a successful read of the
> FIFO then one may or may not want to retry the operation.
> 
> By putting infrequent Errors off in their own bucket they can be
> dealt with on a case by case basis without burdening the frequent
> occurance delivery mechanisms for Exceptions and Interrupts.
> 

[toc] | [prev] | [next] | [standalone]

#118869 — Re: Interrupts in OoO

EricP <ThatWouldBeTelling@thevillage.com> posted:

> On 2026-Oct-25 11:53, Scott Lurndal wrote:
> > MitchAlsup <user5857@newsgrouper.org.invalid> writes:
> >> quadibloc@invalid.com (John Savard) posted:
> >>
> >>> On Sat, 05 Oct 2024 17:57:12 GMT, anton@mips.complang.tuwien.ac.at
> >>> (Anton Ertl) wrote:
> >>>
> >>>> Ok, it's more efficient to insert an interrupt call into the
> >>>> instruction stream in the decoder: all the in-flight instructions will
> >>>> be completed instead of going to waste.  However, the interrupt will
> >>>> usually be serviced later, and, as you point out, if the instruction
> >>>> stream is redirected for some other reason, you may have to replay the
> >>>> interrupt.
> >>>
> >>> A _trap_ can be a problem on an out-of-order machine, since traps
> >>> result from exceptions, and so they have to be tied to the instruction
> >>> stream.
> >>
> >> We have {Traps, Exceptions, Interrupts}.
> >>
> >> A trap is synchronous to the executing code and we want IP pointing at the
> >> subsequent instruction. A trap may send a small amount of data to the service
> >> at the other end. Not all registers have to be preserved. Registers R0..R15
> >> are considered temporaries at the caller->callee boundary. Those registers
> >> either carry a result or are cleared <HW> at the callee->caller boundary::
> >> in My 66000 SBI.
> >>
> >> An Exception is synchronous to the executing code and we want IP pointing
> >> at the offending instruction. What kind of exception is the message to the
> >> service routine. All Thread-State has to be preserved. A Thread should
> >> be able to handle its own exceptions should it communicate same with
> >> Guest OS.
> > 
> > Exceptions are not alway precise.  For example, an exception
> > on a posted store (that's spent time in a store buffer, for
> > example), may have an exception asynchronously.
> 
> This is a terminology problem that comes from processors putting
> all the "alert" notifications through the same delivery mechanism.
> People often use various the terms as interchangeable when really
> they are not.
> 
> In my terminology I have Exceptions, Interrupts, and Errors.
> I take all the well defined stuff and put it in the Exception and
> Interrupt buckets, and take all the weird stuff and call it Errors.

To a very large extent I agree with you. The things you call errors
I call Checks. Checks are never supposed to happen {{like late store
data ECC fails}}.
 
> Exceptions are defined by the ISA, are precise, and synchronous
> to the excepting instruction. There are two kinds, Faults and Traps.
> Most exceptions notifications cannot be deferred or "masked".
> Floating point exceptions are potentially maskable however if
> they are then any auxiliary information about the precise source of
> the FP exceptions will be synchronous with instruction that unmasks
> a pending floating point exception.
> 
> Interrupts are defined as precise and asynchronous.
> Their number and priority mechanisms is defined as model specific.
> Interrupts are potentially maskable and may be deferred.
> The only ISA defined part of interrupts are the master interrupt
> mask control instructions INTEN enable and INTDI disable.
> 
> Exceptions and Interrupts are defined as precise and restartable.
> 
> The mechanism for delivering Exception and Interrupt notifications
> is defined as model specific. In practice this just means that
> operating systems should modularize this layer of their software
> (sometimes called a HAL or Hardware Abstraction Layer).
> 
> Errors are model and error specific, may be synchronous or asynchronous,
> may be precise (Faults or Traps) or imprecise (Aborts).
> Errors may or may not be deferred (maskable).

It is at this point I generally point out that Checks {errors} deliver
control to the highest privilege level {HyperVisor} not the next higher
privilege level {Guest OS when in User}.

> Errors may not be restartable.
> The mechanism for delivering Error notifications is defined as
> model and error specific.

No, while the checks are model specific, the mechanism to deliver them
to HV is specified.
> 

[toc] | [prev] | [next] | [standalone]

#118882 — Re: Interrupts in OoO

MitchAlsup <user5857@newsgrouper.org.invalid> writes:
>EricP <ThatWouldBeTelling@thevillage.com> posted:
>
>> On 2026-Oct-25 11:53, Scott Lurndal wrote:
>> > MitchAlsup <user5857@newsgrouper.org.invalid> writes:
>> >> quadibloc@invalid.com (John Savard) posted:
>> >>
>> >>> On Sat, 05 Oct 2024 17:57:12 GMT, anton@mips.complang.tuwien.ac.at
>> >>> (Anton Ertl) wrote:
>> >>>
>> >>>> Ok, it's more efficient to insert an interrupt call into the
>> >>>> instruction stream in the decoder: all the in-flight instructions will
>> >>>> be completed instead of going to waste.  However, the interrupt will
>> >>>> usually be serviced later, and, as you point out, if the instruction
>> >>>> stream is redirected for some other reason, you may have to replay the
>> >>>> interrupt.
>> >>>
>> >>> A _trap_ can be a problem on an out-of-order machine, since traps
>> >>> result from exceptions, and so they have to be tied to the instruction
>> >>> stream.
>> >>
>> >> We have {Traps, Exceptions, Interrupts}.
>> >>
>> >> A trap is synchronous to the executing code and we want IP pointing at the
>> >> subsequent instruction. A trap may send a small amount of data to the service
>> >> at the other end. Not all registers have to be preserved. Registers R0..R15
>> >> are considered temporaries at the caller->callee boundary. Those registers
>> >> either carry a result or are cleared <HW> at the callee->caller boundary::
>> >> in My 66000 SBI.
>> >>
>> >> An Exception is synchronous to the executing code and we want IP pointing
>> >> at the offending instruction. What kind of exception is the message to the
>> >> service routine. All Thread-State has to be preserved. A Thread should
>> >> be able to handle its own exceptions should it communicate same with
>> >> Guest OS.
>> > 
>> > Exceptions are not alway precise.  For example, an exception
>> > on a posted store (that's spent time in a store buffer, for
>> > example), may have an exception asynchronously.
>> 
>> This is a terminology problem that comes from processors putting
>> all the "alert" notifications through the same delivery mechanism.
>> People often use various the terms as interchangeable when really
>> they are not.
>> 
>> In my terminology I have Exceptions, Interrupts, and Errors.
>> I take all the well defined stuff and put it in the Exception and
>> Interrupt buckets, and take all the weird stuff and call it Errors.
>
>To a very large extent I agree with you. The things you call errors
>I call Checks. Checks are never supposed to happen {{like late store
>data ECC fails}}.

If it is "never supposed to happen", why bother to check at all...

For RAS reasons, if nothing else, one must assume that anything
may happen.

> 
 <snip>

>> Errors are model and error specific, may be synchronous or asynchronous,
>> may be precise (Faults or Traps) or imprecise (Aborts).
>> Errors may or may not be deferred (maskable).
>
>It is at this point I generally point out that Checks {errors} deliver
>control to the highest privilege level {HyperVisor} not the next higher
>privilege level {Guest OS when in User}.

It's better if that is programmable, i.e. the highest privilege level
(even higher than the hypervisor - i.e. the EL3 in ARM64, or SMM for
intel/amd (or perhaps a BMC)).  The higher privilege levels can delegate
the asynchronous "check" exception handling to a lower level, if necessary.

[toc] | [prev] | [next] | [standalone]

#118899 — Re: Interrupts in OoO

scott@slp53.sl.home (Scott Lurndal) posted:

> MitchAlsup <user5857@newsgrouper.org.invalid> writes:
-----------------------------
> >> In my terminology I have Exceptions, Interrupts, and Errors.
> >> I take all the well defined stuff and put it in the Exception and
> >> Interrupt buckets, and take all the weird stuff and call it Errors.
> >
> >To a very large extent I agree with you. The things you call errors
> >I call Checks. Checks are never supposed to happen {{like late store
> >data ECC fails}}.
> 
> If it is "never supposed to happen", why bother to check at all...

Guest OS can run as if uncorrectable ECC can never happen. So, to
Guest OS it is never supposed to happen, and Guest OS is not setup
to deal with the problem if it does.

HyperVisor cannot run banking applications with UC ECC, whereas some
real-time control might be able to; so you punt it all the way up the
privilege ladder.
 
> For RAS reasons, if nothing else, one must assume that anything
> may happen.

But, still, Guest OS should not be burdened with those checks.
> 
> > 
>  <snip>
> 
> >> Errors are model and error specific, may be synchronous or asynchronous,
> >> may be precise (Faults or Traps) or imprecise (Aborts).
> >> Errors may or may not be deferred (maskable).
> >
> >It is at this point I generally point out that Checks {errors} deliver
> >control to the highest privilege level {HyperVisor} not the next higher
> >privilege level {Guest OS when in User}.
> 
> It's better if that is programmable, i.e. the highest privilege level
> (even higher than the hypervisor - i.e. the EL3 in ARM64, or SMM for
> intel/amd (or perhaps a BMC)).  The higher privilege levels can delegate
> the asynchronous "check" exception handling to a lower level, if necessary.

Or abort the lower level that suffered the check, possibly restarting it
from the last checkpoint with new/diffent memory.
 

[toc] | [prev] | [next] | [standalone]

#118901 — Re: Interrupts in OoO

MitchAlsup <user5857@newsgrouper.org.invalid> writes:
>scott@slp53.sl.home (Scott Lurndal) posted:
>
>> MitchAlsup <user5857@newsgrouper.org.invalid> writes:
>-----------------------------
>> >> In my terminology I have Exceptions, Interrupts, and Errors.
>> >> I take all the well defined stuff and put it in the Exception and
>> >> Interrupt buckets, and take all the weird stuff and call it Errors.
>> >
>> >To a very large extent I agree with you. The things you call errors
>> >I call Checks. Checks are never supposed to happen {{like late store
>> >data ECC fails}}.
>> 
>> If it is "never supposed to happen", why bother to check at all...
>
>Guest OS can run as if uncorrectable ECC can never happen. So, to
>Guest OS it is never supposed to happen, and Guest OS is not setup
>to deal with the problem if it does.
>
>HyperVisor cannot run banking applications with UC ECC, whereas some
>real-time control might be able to; so you punt it all the way up the
>privilege ladder.
> 
>> For RAS reasons, if nothing else, one must assume that anything
>> may happen.
>
>But, still, Guest OS should not be burdened with those checks.

Nor should the hypervisor.  All of which are generally hardware
agnostic (KVM, XEN, VMWare, et alia) for most everything except
the virtualization mechanics (AMD's SVM or Intel's VM capabilities).

[toc] | [prev] | [next] | [standalone]

#118917 — Re: Interrupts in OoO

scott@slp53.sl.home (Scott Lurndal) posted:

> MitchAlsup <user5857@newsgrouper.org.invalid> writes:
> >scott@slp53.sl.home (Scott Lurndal) posted:
> >
> >> MitchAlsup <user5857@newsgrouper.org.invalid> writes:
> >-----------------------------
> >> >> In my terminology I have Exceptions, Interrupts, and Errors.
> >> >> I take all the well defined stuff and put it in the Exception and
> >> >> Interrupt buckets, and take all the weird stuff and call it Errors.
> >> >
> >> >To a very large extent I agree with you. The things you call errors
> >> >I call Checks. Checks are never supposed to happen {{like late store
> >> >data ECC fails}}.
> >> 
> >> If it is "never supposed to happen", why bother to check at all...
> >
> >Guest OS can run as if uncorrectable ECC can never happen. So, to
> >Guest OS it is never supposed to happen, and Guest OS is not setup
> >to deal with the problem if it does.
> >
> >HyperVisor cannot run banking applications with UC ECC, whereas some
> >real-time control might be able to; so you punt it all the way up the
> >privilege ladder.
> > 
> >> For RAS reasons, if nothing else, one must assume that anything
> >> may happen.
> >
> >But, still, Guest OS should not be burdened with those checks.
> 
> Nor should the hypervisor.  All of which are generally hardware
> agnostic (KVM, XEN, VMWare, et alia) for most everything except
> the virtualization mechanics (AMD's SVM or Intel's VM capabilities).

So, it all one has is {user, Guest, Host, Hyper}: who SHOULD be burdened
with creating responses to things that are not supposed to happen but do ??

[toc] | [prev] | [next] | [standalone]

#118922 — Re: Interrupts in OoO

On Wed, 28 Oct 2026 15:35:45 GMT, MitchAlsup
<user5857@newsgrouper.org.invalid> wrote:

>scott@slp53.sl.home (Scott Lurndal) posted:
>
>> MitchAlsup <user5857@newsgrouper.org.invalid> writes:
>> >scott@slp53.sl.home (Scott Lurndal) posted:
>> >
>> >> MitchAlsup <user5857@newsgrouper.org.invalid> writes:
>> >-----------------------------
>> >> >> In my terminology I have Exceptions, Interrupts, and Errors.
>> >> >> I take all the well defined stuff and put it in the Exception and
>> >> >> Interrupt buckets, and take all the weird stuff and call it Errors.
>> >> >
>> >> >To a very large extent I agree with you. The things you call errors
>> >> >I call Checks. Checks are never supposed to happen {{like late store
>> >> >data ECC fails}}.
>> >> 
>> >> If it is "never supposed to happen", why bother to check at all...
>> >
>> >Guest OS can run as if uncorrectable ECC can never happen. So, to
>> >Guest OS it is never supposed to happen, and Guest OS is not setup
>> >to deal with the problem if it does.
>> >
>> >HyperVisor cannot run banking applications with UC ECC, whereas some
>> >real-time control might be able to; so you punt it all the way up the
>> >privilege ladder.
>> > 
>> >> For RAS reasons, if nothing else, one must assume that anything
>> >> may happen.
>> >
>> >But, still, Guest OS should not be burdened with those checks.
>> 
>> Nor should the hypervisor.  All of which are generally hardware
>> agnostic (KVM, XEN, VMWare, et alia) for most everything except
>> the virtualization mechanics (AMD's SVM or Intel's VM capabilities).
>
>So, it all one has is {user, Guest, Host, Hyper}: who SHOULD be burdened
>with creating responses to things that are not supposed to happen but do ??

There is another question along with who handles it ... what should be
done?  Uncorrected ECC error (check) means the data has to be
considered corrupt [even if it really is valid] and unusable.

Do you crash the program that can't handle it?
Do you crash the guest OS that can't handle it?
  :

How is the user going to feel about your system?

[toc] | [prev] | [next] | [standalone]

#118927 — Re: Interrupts in OoO

George Neuner <gneuner2@comcast.net> writes:
>On Wed, 28 Oct 2026 15:35:45 GMT, MitchAlsup
><user5857@newsgrouper.org.invalid> wrote:
>
>>scott@slp53.sl.home (Scott Lurndal) posted:
>>
>>> MitchAlsup <user5857@newsgrouper.org.invalid> writes:
>>> >scott@slp53.sl.home (Scott Lurndal) posted:
>>> >
>>> >> MitchAlsup <user5857@newsgrouper.org.invalid> writes:
>>> >-----------------------------
>>> >> >> In my terminology I have Exceptions, Interrupts, and Errors.
>>> >> >> I take all the well defined stuff and put it in the Exception and
>>> >> >> Interrupt buckets, and take all the weird stuff and call it Errors.
>>> >> >
>>> >> >To a very large extent I agree with you. The things you call errors
>>> >> >I call Checks. Checks are never supposed to happen {{like late store
>>> >> >data ECC fails}}.
>>> >> 
>>> >> If it is "never supposed to happen", why bother to check at all...
>>> >
>>> >Guest OS can run as if uncorrectable ECC can never happen. So, to
>>> >Guest OS it is never supposed to happen, and Guest OS is not setup
>>> >to deal with the problem if it does.
>>> >
>>> >HyperVisor cannot run banking applications with UC ECC, whereas some
>>> >real-time control might be able to; so you punt it all the way up the
>>> >privilege ladder.
>>> > 
>>> >> For RAS reasons, if nothing else, one must assume that anything
>>> >> may happen.
>>> >
>>> >But, still, Guest OS should not be burdened with those checks.
>>> 
>>> Nor should the hypervisor.  All of which are generally hardware
>>> agnostic (KVM, XEN, VMWare, et alia) for most everything except
>>> the virtualization mechanics (AMD's SVM or Intel's VM capabilities).
>>
>>So, it all one has is {user, Guest, Host, Hyper}: who SHOULD be burdened
>>with creating responses to things that are not supposed to happen but do ??
>
>There is another question along with who handles it ... what should be
>done?  Uncorrected ECC error (check) means the data has to be
>considered corrupt [even if it really is valid] and unusable.

On ARM64 systems, which supports industry standard interfaces implemented by
both windows and linux, there are four exception levels, in order of
increasing privilege:

   EL0   User code
   EL1   Kernel code (off the shelf, not platform specific)
   EL2   Hypervisor code (off the shelf, not platform specific)
   EL3   Secure monitor code

Platform and processor specific code is encapsulated in EL3, so that
the operating systems and hypervisors can off-the-shelf standard across
all implementations of a given processor family.

In addition, most non-consumer ARM64 systems will have a Baseboard
Management Controller (BMC) and perhaps a CPLD which handle all the
platform and processor specific RAS and management functionality,
with a small shim in the EL3 code, if necessary, to interface
with the BMC (usually over I2C/I3C and/or ethernet).  This is
all provided by the board manufacturer, with the appropriate
security mechanisms and guarantees (e.g. encrypted communications
over I3C or ethernet, etc.)

Beyond that, most processors (intel/amd/arm) used in server-grade
systems have on-SoC mini processors such as Cortex-M3 or M7 cores for handling
boot (including loading firmware from SPI/eMMC controllers prior to
releasing the primary core from reset).  The boot rom is executed by
the aux (e.g. m7) processor, not the main processor and is pretty
small, but sufficient to support trusted boot.

RAS notification and recovery is fairly well defined in the various
related specifications for standard firmware interfaces (e.g. AcPI).

Most of the platform-specific recovery is done by the secure firmware,
BMC or the M7 core(s).   The ACPI standard (used by intel/amd/arm) defines
the standard mechanisms that windows and linux will use to interface
with the proprietary hardware, including a notification mechanism
to the hypervisor (which can then, if required, use the same mechanism
to notify the kernel of an issue).  Recovery is done by standard
non-platform dependent code in the operating systems, if possible.

>
>Do you crash the program that can't handle it?
>Do you crash the guest OS that can't handle it?
>  :
>
>How is the user going to feel about your system?

Recovery from an uncorrectable hardware fault is HARD.

Believe that the user experience is at the forefront of the
designers of _all_ these systems; certainly at the high-end
and for most custom designs.

Worst case, if no recovery path exists for a hardware problem,
the BMC will reset the board.

[toc] | [prev] | [next] | [standalone]

#118931 — Re: Interrupts in OoO

George Neuner <gneuner2@comcast.net> posted:

> On Wed, 28 Oct 2026 15:35:45 GMT, MitchAlsup
> <user5857@newsgrouper.org.invalid> wrote:
> 
> >scott@slp53.sl.home (Scott Lurndal) posted:
> >
> >> MitchAlsup <user5857@newsgrouper.org.invalid> writes:
> >> >scott@slp53.sl.home (Scott Lurndal) posted:
> >> >
> >> >> MitchAlsup <user5857@newsgrouper.org.invalid> writes:
> >> >-----------------------------
> >> >> >> In my terminology I have Exceptions, Interrupts, and Errors.
> >> >> >> I take all the well defined stuff and put it in the Exception and
> >> >> >> Interrupt buckets, and take all the weird stuff and call it Errors.
> >> >> >
> >> >> >To a very large extent I agree with you. The things you call errors
> >> >> >I call Checks. Checks are never supposed to happen {{like late store
> >> >> >data ECC fails}}.
> >> >> 
> >> >> If it is "never supposed to happen", why bother to check at all...
> >> >
> >> >Guest OS can run as if uncorrectable ECC can never happen. So, to
> >> >Guest OS it is never supposed to happen, and Guest OS is not setup
> >> >to deal with the problem if it does.
> >> >
> >> >HyperVisor cannot run banking applications with UC ECC, whereas some
> >> >real-time control might be able to; so you punt it all the way up the
> >> >privilege ladder.
> >> > 
> >> >> For RAS reasons, if nothing else, one must assume that anything
> >> >> may happen.
> >> >
> >> >But, still, Guest OS should not be burdened with those checks.
> >> 
> >> Nor should the hypervisor.  All of which are generally hardware
> >> agnostic (KVM, XEN, VMWare, et alia) for most everything except
> >> the virtualization mechanics (AMD's SVM or Intel's VM capabilities).
> >
> >So, it all one has is {user, Guest, Host, Hyper}: who SHOULD be burdened
> >with creating responses to things that are not supposed to happen but do ??
> 
> There is another question along with who handles it ... what should be
> done?  Uncorrected ECC error (check) means the data has to be
> considered corrupt [even if it really is valid] and unusable.
> 
> Do you crash the program that can't handle it?

Real Time control--just move along
.....
Banking--recover to checkpoint
4,000 hour GB FP application--recover to checkpoint

> Do you crash the guest OS that can't handle it?

PC--take a crash or restart Guest OS
Server--backup to checkpoint and run forward until you catchup with real-time
   :
> How is the user going to feel about your system?

Better than if the error was never detected...

[toc] | [prev] | [next] | [standalone]

Page 1 of 2  [1] 2  Next page →

Back to top | Article view | comp.arch

csiph-web