Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > comp.arch > #111126 > unrolled thread
| Started by | Robert Finch <robfi680@gmail.com> |
|---|---|
| First post | 2025-03-13 00:50 -0400 |
| Last post | 2025-03-17 17:33 -0400 |
| Articles | 20 on this page of 211 — 12 participants |
Back to article view | Back to comp.arch
MSI interrupts Robert Finch <robfi680@gmail.com> - 2025-03-13 00:50 -0400
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-13 16:34 +0000
Re: MSI interrupts Robert Finch <robfi680@gmail.com> - 2025-03-13 13:49 -0400
Re: MSI interrupts scott@slp53.sl.home (Scott Lurndal) - 2025-03-13 18:42 +0000
Re: MSI interrupts Robert Finch <robfi680@gmail.com> - 2025-03-13 15:59 -0400
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-13 20:46 +0000
Re: MSI interrupts scott@slp53.sl.home (Scott Lurndal) - 2025-03-13 21:39 +0000
Re: MSI interrupts Stefan Monnier <monnier@iro.umontreal.ca> - 2025-03-13 17:00 -0400
Re: MSI interrupts scott@slp53.sl.home (Scott Lurndal) - 2025-03-13 23:27 +0000
Re: MSI interrupts scott@slp53.sl.home (Scott Lurndal) - 2025-03-13 21:27 +0000
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-13 19:12 +0000
Re: MSI interrupts scott@slp53.sl.home (Scott Lurndal) - 2025-03-13 18:34 +0000
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-13 19:28 +0000
Re: MSI interrupts scott@slp53.sl.home (Scott Lurndal) - 2025-03-13 21:14 +0000
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-13 21:34 +0000
Re: MSI interrupts scott@slp53.sl.home (Scott Lurndal) - 2025-03-13 23:34 +0000
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-13 23:52 +0000
Re: MSI interrupts scott@slp53.sl.home (Scott Lurndal) - 2025-03-14 14:52 +0000
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-14 15:34 +0000
Re: MSI interrupts scott@slp53.sl.home (Scott Lurndal) - 2025-03-14 17:35 +0000
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-14 18:12 +0000
Re: MSI interrupts scott@slp53.sl.home (Scott Lurndal) - 2025-03-14 18:42 +0000
Re: MSI interrupts Robert Finch <robfi680@gmail.com> - 2025-03-14 22:10 -0400
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-15 17:46 +0000
Re: MSI interrupts Robert Finch <robfi680@gmail.com> - 2025-03-15 16:46 -0400
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-15 22:28 +0000
Re: MSI interrupts Robert Finch <robfi680@gmail.com> - 2025-03-16 17:35 -0400
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-16 22:00 +0000
Re: MSI interrupts scott@slp53.sl.home (Scott Lurndal) - 2025-03-17 13:38 +0000
Re: MSI interrupts Michael S <already5chosen@yahoo.com> - 2025-03-17 16:11 +0200
Re: MSI interrupts Robert Finch <robfi680@gmail.com> - 2025-03-17 11:23 -0400
Re: MSI interrupts scott@slp53.sl.home (Scott Lurndal) - 2025-03-17 15:44 +0000
Re: MSI interrupts EricP <ThatWouldBeTelling@thevillage.com> - 2025-03-17 14:33 -0400
Re: MSI interrupts Robert Finch <robfi680@gmail.com> - 2025-03-17 17:14 -0400
Re: MSI interrupts EricP <ThatWouldBeTelling@thevillage.com> - 2025-03-18 14:51 -0400
Re: MSI interrupts Robert Finch <robfi680@gmail.com> - 2025-03-18 17:59 -0400
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-18 23:00 +0000
Re: MSI interrupts scott@slp53.sl.home (Scott Lurndal) - 2025-03-19 13:49 +0000
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-19 15:54 +0000
Re: MSI interrupts scott@slp53.sl.home (Scott Lurndal) - 2025-03-19 16:14 +0000
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-19 19:03 +0000
Re: MSI interrupts cross@spitfire.i.gajendra.net (Dan Cross) - 2025-03-19 14:03 +0000
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-19 15:59 +0000
Re: MSI interrupts cross@spitfire.i.gajendra.net (Dan Cross) - 2025-03-20 12:44 +0000
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-20 19:41 +0000
Re: MSI interrupts cross@spitfire.i.gajendra.net (Dan Cross) - 2025-03-20 20:25 +0000
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-20 21:55 +0000
Re: MSI interrupts cross@spitfire.i.gajendra.net (Dan Cross) - 2025-03-24 12:37 +0000
Re: MSI interrupts scott@slp53.sl.home (Scott Lurndal) - 2025-03-24 13:59 +0000
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-24 17:47 +0000
Re: MSI interrupts cross@spitfire.i.gajendra.net (Dan Cross) - 2025-03-24 18:49 +0000
Re: MSI interrupts "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> - 2025-03-24 12:12 -0700
Re: MSI interrupts scott@slp53.sl.home (Scott Lurndal) - 2025-03-24 19:46 +0000
Re: MSI interrupts cross@spitfire.i.gajendra.net (Dan Cross) - 2025-03-24 20:14 +0000
Re: MSI interrupts "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> - 2025-03-24 19:02 -0700
Re: MSI interrupts "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> - 2025-03-24 19:05 -0700
Re: MSI interrupts cross@spitfire.i.gajendra.net (Dan Cross) - 2025-03-25 11:17 +0000
Re: MSI interrupts cross@spitfire.i.gajendra.net (Dan Cross) - 2025-03-25 11:14 +0000
Re: MSI interrupts "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> - 2025-03-25 13:32 -0700
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-24 17:36 +0000
Re: MSI interrupts Stefan Monnier <monnier@iro.umontreal.ca> - 2025-03-24 14:02 -0400
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-24 19:09 +0000
Re: MSI interrupts cross@spitfire.i.gajendra.net (Dan Cross) - 2025-03-24 20:28 +0000
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-24 22:29 +0000
Re: MSI interrupts cross@spitfire.i.gajendra.net (Dan Cross) - 2025-03-24 23:27 +0000
Re: MSI interrupts scott@slp53.sl.home (Scott Lurndal) - 2025-03-25 13:43 +0000
Re: MSI interrupts cross@spitfire.i.gajendra.net (Dan Cross) - 2025-03-25 15:08 +0000
Re: MSI interrupts Stefan Monnier <monnier@iro.umontreal.ca> - 2025-03-25 11:48 -0400
Re: MSI interrupts scott@slp53.sl.home (Scott Lurndal) - 2025-03-25 16:29 +0000
Re: MSI interrupts Stefan Monnier <monnier@iro.umontreal.ca> - 2025-03-25 13:28 -0400
Re: MSI interrupts EricP <ThatWouldBeTelling@thevillage.com> - 2025-03-25 13:53 -0400
Re: MSI interrupts scott@slp53.sl.home (Scott Lurndal) - 2025-03-25 20:33 +0000
Re: MSI interrupts cross@spitfire.i.gajendra.net (Dan Cross) - 2025-03-25 17:47 +0000
Re: MSI interrupts Terje Mathisen <terje.mathisen@tmsw.no> - 2025-03-27 10:55 +0100
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-27 20:31 +0000
Re: MSI interrupts "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> - 2025-03-24 19:08 -0700
Re: MSI interrupts cross@spitfire.i.gajendra.net (Dan Cross) - 2025-03-25 11:20 +0000
Re: MSI interrupts "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> - 2025-03-25 13:38 -0700
Re: MSI interrupts cross@spitfire.i.gajendra.net (Dan Cross) - 2025-03-25 22:21 +0000
Re: MSI interrupts "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> - 2025-03-25 16:23 -0700
Re: MSI interrupts cross@spitfire.i.gajendra.net (Dan Cross) - 2025-03-25 23:29 +0000
Re: MSI interrupts "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> - 2025-03-25 16:53 -0700
Re: MSI interrupts cross@spitfire.i.gajendra.net (Dan Cross) - 2025-03-24 18:55 +0000
Re: MSI interrupts EricP <ThatWouldBeTelling@thevillage.com> - 2025-03-24 13:46 -0400
Re: MSI interrupts "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> - 2025-03-24 12:09 -0700
Re: MSI interrupts cross@spitfire.i.gajendra.net (Dan Cross) - 2025-03-24 20:11 +0000
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-24 22:50 +0000
Re: MSI interrupts cross@spitfire.i.gajendra.net (Dan Cross) - 2025-03-24 23:45 +0000
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-25 01:53 +0000
Re: MSI interrupts Stefan Monnier <monnier@iro.umontreal.ca> - 2025-03-25 01:26 -0400
Re: MSI interrupts "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> - 2025-03-25 00:07 -0700
Re: MSI interrupts cross@spitfire.i.gajendra.net (Dan Cross) - 2025-03-25 12:23 +0000
Re: MSI interrupts scott@slp53.sl.home (Scott Lurndal) - 2025-03-25 13:45 +0000
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-25 15:01 +0000
Re: MSI interrupts cross@spitfire.i.gajendra.net (Dan Cross) - 2025-03-25 15:10 +0000
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-25 16:40 +0000
Re: MSI interrupts cross@spitfire.i.gajendra.net (Dan Cross) - 2025-03-25 17:54 +0000
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-25 20:40 +0000
Re: MSI interrupts cross@spitfire.i.gajendra.net (Dan Cross) - 2025-03-25 22:19 +0000
Re: MSI interrupts Stefan Monnier <monnier@iro.umontreal.ca> - 2025-03-25 18:49 -0400
Re: MSI interrupts cross@spitfire.i.gajendra.net (Dan Cross) - 2025-03-25 23:33 +0000
Re: MSI interrupts Stefan Monnier <monnier@iro.umontreal.ca> - 2025-03-25 22:58 -0400
Re: MSI interrupts cross@spitfire.i.gajendra.net (Dan Cross) - 2025-03-26 04:40 +0000
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-26 19:42 +0000
Re: MSI interrupts Stefan Monnier <monnier@iro.umontreal.ca> - 2025-03-27 09:17 -0400
Re: MSI interrupts scott@slp53.sl.home (Scott Lurndal) - 2025-03-27 13:42 +0000
Re: MSI interrupts "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> - 2025-03-25 17:09 -0700
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-26 01:00 +0000
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-26 00:56 +0000
Re: MSI interrupts cross@spitfire.i.gajendra.net (Dan Cross) - 2025-03-26 04:08 +0000
Re: MSI interrupts Stefan Monnier <monnier@iro.umontreal.ca> - 2025-03-26 09:55 -0400
Re: MSI interrupts cross@spitfire.i.gajendra.net (Dan Cross) - 2025-03-26 14:57 +0000
Re: MSI interrupts "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> - 2025-03-26 13:22 -0700
Re: MSI interrupts "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> - 2025-04-03 22:19 -0700
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-26 19:21 +0000
Re: MSI interrupts Stefan Monnier <monnier@iro.umontreal.ca> - 2025-03-26 15:41 -0400
Re: MSI interrupts Stefan Monnier <monnier@iro.umontreal.ca> - 2025-03-26 15:50 -0400
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-26 23:16 +0000
Re: MSI interrupts scott@slp53.sl.home (Scott Lurndal) - 2025-03-26 21:35 +0000
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-26 23:22 +0000
Re: MSI interrupts scott@slp53.sl.home (Scott Lurndal) - 2025-03-26 23:48 +0000
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-27 00:18 +0000
Re: MSI interrupts cross@spitfire.i.gajendra.net (Dan Cross) - 2025-03-27 12:46 +0000
Re: MSI interrupts cross@spitfire.i.gajendra.net (Dan Cross) - 2025-03-27 17:19 +0000
Re: MSI interrupts Stefan Monnier <monnier@iro.umontreal.ca> - 2025-03-27 14:27 -0400
Re: MSI interrupts cross@spitfire.i.gajendra.net (Dan Cross) - 2025-03-27 18:34 +0000
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-27 19:18 +0000
Re: MSI interrupts cross@spitfire.i.gajendra.net (Dan Cross) - 2025-03-28 01:01 +0000
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-28 01:35 +0000
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-28 01:47 +0000
Re: MSI interrupts Stefan Monnier <monnier@iro.umontreal.ca> - 2025-03-27 22:02 -0400
Re: MSI interrupts "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> - 2025-03-27 19:14 -0700
Re: MSI interrupts Stefan Monnier <monnier@iro.umontreal.ca> - 2025-03-29 10:31 -0400
Re: MSI interrupts cross@spitfire.i.gajendra.net (Dan Cross) - 2025-03-28 02:56 +0000
Re: MSI interrupts Stefan Monnier <monnier@iro.umontreal.ca> - 2025-03-27 21:52 -0400
Re: MSI interrupts cross@spitfire.i.gajendra.net (Dan Cross) - 2025-03-28 02:58 +0000
Re: MSI interrupts "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> - 2025-03-27 19:09 -0700
Re: MSI interrupts "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> - 2025-03-27 19:10 -0700
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-27 21:47 +0000
Re: MSI interrupts cross@spitfire.i.gajendra.net (Dan Cross) - 2025-03-28 02:53 +0000
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-28 15:24 +0000
Re: MSI interrupts Stefan Monnier <monnier@iro.umontreal.ca> - 2025-03-28 11:41 -0400
Re: MSI interrupts "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> - 2025-03-28 09:51 -0700
Re: MSI interrupts cross@spitfire.i.gajendra.net (Dan Cross) - 2025-03-29 14:28 +0000
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-29 15:23 +0000
Re: MSI interrupts cross@spitfire.i.gajendra.net (Dan Cross) - 2025-03-29 20:58 +0000
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-29 22:04 +0000
Re: MSI interrupts EricP <ThatWouldBeTelling@thevillage.com> - 2025-03-25 10:54 -0400
Re: MSI interrupts scott@slp53.sl.home (Scott Lurndal) - 2025-03-25 16:31 +0000
Re: MSI interrupts cross@spitfire.i.gajendra.net (Dan Cross) - 2025-03-25 17:44 +0000
Re: MSI interrupts EricP <ThatWouldBeTelling@thevillage.com> - 2025-03-28 14:40 -0400
Re: MSI interrupts cross@spitfire.i.gajendra.net (Dan Cross) - 2025-03-29 20:55 +0000
Re: MSI interrupts EricP <ThatWouldBeTelling@thevillage.com> - 2025-03-30 16:34 -0400
Re: MSI interrupts cross@spitfire.i.gajendra.net (Dan Cross) - 2025-03-31 03:33 +0000
Re: MSI interrupts Robert Finch <robfi680@gmail.com> - 2025-03-31 01:30 -0400
Software synchronization (was: Re: MSI interrupts) cross@spitfire.i.gajendra.net (Dan Cross) - 2025-03-31 14:42 +0000
Threads (was Re: MSI interrupts) scott@slp53.sl.home (Scott Lurndal) - 2025-03-31 15:28 +0000
Re: Threads (was Re: MSI interrupts) Terje Mathisen <terje.mathisen@tmsw.no> - 2025-04-01 16:44 +0200
Re: Threads (was Re: MSI interrupts) scott@slp53.sl.home (Scott Lurndal) - 2025-04-01 15:31 +0000
Re: Threads (was Re: MSI interrupts) "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> - 2025-04-01 12:49 -0700
Re: Threads (was Re: MSI interrupts) scott@slp53.sl.home (Scott Lurndal) - 2025-04-01 20:01 +0000
Re: Threads (was Re: MSI interrupts) "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> - 2025-04-01 13:17 -0700
Re: Threads (was Re: MSI interrupts) scott@slp53.sl.home (Scott Lurndal) - 2025-04-01 22:04 +0000
Re: Threads (was Re: MSI interrupts) anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2025-04-02 06:25 +0000
Re: Threads (was Re: MSI interrupts) cross@spitfire.i.gajendra.net (Dan Cross) - 2025-04-02 13:04 +0000
Re: Threads (was Re: MSI interrupts) Terje Mathisen <terje.mathisen@tmsw.no> - 2025-04-03 16:45 +0200
Re: Threads (was Re: MSI interrupts) scott@slp53.sl.home (Scott Lurndal) - 2025-04-03 15:00 +0000
Re: Threads (was Re: MSI interrupts) cross@spitfire.i.gajendra.net (Dan Cross) - 2025-04-03 17:16 +0000
Re: Threads (was Re: MSI interrupts) scott@slp53.sl.home (Scott Lurndal) - 2025-04-03 17:24 +0000
Re: Threads (was Re: MSI interrupts) cross@spitfire.i.gajendra.net (Dan Cross) - 2025-04-04 01:44 +0000
Centronics (was Re: Threads (was Re: MSI interrupts)) George Neuner <gneuner2@comcast.net> - 2025-04-03 22:22 -0400
Re: Centronics (was Re: Threads (was Re: MSI interrupts)) scott@slp53.sl.home (Scott Lurndal) - 2025-04-04 02:29 +0000
Re: Threads (was Re: MSI interrupts) George Neuner <gneuner2@comcast.net> - 2025-04-03 01:38 -0400
Re: Threads (was Re: MSI interrupts) cross@spitfire.i.gajendra.net (Dan Cross) - 2025-04-03 12:15 +0000
Re: Threads (was Re: MSI interrupts) Robert Swindells <rjs@fdy2.co.uk> - 2025-04-04 10:41 +0000
Re: Threads (was Re: MSI interrupts) George Neuner <gneuner2@comcast.net> - 2025-04-04 23:10 -0400
Re: MSI interrupts EricP <ThatWouldBeTelling@thevillage.com> - 2025-04-01 10:07 -0400
Re: MSI interrupts cross@spitfire.i.gajendra.net (Dan Cross) - 2025-04-01 17:39 +0000
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-19 23:28 +0000
Re: MSI interrupts cross@spitfire.i.gajendra.net (Dan Cross) - 2025-03-20 12:50 +0000
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-20 22:28 +0000
Re: MSI interrupts EricP <ThatWouldBeTelling@thevillage.com> - 2025-03-23 12:16 -0400
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-23 16:48 +0000
Re: MSI interrupts EricP <ThatWouldBeTelling@thevillage.com> - 2025-03-24 11:55 -0400
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-24 18:56 +0000
Re: MSI interrupts "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> - 2025-03-23 17:48 -0700
Re: MSI interrupts cross@spitfire.i.gajendra.net (Dan Cross) - 2025-03-24 18:39 +0000
Re: MSI interrupts EricP <ThatWouldBeTelling@thevillage.com> - 2025-03-20 12:03 -0400
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-20 19:52 +0000
Re: MSI interrupts EricP <ThatWouldBeTelling@thevillage.com> - 2025-03-19 13:07 -0400
Re: MSI interrupts Robert Finch <robfi680@gmail.com> - 2025-03-19 15:47 -0400
Re: MSI interrupts - Instructions Robert Finch <robfi680@gmail.com> - 2025-03-26 06:56 -0400
Re: MSI interrupts - Instructions EricP <ThatWouldBeTelling@thevillage.com> - 2025-03-26 10:33 -0400
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-18 22:49 +0000
Re: MSI interrupts Robert Finch <robfi680@gmail.com> - 2025-03-18 20:39 -0400
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-19 01:22 +0000
Re: MSI interrupts Robert Finch <robfi680@gmail.com> - 2025-03-18 23:22 -0400
Re: MSI interrupts scott@slp53.sl.home (Scott Lurndal) - 2025-03-19 13:54 +0000
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-19 15:56 +0000
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-17 21:49 +0000
Re: MSI interrupts scott@slp53.sl.home (Scott Lurndal) - 2025-03-18 01:34 +0000
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-18 18:51 +0000
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-18 19:19 +0000
Re: MSI interrupts Stefan Monnier <monnier@iro.umontreal.ca> - 2025-03-19 11:20 -0400
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-19 16:00 +0000
Re: MSI interrupts Robert Finch <robfi680@gmail.com> - 2025-03-19 13:27 -0400
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-19 19:17 +0000
Re: MSI interrupts Robert Finch <robfi680@gmail.com> - 2025-03-17 10:12 -0400
Re: MSI interrupts scott@slp53.sl.home (Scott Lurndal) - 2025-03-17 15:53 +0000
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-17 19:06 +0000
Re: MSI interrupts Robert Finch <robfi680@gmail.com> - 2025-03-17 17:33 -0400
Page 8 of 11 — ← Prev page 1 … 6 7 [8] 9 10 11 Next page →
| From | mitchalsup@aol.com (MitchAlsup1) |
|---|---|
| Date | 2025-03-28 15:24 +0000 |
| Message-ID | <cb049d5490b541878e264cedf95168e1@www.novabbs.org> |
| In reply to | #111359 |
On Fri, 28 Mar 2025 2:53:57 +0000, Dan Cross wrote:
> In article <34434320650f5844b18b1c0b684acf43@www.novabbs.org>,
> MitchAlsup1 <mitchalsup@aol.com> wrote:
>>On Thu, 27 Mar 2025 17:19:21 +0000, Dan Cross wrote:
>>> In article <7a093bbb356e3bda3782c15ca27e98a7@www.novabbs.org>,
>>> MitchAlsup1 <mitchalsup@aol.com> wrote:
>>-------------------
>>> Or maybe even just swapping places between two elements in a
>>> linked list. For example:
>>>
>>> void
>>> swap_places(Node **head, Node *a, Node *b)
>>> {
>>> Node *hp, *an, *ap, *bn, *bp;
>>>
>>> assert(head != NULL);
>>> assert(a != NULL);
>>> assert(b != NULL);
>>>
>>> if (a == b)
>>> return;
>>>
>>> esmLOCKprefetch(*head);
>>
>>This should be a load not prefetch--you want the value of *head
>
> I wondered about that. I ASSumed that `esmLOCKprefetch` was
> enough to set up a cache line address monitor on `*head`, but if
> you tell me I need `esmLOCKload`, then I need `esmLOCKload`:
>
> hp = esmLOCKload(*head);
> if (hp == a)
> *head = b; // Note I need to set whatever head points to, not hp.
> else if (hp == b)
> *head = a;
>
> (You can see a vestige of me trying that earlier by the `hp`
> temporary that I left in when I posted that initially, but I
> didn't like how it looked, so I changed it to what was posted.)
>
>>> if (*head == a) // see !
>>> *head = b;
>>> else if (*head == b)
>>> *head = a;
>>
>>There is a ESM rule that states:: all participating cache lines
>>must be touched before any participating cache lines can be
>>modified.
>
> Are these rules in a reference somewhere I can see? I kinda
> reckoned `esmLOCKprefetch` would be enough to "touch" the line,
> but I fully admit I was just guessing. Looks like `prefetch` is
> mostly for the case where you'll store, but don't need to read?
>
>>Also note: Participating cache lines are checked for write permission
>>at touch time, and on cache miss, read with intent to modify.
>
> Ok.
>
>>> an = esmLOCKload(a->next);
>>> ap = esmLOCKload(a->prev);
>>> bn = esmLOCKload(b->next);
>>> bp = esmLOCKload(b->prev);
>>>
>>> b->next = an;
>>> if (an != NULL) {
>>> esmLOCKprefetch(an->prev);
>>> an->prev = b;
>>> }
>>> b->prev = ap;
>>> if (ap != NULL) {
>>> esmLOCKprefetch(ap->next);
>>> ap->next = b;
>>> }
>>>
>>> a->next = bn;
>>> if (bn != NULL) {
>>> esmLOCKprefetch(bn->prev);
>>> bn->prev = a;
>>> }
>>> if (bp != NULL) {
>>> esmLOCKprefetch(bp->next);
>>> bp->next = a;
>>> }
>>> esmLOCKstore(a->prev, bp);
>>> }
>>
>>What I think you want:: (ignoring the 9 participants limit)
>
> You mean 8 participants limit? That counting thing again. ;-}
>
> What I really wanted was an example that exceeded that limit as
> an expository vehicle for understanding what happens when the
> limit is exceeded. What does the hardware do in that case?
Raise OPERATION (instruction) Fault.
>>> void
>>> swap_places(Node **head, Node *a, Node *b)
>>> {
>>> Node *hp, *an, *ap, *bn, *bp;
>>>
>>> assert(head != NULL);
>>> assert(a != NULL);
>>> assert(b != NULL);
>>>
>>> if (a == b)
>>> return;
>>
>>top_of_ATOMIC_event:
>>
>>// this is the recovery point is you don't use esmINTERFERENCE()
>>// the very next instruction begins the event.
>
> Yup. Tracking.
>
>>> esmLOCKprefetch( an = esmLOCKload(a->next) );
>>> esmLOCKprefetch( ap = esmLOCKload(a->prev) );
>>> esmLOCKprefetch( bn = esmLOCKload(b->next) );
>>> esmLOCKprefetch( bp = esmLOCKload(b->prev) );
>
> What exactly does this do? This "touches" whatever lines the
> things that `an`, `ap`, `bn`, and `bp` are all in, as well as
> the lines that the `a` and `b` prev and next pointers point
> into as well, right?
>
>> Node *Ehead = esmLOCKload(*head);
>
> You shoulda just used the `hp` local I left for you for exactly
> this. :-D
>
>>// by placing all the the touching before any manifestation, you put
>>// all the touch latency* in the code before it has tried to damage any
>>// participating memory location. (*) and TLB latency and 2nd party
>>// observation of your event.
>>
>>// this would be the point where you would insert if( esmINTERFERENCE(
>>))
>>// if you wanted control at a known failure point rather than at the
>>// top of the event on failure.
>>
>>> if (Ehead == a)
>>> *head = b;
>>> else if (Ehead == b)
>>> *head = a;
>>>
>>> b->next = an;
>>> if (an != NULL) {
>>> an->prev = b;
>>> }
>>> b->prev = ap;
>>> if (ap != NULL) {
>>> ap->next = b;
>>> }
>>>
>>> a->next = bn;
>>> if (bn != NULL) {
>>> bn->prev = a;
>>> }
>>> if (bp != NULL) {
>>> bp->next = a;
>>> }
>>> esmLOCKstore(a->prev, bp);
>>> }
>>
>>// now manifestation has lowest possible latency (as seen by this core
>>alone)
>
> Doesn't this code now assume that `an->prev` is in the same
> cache line as `an`, and similarly that `bn` is in the same line
> as `bn->prev`? Absent an alignment constraint on the `Node`
> type, that's not guaranteed given the definition posted earlier.
Everything you state is true, I just tried to move the esmLOCKxxxx
up to the setup phase to obey ESM rules.
> That may be an odd and ill-advised thing for a programmer to do
> if they want their list type to work with atomic events, but it
> is possible.
>
> The node pointers and their respective `next` pointers are okay,
> so I wonder if perhaps this might have been written as:
>
> void
> swap_places(Node **head, Node *a, Node *b)
> {
> Node *hp, *an, *ap, *bn, *bp;
>
> assert(head != NULL);
> assert(a != NULL);
> assert(b != NULL);
>
> if (a == b)
> return;
>
> hp = esmLOCKload(*head);
> esmLOCKprefetch(an = esmLOCKload(a->next));
> ap = esmLOCKload(a->prev);
> esmLOCKprefetch(bn = esmLOCKload(b->next));
> bp = esmLOCKload(b->prev);
>
> if (an != NULL) // I see what you did
> esmLOCKprefetch(an->prev);
> if (bn != NULL) {
> esmLOCKprefetch(bn->prev);
> bn->prev = a;
> }
>
> if (hp == a)
> *head = b;
> else if (hp == b)
> *head = a;
>
> b->next = an;
> if (an != NULL)
> an->prev = b;
> b->prev = ap;
> if (ap != NULL)
> ap->next = b;
> // illustrative code
> a->next = bn; // ST Rbp,[Ra,#next]
> if (bp != NULL) // PNE0 Rbp,T
> bp->next = a; // ST Ra,[Rbp,#next]
>
> esmLOCKstore(a->prev, bp);
> }
>
> But now the conditional testing whether or not `an` is `NULL` is
> repeated. Is the additional branch overhead worth it here?
In My 66000 ISA, a compare against zero (or NULL) is just a branch
instruction, so the CMP zero is performed twice, but each use is
but a single Branch-on-condition instruction (or Predicate-on-
Condition instruction).
In the case of using predicates, FETCH/DECODE will simple issue
both then and else clauses into the execution window (else-clause
is empty here) and let the reservation stations handle execution
order. And the condition latency is purely the register dependence
chain. A 6-wide machine should have no trouble in inserting two
copies of the code commented by "illustrative code" above--in
this case 6-instructions or 2 sets of {ST, PNE0, ST}.
In the case of using a real branch, latency per use is likely to
be 2-cycles, moderated by typical branch prediction. The condition
will have resolved early, so we are simply FETCH/DECODE/TAKE bound.
{{That is: PRED should be faster in almost all conceivable cases.}}
> - Dan C.
Generally when I write queue-code, I use a dummy Node front/rear
such that the checks for Null are unnecessary (at the cost of
following 1 more ->next or ->prev). That is Q->head and Q->tail
are never NULL and when the queue is empty there is a Node which
carries the fact the queue is empty (not using NULL as a pointer).
But that is just my style.
[toc] | [prev] | [next] | [standalone]
| From | Stefan Monnier <monnier@iro.umontreal.ca> |
|---|---|
| Date | 2025-03-28 11:41 -0400 |
| Message-ID | <jwvpli1xiw2.fsf-monnier+comp.arch@gnu.org> |
| In reply to | #111363 |
> Generally when I write queue-code, I use a dummy Node front/rear
> such that the checks for Null are unnecessary (at the cost of
> following 1 more ->next or ->prev). That is Q->head and Q->tail
> are never NULL and when the queue is empty there is a Node which
> carries the fact the queue is empty (not using NULL as a pointer).
It's called a "sentinel".
Stefan
[toc] | [prev] | [next] | [standalone]
| From | "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> |
|---|---|
| Date | 2025-03-28 09:51 -0700 |
| Message-ID | <vs6k3a$359tg$1@dont-email.me> |
| In reply to | #111363 |
On 3/28/2025 8:24 AM, MitchAlsup1 wrote: > On Fri, 28 Mar 2025 2:53:57 +0000, Dan Cross wrote: [...] > > Generally when I write queue-code, I use a dummy Node front/rear > such that the checks for Null are unnecessary (at the cost of > following 1 more ->next or ->prev). That is Q->head and Q->tail > are never NULL and when the queue is empty there is a Node which > carries the fact the queue is empty (not using NULL as a pointer). > > But that is just my style. Actually, it's a fairly common practice to use a dummy node in lock/wait-free queues. For instance: https://www.cs.rochester.edu/research/synchronization/pseudocode/queues.html In fact, wrt the C++20 futex stack code I posted to this group uses a dummy address: #define CT_WAIT ((ct_node*)0xDEADBEEF) To define a wait state for the C++20 futex to use wrt avoiding slow-paths. However, this scares me a bit because if the user allocates a node that has that exact address, it will break the algorithm. So, I really should create a dummy node and use that for the wait condition. Have you seen that post "Futex Stack..."?
[toc] | [prev] | [next] | [standalone]
| From | cross@spitfire.i.gajendra.net (Dan Cross) |
|---|---|
| Date | 2025-03-29 14:28 +0000 |
| Message-ID | <vs901i$f7e$1@reader1.panix.com> |
| In reply to | #111363 |
In article <cb049d5490b541878e264cedf95168e1@www.novabbs.org>,
MitchAlsup1 <mitchalsup@aol.com> wrote:
>On Fri, 28 Mar 2025 2:53:57 +0000, Dan Cross wrote:
>> [snip]
>> What I really wanted was an example that exceeded that limit as
>> an expository vehicle for understanding what happens when the
>> limit is exceeded. What does the hardware do in that case?
>
>Raise OPERATION (instruction) Fault.
Ok, very good. The software implications could be profound.
>>>[snip]
>>>// by placing all the the touching before any manifestation, you put
>>>// all the touch latency* in the code before it has tried to damage any
>>>// participating memory location. (*) and TLB latency and 2nd party
>>>// observation of your event.
>>>
>>>// this would be the point where you would insert if( esmINTERFERENCE(
>>>))
>>>// if you wanted control at a known failure point rather than at the
>>>// top of the event on failure.
>>>
>>>> if (Ehead == a)
>>>> *head = b;
>>>> else if (Ehead == b)
>>>> *head = a;
>>>>
>>>> b->next = an;
>>>> if (an != NULL) {
>>>> an->prev = b;
>>>> }
>>>> b->prev = ap;
>>>> if (ap != NULL) {
>>>> ap->next = b;
>>>> }
>>>>
>>>> a->next = bn;
>>>> if (bn != NULL) {
>>>> bn->prev = a;
>>>> }
>>>> if (bp != NULL) {
>>>> bp->next = a;
>>>> }
>>>> esmLOCKstore(a->prev, bp);
>>>> }
>>>
>>>// now manifestation has lowest possible latency (as seen by this core
>>>alone)
>>
>> Doesn't this code now assume that `an->prev` is in the same
>> cache line as `an`, and similarly that `bn` is in the same line
>> as `bn->prev`? Absent an alignment constraint on the `Node`
>> type, that's not guaranteed given the definition posted earlier.
>
>Everything you state is true, I just tried to move the esmLOCKxxxx
>up to the setup phase to obey ESM rules.
Yeah, that makes sense. Describing it as a rule, however,
raises the question of whether one must touch a line before a
store to a different line, or just before a store to that line?
>> That may be an odd and ill-advised thing for a programmer to do
>> if they want their list type to work with atomic events, but it
>> is possible.
>>
>> The node pointers and their respective `next` pointers are okay,
>> so I wonder if perhaps this might have been written as:
>>
>> void
>> swap_places(Node **head, Node *a, Node *b)
>> {
>> Node *hp, *an, *ap, *bn, *bp;
>>
>> assert(head != NULL);
>> assert(a != NULL);
>> assert(b != NULL);
>>
>> if (a == b)
>> return;
>>
>> hp = esmLOCKload(*head);
>> esmLOCKprefetch(an = esmLOCKload(a->next));
>> ap = esmLOCKload(a->prev);
>> esmLOCKprefetch(bn = esmLOCKload(b->next));
>> bp = esmLOCKload(b->prev);
>>
>> if (an != NULL) // I see what you did
>> esmLOCKprefetch(an->prev);
>> if (bn != NULL) {
>> esmLOCKprefetch(bn->prev);
>> bn->prev = a;
>> }
>>
>> if (hp == a)
>> *head = b;
>> else if (hp == b)
>> *head = a;
>>
>> b->next = an;
>> if (an != NULL)
>> an->prev = b;
>> b->prev = ap;
>> if (ap != NULL)
>> ap->next = b;
>> // illustrative code
>> a->next = bn; // ST Rbp,[Ra,#next]
>> if (bp != NULL) // PNE0 Rbp,T
>> bp->next = a; // ST Ra,[Rbp,#next]
>>
>> esmLOCKstore(a->prev, bp);
>> }
>>
>> But now the conditional testing whether or not `an` is `NULL` is
>> repeated. Is the additional branch overhead worth it here?
>
>In My 66000 ISA, a compare against zero (or NULL) is just a branch
>instruction, so the CMP zero is performed twice, but each use is
>but a single Branch-on-condition instruction (or Predicate-on-
>Condition instruction).
>
>In the case of using predicates, FETCH/DECODE will simple issue
>both then and else clauses into the execution window (else-clause
>is empty here) and let the reservation stations handle execution
>order. And the condition latency is purely the register dependence
>chain. A 6-wide machine should have no trouble in inserting two
>copies of the code commented by "illustrative code" above--in
>this case 6-instructions or 2 sets of {ST, PNE0, ST}.
>
>In the case of using a real branch, latency per use is likely to
>be 2-cycles, moderated by typical branch prediction. The condition
>will have resolved early, so we are simply FETCH/DECODE/TAKE bound.
>
>{{That is: PRED should be faster in almost all conceivable cases.}}
Okay, that all makes sense. Thanks for the detailed
explanation. I agree it's very slick; is this documented
somewhere publicly? If not, why not?
>Generally when I write queue-code, I use a dummy Node front/rear
>such that the checks for Null are unnecessary (at the cost of
>following 1 more ->next or ->prev). That is Q->head and Q->tail
>are never NULL and when the queue is empty there is a Node which
>carries the fact the queue is empty (not using NULL as a pointer).
>
>But that is just my style.
Ok.
- Dan C.
[toc] | [prev] | [next] | [standalone]
| From | mitchalsup@aol.com (MitchAlsup1) |
|---|---|
| Date | 2025-03-29 15:23 +0000 |
| Message-ID | <1b9a9644c3f5cbd2985b89443041e01a@www.novabbs.org> |
| In reply to | #111369 |
On Sat, 29 Mar 2025 14:28:02 +0000, Dan Cross wrote:
> In article <cb049d5490b541878e264cedf95168e1@www.novabbs.org>,
> MitchAlsup1 <mitchalsup@aol.com> wrote:
>>On Fri, 28 Mar 2025 2:53:57 +0000, Dan Cross wrote:
>>> [snip]
>>> What I really wanted was an example that exceeded that limit as
>>> an expository vehicle for understanding what happens when the
>>> limit is exceeded. What does the hardware do in that case?
>>
>>Raise OPERATION (instruction) Fault.
>
> Ok, very good. The software implications could be profound.
>
>>>>[snip]---------
>>>
>>> Doesn't this code now assume that `an->prev` is in the same
>>> cache line as `an`, and similarly that `bn` is in the same line
>>> as `bn->prev`? Absent an alignment constraint on the `Node`
>>> type, that's not guaranteed given the definition posted earlier.
>>
>>Everything you state is true, I just tried to move the esmLOCKxxxx
>>up to the setup phase to obey ESM rules.
>
> Yeah, that makes sense. Describing it as a rule, however,
> raises the question of whether one must touch a line before a
> store to a different line, or just before a store to that line?
Right from the spec::
"There is an order of instructions imposed upon software <i.e.,
compiler> where:
• all participating inbound memory reference instructions shall be
performed prior to manifestation;
• non-participating inbound memory reference instructions should be
performed prior to manifestation;
• the <optional> query instruction denotes the boundary between setup
and manifestation;
• the first Outbound to a participating cache line <also> begins
manifestation
• the only Outbound with Lock bit set (L ≡ 1) completes the event
The processor monitors the instruction sequence of the ATOMIC event and
will raise the OPERATION exception when the imposed order is violated."
Basically, all participating page-faults happen prior to any
modification
attempts.
>
>>> That may be an odd and ill-advised thing for a programmer to do
>>> if they want their list type to work with atomic events, but it
>>> is possible.
>>>
>>> The node pointers and their respective `next` pointers are okay,
>>> so I wonder if perhaps this might have been written as:
>>>
>>> void
>>> swap_places(Node **head, Node *a, Node *b)
>>> {
>>> Node *hp, *an, *ap, *bn, *bp;
>>>
>>> assert(head != NULL);
>>> assert(a != NULL);
>>> assert(b != NULL);
>>>
>>> if (a == b)
>>> return;
>>>
>>> hp = esmLOCKload(*head);
>>> esmLOCKprefetch(an = esmLOCKload(a->next));
>>> ap = esmLOCKload(a->prev);
>>> esmLOCKprefetch(bn = esmLOCKload(b->next));
>>> bp = esmLOCKload(b->prev);
>>>
>>> if (an != NULL) // I see what you did
>>> esmLOCKprefetch(an->prev);
>>> if (bn != NULL) {
>>> esmLOCKprefetch(bn->prev);
>>> bn->prev = a;
>>> }
>>>
>>> if (hp == a)
>>> *head = b;
>>> else if (hp == b)
>>> *head = a;
>>>
>>> b->next = an;
>>> if (an != NULL)
>>> an->prev = b;
>>> b->prev = ap;
>>> if (ap != NULL)
>>> ap->next = b;
>>> // illustrative code
>>> a->next = bn; // ST Rbp,[Ra,#next]
>>> if (bp != NULL) // PNE0 Rbp,T
>>> bp->next = a; // ST Ra,[Rbp,#next]
>>>
>>> esmLOCKstore(a->prev, bp);
>>> }
>>>
>>> But now the conditional testing whether or not `an` is `NULL` is
>>> repeated. Is the additional branch overhead worth it here?
>>
>>In My 66000 ISA, a compare against zero (or NULL) is just a branch
>>instruction, so the CMP zero is performed twice, but each use is
>>but a single Branch-on-condition instruction (or Predicate-on-
>>Condition instruction).
>>
>>In the case of using predicates, FETCH/DECODE will simple issue
>>both then and else clauses into the execution window (else-clause
>>is empty here) and let the reservation stations handle execution
>>order. And the condition latency is purely the register dependence
>>chain. A 6-wide machine should have no trouble in inserting two
>>copies of the code commented by "illustrative code" above--in
>>this case 6-instructions or 2 sets of {ST, PNE0, ST}.
>>
>>In the case of using a real branch, latency per use is likely to
>>be 2-cycles, moderated by typical branch prediction. The condition
>>will have resolved early, so we are simply FETCH/DECODE/TAKE bound.
>>
>>{{That is: PRED should be faster in almost all conceivable cases.}}
>
> Okay, that all makes sense. Thanks for the detailed
> explanation. I agree it's very slick; is this documented
> somewhere publicly? If not, why not?
Documented: Yes 21 pages.
Public: I keep holding back as if I were to attempt to patent certain
aspects. But I don't seem to have the energy to do such. I as if you
wanted to see it.
>>Generally when I write queue-code, I use a dummy Node front/rear
>>such that the checks for Null are unnecessary (at the cost of
>>following 1 more ->next or ->prev). That is Q->head and Q->tail
>>are never NULL and when the queue is empty there is a Node which
>>carries the fact the queue is empty (not using NULL as a pointer).
>>
>>But that is just my style.
>
> Ok.
>
> - Dan C.
[toc] | [prev] | [next] | [standalone]
| From | cross@spitfire.i.gajendra.net (Dan Cross) |
|---|---|
| Date | 2025-03-29 20:58 +0000 |
| Message-ID | <vs9mu4$79f$2@reader1.panix.com> |
| In reply to | #111371 |
In article <1b9a9644c3f5cbd2985b89443041e01a@www.novabbs.org>,
MitchAlsup1 <mitchalsup@aol.com> wrote:
>On Sat, 29 Mar 2025 14:28:02 +0000, Dan Cross wrote:
>> In article <cb049d5490b541878e264cedf95168e1@www.novabbs.org>,
>> MitchAlsup1 <mitchalsup@aol.com> wrote:
>>>On Fri, 28 Mar 2025 2:53:57 +0000, Dan Cross wrote:
>>>> [snip]
>>>> What I really wanted was an example that exceeded that limit as
>>>> an expository vehicle for understanding what happens when the
>>>> limit is exceeded. What does the hardware do in that case?
>>>
>>>Raise OPERATION (instruction) Fault.
>>
>> Ok, very good. The software implications could be profound.
>>
>>>>>[snip]---------
>>>>
>>>> Doesn't this code now assume that `an->prev` is in the same
>>>> cache line as `an`, and similarly that `bn` is in the same line
>>>> as `bn->prev`? Absent an alignment constraint on the `Node`
>>>> type, that's not guaranteed given the definition posted earlier.
>>>
>>>Everything you state is true, I just tried to move the esmLOCKxxxx
>>>up to the setup phase to obey ESM rules.
>>
>> Yeah, that makes sense. Describing it as a rule, however,
>> raises the question of whether one must touch a line before a
>> store to a different line, or just before a store to that line?
>
>Right from the spec::
>
>"There is an order of instructions imposed upon software <i.e.,
>compiler> where:
>
>• all participating inbound memory reference instructions shall be
>performed prior to manifestation;
>• non-participating inbound memory reference instructions should be
>performed prior to manifestation;
>• the <optional> query instruction denotes the boundary between setup
>and manifestation;
>• the first Outbound to a participating cache line <also> begins
>manifestation
>• the only Outbound with Lock bit set (L ≡ 1) completes the event
>
>The processor monitors the instruction sequence of the ATOMIC event and
>will raise the OPERATION exception when the imposed order is violated."
>
>Basically, all participating page-faults happen prior to any
>modification
>attempts.
Ok.
>>>> That may be an odd and ill-advised thing for a programmer to do
>>>> if they want their list type to work with atomic events, but it
>>>> is possible.
>>>>
>>>> The node pointers and their respective `next` pointers are okay,
>>>> so I wonder if perhaps this might have been written as:
>>>>
>>>> void
>>>> swap_places(Node **head, Node *a, Node *b)
>>>> {
>>>> Node *hp, *an, *ap, *bn, *bp;
>>>>
>>>> assert(head != NULL);
>>>> assert(a != NULL);
>>>> assert(b != NULL);
>>>>
>>>> if (a == b)
>>>> return;
>>>>
>>>> hp = esmLOCKload(*head);
>>>> esmLOCKprefetch(an = esmLOCKload(a->next));
>>>> ap = esmLOCKload(a->prev);
>>>> esmLOCKprefetch(bn = esmLOCKload(b->next));
>>>> bp = esmLOCKload(b->prev);
>>>>
>>>> if (an != NULL) // I see what you did
>>>> esmLOCKprefetch(an->prev);
>>>> if (bn != NULL) {
>>>> esmLOCKprefetch(bn->prev);
>>>> bn->prev = a;
>>>> }
>>>>
>>>> if (hp == a)
>>>> *head = b;
>>>> else if (hp == b)
>>>> *head = a;
>>>>
>>>> b->next = an;
>>>> if (an != NULL)
>>>> an->prev = b;
>>>> b->prev = ap;
>>>> if (ap != NULL)
>>>> ap->next = b;
>>>> // illustrative code
>>>> a->next = bn; // ST Rbp,[Ra,#next]
>>>> if (bp != NULL) // PNE0 Rbp,T
>>>> bp->next = a; // ST Ra,[Rbp,#next]
>>>>
>>>> esmLOCKstore(a->prev, bp);
>>>> }
>>>>
>>>> But now the conditional testing whether or not `an` is `NULL` is
>>>> repeated. Is the additional branch overhead worth it here?
>>>
>>>In My 66000 ISA, a compare against zero (or NULL) is just a branch
>>>instruction, so the CMP zero is performed twice, but each use is
>>>but a single Branch-on-condition instruction (or Predicate-on-
>>>Condition instruction).
>>>
>>>In the case of using predicates, FETCH/DECODE will simple issue
>>>both then and else clauses into the execution window (else-clause
>>>is empty here) and let the reservation stations handle execution
>>>order. And the condition latency is purely the register dependence
>>>chain. A 6-wide machine should have no trouble in inserting two
>>>copies of the code commented by "illustrative code" above--in
>>>this case 6-instructions or 2 sets of {ST, PNE0, ST}.
>>>
>>>In the case of using a real branch, latency per use is likely to
>>>be 2-cycles, moderated by typical branch prediction. The condition
>>>will have resolved early, so we are simply FETCH/DECODE/TAKE bound.
>>>
>>>{{That is: PRED should be faster in almost all conceivable cases.}}
>>
>> Okay, that all makes sense. Thanks for the detailed
>> explanation. I agree it's very slick; is this documented
>> somewhere publicly? If not, why not?
>
>Documented: Yes 21 pages.
>Public: I keep holding back as if I were to attempt to patent certain
>aspects. But I don't seem to have the energy to do such. I as if you
>wanted to see it.
(I assume you meant "ask if you want to see it"?)
Sure, I think it would be interesting. Should I send you an
email?
- Dan C.
[toc] | [prev] | [next] | [standalone]
| From | mitchalsup@aol.com (MitchAlsup1) |
|---|---|
| Date | 2025-03-29 22:04 +0000 |
| Message-ID | <b247ac1444c0cfb4c79cf3089b4bab3e@www.novabbs.org> |
| In reply to | #111373 |
On Sat, 29 Mar 2025 20:58:44 +0000, Dan Cross wrote: > In article <1b9a9644c3f5cbd2985b89443041e01a@www.novabbs.org>, > MitchAlsup1 <mitchalsup@aol.com> wrote: ------------ >>> Okay, that all makes sense. Thanks for the detailed >>> explanation. I agree it's very slick; is this documented >>> somewhere publicly? If not, why not? >> >>Documented: Yes 21 pages. >>Public: I keep holding back as if I were to attempt to patent certain >>aspects. But I don't seem to have the energy to do such. I as if you >>wanted to see it. > > (I assume you meant "ask if you want to see it"?) > > Sure, I think it would be interesting. Should I send you an > email? Yes, exactly. > - Dan C.
[toc] | [prev] | [next] | [standalone]
| From | EricP <ThatWouldBeTelling@thevillage.com> |
|---|---|
| Date | 2025-03-25 10:54 -0400 |
| Message-ID | <8uzEP.115550$Xq5f.66883@fx38.iad> |
| In reply to | #111277 |
Dan Cross wrote: > In article <kVgEP.1277108$_N6e.605199@fx17.iad>, > EricP <ThatWouldBeTelling@thevillage.com> wrote: >> Dan Cross wrote: >>> In article <fe9715fa347144df1e584463375107cf@www.novabbs.org>, >>> MitchAlsup1 <mitchalsup@aol.com> wrote: >>>> On Thu, 20 Mar 2025 12:44:08 +0000, Dan Cross wrote: >>>>> Sometimes you really don't want to be interrupted. >>>> And sometimes you don't want to be interrupted unless the >>>> "house is on fire"; I cannot see a time when "the house is >>>> on fire" that you would not want to take the interrupt. >>>> >>>> Is there one ?!? >>> Consider a thread that takes a spinlock; suppose some >>> high-priority interrupt comes in while the thread is holding >>> that lock. In response to the interrupt, software decides to >>> suspend the thread and switch some other thread; that thread >>> wants to lock the spin lock that the now-descheduled thread is >>> holding: a classic deadlock scenario. >> Terminology: mutexes coordinate mutual exclusion between threads, >> spinlocks coordinate mutual exclusion between cpu cores. >> Windows "critical sections" are mutexes with a fast path. > > A spin lock is simply any lock where you spin trying to acquire > the lock, as opposed to a blocking synchronization protocol. > Here I'm using the terminology of Herlihy and Shavit [Her08]. I'm using terminology in common use on multiprocessor systems like VMS since 1980's and later WinNT. Each OS data structure has different reentrancy requirements. - Ones accessed by threads are guarded by mutexes. Example on Windows would be the page tables as paging is done by threads. - Ones accessed by OS software interrupt level are guarded by spinlocks. Example on Windows is the Scheduler tables. - Ones accessed by HW interrupts are guarded by interrupt spinlocks. Example on Windows is device interrupt service routines. > Traditional Unix "sleep" and "wakeup" are an example of a > blocking protocol, where a thread may "sleep" on some "channel", > yielding the locking thread while the lock cannot be acquired, > presumably scheduling something else until the thread is later > marked runnable by virtual of something else calling "wakeup" on > the sleep channel. > > But note that I am deliberately simplifying in order to > construct a particular scenario in which a light-weight > synchronization primitive is being used for mutual exclusion > between concurrent entities, hence mentioning spinlocks > specifically. > > Suggesting that spin locks iare only applicable to > multiprocessing scenarios is flatly incorrect. I didn't. I am saying the OS by design guards *ITS* different data structures with different mechanisms, and those mechanism must only be used in a very controlled manner. And if you want to do something that requires access to one of *ITS* data structures then you must obey *ITS* rules. In Windows and VMS and from what I understand of *nix spinlocks guard most of the core OS data structures with spinlocks acquired at the software interrupt level. If you are not at that level then the spinlock will not provide the necessary synchronization for access to that data. > Software may use > the technique to spin on a "busy" bit on some IP in a device for > example, even on a uniprocessor system. > >>> A valid response here might be, "don't context switch from the >>> interrupt handler; use a DPC instead". That may be valid, but >>> it puts a constraint on the software designer that may be onerus >>> in practice: suppose the interrupt happens in the scheduler, >>> while examining a run queue or something. A DPC object must be >>> available, etc. >> That is exactly what DPC/SoftIrq are design to do - allow the bulk of >> the kernel, like the scheduler, non-paged heap management, IO pre and >> post processing, to be interrupted without causing reentrancy. >> The higher level device interrupt enqueues a request to the lower software >> interrupt level, which is processed when it will not cause reentrancy. >> >> That constraint is by design and any good OS will crash if violated. > > Yes, I'm aware of the technique. That wasn't the point. > >>> Further, software must now consider the complexity of >>> potentially interruptable critical sections. From the >>> standpoint of reasoning about already-complex concurrency issues >>> it's simpler to be able to assert that (almost) all interrupt >>> delivery can be cheaply disabled entirely, save for very >>> special, specific, scenarios like NMIs. Potentially switching >>> away from a thread holding a spinlock sort of defeats the >>> purpose of a spinlock in the first place, which is a mutex >>> primitive designed to avoid the overhead of switching. >> This is why I mentioned the terminology thing: threads do not hold >> spinlocks, they hold mutexes. > > See above. Threads can certainly "hold" a spin lock, as they > can hold any kind of lock. To quote from sec 7.6.1 of [Val96], > page 202: When a particular OS defined spinlock is used to guard a particular OS defined data structure, exactly WHEN the OS allows this decides how the OS avoids all the nasty interrupt pitfalls you identified. > |On a uniprocessor, if a thread tries to acquire a spin lock > |that is already held, it will loop forever. Multiprocessor > |algorithms, however, must operate correctly regardless of the > |number of processors, which means that they should handle the > |uniprocessor case as well. This requires strict adherence to > |the rule that threads not relinquish control of the CPU while > |holding a spin lock. Yes, which is what I was saying. I am describing HOW that is accomplished inside an OS. The way you prevent a thread from relinquishing control while holding a spinlock is to first switch hats from thread level to software interrupt level, which blocks the scheduler because it also runs at SWIL, then acquire the spinlock. Raising the interrupt priority level from passive (aka thread) level to SWI level (called dispatch level on Windows) blocks the thread switch. This ensures non-reentrant algorithms are serialized. The core may then acquire spinlocks guarding data structures. >> Threads and mutexes are a fiction created >> by the OS scheduler, and switching threads while waiting for a mutex >> is exactly what they are supposed to do. >> >> Spinlocks are used by cores to coordinate access to shared memory by >> things like a OS scheduler looking at list of threads waiting for a mutex. > > Spinlocks, of the form I was describing, are an optimization for > cases where the critical section guarded by the lock is short, > and the overhead of invoking a scheduler and context switching > to some other thread is higher than just spinning until the lock > becomes available again. The whole point is to avoid blocking > and switching. > > E.g., as [Vaj11] puts it (page 98): > > |Spin locks have the advantage of preventing pre-emption by the > |OS (due to blocking while waiting for the lock) > > That is, the advantage is because the thread does not block and > yield. And I am describing HOW the OS accomplishes this. Disabling the scheduler is the same as raising the priority to SWI level. Once the scheduler is disabled THEN you may acquire a spinlock that guards a shared data structure accessed at SWI level. But you may NOT access a shared data structure accessed at interrupt level. For that you'd need an interrupt spinlock acquired at the device ISR priority interrupt level. And note that the ISR interrupt priority level is assigned by the OS itself, usually when the device is mounted. >> I like to think of it as all having to do with hats. >> The cpu is wearing one of three hats: a thread hat when it pretends to be >> a time shared execution machine; a core hat when it acts as a single >> execution machine running non-reentrant things like a scheduler which >> creates the fiction of threads and processes (multiple virtual spaces); >> and an interrupt hat when executing nestable reentrant interrupts. >> >> The interrupt mechanism coordinates exactly when and how we switch hats. > > I suppose you are lumping system calls into the overall bucket > of "interrupts" in this model, regardless of whether one might > use a dedicated supervisor call instruction (e.g., something > like x86 SYSENTER or SYSCALL or ARM SVC or RISC-V's ECALL)? Syscall switches a thread from user mode to supervisor mode. > The hat analogy feels strained to me, and too colloquial to be > useful. I offered it because it answers your prior questions about how OS avoid the kinds of deadlocks and race conditions you described. If something you want to do requires a certain hat and you are not wearing that hat then you have to get one. Inside an HW interrupt service routine and want to run a thread? ISR (interrupt hat) posts a Dpc to software interrupt level (SWI hat) Dispatcher, which eventually runs calls a routine that accesses scheduler tables guarded by a spinlock (that requires the SWI hat), that selects your thread so when OS returns (thread hat) your application code executes. > - Dan C. > > References: > > [Her08] Maurice Herlihy and Nir Shavit. 2008. _The Art of > Multiprocessor Programming_. Morgan Kaufmann, Burlington, MA. > > [Vah96] Uresh Vahalia. 1996. _Unix Internals: The New > Frontiers_. Prentice Hall, Upper Saddle River, NJ. > > [Vaj11] Andras Vajda. 2011. _Programming Many-Core Chips_. > Springer, New York, NY.
[toc] | [prev] | [next] | [standalone]
| From | scott@slp53.sl.home (Scott Lurndal) |
|---|---|
| Date | 2025-03-25 16:31 +0000 |
| Message-ID | <0UAEP.1333399$_N6e.909079@fx17.iad> |
| In reply to | #111297 |
EricP <ThatWouldBeTelling@thevillage.com> writes: >Dan Cross wrote: >I didn't. >I am saying the OS by design guards *ITS* different data structures >with different mechanisms, and those mechanism must only be used in >a very controlled manner. And if you want to do something that requires >access to one of *ITS* data structures then you must obey *ITS* rules. > https://en.wikipedia.org/wiki/Incompatible_Timesharing_System
[toc] | [prev] | [next] | [standalone]
| From | cross@spitfire.i.gajendra.net (Dan Cross) |
|---|---|
| Date | 2025-03-25 17:44 +0000 |
| Message-ID | <vruq1p$cv2$1@reader1.panix.com> |
| In reply to | #111297 |
In article <8uzEP.115550$Xq5f.66883@fx38.iad>, EricP <ThatWouldBeTelling@thevillage.com> wrote: >Dan Cross wrote: >> In article <kVgEP.1277108$_N6e.605199@fx17.iad>, >> EricP <ThatWouldBeTelling@thevillage.com> wrote: >>> Dan Cross wrote: >>>> In article <fe9715fa347144df1e584463375107cf@www.novabbs.org>, >>>> MitchAlsup1 <mitchalsup@aol.com> wrote: >>>>> On Thu, 20 Mar 2025 12:44:08 +0000, Dan Cross wrote: >>>>>> Sometimes you really don't want to be interrupted. >>>>> And sometimes you don't want to be interrupted unless the >>>>> "house is on fire"; I cannot see a time when "the house is >>>>> on fire" that you would not want to take the interrupt. >>>>> >>>>> Is there one ?!? >>>> Consider a thread that takes a spinlock; suppose some >>>> high-priority interrupt comes in while the thread is holding >>>> that lock. In response to the interrupt, software decides to >>>> suspend the thread and switch some other thread; that thread >>>> wants to lock the spin lock that the now-descheduled thread is >>>> holding: a classic deadlock scenario. >>> >>> Terminology: mutexes coordinate mutual exclusion between threads, >>> spinlocks coordinate mutual exclusion between cpu cores. >>> Windows "critical sections" are mutexes with a fast path. >> >> A spin lock is simply any lock where you spin trying to acquire >> the lock, as opposed to a blocking synchronization protocol. >> Here I'm using the terminology of Herlihy and Shavit [Her08]. > >I'm using terminology in common use on multiprocessor systems like >VMS since 1980's and later WinNT. > >Each OS data structure has different reentrancy requirements. >- Ones accessed by threads are guarded by mutexes. > Example on Windows would be the page tables as paging is done by threads. >- Ones accessed by OS software interrupt level are guarded by spinlocks. > Example on Windows is the Scheduler tables. >- Ones accessed by HW interrupts are guarded by interrupt spinlocks. > Example on Windows is device interrupt service routines. I'm not sure what you're saying here, or rather, how it is relevant. Your earlier statement was that "mutexes coordinate mutual exclusion between threads, spinlocks coordinate mutual exclusion between cpu cores." But this is not the case. Taking VMS as an example, from [Gol94], sec 9.5 (page 255): |A thread of execution running on one processor acquires a |spinlock to serialize access to the data with threads of |execution running on other processors. Before acquiring |the spinlock, the thread of executions raises IPL to block |accesses by other threads of execution running on the same |processor. The IPL value is determined by the spinlock being |locked. Goldberg and Saravanan seems quite explicit that threads can acquire and hold spinlocks on OpenVMS, which seems to be something that you were objecting to? Turning to Windows, [Rus12] says about spin locks on page 179: |Testing and acquiring the lock in one instruction prevents a |second thread from grabbing the lock between the time the first |thread tests the variable and the time it acquires the lock. Note that, again, the language here is thread-centric. By the way, regarding terminology, on page 177 [Rus12] also says, |Sections of code that access a nonshareable resource are called |_critical_ sections. To ensure correct code, only one thread |at a time can execute in a critical section. This is the generic definition of a critical section and note that this is the context in which we are using the term throughout this thread. What you referred to earlier as a "critical section" in Windows is a _userspace_ synchronization primitive with a kernel assist and mostly unrelated to concerns around interruptibility. From the description on page 201 of [Rus12], they seem similar to futexes on Linux, though somewhat more limited: Russinovich et al states that they cannot be used between processes, while a Linux futex can if it's in a region of memory shared between cooperating processes. >> Traditional Unix "sleep" and "wakeup" are an example of a >> blocking protocol, where a thread may "sleep" on some "channel", >> yielding the locking thread while the lock cannot be acquired, >> presumably scheduling something else until the thread is later >> marked runnable by virtual of something else calling "wakeup" on >> the sleep channel. >> >> But note that I am deliberately simplifying in order to >> construct a particular scenario in which a light-weight >> synchronization primitive is being used for mutual exclusion >> between concurrent entities, hence mentioning spinlocks >> specifically. >> >> Suggesting that spin locks iare only applicable to >> multiprocessing scenarios is flatly incorrect. > >I didn't. Your exacty words were, "spinlocks coordinate mutual exclusion between cpu cores." See the text quoted above. This is incorrect in that it ignores that threads are, in contxt, the software primitive for the computational entities that are doing the acquiring, and while the "between cpu core" part is the usual case, it is not the only case. >I am saying the OS by design guards *ITS* different data structures >with different mechanisms, and those mechanism must only be used in >a very controlled manner. And if you want to do something that requires >access to one of *ITS* data structures then you must obey *ITS* rules. This is axiomatic. But I fail to see how it's relevant to the topic at hand. In context we're discussing scenarios where a system may want to disable interrupts entirely, and whether a particular aspect of a hardware design is sufficient to implement critical sections generally. >In Windows and VMS and from what I understand of *nix spinlocks guard most >of the core OS data structures with spinlocks acquired at the software >interrupt level. If you are not at that level then the spinlock will not >provide the necessary synchronization for access to that data. Precisely. >> Software may use >> the technique to spin on a "busy" bit on some IP in a device for >> example, even on a uniprocessor system. >> >>>> A valid response here might be, "don't context switch from the >>>> interrupt handler; use a DPC instead". That may be valid, but >>>> it puts a constraint on the software designer that may be onerus >>>> in practice: suppose the interrupt happens in the scheduler, >>>> while examining a run queue or something. A DPC object must be >>>> available, etc. >>> That is exactly what DPC/SoftIrq are design to do - allow the bulk of >>> the kernel, like the scheduler, non-paged heap management, IO pre and >>> post processing, to be interrupted without causing reentrancy. >>> The higher level device interrupt enqueues a request to the lower software >>> interrupt level, which is processed when it will not cause reentrancy. >>> >>> That constraint is by design and any good OS will crash if violated. >> >> Yes, I'm aware of the technique. That wasn't the point. >> >>>> Further, software must now consider the complexity of >>>> potentially interruptable critical sections. From the >>>> standpoint of reasoning about already-complex concurrency issues >>>> it's simpler to be able to assert that (almost) all interrupt >>>> delivery can be cheaply disabled entirely, save for very >>>> special, specific, scenarios like NMIs. Potentially switching >>>> away from a thread holding a spinlock sort of defeats the >>>> purpose of a spinlock in the first place, which is a mutex >>>> primitive designed to avoid the overhead of switching. (It occurs to me that perhaps this is why you are going on about how VMS and windows uses spin locks, and how they interact with interrupt priority levels.) >>> This is why I mentioned the terminology thing: threads do not hold >>> spinlocks, they hold mutexes. >> >> See above. Threads can certainly "hold" a spin lock, as they >> can hold any kind of lock. To quote from sec 7.6.1 of [Val96], >> page 202: > >When a particular OS defined spinlock is used to guard a particular OS >defined data structure, exactly WHEN the OS allows this decides how >the OS avoids all the nasty interrupt pitfalls you identified. How is this relevant to what you wrote earlier and my response, quoted just above? My statement was that threads can hold spin locks. This is obviously true. >> |On a uniprocessor, if a thread tries to acquire a spin lock >> |that is already held, it will loop forever. Multiprocessor >> |algorithms, however, must operate correctly regardless of the >> |number of processors, which means that they should handle the >> |uniprocessor case as well. This requires strict adherence to >> |the rule that threads not relinquish control of the CPU while >> |holding a spin lock. > >Yes, which is what I was saying. >I am describing HOW that is accomplished inside an OS. Why? First, it's well-known, and second, that's not relevant to the discussion at hand. But it also doesn't seem like that's what you're saying; you asserted that a thread (a software thread in this context) does not "hold" a spin lock; that's incorrect. >The way you prevent a thread from relinquishing control while holding a >spinlock is to first switch hats from thread level to software interrupt >level, which blocks the scheduler because it also runs at SWIL, >then acquire the spinlock. > >Raising the interrupt priority level from passive (aka thread) level >to SWI level (called dispatch level on Windows) blocks the thread switch. >This ensures non-reentrant algorithms are serialized. >The core may then acquire spinlocks guarding data structures. Just so. Earlier you said that threads cannot hold spin locks; in the text quoted just above, you said that they do. As for this specific scenario of specific interrupt priority levels, I would go further and say that is just a detail. In the simplest systems, ie, the kind we might reach for as an expository example when discussing these kinds of issues, one just disables interrupts completely when holding a spinlock. >>> Threads and mutexes are a fiction created >>> by the OS scheduler, and switching threads while waiting for a mutex >>> is exactly what they are supposed to do. >>> >>> Spinlocks are used by cores to coordinate access to shared memory by >>> things like a OS scheduler looking at list of threads waiting for a mutex. >> >> Spinlocks, of the form I was describing, are an optimization for >> cases where the critical section guarded by the lock is short, >> and the overhead of invoking a scheduler and context switching >> to some other thread is higher than just spinning until the lock >> becomes available again. The whole point is to avoid blocking >> and switching. >> >> E.g., as [Vaj11] puts it (page 98): >> >> |Spin locks have the advantage of preventing pre-emption by the >> |OS (due to blocking while waiting for the lock) >> >> That is, the advantage is because the thread does not block and >> yield. > >And I am describing HOW the OS accomplishes this. Well, what you are describing is how _a_ specific family of operating systems that you are familiar with accomplishes this, and that's all well and good, but it's not all that relevant to discussing the pitfalls of this "ATOMIC event" proposal or addressing the OP's questions about scenarios in which interrupt disable is used. >Disabling the scheduler is the same as raising the priority to SWI level. >Once the scheduler is disabled THEN you may acquire a spinlock >that guards a shared data structure accessed at SWI level. > >But you may NOT access a shared data structure accessed at interrupt level. >For that you'd need an interrupt spinlock acquired at the device ISR >priority interrupt level. And note that the ISR interrupt priority level >is assigned by the OS itself, usually when the device is mounted. You are presuming a very specific system architecture and going into great detail about its implementation. That's fine, and perhaps useful I suppose, but again, I don't think it's very relevant to the context of this discussion. >>> I like to think of it as all having to do with hats. >>> The cpu is wearing one of three hats: a thread hat when it pretends to be >>> a time shared execution machine; a core hat when it acts as a single >>> execution machine running non-reentrant things like a scheduler which >>> creates the fiction of threads and processes (multiple virtual spaces); >>> and an interrupt hat when executing nestable reentrant interrupts. >>> >>> The interrupt mechanism coordinates exactly when and how we switch hats. >> >> I suppose you are lumping system calls into the overall bucket >> of "interrupts" in this model, regardless of whether one might >> use a dedicated supervisor call instruction (e.g., something >> like x86 SYSENTER or SYSCALL or ARM SVC or RISC-V's ECALL)? > >Syscall switches a thread from user mode to supervisor mode. Your statement was that "the interrupt mechanism coordinates exactly when and how we switch hats". But that's incomplete; if a system call blocks, for example, the typical behavior is to yield to the scheduler to pick something else to do. Clearly, that is another way that coordinates this business of changing one's hat. >> The hat analogy feels strained to me, and too colloquial to be >> useful. > >I offered it because it answers your prior questions about how >OS avoid the kinds of deadlocks and race conditions you described. I'm afraid you answered a question that I did not ask, then. I am discussing the specifics of this architectural proposal, and specifically two aspects of it: one is addressing the proposer's questions about scenarios where one might want to disable interrupts entirely. Another, which has emerged somewhat more recently in the thread, is this mechanism which seems to be a semi-transactional elaboration of LL-SC primitives. Any questions I did ask were Socratic in nature, and in the context of these architectural discussions. I did not ask any questions about basic design or implementation of synchronization primitives. - Dan C. References: [Gol94] Ruth E. Goldberg and Saro Saravanan. 1994. _OpenVMS AXP Internals and Data Structures_ (Vers. 1.5). Digital Press, Newton, MA. [Rus12] Mark Russinovich, David A. Solomon and Alex Ionescu. 2012. _Windows Internals: Part 1_. Microsoft Press, Redmond, WA.
[toc] | [prev] | [next] | [standalone]
| From | EricP <ThatWouldBeTelling@thevillage.com> |
|---|---|
| Date | 2025-03-28 14:40 -0400 |
| Message-ID | <t3CFP.529522$f81.217510@fx48.iad> |
| In reply to | #111306 |
Dan Cross wrote: > In article <8uzEP.115550$Xq5f.66883@fx38.iad>, > EricP <ThatWouldBeTelling@thevillage.com> wrote: >> Dan Cross wrote: >>> In article <kVgEP.1277108$_N6e.605199@fx17.iad>, >>> EricP <ThatWouldBeTelling@thevillage.com> wrote: >>>> Dan Cross wrote: >>>>> In article <fe9715fa347144df1e584463375107cf@www.novabbs.org>, >>>>> MitchAlsup1 <mitchalsup@aol.com> wrote: >>>>>> On Thu, 20 Mar 2025 12:44:08 +0000, Dan Cross wrote: >>>>>>> Sometimes you really don't want to be interrupted. >>>>>> And sometimes you don't want to be interrupted unless the >>>>>> "house is on fire"; I cannot see a time when "the house is >>>>>> on fire" that you would not want to take the interrupt. >>>>>> >>>>>> Is there one ?!? >>>>> Consider a thread that takes a spinlock; suppose some >>>>> high-priority interrupt comes in while the thread is holding >>>>> that lock. In response to the interrupt, software decides to >>>>> suspend the thread and switch some other thread; that thread >>>>> wants to lock the spin lock that the now-descheduled thread is >>>>> holding: a classic deadlock scenario. >>>> Terminology: mutexes coordinate mutual exclusion between threads, >>>> spinlocks coordinate mutual exclusion between cpu cores. >>>> Windows "critical sections" are mutexes with a fast path. >>> A spin lock is simply any lock where you spin trying to acquire >>> the lock, as opposed to a blocking synchronization protocol. >>> Here I'm using the terminology of Herlihy and Shavit [Her08]. >> I'm using terminology in common use on multiprocessor systems like >> VMS since 1980's and later WinNT. >> >> Each OS data structure has different reentrancy requirements. >> - Ones accessed by threads are guarded by mutexes. >> Example on Windows would be the page tables as paging is done by threads. >> - Ones accessed by OS software interrupt level are guarded by spinlocks. >> Example on Windows is the Scheduler tables. >> - Ones accessed by HW interrupts are guarded by interrupt spinlocks. >> Example on Windows is device interrupt service routines. > > I'm not sure what you're saying here, or rather, how it is > relevant. > > Your earlier statement was that "mutexes coordinate mutual > exclusion between threads, spinlocks coordinate mutual exclusion > between cpu cores." But this is not the case. > > Taking VMS as an example, from [Gol94], sec 9.5 (page 255): > > |A thread of execution running on one processor acquires a > |spinlock to serialize access to the data with threads of > |execution running on other processors. Before acquiring > |the spinlock, the thread of executions raises IPL to block > |accesses by other threads of execution running on the same > |processor. The IPL value is determined by the spinlock being > |locked. > > Goldberg and Saravanan seems quite explicit that threads can > acquire and hold spinlocks on OpenVMS, which seems to be > something that you were objecting to? It took me a while to find copy of that book online. That 1994 book relates to VMS before they added kernel threads to the OS. At that point VMS scheduled processes, an execution context plus virtual space, like most *nix (except I think Solaris which had kernel threads). It says on pg 3: "The term thread of execution as used in this book refers to a computational entity, an agent of execution." This refers generally to the various code sequences from different context that may be intertwined on a processors core for interrupts, exceptions, Dpc (called driver forks in VMS), process AST's (software interrupts), and applications. In a later book on VMS after they added kernel scheduled threads they consistently use the two terms "thread of execution" and "kernel thread" to distinguish the two meanings. I find the phrase "thread of execution" ambiguous and try not use it. After the term thread had been pretty much adopted to mean kernel thread as an OS scheduled execution context, I would have preferred if they used something like "strand of execution" to avoid the overloading. > Turning to Windows, [Rus12] says about spin locks on page 179: > > |Testing and acquiring the lock in one instruction prevents a > |second thread from grabbing the lock between the time the first > |thread tests the variable and the time it acquires the lock. > > Note that, again, the language here is thread-centric. He says on pg 12: "A thread is the entity within a process that Windows schedules for execution." meaning "kernel thread". He also uses the term thread loosely as you can see on pg 13 when he says: "Fibers allow an application to schedule its own “threads” of execution" He also says pg 87 (to paraphrase) that scheduling priority is an attribute of a thread whereas IRQL is an attribute of a processor. Which is the distinction I am trying to make: when you raise IRQL it stops being a thread executing and starts being a processor core. > By the way, regarding terminology, on page 177 [Rus12] also > says, > > |Sections of code that access a nonshareable resource are called > |_critical_ sections. To ensure correct code, only one thread > |at a time can execute in a critical section. > > This is the generic definition of a critical section and note > that this is the context in which we are using the term > throughout this thread. Yes, I was trying to note the ambiguous terminology so we could avoid it. That page is referring to kernel threads as scheduled entities because he says "if the operating system performed a context switch to the second thread...". In a Windows context the term "critical section" means a specific Win32 user mode mechanism used to synchronize access for scheduled threads. As opposed to books on multi-processors often use the term "critical section" in a generic way to mean a shared section of memory accessed by multiple processors, guarded by some form of mutual exclusion, which could apply to shared memory guarded by OS spinlocks but equally to shared application memory with its own synchronization. > What you referred to earlier as a "critical section" in Windows > is a _userspace_ synchronization primitive with a kernel assist > and mostly unrelated to concerns around interruptibility. From > the description on page 201 of [Rus12], they seem similar to > futexes on Linux, though somewhat more limited: Russinovich et > al states that they cannot be used between processes, while a > Linux futex can if it's in a region of memory shared between > cooperating processes. Yes. >>> Traditional Unix "sleep" and "wakeup" are an example of a >>> blocking protocol, where a thread may "sleep" on some "channel", >>> yielding the locking thread while the lock cannot be acquired, >>> presumably scheduling something else until the thread is later >>> marked runnable by virtual of something else calling "wakeup" on >>> the sleep channel. >>> >>> But note that I am deliberately simplifying in order to >>> construct a particular scenario in which a light-weight >>> synchronization primitive is being used for mutual exclusion >>> between concurrent entities, hence mentioning spinlocks >>> specifically. >>> >>> Suggesting that spin locks iare only applicable to >>> multiprocessing scenarios is flatly incorrect. >> I didn't. > > Your exacty words were, "spinlocks coordinate mutual exclusion > between cpu cores." See the text quoted above. This is > incorrect in that it ignores that threads are, in contxt, the > software primitive for the computational entities that are doing > the acquiring, and while the "between cpu core" part is the > usual case, it is not the only case. I believe I'm being consistent and precise. However a *core's* interrupt level gets raised >= Dispatch level, either by an interrupt of a lower priority strand of execution, or if a kernel thread making a SysCall, it shuts off the scheduler. At that point the core is no longer executing as a kernel thread - it is executing as a processor core. If it then acquires an OS spinlock, it is synchronizing between cores. >> I am saying the OS by design guards *ITS* different data structures >> with different mechanisms, and those mechanism must only be used in >> a very controlled manner. And if you want to do something that requires >> access to one of *ITS* data structures then you must obey *ITS* rules. > > This is axiomatic. > > But I fail to see how it's relevant to the topic at hand. In > context we're discussing scenarios where a system may want to > disable interrupts entirely, and whether a particular aspect of > a hardware design is sufficient to implement critical sections > generally. You seemed to be switching between talking about general spinlocks as "any code that spin-waits" whereas I am just referring to OS spinlocks executing at a specific raised IRQL, guarding shared OS data structures. >> In Windows and VMS and from what I understand of *nix spinlocks guard most >> of the core OS data structures with spinlocks acquired at the software >> interrupt level. If you are not at that level then the spinlock will not >> provide the necessary synchronization for access to that data. > > Precisely. Yeah. We are synchonized! >>> Software may use >>> the technique to spin on a "busy" bit on some IP in a device for >>> example, even on a uniprocessor system. >>> >>>>> A valid response here might be, "don't context switch from the >>>>> interrupt handler; use a DPC instead". That may be valid, but >>>>> it puts a constraint on the software designer that may be onerus >>>>> in practice: suppose the interrupt happens in the scheduler, >>>>> while examining a run queue or something. A DPC object must be >>>>> available, etc. >>>> That is exactly what DPC/SoftIrq are design to do - allow the bulk of >>>> the kernel, like the scheduler, non-paged heap management, IO pre and >>>> post processing, to be interrupted without causing reentrancy. >>>> The higher level device interrupt enqueues a request to the lower software >>>> interrupt level, which is processed when it will not cause reentrancy. >>>> >>>> That constraint is by design and any good OS will crash if violated. >>> Yes, I'm aware of the technique. That wasn't the point. >>> >>>>> Further, software must now consider the complexity of >>>>> potentially interruptable critical sections. From the >>>>> standpoint of reasoning about already-complex concurrency issues >>>>> it's simpler to be able to assert that (almost) all interrupt >>>>> delivery can be cheaply disabled entirely, save for very >>>>> special, specific, scenarios like NMIs. Potentially switching >>>>> away from a thread holding a spinlock sort of defeats the >>>>> purpose of a spinlock in the first place, which is a mutex >>>>> primitive designed to avoid the overhead of switching. > > (It occurs to me that perhaps this is why you are going on about > how VMS and windows uses spin locks, and how they interact with > interrupt priority levels.) I find that because WinNT was a clean-sheet design, much of its terminology for things like threads, processes, mutexes, Dpc's, is less ambiguous, particularly when talking about interrupts and scheduling. Whereas with VMS and *nix that terminology can get messy because they started out with "process" as the unit of scheduled execution and later glued threads on, and then Unix glued threads on as a new flavor of forking that doesn't really fork, and Linux with its interrupt handling history of back half's vs tasklettes vs SoftIrq. >>>> This is why I mentioned the terminology thing: threads do not hold >>>> spinlocks, they hold mutexes. >>> See above. Threads can certainly "hold" a spin lock, as they >>> can hold any kind of lock. To quote from sec 7.6.1 of [Val96], >>> page 202: >> When a particular OS defined spinlock is used to guard a particular OS >> defined data structure, exactly WHEN the OS allows this decides how >> the OS avoids all the nasty interrupt pitfalls you identified. > > How is this relevant to what you wrote earlier and my response, > quoted just above? My statement was that threads can hold spin > locks. This is obviously true. If you means a kernel thread doing its own spin-wait, sure. But that is a "wait"the OS know nothing about. As far as OS is concerned that thread is just executing its applications. Whereas an OS spinlock requires IRQL >= Dispatch, at which point it is no longer executing as a kernel thread, it is a core. >>> |On a uniprocessor, if a thread tries to acquire a spin lock >>> |that is already held, it will loop forever. Multiprocessor >>> |algorithms, however, must operate correctly regardless of the >>> |number of processors, which means that they should handle the >>> |uniprocessor case as well. This requires strict adherence to >>> |the rule that threads not relinquish control of the CPU while >>> |holding a spin lock. >> Yes, which is what I was saying. >> I am describing HOW that is accomplished inside an OS. > > Why? First, it's well-known, and second, that's not relevant to > the discussion at hand. > > But it also doesn't seem like that's what you're saying; you > asserted that a thread (a software thread in this context) does > not "hold" a spin lock; that's incorrect. Answered above. >> The way you prevent a thread from relinquishing control while holding a >> spinlock is to first switch hats from thread level to software interrupt >> level, which blocks the scheduler because it also runs at SWIL, >> then acquire the spinlock. >> >> Raising the interrupt priority level from passive (aka thread) level >> to SWI level (called dispatch level on Windows) blocks the thread switch. >> This ensures non-reentrant algorithms are serialized. >> The core may then acquire spinlocks guarding data structures. > > Just so. Earlier you said that threads cannot hold spin locks; > in the text quoted just above, you said that they do. > > As for this specific scenario of specific interrupt priority > levels, I would go further and say that is just a detail. In > the simplest systems, ie, the kind we might reach for as an > expository example when discussing these kinds of issues, one > just disables interrupts completely when holding a spinlock. As I think you noted elsewhere, disabling interrupts is the same as raising the interrupt level to maximum. The general intent is to not do that as that limits concurrency and causes higher interrupt latencies. >>>> Threads and mutexes are a fiction created >>>> by the OS scheduler, and switching threads while waiting for a mutex >>>> is exactly what they are supposed to do. >>>> >>>> Spinlocks are used by cores to coordinate access to shared memory by >>>> things like a OS scheduler looking at list of threads waiting for a mutex. >>> Spinlocks, of the form I was describing, are an optimization for >>> cases where the critical section guarded by the lock is short, >>> and the overhead of invoking a scheduler and context switching >>> to some other thread is higher than just spinning until the lock >>> becomes available again. The whole point is to avoid blocking >>> and switching. >>> >>> E.g., as [Vaj11] puts it (page 98): >>> >>> |Spin locks have the advantage of preventing pre-emption by the >>> |OS (due to blocking while waiting for the lock) >>> >>> That is, the advantage is because the thread does not block and >>> yield. >> And I am describing HOW the OS accomplishes this. > > Well, what you are describing is how _a_ specific family of > operating systems that you are familiar with accomplishes this, > and that's all well and good, but it's not all that relevant to > discussing the pitfalls of this "ATOMIC event" proposal or > addressing the OP's questions about scenarios in which interrupt > disable is used. > >> Disabling the scheduler is the same as raising the priority to SWI level. >> Once the scheduler is disabled THEN you may acquire a spinlock >> that guards a shared data structure accessed at SWI level. >> >> But you may NOT access a shared data structure accessed at interrupt level. >> For that you'd need an interrupt spinlock acquired at the device ISR >> priority interrupt level. And note that the ISR interrupt priority level >> is assigned by the OS itself, usually when the device is mounted. > > You are presuming a very specific system architecture and going > into great detail about its implementation. That's fine, and > perhaps useful I suppose, but again, I don't think it's very > relevant to the context of this discussion. > >>>> I like to think of it as all having to do with hats. >>>> The cpu is wearing one of three hats: a thread hat when it pretends to be >>>> a time shared execution machine; a core hat when it acts as a single >>>> execution machine running non-reentrant things like a scheduler which >>>> creates the fiction of threads and processes (multiple virtual spaces); >>>> and an interrupt hat when executing nestable reentrant interrupts. >>>> >>>> The interrupt mechanism coordinates exactly when and how we switch hats. >>> I suppose you are lumping system calls into the overall bucket >>> of "interrupts" in this model, regardless of whether one might >>> use a dedicated supervisor call instruction (e.g., something >>> like x86 SYSENTER or SYSCALL or ARM SVC or RISC-V's ECALL)? >> Syscall switches a thread from user mode to supervisor mode. > > Your statement was that "the interrupt mechanism coordinates > exactly when and how we switch hats". But that's incomplete; if > a system call blocks, for example, the typical behavior is to > yield to the scheduler to pick something else to do. Clearly, > that is another way that coordinates this business of changing > one's hat. Yes, I believe that is how Linux worked originally. That just moves the point where it changes privilege mode and raising of priority level that disables the scheduler to be the OS SysCall entry point. They later changed to what I described, which is seems to be how many other OS also work, because it allows more kernel mode concurrency. >>> The hat analogy feels strained to me, and too colloquial to be >>> useful. >> I offered it because it answers your prior questions about how >> OS avoid the kinds of deadlocks and race conditions you described. > > I'm afraid you answered a question that I did not ask, then. > > I am discussing the specifics of this architectural proposal, > and specifically two aspects of it: one is addressing the > proposer's questions about scenarios where one might want to > disable interrupts entirely. > > Another, which has emerged somewhat more recently in the thread, > is this mechanism which seems to be a semi-transactional > elaboration of LL-SC primitives. > > Any questions I did ask were Socratic in nature, and in the > context of these architectural discussions. I did not ask any > questions about basic design or implementation of > synchronization primitives. You claimed I was wrong when I said spinlocks synchronize between cores. So I have repeatedly shown why I am correct. > - Dan C. > > References: > > [Gol94] Ruth E. Goldberg and Saro Saravanan. 1994. _OpenVMS AXP > Internals and Data Structures_ (Vers. 1.5). Digital Press, > Newton, MA. > > [Rus12] Mark Russinovich, David A. Solomon and Alex Ionescu. > 2012. _Windows Internals: Part 1_. Microsoft Press, Redmond, WA.
[toc] | [prev] | [next] | [standalone]
| From | cross@spitfire.i.gajendra.net (Dan Cross) |
|---|---|
| Date | 2025-03-29 20:55 +0000 |
| Message-ID | <vs9moh$79f$1@reader1.panix.com> |
| In reply to | #111368 |
In article <t3CFP.529522$f81.217510@fx48.iad>, EricP <ThatWouldBeTelling@thevillage.com> wrote: >Dan Cross wrote: >> In article <8uzEP.115550$Xq5f.66883@fx38.iad>, >> EricP <ThatWouldBeTelling@thevillage.com> wrote: >>> Dan Cross wrote: >>>> In article <kVgEP.1277108$_N6e.605199@fx17.iad>, >>>> EricP <ThatWouldBeTelling@thevillage.com> wrote: >>>>> Dan Cross wrote: >>>>>> [snip] >>>>>> Consider a thread that takes a spinlock; suppose some >>>>>> high-priority interrupt comes in while the thread is holding >>>>>> that lock. In response to the interrupt, software decides to >>>>>> suspend the thread and switch some other thread; that thread >>>>>> wants to lock the spin lock that the now-descheduled thread is >>>>>> holding: a classic deadlock scenario. >>>>> Terminology: mutexes coordinate mutual exclusion between threads, >>>>> spinlocks coordinate mutual exclusion between cpu cores. >>>>> Windows "critical sections" are mutexes with a fast path. >>>> A spin lock is simply any lock where you spin trying to acquire >>>> the lock, as opposed to a blocking synchronization protocol. >>>> Here I'm using the terminology of Herlihy and Shavit [Her08]. >>> I'm using terminology in common use on multiprocessor systems like >>> VMS since 1980's and later WinNT. >>> >>> Each OS data structure has different reentrancy requirements. >>> - Ones accessed by threads are guarded by mutexes. >>> Example on Windows would be the page tables as paging is done by threads. >>> - Ones accessed by OS software interrupt level are guarded by spinlocks. >>> Example on Windows is the Scheduler tables. >>> - Ones accessed by HW interrupts are guarded by interrupt spinlocks. >>> Example on Windows is device interrupt service routines. >> >> I'm not sure what you're saying here, or rather, how it is >> relevant. >> >> Your earlier statement was that "mutexes coordinate mutual >> exclusion between threads, spinlocks coordinate mutual exclusion >> between cpu cores." But this is not the case. >> >> Taking VMS as an example, from [Gol94], sec 9.5 (page 255): >> >> |A thread of execution running on one processor acquires a >> |spinlock to serialize access to the data with threads of >> |execution running on other processors. Before acquiring >> |the spinlock, the thread of executions raises IPL to block >> |accesses by other threads of execution running on the same >> |processor. The IPL value is determined by the spinlock being >> |locked. >> >> Goldberg and Saravanan seems quite explicit that threads can >> acquire and hold spinlocks on OpenVMS, which seems to be >> something that you were objecting to? > >It took me a while to find copy of that book online. >That 1994 book relates to VMS before they added kernel threads to the OS. Before you said you were using terminology commonly used on systems "since [the] 1980s". The systems you cited were VMS and Windows. I quoted references about both systems from the mid 1990s and mid 2000s, respectively. If that terminology were common in the 1980s, as you suggested, surely it would be in a book from 1994? Regardless, see below. >At that point VMS scheduled processes, an execution context plus virtual >space, like most *nix (except I think Solaris which had kernel threads). > >It says on pg 3: "The term thread of execution as used in this book >refers to a computational entity, an agent of execution." >This refers generally to the various code sequences from different context >that may be intertwined on a processors core for interrupts, exceptions, >Dpc (called driver forks in VMS), process AST's (software interrupts), >and applications. > >In a later book on VMS after they added kernel scheduled threads they >consistently use the two terms "thread of execution" and "kernel thread" >to distinguish the two meanings. Are you referring to the 1997 revision of Goldenberg et al, that covers VMS 7.0? [Gol97] In that context, VMS "kernel threads" refer to threads of execution within a program that are managed and scheduled by the VMS executive (aka kernel). This is described on page 151, section 5.1: |A thread of execution is a sequential flow of instruction |executiuon. A traditional program is single-threaded, |consisting of one main thread of execution. In contrast, a |multithreaded program divides its work among multiple threads |that can run concurrently. OpenVMS supplies DECthreads, a |run-time library, to facilitate the creation, synchronization, |and deletion of multiple user-mode thredas. | |OpenVMS Alpha Version 7.0 supports multiple execution contexts |within a process: each execution context has its own hardware |context and stacks, and canexecute independently of the others; |all execution context with a process share the same virtual |address space. The kernel _kernel thread_ refers to one of |those execution contexts and the thread of execution running |within it. This volume uses the term _multithreaded_ to refer |to a process with multiple kernel threads. | |The kernel thread is now the basis for OpenVMS scheduling. |[...] It continues on page 152: |The term _kernel thread_ is a somewhat confusing name, in that |instructions executing in a kernel thread can run in any of the |four access modes, not only kernel mode. The term arose to |distinguish the current multithreaded model, supported within |the OpenVMS executive (or kernel), from the previous model, |supported only by DECthreads. Sadly, the later books on OpenVMS internals are more focused on specific subsystems (scheduling and process management; memory management) than the ealier volumes. Details of the implementation of synchronization primitives are almost entirely missing. There is discussion about how those primitives are _used_, however. [Gol03] mentions spinlocks, such as the SCHED spinlock, in the context of memory management data structures. E.g., on page 51: |The swappability of the PHD [Process Header] result in several |different methods for synchronizing access to fields within it. |Because a PHD can be inswapped to a different balance set slot |than it last occuped, accesses to a PHD that use its system |space address must be synchronized against swapper |interference. Accesses from a kernel thread to its own PHD can |be made with the SCHED spinlock held to block any rescheduling |and possible swapping of the process. Holding the MMG spinlock |is another way to block swapping. Notably here, that the language is again in terms of (software) threads. It seems clear enough that in the VMS context, spinlocks are conceptually held by threads, not cores. >I find the phrase "thread of execution" ambiguous and try not use it. >After the term thread had been pretty much adopted to mean kernel thread >as an OS scheduled execution context, I would have preferred if they used >something like "strand of execution" to avoid the overloading. I disagree with that characterization. "Thread" generically means what they said it means above: a sequential flow of instruction execution. Generically, on most systems, "threads" are just register state and a stack in some address address space. Fibers/Strands/etc tend to refer to lightweight cooperatively scheduled concurrency structures. Anyway, I don't think that much changes the point I was making. I don't see any evidence of >> Turning to Windows, [Rus12] says about spin locks on page 179: >> >> |Testing and acquiring the lock in one instruction prevents a >> |second thread from grabbing the lock between the time the first >> |thread tests the variable and the time it acquires the lock. >> >> Note that, again, the language here is thread-centric. > >He says on pg 12: "A thread is the entity within a process that >Windows schedules for execution." meaning "kernel thread". > >He also uses the term thread loosely as you can see on pg 13 when he says: >"Fibers allow an application to schedule its own “threads” of execution" I don't see how that changes the point. >He also says pg 87 (to paraphrase) that scheduling priority is an >attribute of a thread whereas IRQL is an attribute of a processor. >Which is the distinction I am trying to make: when you raise IRQL >it stops being a thread executing and starts being a processor core. The exact paragraph I suspect you are referring to is this: |IRQL priority levels have a completely different meaning than |thread-scheduling priorities (which are described in chapter |5). A scheduling priority is an attribute of a thread, whereas |an IRQL is an attribute of an interrupt source, such as a |keyboard or a mouse. In addition, each processor has an IRQL |setting that chnages as operating system code executes. Here, they simply discuss the difference between the IRQL and thread priorities If one looks past software and looks at the hardware for a moment, we know that (for instance) the x86 LAPIC has a Task Priority Register (TPR) can be used to set what asynchronous interrupts the current task is willing to accept, where of such interrupts is a function of its vector number. Well, it's a little more complex than that in that x86_64 buckets vectors into 16 "classes" and the threshold for interrupt acceptance is actually based on the 4-bit "task-priority class" subfield of that register, but that's the gist of it. But that's obviously different than, say, the OS's notion of a thread (or, if you prefer, some schedulable execution context)'s priority, which is an abstraction it creates and maintains for its own consumption. Clearly, the IRQL is an abstraction of that hardware concept. Anyway, it seems clear that the above quoted text is merely explaining that these are not the same thing, and that readers should take care not to confuse them. I don't see how that has any bearing at all on your assertion that spinlocks are conceptualized in terms of cores and not (software) threads in Windows, let alove VMS. Indeed, taking in the larger context in this section, I see no evidence that Russinovich Russinovich et al (multiple authors, not just one) talk about anything other than (kernel-mode) threads holding locks. From pages 86 and 87, |Interrupts are serviced in priority order, and a |higher-priority interrupt preempts the servicing of a |lower-priority interrupt. When a high-priority interrupt |occurs, the processor saves the interrupted thread's state and |invokes the trap dispatchers associated with the interrupt. |The trap dispatcher raises the IRQL and calls the interrupt's |service routine. After the service routine executes, the |interrupt dispatcher lowers the processor's IRQL to where it |was before the interrupt occurred and then loads the saved |machine state. The interrupted thread resumes executing where |it left off. This just discusses how _interrupts_ are processed. Note that on page 86, they write: |The kernel represents IRQLs internally as a number from 0 |through 31 on x86 and from 0 to 15 on x64 and IA64, with higher |numbers representing higher-priority interrupts. Note how these numbers correspond to the hardware. E.g., in the case of x86_64, 0-15 are the priority thresholds representable in the 4-bit "task-priority class" field in the TPR, settable by software via a move to CR8. But continuing on after the paragraph you cited on page 87, they write: |Each processor's IRQL setting determines which interrupts that |processor can receive. IRQLs are also used to synchronize |access to kernel-mode data structures. (You'll find out more |about synchronization later in this chapter.) As a kernel-mode |thread runs, it raises or lowers the processor's IRQL either |directly by calling _KeRaiseIrql_ and _KeLowerIrql_ or, more |commonly, indirectly via class to functions that acquire kernel |synchronization objects. In other words, kernel-mode threads raise and lower the IRQL as they interact with kernel synchronization objects, such as spinlocks. Continuing on page 88, one reads: |A kernel-mode thread raises and lowers the IRQL of the |processor on which it's running, depending on what it's trying |to do. Again, note that the language describing the actors in these scenarios is centered on software abstractions. It is kernel-mode _threads_ that are acquiring and holding spinlocks, changing the IRQL, etc. Not cores. >> By the way, regarding terminology, on page 177 [Rus12] also >> says, >> >> |Sections of code that access a nonshareable resource are called >> |_critical_ sections. To ensure correct code, only one thread >> |at a time can execute in a critical section. >> >> This is the generic definition of a critical section and note >> that this is the context in which we are using the term >> throughout this thread. > >Yes, I was trying to note the ambiguous terminology so we could avoid it. > >That page is referring to kernel threads as scheduled entities >because he says "if the operating system performed a context >switch to the second thread...". In a Windows context the term >"critical section" means a specific Win32 user mode mechanism >used to synchronize access for scheduled threads. I don't think so. I mentioned the mechanism that windows calls a "critical section" later, but we can see that that is not the meaning here by looking at the surrounding context. For instance, on page 176: |The concept of _mutual exclusion_ is a crucial one in operating |systems development. It refers to the guarantee that one, and |only one, thread cn access a particular resource at a time. Given the reference to "operating system development", it seems clear this is referring to a "thread" in a generic sense. Note that the sentence you quoted above is somewhat misleading out of context. It is part of a larger paragraph explaining an illustrative example about the consequences of failing to maintain proper mutual between threads of execution generally. This paragraph, and that sentence fragment in particular, are explaining how the specific example, which shows a conflict between code running on two separate processors can also affect a single processor. The exact text is, |Because the second thread obtained the value of the queue tail |pointer before the first thread finished updating it, the |second thread inserted its data into the same location that the |first thread used, overwriting data and leaving one queue |location empty. *Even though Figure 3-24 illustrates what |could happen on a multiprocessor system, the same error could |occur on a single-processor system if the operating system |performed a context switch to the second thread before the |first thread updated the queue tail pointer* (Emphasis mine.) Mentioning thread context switching here doesn't seem particular indicative of user-mode versus kernel-mode threads; it's common to talk about thread context switch in the context of a kernel. For example, Unix `swtch` has been switching between thread contexts since the early 1970s (Unix has had a multithreaded kernel since the earliest editions, where processes were conceptualized as either running in user-mode, or "in the kernel", as part of a thread maintained by the kernel for each process; just to be extra confusing, these are often called "kernel threads", meaning threads internal to the kernel. A "context switch" between user processes thus always involved a trap so that the process was running in the kernel, then a thread switch from one process's kernel thread to that of another, followed by an eventual return to userspace). Anyway, continuing with Russinovich et al, later on page 177 they write, |The issue of mutual exclusion, although important for all |operating systems, is especially important (and intricate) for |a _tightly coupled, symmetric multiprocessing_ (SMP) operating |system such as Windows, in which the same system code runs |simultaneously on more than one processor, sharing certain data |structures stored in global memory. In Windows, it is the |kernel's job to provide mechanisms that system code can use to |prevent two threads from modifying the same structure at the |same time. The kernel provides mutual-exclusion primitives |that it and the rest of the executive use to synchronize their |access to global data structures. Taken together, this all makes it clear that, in this section, these authors are talking about these concepts generally as a preface to discussing them within the context of Windows specifically. >As opposed to books on multi-processors often use the term >"critical section" in a generic way to mean a shared section of memory >accessed by multiple processors, guarded by some form of mutual exclusion, >which could apply to shared memory guarded by OS spinlocks >but equally to shared application memory with its own synchronization. In this context, "critical section" usually refers to a sequence of instructions that are modifying some shared resource, but must do so atomically. >> What you referred to earlier as a "critical section" in Windows >> is a _userspace_ synchronization primitive with a kernel assist >> and mostly unrelated to concerns around interruptibility. From >> the description on page 201 of [Rus12], they seem similar to >> futexes on Linux, though somewhat more limited: Russinovich et >> al states that they cannot be used between processes, while a >> Linux futex can if it's in a region of memory shared between >> cooperating processes. > >Yes. > >>>> Traditional Unix "sleep" and "wakeup" are an example of a >>>> blocking protocol, where a thread may "sleep" on some "channel", >>>> yielding the locking thread while the lock cannot be acquired, >>>> presumably scheduling something else until the thread is later >>>> marked runnable by virtual of something else calling "wakeup" on >>>> the sleep channel. >>>> >>>> But note that I am deliberately simplifying in order to >>>> construct a particular scenario in which a light-weight >>>> synchronization primitive is being used for mutual exclusion >>>> between concurrent entities, hence mentioning spinlocks >>>> specifically. >>>> >>>> Suggesting that spin locks iare only applicable to >>>> multiprocessing scenarios is flatly incorrect. >>> I didn't. >> >> Your exacty words were, "spinlocks coordinate mutual exclusion >> between cpu cores." See the text quoted above. This is >> incorrect in that it ignores that threads are, in contxt, the >> software primitive for the computational entities that are doing >> the acquiring, and while the "between cpu core" part is the >> usual case, it is not the only case. > >I believe I'm being consistent and precise. >However a *core's* interrupt level gets raised >= Dispatch level, either by >an interrupt of a lower priority strand of execution, or if a kernel thread >making a SysCall, it shuts off the scheduler. At that point the core is no >longer executing as a kernel thread - it is executing as a processor core. A processor core is always a processor core, full stop. A core does not "execute _as_ a kernel thread"; a "kernel thread" is a software abstraction, something the core has no conception of; the core just executes instructions, handles interrupts (modulo the state that it's put into by software) and exceptions, and does other processor things. In the context of Windows, a "kernel thread" is an entity that is managed and scheduled by the operating system. The operating system manages scheduling threads onto cores, switching between them, etc, but the processor itself is no more aware of that that an electrical power plant is aware whether it is powering a toaster versus a dishwasher. Software mutual exclusion primitives like spinlocks, are by definition the purview of software; as such, the core has no insight into them. Systems software elevates interrupt priority levels (or disables interrupts completely) when holding spinlocks as a necessary part correctness of a software locking protocol, in order to ensure correct serialization of access between multiple software agents. But in particular, changing interrupt priorities when holding a spinlock is actually done to ensure correctness of the protocol on a _single_ processor, not _between_ processors. >If it then acquires an OS spinlock, it is synchronizing between cores. Not true at all. This would preclude using spinlocks on a uniprocessor system, which is an unreasonable constraint: consider an operating system distributed as a single image that should run correctly on both uni- and multi-processor systems. Recall the reference to [Vah96] earlier, that goes into this. Here it is again: |On a uniprocessor, if a thread tries to acquire a spin lock |that is already held, it will loop forever. Multiprocessor |algorithms, however, must operate correctly regardless of the |number of processors, which means that they should handle the |uniprocessor case as well. This requires strict adherence to |the rule that threads not relinquish control of the CPU while |holding a spin lock. >>> I am saying the OS by design guards *ITS* different data structures >>> with different mechanisms, and those mechanism must only be used in >>> a very controlled manner. And if you want to do something that requires >>> access to one of *ITS* data structures then you must obey *ITS* rules. >> >> This is axiomatic. >> >> But I fail to see how it's relevant to the topic at hand. In >> context we're discussing scenarios where a system may want to >> disable interrupts entirely, and whether a particular aspect of >> a hardware design is sufficient to implement critical sections >> generally. > >You seemed to be switching between talking about general spinlocks as >"any code that spin-waits" whereas I am just referring to OS spinlocks >executing at a specific raised IRQL, guarding shared OS data structures. Going back to the original context, I wrote: >> Consider a thread that takes a spinlock; suppose some >> high-priority interrupt comes in while the thread is holding >> that lock. In response to the interrupt, software decides to >> suspend the thread and switch some other thread; that thread >> wants to lock the spin lock that the now-descheduled thread is >> holding: a classic deadlock scenario. To which you responded: > Terminology: mutexes coordinate mutual exclusion between threads, > spinlocks coordinate mutual exclusion between cpu cores. > Windows "critical sections" are mutexes with a fast path. Which strikes me as pretty non-standard: everything cited so far pretty much acknowledges that spinlocks coordinate between threads, independent of the specific meaning of "thread" in context a given context. Indeed, Vahalia explicitly addresses the uniprocessor case for spinlocks. Beyond that, I still fail to see how that's relevant to the original context: scenarios under which one may want to disable interrupts entirely. >>> In Windows and VMS and from what I understand of *nix spinlocks guard most >>> of the core OS data structures with spinlocks acquired at the software >>> interrupt level. If you are not at that level then the spinlock will not >>> provide the necessary synchronization for access to that data. >> >> Precisely. > >Yeah. We are synchonized! Yay! >>>>>> Further, software must now consider the complexity of >>>>>> potentially interruptable critical sections. From the >>>>>> standpoint of reasoning about already-complex concurrency issues >>>>>> it's simpler to be able to assert that (almost) all interrupt >>>>>> delivery can be cheaply disabled entirely, save for very >>>>>> special, specific, scenarios like NMIs. Potentially switching >>>>>> away from a thread holding a spinlock sort of defeats the >>>>>> purpose of a spinlock in the first place, which is a mutex >>>>>> primitive designed to avoid the overhead of switching. >> >> (It occurs to me that perhaps this is why you are going on about >> how VMS and windows uses spin locks, and how they interact with >> interrupt priority levels.) > >I find that because WinNT was a clean-sheet design, much of its terminology >for things like threads, processes, mutexes, Dpc's, is less ambiguous, >particularly when talking about interrupts and scheduling. > >Whereas with VMS and *nix that terminology can get messy because they >started out with "process" as the unit of scheduled execution and later >glued threads on, and then Unix glued threads on as a new flavor of forking >that doesn't really fork, and Linux with its interrupt handling history of >back half's vs tasklettes vs SoftIrq. Ok. >>>>> This is why I mentioned the terminology thing: threads do not hold >>>>> spinlocks, they hold mutexes. >>>> See above. Threads can certainly "hold" a spin lock, as they >>>> can hold any kind of lock. To quote from sec 7.6.1 of [Val96], >>>> page 202: >>> When a particular OS defined spinlock is used to guard a particular OS >>> defined data structure, exactly WHEN the OS allows this decides how >>> the OS avoids all the nasty interrupt pitfalls you identified. >> >> How is this relevant to what you wrote earlier and my response, >> quoted just above? My statement was that threads can hold spin >> locks. This is obviously true. > >If you means a kernel thread doing its own spin-wait, sure. But that is a >"wait"the OS know nothing about. As far as OS is concerned that thread is >just executing its applications. > >Whereas an OS spinlock requires IRQL >= Dispatch, at which point it >is no longer executing as a kernel thread, it is a core. Please provide a citation to something that uses these terms in this sense. I see no evidence that this terminology is common anywhere, but have cited plenty of evidence using the terminology that threads can hold spinlocks. >>>> |On a uniprocessor, if a thread tries to acquire a spin lock >>>> |that is already held, it will loop forever. Multiprocessor >>>> |algorithms, however, must operate correctly regardless of the >>>> |number of processors, which means that they should handle the >>>> |uniprocessor case as well. This requires strict adherence to >>>> |the rule that threads not relinquish control of the CPU while >>>> |holding a spin lock. >>> Yes, which is what I was saying. >>> I am describing HOW that is accomplished inside an OS. >> >> Why? First, it's well-known, and second, that's not relevant to >> the discussion at hand. >> >> But it also doesn't seem like that's what you're saying; you >> asserted that a thread (a software thread in this context) does >> not "hold" a spin lock; that's incorrect. > >Answered above. Not really. You made an assertion, but that assertion was not backed up with any evidence, and moreover, disconfirming evidence has been provided. Threads can clearly hold spinlocks; suggesting otherwise is simply incorrect. If you still disagree post evidence. >>> The way you prevent a thread from relinquishing control while holding a >>> spinlock is to first switch hats from thread level to software interrupt >>> level, which blocks the scheduler because it also runs at SWIL, >>> then acquire the spinlock. >>> >>> Raising the interrupt priority level from passive (aka thread) level >>> to SWI level (called dispatch level on Windows) blocks the thread switch. >>> This ensures non-reentrant algorithms are serialized. >>> The core may then acquire spinlocks guarding data structures. >> >> Just so. Earlier you said that threads cannot hold spin locks; >> in the text quoted just above, you said that they do. >> >> As for this specific scenario of specific interrupt priority >> levels, I would go further and say that is just a detail. In >> the simplest systems, ie, the kind we might reach for as an >> expository example when discussing these kinds of issues, one >> just disables interrupts completely when holding a spinlock. > >As I think you noted elsewhere, disabling interrupts is the same as >raising the interrupt level to maximum. > >The general intent is to not do that as that limits concurrency >and causes higher interrupt latencies. Did you read what you responded to? >>>>> I like to think of it as all having to do with hats. >>>>> The cpu is wearing one of three hats: a thread hat when it pretends to be >>>>> a time shared execution machine; a core hat when it acts as a single >>>>> execution machine running non-reentrant things like a scheduler which >>>>> creates the fiction of threads and processes (multiple virtual spaces); >>>>> and an interrupt hat when executing nestable reentrant interrupts. >>>>> >>>>> The interrupt mechanism coordinates exactly when and how we switch hats. >>>> I suppose you are lumping system calls into the overall bucket >>>> of "interrupts" in this model, regardless of whether one might >>>> use a dedicated supervisor call instruction (e.g., something >>>> like x86 SYSENTER or SYSCALL or ARM SVC or RISC-V's ECALL)? >>> Syscall switches a thread from user mode to supervisor mode. >> >> Your statement was that "the interrupt mechanism coordinates >> exactly when and how we switch hats". But that's incomplete; if >> a system call blocks, for example, the typical behavior is to >> yield to the scheduler to pick something else to do. Clearly, >> that is another way that coordinates this business of changing >> one's hat. > >Yes, I believe that is how Linux worked originally. >That just moves the point where it changes privilege mode and raising of >priority level that disables the scheduler to be the OS SysCall entry point. System call entry does not "disable the scheduler" in any meaningful way. In fact, it's quite common to invoke the scheduler as a consequence of user-initiated system calls; most Unix and Linux system calls are synchronous. >They later changed to what I described, which is seems to be how many >other OS also work, because it allows more kernel mode concurrency. Consider a single-threaded process that issues a synchronous `read` call against a blocking socket descriptor referring to a connected TCP stream. Assuming no data is currently immediately available for consumption, the system call will block; the calling task will be marked as blocked waiting on data to arrive on the socket and the scheduler will be invoked on that core. No interrupts involved. This has nothing to do with "kernel-mode concurrency." Or, for that matter, consider `vfork`, which always executes the scheduler. Exercise left to the reader. >>>> The hat analogy feels strained to me, and too colloquial to be >>>> useful. >>> I offered it because it answers your prior questions about how >>> OS avoid the kinds of deadlocks and race conditions you described. >> >> I'm afraid you answered a question that I did not ask, then. >> >> I am discussing the specifics of this architectural proposal, >> and specifically two aspects of it: one is addressing the >> proposer's questions about scenarios where one might want to >> disable interrupts entirely. >> >> Another, which has emerged somewhat more recently in the thread, >> is this mechanism which seems to be a semi-transactional >> elaboration of LL-SC primitives. >> >> Any questions I did ask were Socratic in nature, and in the >> context of these architectural discussions. I did not ask any >> questions about basic design or implementation of >> synchronization primitives. > >You claimed I was wrong when I said spinlocks synchronize between cores. >So I have repeatedly shown why I am correct. Well, you have asserted the same thing several times, despite being presented with disconfirming evidence. But you have not shown any evidence supporting your assertions or why you think you are correct. - Dan C. References: [Gol94] Ruth E. Goldenberg and Saro Saravanan. 1994. _OpenVMS AXP Internals and Data Structures_ (Vers. 1.5). Digital Press, Newton, MA. [Gol97] Ruth E. Goldenberg, Denise E. Dumas and Saro Saravanan. 1997. _OpenVMS Alpha Internals: Scheduling and Process Control_ (Vers. 7.0). Digital Press, Newton, MA. [Gol03] Ruth E. Goldenberg. 2003. _OpenVMS Alpha Internals and Data Structures: Memory Management_. Digital Press, Boston, MA. [Rus12] Mark Russinovich, David A. Solomon and Alex Ionescu. 2012. _Windows Internals: Part 1_. Microsoft Press, Redmond, WA. [Vah96] Uresh Vahalia. 1996. _Unix Internals: The New Frontiers_. Prentice Hall, Upper Saddle River, NJ.
[toc] | [prev] | [next] | [standalone]
| From | EricP <ThatWouldBeTelling@thevillage.com> |
|---|---|
| Date | 2025-03-30 16:34 -0400 |
| Message-ID | <tWhGP.1790040$TBhc.304536@fx16.iad> |
| In reply to | #111372 |
Dan Cross wrote: > In article <t3CFP.529522$f81.217510@fx48.iad>, > EricP <ThatWouldBeTelling@thevillage.com> wrote: >> Dan Cross wrote: >>> In article <8uzEP.115550$Xq5f.66883@fx38.iad>, >>> EricP <ThatWouldBeTelling@thevillage.com> wrote: >>>> Dan Cross wrote: >>>>> In article <kVgEP.1277108$_N6e.605199@fx17.iad>, >>>>> EricP <ThatWouldBeTelling@thevillage.com> wrote: >>>>>> Dan Cross wrote: >>>>>>> [snip] >>>>>>> Consider a thread that takes a spinlock; suppose some >>>>>>> high-priority interrupt comes in while the thread is holding >>>>>>> that lock. In response to the interrupt, software decides to >>>>>>> suspend the thread and switch some other thread; that thread >>>>>>> wants to lock the spin lock that the now-descheduled thread is >>>>>>> holding: a classic deadlock scenario. >>>>>> Terminology: mutexes coordinate mutual exclusion between threads, >>>>>> spinlocks coordinate mutual exclusion between cpu cores. >>>>>> Windows "critical sections" are mutexes with a fast path. >>>>> A spin lock is simply any lock where you spin trying to acquire >>>>> the lock, as opposed to a blocking synchronization protocol. >>>>> Here I'm using the terminology of Herlihy and Shavit [Her08]. >>>> I'm using terminology in common use on multiprocessor systems like >>>> VMS since 1980's and later WinNT. >>>> >>>> Each OS data structure has different reentrancy requirements. >>>> - Ones accessed by threads are guarded by mutexes. >>>> Example on Windows would be the page tables as paging is done by threads. >>>> - Ones accessed by OS software interrupt level are guarded by spinlocks. >>>> Example on Windows is the Scheduler tables. >>>> - Ones accessed by HW interrupts are guarded by interrupt spinlocks. >>>> Example on Windows is device interrupt service routines. >>> I'm not sure what you're saying here, or rather, how it is >>> relevant. >>> >>> Your earlier statement was that "mutexes coordinate mutual >>> exclusion between threads, spinlocks coordinate mutual exclusion >>> between cpu cores." But this is not the case. >>> >>> Taking VMS as an example, from [Gol94], sec 9.5 (page 255): >>> >>> |A thread of execution running on one processor acquires a >>> |spinlock to serialize access to the data with threads of >>> |execution running on other processors. Before acquiring >>> |the spinlock, the thread of executions raises IPL to block >>> |accesses by other threads of execution running on the same >>> |processor. The IPL value is determined by the spinlock being >>> |locked. >>> >>> Goldberg and Saravanan seems quite explicit that threads can >>> acquire and hold spinlocks on OpenVMS, which seems to be >>> something that you were objecting to? >> It took me a while to find copy of that book online. >> That 1994 book relates to VMS before they added kernel threads to the OS. > > Before you said you were using terminology commonly used on > systems "since [the] 1980s". The systems you cited were VMS and > Windows. I quoted references about both systems from the mid > 1990s and mid 2000s, respectively. If that terminology were > common in the 1980s, as you suggested, surely it would be in a > book from 1994? Regardless, see below. SMP OS (synchronized with spinlocks) was added to VMS is 1988. VMS clusters were released in 1983 with its locking system and cluster read-write locks can be used to build other synchronization structures like mutexes, semaphores, quorums. I don't know when the term "thread" meaning "scheduled units of execution within a common address space" first show up. I found a reference to it on Unix in 1989. Previously the term "task" was used to mean the same thing, such as Ada tasks which on DEC Ada in 1986 were similar to what are now called user mode threads. It looks to me that the DEC Ada runtime library was later morphed into DECThreads user mode library released in 1991. >> At that point VMS scheduled processes, an execution context plus virtual >> space, like most *nix (except I think Solaris which had kernel threads). >> >> It says on pg 3: "The term thread of execution as used in this book >> refers to a computational entity, an agent of execution." >> This refers generally to the various code sequences from different context >> that may be intertwined on a processors core for interrupts, exceptions, >> Dpc (called driver forks in VMS), process AST's (software interrupts), >> and applications. >> >> In a later book on VMS after they added kernel scheduled threads they >> consistently use the two terms "thread of execution" and "kernel thread" >> to distinguish the two meanings. > > Are you referring to the 1997 revision of Goldenberg et al, > that covers VMS 7.0? [Gol97] In that context, VMS "kernel > threads" refer to threads of execution within a program that > are managed and scheduled by the VMS executive (aka kernel). I saw reference to 1997 internals book and that kernel threads were added. However the only other OpenVMS internals document I could find was OpenVMS Alpha Internals and Data Structures - Memory Management, Goldenberg et al, 2003. > This is described on page 151, section 5.1: > > |A thread of execution is a sequential flow of instruction > |executiuon. A traditional program is single-threaded, > |consisting of one main thread of execution. In contrast, a > |multithreaded program divides its work among multiple threads > |that can run concurrently. OpenVMS supplies DECthreads, a > |run-time library, to facilitate the creation, synchronization, > |and deletion of multiple user-mode thredas. > | > |OpenVMS Alpha Version 7.0 supports multiple execution contexts > |within a process: each execution context has its own hardware > |context and stacks, and canexecute independently of the others; > |all execution context with a process share the same virtual > |address space. The kernel _kernel thread_ refers to one of > |those execution contexts and the thread of execution running > |within it. This volume uses the term _multithreaded_ to refer > |to a process with multiple kernel threads. > | > |The kernel thread is now the basis for OpenVMS scheduling. > |[...] This "kernel thread" is the definition I have been using: unit of execution *scheduled* by the operating system. > It continues on page 152: > > |The term _kernel thread_ is a somewhat confusing name, in that > |instructions executing in a kernel thread can run in any of the > |four access modes, not only kernel mode. The term arose to > |distinguish the current multithreaded model, supported within > |the OpenVMS executive (or kernel), from the previous model, > |supported only by DECthreads. > > Sadly, the later books on OpenVMS internals are more focused on > specific subsystems (scheduling and process management; memory > management) than the ealier volumes. Details of the > implementation of synchronization primitives are almost entirely > missing. > > There is discussion about how those primitives are _used_, > however. [Gol03] mentions spinlocks, such as the SCHED > spinlock, in the context of memory management data structures. > E.g., on page 51: > > |The swappability of the PHD [Process Header] result in several > |different methods for synchronizing access to fields within it. > |Because a PHD can be inswapped to a different balance set slot > |than it last occuped, accesses to a PHD that use its system > |space address must be synchronized against swapper > |interference. Accesses from a kernel thread to its own PHD can > |be made with the SCHED spinlock held to block any rescheduling > |and possible swapping of the process. Holding the MMG spinlock > |is another way to block swapping. > > Notably here, that the language is again in terms of (software) > threads. It seems clear enough that in the VMS context, > spinlocks are conceptually held by threads, not cores. Also Gol91 VAX VMS Internals and Data Structures v5.2 1991 All three, Gol91, Gol94, Rus12 have a section on spinlocks and explicitly say spinlocks synchronize processors, not threads. Gol91 pg 172, Gol94 pg 254: "A spinlock is acquired by a processor to synchronize access to data shared by members of an SMP system." Rus12 pg 179: "The mechanism the kernel uses to achieve multiprocessor mutual exclusion is called a spinlock." Furthermore a search for the word "spinlock " in all three documents finds repeated references to "processor spinlock". There are NO usages of "process spinlock" on VMS or "thread spinlock" on WinNT. It is mutexes that synchronize VMS processes or WNT threads. The difference is spinlocks do not invoke the scheduler to trigger an executable context switch, whereas mutexes may invoke the scheduler. There is a section on mutexes in Gol91 pg 196, Gol04 pg 277, Rus12 pg 183 Low-IRQL Synchronization Gol91 pg 25, Gol94 pg 29 "Thus, the VMS executive requires a third synchronization tool to allow synchronized access to pageable data structures. This tool must also allow a process to be removed from execution while it maintains ownership of the structure in question. One synchronization tool that fulfills these requirements is called a mutual exclusion semaphore (mutex)." Gol91 pg 196, Gol94 pg 277 "However, in some situations requiring synchronization, elevated IPL is an unacceptable technique." ... "Typically, the threads of execution whose accesses are synchronized through a mutex are process context threads." Rus12 pg 183 Low-IRQL Synchronization "Because waiting for a spinlock literally stalls a processor, spinlocks can be used only under the following strictly limited circumstances" ... "There are several additional synchronization mechanisms for use when spinlocks are not suitable: - Kernel dispatcher objects - Fast mutexes and guarded mutexes - Pushlocks - Executive resources Additionally, user-mode code, which also executes at low IRQL, must be able to have its own locking primitives. Windows supports various user-mode-specific primitives: - Condition variables (CondVars) - Slim Reader-Writer Locks (SRW Locks) - Run-once initialization (InitOnce) - Critical sections" These books consistently refer to spinlocks synchronize *processors*, aka cores, at raised IPL (>= dispatch), and mutexes synchronize scheduled processes/threads at lower IPL (< dispatch). The reason is that when a kernel thread, executing a system call, raises the processor >= dispatch, which is required to acquire a spinlock, it stops being a scheduled thread, and may have temporarily switched kernel stacks as processor interrupt levels require their own stacks. At raised IPL many of the OS data structures associated with a kernel thread cannot be touched as they might trigger a page fault which can only be serviced by a kernel thread which can sleep, and not by processors/cores which can only spin-wait. (Both VMS and WNT will crash/panic if code page faults at IPL >= dispatch for exactly this reason.) All of this is for controlling access to non-reentrant shared data. Non-reentrant data shared between processors/cores is controlled by IPL and blocked by spinlocks. Non-reentrant data shared between kernel threads is controlled with scheduler synchronization and thread switching.
[toc] | [prev] | [next] | [standalone]
| From | cross@spitfire.i.gajendra.net (Dan Cross) |
|---|---|
| Date | 2025-03-31 03:33 +0000 |
| Message-ID | <vsd2ep$5c9$1@reader1.panix.com> |
| In reply to | #111378 |
In article <tWhGP.1790040$TBhc.304536@fx16.iad>, EricP <ThatWouldBeTelling@thevillage.com> wrote: >Dan Cross wrote: >> In article <t3CFP.529522$f81.217510@fx48.iad>, >> EricP <ThatWouldBeTelling@thevillage.com> wrote: >>> Dan Cross wrote: >>>> In article <8uzEP.115550$Xq5f.66883@fx38.iad>, >>>> EricP <ThatWouldBeTelling@thevillage.com> wrote: >>>>> Dan Cross wrote: >>>>>> In article <kVgEP.1277108$_N6e.605199@fx17.iad>, >>>>>> EricP <ThatWouldBeTelling@thevillage.com> wrote: >>>>>>> Dan Cross wrote: >>>>>>>> [snip] >>>>>>>> Consider a thread that takes a spinlock; suppose some >>>>>>>> high-priority interrupt comes in while the thread is holding >>>>>>>> that lock. In response to the interrupt, software decides to >>>>>>>> suspend the thread and switch some other thread; that thread >>>>>>>> wants to lock the spin lock that the now-descheduled thread is >>>>>>>> holding: a classic deadlock scenario. >>>>>>> Terminology: mutexes coordinate mutual exclusion between threads, >>>>>>> spinlocks coordinate mutual exclusion between cpu cores. >>>>>>> Windows "critical sections" are mutexes with a fast path. >>>>>> A spin lock is simply any lock where you spin trying to acquire >>>>>> the lock, as opposed to a blocking synchronization protocol. >>>>>> Here I'm using the terminology of Herlihy and Shavit [Her08]. >>>>> I'm using terminology in common use on multiprocessor systems like >>>>> VMS since 1980's and later WinNT. >>>>> >>>>> Each OS data structure has different reentrancy requirements. >>>>> - Ones accessed by threads are guarded by mutexes. >>>>> Example on Windows would be the page tables as paging is done by threads. >>>>> - Ones accessed by OS software interrupt level are guarded by spinlocks. >>>>> Example on Windows is the Scheduler tables. >>>>> - Ones accessed by HW interrupts are guarded by interrupt spinlocks. >>>>> Example on Windows is device interrupt service routines. >>>> I'm not sure what you're saying here, or rather, how it is >>>> relevant. >>>> >>>> Your earlier statement was that "mutexes coordinate mutual >>>> exclusion between threads, spinlocks coordinate mutual exclusion >>>> between cpu cores." But this is not the case. >>>> >>>> Taking VMS as an example, from [Gol94], sec 9.5 (page 255): >>>> >>>> |A thread of execution running on one processor acquires a >>>> |spinlock to serialize access to the data with threads of >>>> |execution running on other processors. Before acquiring >>>> |the spinlock, the thread of executions raises IPL to block >>>> |accesses by other threads of execution running on the same >>>> |processor. The IPL value is determined by the spinlock being >>>> |locked. >>>> >>>> Goldberg and Saravanan seems quite explicit that threads can >>>> acquire and hold spinlocks on OpenVMS, which seems to be >>>> something that you were objecting to? >>> It took me a while to find copy of that book online. >>> That 1994 book relates to VMS before they added kernel threads to the OS. >> >> Before you said you were using terminology commonly used on >> systems "since [the] 1980s". The systems you cited were VMS and >> Windows. I quoted references about both systems from the mid >> 1990s and mid 2000s, respectively. If that terminology were >> common in the 1980s, as you suggested, surely it would be in a >> book from 1994? Regardless, see below. > >SMP OS (synchronized with spinlocks) was added to VMS is 1988. >VMS clusters were released in 1983 with its locking system and >cluster read-write locks can be used to build other synchronization >structures like mutexes, semaphores, quorums. > >I don't know when the term "thread" meaning "scheduled units of execution >within a common address space" first show up. I found a reference to it on >Unix in 1989. Previously the term "task" was used to mean the same thing, >such as Ada tasks which on DEC Ada in 1986 were similar to what are now >called user mode threads. It looks to me that the DEC Ada runtime library >was later morphed into DECThreads user mode library released in 1991. The point is that if, as you assert, Windows and VMS both use the terminology of _cores_ (or processors) acquiring and holding spinlocks instead of threads doing so, there should be some reference to that in those books. There is not. The terminology on those systems is consistent: threads are the things that hold spinlocks; it's axiomatic that those threads are running on processors, since the whole point is that it is faster to busy-wait on the spinlock rather than potentially context switch as with a different mutual exclusion primtiive, like what VMS calles a mutex (which is a specific software object, not the generic concept of a "mutex" as a mutual exclusion primitive. In the latter context, a spinlock is a kind of mutex; in the former, it's different). >>> At that point VMS scheduled processes, an execution context plus virtual >>> space, like most *nix (except I think Solaris which had kernel threads). >>> >>> It says on pg 3: "The term thread of execution as used in this book >>> refers to a computational entity, an agent of execution." >>> This refers generally to the various code sequences from different context >>> that may be intertwined on a processors core for interrupts, exceptions, >>> Dpc (called driver forks in VMS), process AST's (software interrupts), >>> and applications. >>> >>> In a later book on VMS after they added kernel scheduled threads they >>> consistently use the two terms "thread of execution" and "kernel thread" >>> to distinguish the two meanings. >> >> Are you referring to the 1997 revision of Goldenberg et al, >> that covers VMS 7.0? [Gol97] In that context, VMS "kernel >> threads" refer to threads of execution within a program that >> are managed and scheduled by the VMS executive (aka kernel). > >I saw reference to 1997 internals book and that kernel threads were added. >However the only other OpenVMS internals document I could find was >OpenVMS Alpha Internals and Data Structures - Memory Management, >Goldenberg et al, 2003. Ok, I have that one, too. >> This is described on page 151, section 5.1: >> >> |A thread of execution is a sequential flow of instruction >> |executiuon. A traditional program is single-threaded, >> |consisting of one main thread of execution. In contrast, a >> |multithreaded program divides its work among multiple threads >> |that can run concurrently. OpenVMS supplies DECthreads, a >> |run-time library, to facilitate the creation, synchronization, >> |and deletion of multiple user-mode thredas. >> | >> |OpenVMS Alpha Version 7.0 supports multiple execution contexts >> |within a process: each execution context has its own hardware >> |context and stacks, and canexecute independently of the others; >> |all execution context with a process share the same virtual >> |address space. The kernel _kernel thread_ refers to one of >> |those execution contexts and the thread of execution running >> |within it. This volume uses the term _multithreaded_ to refer >> |to a process with multiple kernel threads. >> | >> |The kernel thread is now the basis for OpenVMS scheduling. >> |[...] > >This "kernel thread" is the definition I have been using: >unit of execution *scheduled* by the operating system. But that's not particularly relevant. The interesting bit vis spinlocks in the kernel is what threads are running in kernel mode, and how those threads synchronize between themselves to control access to kernel objects. >> It continues on page 152: >> >> |The term _kernel thread_ is a somewhat confusing name, in that >> |instructions executing in a kernel thread can run in any of the >> |four access modes, not only kernel mode. The term arose to >> |distinguish the current multithreaded model, supported within >> |the OpenVMS executive (or kernel), from the previous model, >> |supported only by DECthreads. >> >> Sadly, the later books on OpenVMS internals are more focused on >> specific subsystems (scheduling and process management; memory >> management) than the ealier volumes. Details of the >> implementation of synchronization primitives are almost entirely >> missing. >> >> There is discussion about how those primitives are _used_, >> however. [Gol03] mentions spinlocks, such as the SCHED >> spinlock, in the context of memory management data structures. >> E.g., on page 51: >> >> |The swappability of the PHD [Process Header] result in several >> |different methods for synchronizing access to fields within it. >> |Because a PHD can be inswapped to a different balance set slot >> |than it last occuped, accesses to a PHD that use its system >> |space address must be synchronized against swapper >> |interference. Accesses from a kernel thread to its own PHD can >> |be made with the SCHED spinlock held to block any rescheduling >> |and possible swapping of the process. Holding the MMG spinlock >> |is another way to block swapping. >> >> Notably here, that the language is again in terms of (software) >> threads. It seems clear enough that in the VMS context, >> spinlocks are conceptually held by threads, not cores. > >Also Gol91 VAX VMS Internals and Data Structures v5.2 1991 > >All three, Gol91, Gol94, Rus12 have a section on spinlocks and >explicitly say spinlocks synchronize processors, not threads. > >Gol91 pg 172, Gol94 pg 254: "A spinlock is acquired by a processor to >synchronize access to data shared by members of an SMP system." Also [Gol91], page 173: |A resource synchronized through elevated IPL on a uniprocessor |is synchronzied through a combination of spinlock and elevated |IPL on an SMP system. *A thread of execution* running on one |processor acquires a spinlock to serialized access to the data |with *threads of execution* running on other processors. |Before acquiring, the _thread of execution_ raises IPL to block |accesses *by other threads of execution* running on the *same* |processor. Emphasis added. Note that the primitives that do the acquiring and holding are threads. (Note, I already quoted this text from [Gol94] in an earlier response; page 255.) >Rus12 pg 179: "The mechanism the kernel uses to achieve multiprocessor >mutual exclusion is called a spinlock." Yes, I already pointed out that this is the usual case. But you are still ignoring the uniprocessor case: spinlocks must work there, as well. But more relevantly, this does not imply that _processors_ "own" spinlocks in the way you have asserted. That is done by threads. A thread does not magically "turn into" a processor when it acquires a spinlock; it remains a thread, albeit one that is now holding a spinlock that it wasn't before. >Furthermore a search for the word "spinlock " in all three documents >finds repeated references to "processor spinlock". There are NO usages >of "process spinlock" on VMS or "thread spinlock" on WinNT. I don't know what "process spinlock" you are referring to on VMS or why you are mentioning it; this seems to be a strawman. With respect to Windows, unless you have some very specific meaning in mind for the term "thread spinlock", but if one takes this to be a generic search for the two terms "thread" and "spinlock" to see if they occur together, this is demonstrably false: I have repeatedly provided specific references and quotations showing that Windows refers to _threads_ acquiring spinlocks. >It is mutexes that synchronize VMS processes or WNT threads. I don't know what processes on VMS have to do with it. The available documentation quite clearly says that spinlocks are one of the synchronization mechanisms used by VMS to serialize access to critical sections among multiple threads of execution, both on the same processor and on other processors in multiprocessor configurations. Remember: my objection to your terminology is that cores/processors are the things that hold spinlocks. >The difference is spinlocks do not invoke the scheduler to trigger >an executable context switch, whereas mutexes may invoke the scheduler. That's obvious, and I've stated as much repeatedly. Refer to my earlier posts where I drew a distinction between blocking lock protocols as opposed to spinning protocols. Actually, to quote myself from earlier in the thread: |A spin lock is simply any lock where you spin trying to acquire |the lock, as opposed to a blocking synchronization protocol. |Here I'm using the terminology of Herlihy and Shavit [Her08]. | |Traditional Unix "sleep" and "wakeup" are an example of a |blocking protocol, where a thread may "sleep" on some |"channel", yielding the locking thread while the lock cannot be |acquired, presumably scheduling something else until the thread |is later marked runnable by virtual of something else calling |"wakeup" on the sleep channel. >There is a section on mutexes in >Gol91 pg 196, Gol04 pg 277, Rus12 pg 183 Low-IRQL Synchronization This is all presented wildly out of context, and doesn't support your earlier assertion, that spinlocks are held by cores and not threads. >Gol91 pg 25, Gol94 pg 29 >"Thus, the VMS executive requires a third synchronization tool to allow >synchronized access to pageable data structures. This tool must also allow >a process to be removed from execution while it maintains ownership of the >structure in question. One synchronization tool that fulfills these >requirements is called a mutual exclusion semaphore (mutex)." Yup. Use of VMS "mutex" primitives is a blocking protocol, as described above. >Gol91 pg 196, Gol94 pg 277 >"However, in some situations requiring synchronization, elevated IPL is an >unacceptable technique." >... >"Typically, the threads of execution whose accesses are synchronized >through a mutex are process context threads." > >Rus12 pg 183 Low-IRQL Synchronization >"Because waiting for a spinlock literally stalls a processor, spinlocks >can be used only under the following strictly limited circumstances" You omitted important context that makes this clearer. This whole section begins, "Executive software outside the kernel also needs to synchronize access to global data structures in a multiprocessor environment." Here, they're describing other mechanisms for achieving synchronization between competing software running on different CPUs. Here's the full quote corresponding to the snippet above: |Spinlocks only partially fill the executive's need for |synchronization mechanisms, however. Because waiting for a |spinlock literally stalls a processor, spinlocks can only be |used under the following strictly limited circumstances: | |* The protected resource must be access quickly and without | complicated interactions with other code. |* The critical section code can't be paged out of memory, | can't make references to pageable data, can't call external | procedures (including system services), and can't generate | interrupts or exceptions. (One presumes they mean synchronous interrupts, here; as we've seen, asynchronous interrupts are blocked by means of setting the IRQL.) But this is all well-known; and the first point is also something I've repeatedly throughout this discussion. Quoting myself again: |Spinlocks, of the form I was describing, are an optimization |for cases where the critical section guarded by the lock is |short, and the overhead of invoking a scheduler and context |switching to some other thread is higher than just spinning |until the lock becomes available again. The whole point is |to avoid blocking and switching. These criteria from Russinovich et al are entirely consistent with this. >"There are several additional synchronization mechanisms for use when >spinlocks are not suitable: >- Kernel dispatcher objects >- Fast mutexes and guarded mutexes >- Pushlocks >- Executive resources > >Additionally, user-mode code, which also executes at low IRQL, must be able >to have its own locking primitives. Windows supports various >user-mode-specific primitives: >- Condition variables (CondVars) >- Slim Reader-Writer Locks (SRW Locks) >- Run-once initialization (InitOnce) >- Critical sections" > >These books consistently refer to spinlocks synchronize *processors*, No, they really do not, as the other direct quotations I have provided show. Spinlocks synchronize threads of execution, either on uniprocessor or multiprocessor systems. On the latter, they synchronize between multiple threads of execution either locally or between processor cores; on uniprocessors. Really, this is standard and normal, and is pretty much how all relevant systems describe it, including Windows and VMS. >aka cores, at raised IPL (>= dispatch), and mutexes synchronize scheduled >processes/threads at lower IPL (< dispatch). See above. >The reason is that when a kernel thread, executing a system call, >raises the processor >= dispatch, which is required to acquire a spinlock, >it stops being a scheduled thread, and may have temporarily switched >kernel stacks as processor interrupt levels require their own stacks. I think perhaps you mean that processor _modes_, which on x86_64 means the processor's current protection ring, require their own stacks. Windows supports three broad categories of stacks: for each kernel-managed thread, a user stack and a kernel stack, and for each physical processor, a DPC stack (since, as Russinovich et al put it, "DPC's run in arbitrary thread context.") When a thread is running in user mode (ring 3), it's running on its user-mode stack; when running in kernel mode (ring 0), it's running on its kernel-mode stack. DPCs are run on their own stack (presumably switched to via some kind of trampoline). Systems running on x86 usually also have a handful of special stacks for specific types of interrupts (NMIs, double-fault handlers, debug interrupts, and machine-check exceptions come to mind). A field in the IDT entry for any given interrupt/exception vector points to the index in the interrupt stack table in the TSS that should be used for that interrupt/exception; if not specified, the default ring 0 stack pointed to by the RSP0 entry is used. If an trap occurs while running in ring 3, the appropriate stack for the trap will be selected and the processor will switch to it automatically. If already running in ring 0, this won't happen unless the trap vector entry in the IDT specifies a non-zero IST index, in which case that stack will be selected and switched to unconditionally. Note that there are also fields in the TSS for stacks for rings 1 and 2, but almost no one uses those (Windows does not), in large part because the page table format used by Intel and AMD only distinguishes between 0 and 3 for page-level permissions. The sysenter/syscall mechanism is a little more involved here (in particular, it does not automatically load the stack pointer register to point to the kernel-mode stack), and the system call entry point (e.g., the address pointed to by the LSTAR MSR on x64 when considering SYSCALL) must be carefully written. Since on entry to the code pointed at by LSTAR the stack pointer for the currently executing thread is not changed, %rsp still points to the user-mode stack. But the processor is in kernel mode, so interrupt delivery won't generally trigger switching to the stack in RSP0 in the TSS (if specified in the IDT entry for a given trap, switching to one of the other stacks in the IST will happen, but those are typically reserved for a limited set of traps that require special handling with respect to stacks; there are only seven IST entries in the TSS anyway). So a system may inhibit interrupt delivery for a short time until it has saved user-mode state and set up it's kernel mode stack. But systems need not inhibit interrupt delivery while servicing most system calls, and indeed, they often do not. Importantly, inhibiting interrupt delivery initially isn't to avoid being rescheduled, but rather to protect the kernel from faulting if the user mode stack is trashed (and to avoid accidentally leaking kernel data into user mode via taking an interrupt on the user-mode stack). >At raised IPL many of the OS data structures associated with a kernel thread >cannot be touched as they might trigger a page fault which can only be >serviced by a kernel thread which can sleep, and not by processors/cores >which can only spin-wait. (Both VMS and WNT will crash/panic if code >page faults at IPL >= dispatch for exactly this reason.) No, that's not really the issue. A page fault is a synchronous exception; similarly with a software interrupt (e.g., the effect of the `INT` instruction on x86). These are always delivered regardless of what the current thread has set the interrupt priority threshold to on the current processor, or whether it has locally inhibited interrupt delivery (e.g., via the `CLI` instruction on x86_64). If such were to be allowed while the current thread is holding a spinlock, then the trap handler could try and acquire that same spinlock (perhaps indirectly), which would result in a deadlock. Similarly with why threads holding spinlocks shouldn't invoke system calls themselves. The statement, "...and not by processors/cores which can only spin-wait" is not meaningful. Again "processors" and "cores" are hardware devices, not software objects. They are not restricted to spin-waiting; they just execute instructions and do other processor-y things. You continue to try and force these terms to mean something else but still have not produced any reference that uses those terms in that manner. >All of this is for controlling access to non-reentrant shared data. >Non-reentrant data shared between processors/cores is controlled by >IPL and blocked by spinlocks. Non-reentrant data shared between kernel >threads is controlled with scheduler synchronization and thread switching. I'm afraid you've got the details rather muddled. For instance, saying that, "non-reentrant data shared between processors/cores is controlled by IPL and blocked by spinlocks" is rather meaningless; those words just don't related to each other that way. The correct thing to say is that spinlocks are mutual exclusion structures that can be used to serialize access to data shared between multiple concurrent threads of execution, running either on the same processor or in parallel across multiple processors. You also don't explain how "non-reentrant data shared between kernel threads" is appreciably different from, "non-reentrant data shared between processors/cores." For that matter, one does not usually speak of _data_ as being reentrant; rather, reentrancy is a concept one usually applies to programs. I think this stems from this idea you seem have that threads somehow turn into "processors/cores", whatever that means, as these are obviously not the hardware devices, when interrupts are disabled on the CPUs they are running on. Near as I can tell this is your own unique invention, and nothing else uses that terminology; use by the systems you cited is not supported by evidence. Consequently, I find your repeated assertions about these terms rather strange. - Dan C.
[toc] | [prev] | [next] | [standalone]
| From | Robert Finch <robfi680@gmail.com> |
|---|---|
| Date | 2025-03-31 01:30 -0400 |
| Message-ID | <vsd9ak$3e0u4$1@dont-email.me> |
| In reply to | #111380 |
On 2025-03-30 11:33 p.m., Dan Cross wrote: > In article <tWhGP.1790040$TBhc.304536@fx16.iad>, > EricP <ThatWouldBeTelling@thevillage.com> wrote: >> Dan Cross wrote: >>> In article <t3CFP.529522$f81.217510@fx48.iad>, >>> EricP <ThatWouldBeTelling@thevillage.com> wrote: >>>> Dan Cross wrote: >>>>> In article <8uzEP.115550$Xq5f.66883@fx38.iad>, >>>>> EricP <ThatWouldBeTelling@thevillage.com> wrote: >>>>>> Dan Cross wrote: >>>>>>> In article <kVgEP.1277108$_N6e.605199@fx17.iad>, >>>>>>> EricP <ThatWouldBeTelling@thevillage.com> wrote: >>>>>>>> Dan Cross wrote: >>>>>>>>> [snip] >>>>>>>>> Consider a thread that takes a spinlock; suppose some >>>>>>>>> high-priority interrupt comes in while the thread is holding >>>>>>>>> that lock. In response to the interrupt, software decides to >>>>>>>>> suspend the thread and switch some other thread; that thread >>>>>>>>> wants to lock the spin lock that the now-descheduled thread is >>>>>>>>> holding: a classic deadlock scenario. >>>>>>>> Terminology: mutexes coordinate mutual exclusion between threads, >>>>>>>> spinlocks coordinate mutual exclusion between cpu cores. >>>>>>>> Windows "critical sections" are mutexes with a fast path. >>>>>>> A spin lock is simply any lock where you spin trying to acquire >>>>>>> the lock, as opposed to a blocking synchronization protocol. >>>>>>> Here I'm using the terminology of Herlihy and Shavit [Her08]. >>>>>> I'm using terminology in common use on multiprocessor systems like >>>>>> VMS since 1980's and later WinNT. >>>>>> >>>>>> Each OS data structure has different reentrancy requirements. >>>>>> - Ones accessed by threads are guarded by mutexes. >>>>>> Example on Windows would be the page tables as paging is done by threads. >>>>>> - Ones accessed by OS software interrupt level are guarded by spinlocks. >>>>>> Example on Windows is the Scheduler tables. >>>>>> - Ones accessed by HW interrupts are guarded by interrupt spinlocks. >>>>>> Example on Windows is device interrupt service routines. >>>>> I'm not sure what you're saying here, or rather, how it is >>>>> relevant. >>>>> >>>>> Your earlier statement was that "mutexes coordinate mutual >>>>> exclusion between threads, spinlocks coordinate mutual exclusion >>>>> between cpu cores." But this is not the case. >>>>> >>>>> Taking VMS as an example, from [Gol94], sec 9.5 (page 255): >>>>> >>>>> |A thread of execution running on one processor acquires a >>>>> |spinlock to serialize access to the data with threads of >>>>> |execution running on other processors. Before acquiring >>>>> |the spinlock, the thread of executions raises IPL to block >>>>> |accesses by other threads of execution running on the same >>>>> |processor. The IPL value is determined by the spinlock being >>>>> |locked. >>>>> >>>>> Goldberg and Saravanan seems quite explicit that threads can >>>>> acquire and hold spinlocks on OpenVMS, which seems to be >>>>> something that you were objecting to? >>>> It took me a while to find copy of that book online. >>>> That 1994 book relates to VMS before they added kernel threads to the OS. >>> >>> Before you said you were using terminology commonly used on >>> systems "since [the] 1980s". The systems you cited were VMS and >>> Windows. I quoted references about both systems from the mid >>> 1990s and mid 2000s, respectively. If that terminology were >>> common in the 1980s, as you suggested, surely it would be in a >>> book from 1994? Regardless, see below. >> >> SMP OS (synchronized with spinlocks) was added to VMS is 1988. >> VMS clusters were released in 1983 with its locking system and >> cluster read-write locks can be used to build other synchronization >> structures like mutexes, semaphores, quorums. >> >> I don't know when the term "thread" meaning "scheduled units of execution >> within a common address space" first show up. I found a reference to it on >> Unix in 1989. Previously the term "task" was used to mean the same thing, >> such as Ada tasks which on DEC Ada in 1986 were similar to what are now >> called user mode threads. It looks to me that the DEC Ada runtime library >> was later morphed into DECThreads user mode library released in 1991. > > The point is that if, as you assert, Windows and VMS both use > the terminology of _cores_ (or processors) acquiring and holding > spinlocks instead of threads doing so, there should be some > reference to that in those books. There is not. The > terminology on those systems is consistent: threads are the > things that hold spinlocks; it's axiomatic that those threads > are running on processors, since the whole point is that it is > faster to busy-wait on the spinlock rather than potentially > context switch as with a different mutual exclusion primtiive, > like what VMS calles a mutex (which is a specific software > object, not the generic concept of a "mutex" as a mutual > exclusion primitive. In the latter context, a spinlock is a > kind of mutex; in the former, it's different). > >>>> At that point VMS scheduled processes, an execution context plus virtual >>>> space, like most *nix (except I think Solaris which had kernel threads). >>>> >>>> It says on pg 3: "The term thread of execution as used in this book >>>> refers to a computational entity, an agent of execution." >>>> This refers generally to the various code sequences from different context >>>> that may be intertwined on a processors core for interrupts, exceptions, >>>> Dpc (called driver forks in VMS), process AST's (software interrupts), >>>> and applications. >>>> >>>> In a later book on VMS after they added kernel scheduled threads they >>>> consistently use the two terms "thread of execution" and "kernel thread" >>>> to distinguish the two meanings. >>> >>> Are you referring to the 1997 revision of Goldenberg et al, >>> that covers VMS 7.0? [Gol97] In that context, VMS "kernel >>> threads" refer to threads of execution within a program that >>> are managed and scheduled by the VMS executive (aka kernel). >> >> I saw reference to 1997 internals book and that kernel threads were added. >> However the only other OpenVMS internals document I could find was >> OpenVMS Alpha Internals and Data Structures - Memory Management, >> Goldenberg et al, 2003. > > Ok, I have that one, too. > >>> This is described on page 151, section 5.1: >>> >>> |A thread of execution is a sequential flow of instruction >>> |executiuon. A traditional program is single-threaded, >>> |consisting of one main thread of execution. In contrast, a >>> |multithreaded program divides its work among multiple threads >>> |that can run concurrently. OpenVMS supplies DECthreads, a >>> |run-time library, to facilitate the creation, synchronization, >>> |and deletion of multiple user-mode thredas. >>> | >>> |OpenVMS Alpha Version 7.0 supports multiple execution contexts >>> |within a process: each execution context has its own hardware >>> |context and stacks, and canexecute independently of the others; >>> |all execution context with a process share the same virtual >>> |address space. The kernel _kernel thread_ refers to one of >>> |those execution contexts and the thread of execution running >>> |within it. This volume uses the term _multithreaded_ to refer >>> |to a process with multiple kernel threads. >>> | >>> |The kernel thread is now the basis for OpenVMS scheduling. >>> |[...] >> >> This "kernel thread" is the definition I have been using: >> unit of execution *scheduled* by the operating system. > > But that's not particularly relevant. The interesting bit vis > spinlocks in the kernel is what threads are running in kernel > mode, and how those threads synchronize between themselves to > control access to kernel objects. > >>> It continues on page 152: >>> >>> |The term _kernel thread_ is a somewhat confusing name, in that >>> |instructions executing in a kernel thread can run in any of the >>> |four access modes, not only kernel mode. The term arose to >>> |distinguish the current multithreaded model, supported within >>> |the OpenVMS executive (or kernel), from the previous model, >>> |supported only by DECthreads. >>> >>> Sadly, the later books on OpenVMS internals are more focused on >>> specific subsystems (scheduling and process management; memory >>> management) than the ealier volumes. Details of the >>> implementation of synchronization primitives are almost entirely >>> missing. >>> >>> There is discussion about how those primitives are _used_, >>> however. [Gol03] mentions spinlocks, such as the SCHED >>> spinlock, in the context of memory management data structures. >>> E.g., on page 51: >>> >>> |The swappability of the PHD [Process Header] result in several >>> |different methods for synchronizing access to fields within it. >>> |Because a PHD can be inswapped to a different balance set slot >>> |than it last occuped, accesses to a PHD that use its system >>> |space address must be synchronized against swapper >>> |interference. Accesses from a kernel thread to its own PHD can >>> |be made with the SCHED spinlock held to block any rescheduling >>> |and possible swapping of the process. Holding the MMG spinlock >>> |is another way to block swapping. >>> >>> Notably here, that the language is again in terms of (software) >>> threads. It seems clear enough that in the VMS context, >>> spinlocks are conceptually held by threads, not cores. >> >> Also Gol91 VAX VMS Internals and Data Structures v5.2 1991 >> >> All three, Gol91, Gol94, Rus12 have a section on spinlocks and >> explicitly say spinlocks synchronize processors, not threads. >> >> Gol91 pg 172, Gol94 pg 254: "A spinlock is acquired by a processor to >> synchronize access to data shared by members of an SMP system." > > Also [Gol91], page 173: > > |A resource synchronized through elevated IPL on a uniprocessor > |is synchronzied through a combination of spinlock and elevated > |IPL on an SMP system. *A thread of execution* running on one > |processor acquires a spinlock to serialized access to the data > |with *threads of execution* running on other processors. > |Before acquiring, the _thread of execution_ raises IPL to block > |accesses *by other threads of execution* running on the *same* > |processor. > > Emphasis added. Note that the primitives that do the acquiring > and holding are threads. > > (Note, I already quoted this text from [Gol94] in an earlier > response; page 255.) > >> Rus12 pg 179: "The mechanism the kernel uses to achieve multiprocessor >> mutual exclusion is called a spinlock." > > Yes, I already pointed out that this is the usual case. But you > are still ignoring the uniprocessor case: spinlocks must work > there, as well. > > But more relevantly, this does not imply that _processors_ "own" > spinlocks in the way you have asserted. That is done by > threads. A thread does not magically "turn into" a processor > when it acquires a spinlock; it remains a thread, albeit one > that is now holding a spinlock that it wasn't before. > >> Furthermore a search for the word "spinlock " in all three documents >> finds repeated references to "processor spinlock". There are NO usages >> of "process spinlock" on VMS or "thread spinlock" on WinNT. > > I don't know what "process spinlock" you are referring to on VMS > or why you are mentioning it; this seems to be a strawman. > > With respect to Windows, unless you have some very specific > meaning in mind for the term "thread spinlock", but if one takes > this to be a generic search for the two terms "thread" and > "spinlock" to see if they occur together, this is demonstrably > false: I have repeatedly provided specific references and > quotations showing that Windows refers to _threads_ acquiring > spinlocks. > >> It is mutexes that synchronize VMS processes or WNT threads. > > I don't know what processes on VMS have to do with it. The > available documentation quite clearly says that spinlocks are > one of the synchronization mechanisms used by VMS to serialize > access to critical sections among multiple threads of execution, > both on the same processor and on other processors in > multiprocessor configurations. Remember: my objection to your > terminology is that cores/processors are the things that hold > spinlocks. > >> The difference is spinlocks do not invoke the scheduler to trigger >> an executable context switch, whereas mutexes may invoke the scheduler. > > That's obvious, and I've stated as much repeatedly. Refer to my > earlier posts where I drew a distinction between blocking > lock protocols as opposed to spinning protocols. Actually, to > quote myself from earlier in the thread: > > |A spin lock is simply any lock where you spin trying to acquire > |the lock, as opposed to a blocking synchronization protocol. > |Here I'm using the terminology of Herlihy and Shavit [Her08]. > | > |Traditional Unix "sleep" and "wakeup" are an example of a > |blocking protocol, where a thread may "sleep" on some > |"channel", yielding the locking thread while the lock cannot be > |acquired, presumably scheduling something else until the thread > |is later marked runnable by virtual of something else calling > |"wakeup" on the sleep channel. > >> There is a section on mutexes in >> Gol91 pg 196, Gol04 pg 277, Rus12 pg 183 Low-IRQL Synchronization > > This is all presented wildly out of context, and doesn't support > your earlier assertion, that spinlocks are held by cores and not > threads. > >> Gol91 pg 25, Gol94 pg 29 >> "Thus, the VMS executive requires a third synchronization tool to allow >> synchronized access to pageable data structures. This tool must also allow >> a process to be removed from execution while it maintains ownership of the >> structure in question. One synchronization tool that fulfills these >> requirements is called a mutual exclusion semaphore (mutex)." > > Yup. Use of VMS "mutex" primitives is a blocking protocol, as > described above. > >> Gol91 pg 196, Gol94 pg 277 >> "However, in some situations requiring synchronization, elevated IPL is an >> unacceptable technique." >> ... >> "Typically, the threads of execution whose accesses are synchronized >> through a mutex are process context threads." >> >> Rus12 pg 183 Low-IRQL Synchronization >> "Because waiting for a spinlock literally stalls a processor, spinlocks >> can be used only under the following strictly limited circumstances" > > You omitted important context that makes this clearer. This > whole section begins, "Executive software outside the kernel > also needs to synchronize access to global data structures in > a multiprocessor environment." Here, they're describing other > mechanisms for achieving synchronization between competing > software running on different CPUs. > > Here's the full quote corresponding to the snippet above: > > |Spinlocks only partially fill the executive's need for > |synchronization mechanisms, however. Because waiting for a > |spinlock literally stalls a processor, spinlocks can only be > |used under the following strictly limited circumstances: > | > |* The protected resource must be access quickly and without > | complicated interactions with other code. > |* The critical section code can't be paged out of memory, > | can't make references to pageable data, can't call external > | procedures (including system services), and can't generate > | interrupts or exceptions. > > (One presumes they mean synchronous interrupts, here; as we've > seen, asynchronous interrupts are blocked by means of setting > the IRQL.) > > But this is all well-known; and the first point is also > something I've repeatedly throughout this discussion. Quoting > myself again: > > |Spinlocks, of the form I was describing, are an optimization > |for cases where the critical section guarded by the lock is > |short, and the overhead of invoking a scheduler and context > |switching to some other thread is higher than just spinning > |until the lock becomes available again. The whole point is > |to avoid blocking and switching. > > These criteria from Russinovich et al are entirely consistent > with this. > >> "There are several additional synchronization mechanisms for use when >> spinlocks are not suitable: >> - Kernel dispatcher objects >> - Fast mutexes and guarded mutexes >> - Pushlocks >> - Executive resources >> >> Additionally, user-mode code, which also executes at low IRQL, must be able >> to have its own locking primitives. Windows supports various >> user-mode-specific primitives: >> - Condition variables (CondVars) >> - Slim Reader-Writer Locks (SRW Locks) >> - Run-once initialization (InitOnce) >> - Critical sections" >> >> These books consistently refer to spinlocks synchronize *processors*, > > No, they really do not, as the other direct quotations I have > provided show. > > Spinlocks synchronize threads of execution, either on > uniprocessor or multiprocessor systems. On the latter, they > synchronize between multiple threads of execution either locally > or between processor cores; on uniprocessors. > > Really, this is standard and normal, and is pretty much how all > relevant systems describe it, including Windows and VMS. > >> aka cores, at raised IPL (>= dispatch), and mutexes synchronize scheduled >> processes/threads at lower IPL (< dispatch). > > See above. > >> The reason is that when a kernel thread, executing a system call, >> raises the processor >= dispatch, which is required to acquire a spinlock, >> it stops being a scheduled thread, and may have temporarily switched >> kernel stacks as processor interrupt levels require their own stacks. > > I think perhaps you mean that processor _modes_, which on x86_64 > means the processor's current protection ring, require their own > stacks. Windows supports three broad categories of stacks: for > each kernel-managed thread, a user stack and a kernel stack, and > for each physical processor, a DPC stack (since, as Russinovich > et al put it, "DPC's run in arbitrary thread context.") > > When a thread is running in user mode (ring 3), it's running on > its user-mode stack; when running in kernel mode (ring 0), it's > running on its kernel-mode stack. DPCs are run on their own > stack (presumably switched to via some kind of trampoline). > Systems running on x86 usually also have a handful of special > stacks for specific types of interrupts (NMIs, double-fault > handlers, debug interrupts, and machine-check exceptions come to > mind). A field in the IDT entry for any given > interrupt/exception vector points to the index in the interrupt > stack table in the TSS that should be used for that > interrupt/exception; if not specified, the default ring 0 stack > pointed to by the RSP0 entry is used. If an trap occurs while > running in ring 3, the appropriate stack for the trap will be > selected and the processor will switch to it automatically. If > already running in ring 0, this won't happen unless the trap > vector entry in the IDT specifies a non-zero IST index, in which > case that stack will be selected and switched to > unconditionally. Note that there are also fields in the TSS for > stacks for rings 1 and 2, but almost no one uses those (Windows > does not), in large part because the page table format used by > Intel and AMD only distinguishes between 0 and 3 for page-level > permissions. > > The sysenter/syscall mechanism is a little more involved here > (in particular, it does not automatically load the stack pointer > register to point to the kernel-mode stack), and the system call > entry point (e.g., the address pointed to by the LSTAR MSR on > x64 when considering SYSCALL) must be carefully written. > > Since on entry to the code pointed at by LSTAR the stack pointer > for the currently executing thread is not changed, %rsp still > points to the user-mode stack. But the processor is in kernel > mode, so interrupt delivery won't generally trigger switching to > the stack in RSP0 in the TSS (if specified in the IDT entry for > a given trap, switching to one of the other stacks in the IST > will happen, but those are typically reserved for a limited set > of traps that require special handling with respect to stacks; > there are only seven IST entries in the TSS anyway). So a > system may inhibit interrupt delivery for a short time until it > has saved user-mode state and set up it's kernel mode stack. > But systems need not inhibit interrupt delivery while servicing > most system calls, and indeed, they often do not. Importantly, > inhibiting interrupt delivery initially isn't to avoid being > rescheduled, but rather to protect the kernel from faulting if > the user mode stack is trashed (and to avoid accidentally > leaking kernel data into user mode via taking an interrupt on > the user-mode stack). > >> At raised IPL many of the OS data structures associated with a kernel thread >> cannot be touched as they might trigger a page fault which can only be >> serviced by a kernel thread which can sleep, and not by processors/cores >> which can only spin-wait. (Both VMS and WNT will crash/panic if code >> page faults at IPL >= dispatch for exactly this reason.) > > No, that's not really the issue. > > A page fault is a synchronous exception; similarly with a > software interrupt (e.g., the effect of the `INT` instruction on > x86). These are always delivered regardless of what the current > thread has set the interrupt priority threshold to on the > current processor, or whether it has locally inhibited interrupt > delivery (e.g., via the `CLI` instruction on x86_64). If such > were to be allowed while the current thread is holding a > spinlock, then the trap handler could try and acquire that same > spinlock (perhaps indirectly), which would result in a deadlock. > Similarly with why threads holding spinlocks shouldn't invoke > system calls themselves. > > The statement, "...and not by processors/cores which can only > spin-wait" is not meaningful. Again "processors" and "cores" > are hardware devices, not software objects. They are not > restricted to spin-waiting; they just execute instructions and > do other processor-y things. > > You continue to try and force these terms to mean something else > but still have not produced any reference that uses those terms > in that manner. > >> All of this is for controlling access to non-reentrant shared data. >> Non-reentrant data shared between processors/cores is controlled by >> IPL and blocked by spinlocks. Non-reentrant data shared between kernel >> threads is controlled with scheduler synchronization and thread switching. > > I'm afraid you've got the details rather muddled. > > For instance, saying that, "non-reentrant data shared between > processors/cores is controlled by IPL and blocked by spinlocks" > is rather meaningless; those words just don't related to each > other that way. > > The correct thing to say is that spinlocks are mutual exclusion > structures that can be used to serialize access to data shared > between multiple concurrent threads of execution, running either > on the same processor or in parallel across multiple processors. > > You also don't explain how "non-reentrant data shared between > kernel threads" is appreciably different from, "non-reentrant > data shared between processors/cores." For that matter, one > does not usually speak of _data_ as being reentrant; rather, > reentrancy is a concept one usually applies to programs. > > I think this stems from this idea you seem have that threads > somehow turn into "processors/cores", whatever that means, as > these are obviously not the hardware devices, when interrupts > are disabled on the CPUs they are running on. Near as I can > tell this is your own unique invention, and nothing else uses > that terminology; use by the systems you cited is not supported > by evidence. > > Consequently, I find your repeated assertions about these terms > rather strange. > > - Dan C. > Having some trouble following the discussion as I do not have a lot of experience working with software threads. I know they need synchronization means. Are you talking about two different kinds of “threads”? RISCV at least refers to ‘harts’ which I think stands for hardware threads. A CPU core may support multiple harts, which implies multiple copies of the processor state. In a multi-core system there could be multiple harts even though each core is only supporting a single one. I am under the impression hardware and software threads are not the same thing. I hope I got the lingo correct.
[toc] | [prev] | [next] | [standalone]
| From | cross@spitfire.i.gajendra.net (Dan Cross) |
|---|---|
| Date | 2025-03-31 14:42 +0000 |
| Subject | Software synchronization (was: Re: MSI interrupts) |
| Message-ID | <vse9jo$k4$1@reader1.panix.com> |
| In reply to | #111381 |
In article <vsd9ak$3e0u4$1@dont-email.me>, Robert Finch <robfi680@gmail.com> wrote: >[Big snip] >Having some trouble following the discussion as I do not have a lot of >experience working with software threads. I know they need >synchronization means. > >Are you talking about two different kinds of “threads”? RISCV at least >refers to ‘harts’ which I think stands for hardware threads. A CPU core >may support multiple harts, which implies multiple copies of the >processor state. In a multi-core system there could be multiple harts >even though each core is only supporting a single one. I am under the >impression hardware and software threads are not the same thing. >I hope I got the lingo correct. Throughout this discussion, I have been using the terms "processor", "core", and "CPU" to refer to what Intel calls a "logical processor", which may be a core, or a thread in an SMT configuration. RISC-V "harts" are basically the same thing as x86 LPs. The way to think about this from the software perspective is that (software) threads are a unit of logical concurrency, while hardware threads (or cores, processors, whatever) are units of physical parallelism. Note that concurrency and parallelism are two different things: one can have concurrent programs running on a uniprocessor system. Here, for example, is a talk from Rob Pike where he discusses this in the context of the Go programming language: https://go.dev/blog/waza-talk. For a more formal treatment of the subject, consider: https://www.cs.cmu.edu/afs/cs/academic/class/15210-f15/www/tapp.html#_parallelism_versus_concurrency though this is a bit more approachable: https://existentialtype.wordpress.com/2011/03/17/parallelism-is-not-concurrency/ Note, however, that concurrency is not required for pallelism, as Leiserson notes here: https://www.cs.cmu.edu/~guyb/spaa/2012/workshop/leiserson-talk.pdf - Dan C.
[toc] | [prev] | [next] | [standalone]
| From | scott@slp53.sl.home (Scott Lurndal) |
|---|---|
| Date | 2025-03-31 15:28 +0000 |
| Subject | Threads (was Re: MSI interrupts) |
| Message-ID | <KwyGP.1425325$eNx6.27503@fx14.iad> |
| In reply to | #111381 |
Robert Finch <robfi680@gmail.com> writes: >On 2025-03-30 11:33 p.m., Dan Cross wrote: >> I think this stems from this idea you seem have that threads >> somehow turn into "processors/cores", whatever that means, as >> these are obviously not the hardware devices, when interrupts >> are disabled on the CPUs they are running on. Near as I can >> tell this is your own unique invention, and nothing else uses >> that terminology; use by the systems you cited is not supported >> by evidence. >> >> Consequently, I find your repeated assertions about these terms >> rather strange. >> >> - Dan C. >> >Having some trouble following the discussion as I do not have a lot of >experience working with software threads. I know they need >synchronization means. > >Are you talking about two different kinds of “threads”? RISCV at least >refers to ‘harts’ which I think stands for hardware threads. A CPU core >may support multiple harts, which implies multiple copies of the >processor state. In a multi-core system there could be multiple harts >even though each core is only supporting a single one. I am under the >impression hardware and software threads are not the same thing. >I hope I got the lingo correct. In the abstract, a thread can be considered a sequence of instructions executed with a consistent processor state (registers, flags, MMU). Threads, using that definition, can be implemented fully in software (using OS facilities for coordination between entities). User-mode threads are an example of this, and were implemented in systems that did not have Operating system support for threads as a schedulable entity. Operating systems later added explicit support for multiple threads of execution to share a single address space by defining a thread schedulable entity encapsulating the thread context (general register state, et alia). POSIX Pthreads was the standard on the Unix side. Subsequently, the hardware vendors realized that they could better utilize the hardware by creating the concept of a 'hardware thread' where the processor resources for a core could be divvied up and shared by multiple hardware contexts (Intel called it hyperthreading).
[toc] | [prev] | [next] | [standalone]
| From | Terje Mathisen <terje.mathisen@tmsw.no> |
|---|---|
| Date | 2025-04-01 16:44 +0200 |
| Subject | Re: Threads (was Re: MSI interrupts) |
| Message-ID | <vsgu3r$3blp9$1@dont-email.me> |
| In reply to | #111384 |
Scott Lurndal wrote: > Robert Finch <robfi680@gmail.com> writes: >> On 2025-03-30 11:33 p.m., Dan Cross wrote: > >>> I think this stems from this idea you seem have that threads >>> somehow turn into "processors/cores", whatever that means, as >>> these are obviously not the hardware devices, when interrupts >>> are disabled on the CPUs they are running on. Near as I can >>> tell this is your own unique invention, and nothing else uses >>> that terminology; use by the systems you cited is not supported >>> by evidence. >>> >>> Consequently, I find your repeated assertions about these terms >>> rather strange. >>> >>> - Dan C. >>> >> Having some trouble following the discussion as I do not have a lot of >> experience working with software threads. I know they need >> synchronization means. >> >> Are you talking about two different kinds of “threads� RISCV at least >> refers to ‘harts’ which I think stands for hardware threads. A CPU core >> may support multiple harts, which implies multiple copies of the >> processor state. In a multi-core system there could be multiple harts >> even though each core is only supporting a single one. I am under the >> impression hardware and software threads are not the same thing. >> I hope I got the lingo correct. > > In the abstract, a thread can be considered a sequence of instructions > executed with a consistent processor state (registers, flags, MMU). > > Threads, using that definition, can be implemented fully in software > (using OS facilities for coordination between entities). User-mode > threads are an example of this, and were implemented in systems that > did not have Operating system support for threads as a schedulable > entity. > > Operating systems later added explicit support for multiple threads > of execution to share a single address space by defining a thread > schedulable entity encapsulating the thread context (general register > state, et alia). POSIX Pthreads was the standard on the Unix side. > > Subsequently, the hardware vendors realized that they could better > utilize the hardware by creating the concept of a 'hardware thread' > where the processor resources for a core could be divvied up and > shared by multiple hardware contexts (Intel called it hyperthreading). > In my programmer world view, a pure user software thread isn't that interesting, it typically allows the programmer to control (more or less) when it can be interrupted. A harware thread is anything which could, at any time (even in the middle of a long-running instruction like a block move), interrupt the currently executing thread and potentially modify anything in the local context which isn't currently locked down. I.e. just like a separate processes running on another core, except that (nearly) nothing is really private. This makes for a much more "interesting" programing environment. Terje -- - <Terje.Mathisen at tmsw.no> "almost all programming can be viewed as an exercise in caching"
[toc] | [prev] | [next] | [standalone]
| From | scott@slp53.sl.home (Scott Lurndal) |
|---|---|
| Date | 2025-04-01 15:31 +0000 |
| Subject | Re: Threads (was Re: MSI interrupts) |
| Message-ID | <0FTGP.567700$f81.368330@fx48.iad> |
| In reply to | #111393 |
Terje Mathisen <terje.mathisen@tmsw.no> writes: >Scott Lurndal wrote: >> In the abstract, a thread can be considered a sequence of instructions >> executed with a consistent processor state (registers, flags, MMU). >>=20 >> Threads, using that definition, can be implemented fully in software >> (using OS facilities for coordination between entities). User-mode >> threads are an example of this, and were implemented in systems that >> did not have Operating system support for threads as a schedulable >> entity. >>=20 >> Operating systems later added explicit support for multiple threads >> of execution to share a single address space by defining a thread >> schedulable entity encapsulating the thread context (general register >> state, et alia). POSIX Pthreads was the standard on the Unix side. >>=20 >> Subsequently, the hardware vendors realized that they could better >> utilize the hardware by creating the concept of a 'hardware thread' >> where the processor resources for a core could be divvied up and >> shared by multiple hardware contexts (Intel called it hyperthreading). >>=20 > >In my programmer world view, a pure user software thread isn't that=20 >interesting, it typically allows the programmer to control (more or=20 >less) when it can be interrupted. Using the pre-pthreads posix facilities (signals and getcontext/setcontext) to support user-level threads wasn't uncommon. Unix SVR4ES/MP actually had both kernel threads (called lightweight processes (LWP)) and user-level threads in an M-N setup (M user threads multiplexed on N kernel threads). Didn't turn out to be particularly useful.
[toc] | [prev] | [next] | [standalone]
| From | "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> |
|---|---|
| Date | 2025-04-01 12:49 -0700 |
| Subject | Re: Threads (was Re: MSI interrupts) |
| Message-ID | <vshg06$3slc6$1@dont-email.me> |
| In reply to | #111394 |
On 4/1/2025 8:31 AM, Scott Lurndal wrote: > Terje Mathisen <terje.mathisen@tmsw.no> writes: >> Scott Lurndal wrote: > >>> In the abstract, a thread can be considered a sequence of instructions >>> executed with a consistent processor state (registers, flags, MMU). >>> =20 >>> Threads, using that definition, can be implemented fully in software >>> (using OS facilities for coordination between entities). User-mode >>> threads are an example of this, and were implemented in systems that >>> did not have Operating system support for threads as a schedulable >>> entity. >>> =20 >>> Operating systems later added explicit support for multiple threads >>> of execution to share a single address space by defining a thread >>> schedulable entity encapsulating the thread context (general register >>> state, et alia). POSIX Pthreads was the standard on the Unix side. >>> =20 >>> Subsequently, the hardware vendors realized that they could better >>> utilize the hardware by creating the concept of a 'hardware thread' >>> where the processor resources for a core could be divvied up and >>> shared by multiple hardware contexts (Intel called it hyperthreading). >>> =20 >> >> In my programmer world view, a pure user software thread isn't that=20 >> interesting, it typically allows the programmer to control (more or=20 >> less) when it can be interrupted. > > Using the pre-pthreads posix facilities (signals and getcontext/setcontext) > to support user-level threads wasn't uncommon. Unix SVR4ES/MP actually had > both kernel threads (called lightweight processes (LWP)) and user-level > threads in an M-N setup (M user threads multiplexed on N kernel threads). > > Didn't turn out to be particularly useful. Are you referring to green threads? Or Fibers?
[toc] | [prev] | [next] | [standalone]
Page 8 of 11 — ← Prev page 1 … 6 7 [8] 9 10 11 Next page →
Back to top | Article view | comp.arch
csiph-web