Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > comp.arch > #111126 > unrolled thread
| Started by | Robert Finch <robfi680@gmail.com> |
|---|---|
| First post | 2025-03-13 00:50 -0400 |
| Last post | 2025-03-17 17:33 -0400 |
| Articles | 20 on this page of 211 — 12 participants |
Back to article view | Back to comp.arch
MSI interrupts Robert Finch <robfi680@gmail.com> - 2025-03-13 00:50 -0400
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-13 16:34 +0000
Re: MSI interrupts Robert Finch <robfi680@gmail.com> - 2025-03-13 13:49 -0400
Re: MSI interrupts scott@slp53.sl.home (Scott Lurndal) - 2025-03-13 18:42 +0000
Re: MSI interrupts Robert Finch <robfi680@gmail.com> - 2025-03-13 15:59 -0400
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-13 20:46 +0000
Re: MSI interrupts scott@slp53.sl.home (Scott Lurndal) - 2025-03-13 21:39 +0000
Re: MSI interrupts Stefan Monnier <monnier@iro.umontreal.ca> - 2025-03-13 17:00 -0400
Re: MSI interrupts scott@slp53.sl.home (Scott Lurndal) - 2025-03-13 23:27 +0000
Re: MSI interrupts scott@slp53.sl.home (Scott Lurndal) - 2025-03-13 21:27 +0000
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-13 19:12 +0000
Re: MSI interrupts scott@slp53.sl.home (Scott Lurndal) - 2025-03-13 18:34 +0000
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-13 19:28 +0000
Re: MSI interrupts scott@slp53.sl.home (Scott Lurndal) - 2025-03-13 21:14 +0000
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-13 21:34 +0000
Re: MSI interrupts scott@slp53.sl.home (Scott Lurndal) - 2025-03-13 23:34 +0000
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-13 23:52 +0000
Re: MSI interrupts scott@slp53.sl.home (Scott Lurndal) - 2025-03-14 14:52 +0000
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-14 15:34 +0000
Re: MSI interrupts scott@slp53.sl.home (Scott Lurndal) - 2025-03-14 17:35 +0000
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-14 18:12 +0000
Re: MSI interrupts scott@slp53.sl.home (Scott Lurndal) - 2025-03-14 18:42 +0000
Re: MSI interrupts Robert Finch <robfi680@gmail.com> - 2025-03-14 22:10 -0400
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-15 17:46 +0000
Re: MSI interrupts Robert Finch <robfi680@gmail.com> - 2025-03-15 16:46 -0400
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-15 22:28 +0000
Re: MSI interrupts Robert Finch <robfi680@gmail.com> - 2025-03-16 17:35 -0400
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-16 22:00 +0000
Re: MSI interrupts scott@slp53.sl.home (Scott Lurndal) - 2025-03-17 13:38 +0000
Re: MSI interrupts Michael S <already5chosen@yahoo.com> - 2025-03-17 16:11 +0200
Re: MSI interrupts Robert Finch <robfi680@gmail.com> - 2025-03-17 11:23 -0400
Re: MSI interrupts scott@slp53.sl.home (Scott Lurndal) - 2025-03-17 15:44 +0000
Re: MSI interrupts EricP <ThatWouldBeTelling@thevillage.com> - 2025-03-17 14:33 -0400
Re: MSI interrupts Robert Finch <robfi680@gmail.com> - 2025-03-17 17:14 -0400
Re: MSI interrupts EricP <ThatWouldBeTelling@thevillage.com> - 2025-03-18 14:51 -0400
Re: MSI interrupts Robert Finch <robfi680@gmail.com> - 2025-03-18 17:59 -0400
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-18 23:00 +0000
Re: MSI interrupts scott@slp53.sl.home (Scott Lurndal) - 2025-03-19 13:49 +0000
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-19 15:54 +0000
Re: MSI interrupts scott@slp53.sl.home (Scott Lurndal) - 2025-03-19 16:14 +0000
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-19 19:03 +0000
Re: MSI interrupts cross@spitfire.i.gajendra.net (Dan Cross) - 2025-03-19 14:03 +0000
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-19 15:59 +0000
Re: MSI interrupts cross@spitfire.i.gajendra.net (Dan Cross) - 2025-03-20 12:44 +0000
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-20 19:41 +0000
Re: MSI interrupts cross@spitfire.i.gajendra.net (Dan Cross) - 2025-03-20 20:25 +0000
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-20 21:55 +0000
Re: MSI interrupts cross@spitfire.i.gajendra.net (Dan Cross) - 2025-03-24 12:37 +0000
Re: MSI interrupts scott@slp53.sl.home (Scott Lurndal) - 2025-03-24 13:59 +0000
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-24 17:47 +0000
Re: MSI interrupts cross@spitfire.i.gajendra.net (Dan Cross) - 2025-03-24 18:49 +0000
Re: MSI interrupts "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> - 2025-03-24 12:12 -0700
Re: MSI interrupts scott@slp53.sl.home (Scott Lurndal) - 2025-03-24 19:46 +0000
Re: MSI interrupts cross@spitfire.i.gajendra.net (Dan Cross) - 2025-03-24 20:14 +0000
Re: MSI interrupts "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> - 2025-03-24 19:02 -0700
Re: MSI interrupts "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> - 2025-03-24 19:05 -0700
Re: MSI interrupts cross@spitfire.i.gajendra.net (Dan Cross) - 2025-03-25 11:17 +0000
Re: MSI interrupts cross@spitfire.i.gajendra.net (Dan Cross) - 2025-03-25 11:14 +0000
Re: MSI interrupts "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> - 2025-03-25 13:32 -0700
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-24 17:36 +0000
Re: MSI interrupts Stefan Monnier <monnier@iro.umontreal.ca> - 2025-03-24 14:02 -0400
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-24 19:09 +0000
Re: MSI interrupts cross@spitfire.i.gajendra.net (Dan Cross) - 2025-03-24 20:28 +0000
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-24 22:29 +0000
Re: MSI interrupts cross@spitfire.i.gajendra.net (Dan Cross) - 2025-03-24 23:27 +0000
Re: MSI interrupts scott@slp53.sl.home (Scott Lurndal) - 2025-03-25 13:43 +0000
Re: MSI interrupts cross@spitfire.i.gajendra.net (Dan Cross) - 2025-03-25 15:08 +0000
Re: MSI interrupts Stefan Monnier <monnier@iro.umontreal.ca> - 2025-03-25 11:48 -0400
Re: MSI interrupts scott@slp53.sl.home (Scott Lurndal) - 2025-03-25 16:29 +0000
Re: MSI interrupts Stefan Monnier <monnier@iro.umontreal.ca> - 2025-03-25 13:28 -0400
Re: MSI interrupts EricP <ThatWouldBeTelling@thevillage.com> - 2025-03-25 13:53 -0400
Re: MSI interrupts scott@slp53.sl.home (Scott Lurndal) - 2025-03-25 20:33 +0000
Re: MSI interrupts cross@spitfire.i.gajendra.net (Dan Cross) - 2025-03-25 17:47 +0000
Re: MSI interrupts Terje Mathisen <terje.mathisen@tmsw.no> - 2025-03-27 10:55 +0100
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-27 20:31 +0000
Re: MSI interrupts "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> - 2025-03-24 19:08 -0700
Re: MSI interrupts cross@spitfire.i.gajendra.net (Dan Cross) - 2025-03-25 11:20 +0000
Re: MSI interrupts "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> - 2025-03-25 13:38 -0700
Re: MSI interrupts cross@spitfire.i.gajendra.net (Dan Cross) - 2025-03-25 22:21 +0000
Re: MSI interrupts "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> - 2025-03-25 16:23 -0700
Re: MSI interrupts cross@spitfire.i.gajendra.net (Dan Cross) - 2025-03-25 23:29 +0000
Re: MSI interrupts "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> - 2025-03-25 16:53 -0700
Re: MSI interrupts cross@spitfire.i.gajendra.net (Dan Cross) - 2025-03-24 18:55 +0000
Re: MSI interrupts EricP <ThatWouldBeTelling@thevillage.com> - 2025-03-24 13:46 -0400
Re: MSI interrupts "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> - 2025-03-24 12:09 -0700
Re: MSI interrupts cross@spitfire.i.gajendra.net (Dan Cross) - 2025-03-24 20:11 +0000
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-24 22:50 +0000
Re: MSI interrupts cross@spitfire.i.gajendra.net (Dan Cross) - 2025-03-24 23:45 +0000
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-25 01:53 +0000
Re: MSI interrupts Stefan Monnier <monnier@iro.umontreal.ca> - 2025-03-25 01:26 -0400
Re: MSI interrupts "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> - 2025-03-25 00:07 -0700
Re: MSI interrupts cross@spitfire.i.gajendra.net (Dan Cross) - 2025-03-25 12:23 +0000
Re: MSI interrupts scott@slp53.sl.home (Scott Lurndal) - 2025-03-25 13:45 +0000
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-25 15:01 +0000
Re: MSI interrupts cross@spitfire.i.gajendra.net (Dan Cross) - 2025-03-25 15:10 +0000
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-25 16:40 +0000
Re: MSI interrupts cross@spitfire.i.gajendra.net (Dan Cross) - 2025-03-25 17:54 +0000
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-25 20:40 +0000
Re: MSI interrupts cross@spitfire.i.gajendra.net (Dan Cross) - 2025-03-25 22:19 +0000
Re: MSI interrupts Stefan Monnier <monnier@iro.umontreal.ca> - 2025-03-25 18:49 -0400
Re: MSI interrupts cross@spitfire.i.gajendra.net (Dan Cross) - 2025-03-25 23:33 +0000
Re: MSI interrupts Stefan Monnier <monnier@iro.umontreal.ca> - 2025-03-25 22:58 -0400
Re: MSI interrupts cross@spitfire.i.gajendra.net (Dan Cross) - 2025-03-26 04:40 +0000
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-26 19:42 +0000
Re: MSI interrupts Stefan Monnier <monnier@iro.umontreal.ca> - 2025-03-27 09:17 -0400
Re: MSI interrupts scott@slp53.sl.home (Scott Lurndal) - 2025-03-27 13:42 +0000
Re: MSI interrupts "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> - 2025-03-25 17:09 -0700
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-26 01:00 +0000
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-26 00:56 +0000
Re: MSI interrupts cross@spitfire.i.gajendra.net (Dan Cross) - 2025-03-26 04:08 +0000
Re: MSI interrupts Stefan Monnier <monnier@iro.umontreal.ca> - 2025-03-26 09:55 -0400
Re: MSI interrupts cross@spitfire.i.gajendra.net (Dan Cross) - 2025-03-26 14:57 +0000
Re: MSI interrupts "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> - 2025-03-26 13:22 -0700
Re: MSI interrupts "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> - 2025-04-03 22:19 -0700
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-26 19:21 +0000
Re: MSI interrupts Stefan Monnier <monnier@iro.umontreal.ca> - 2025-03-26 15:41 -0400
Re: MSI interrupts Stefan Monnier <monnier@iro.umontreal.ca> - 2025-03-26 15:50 -0400
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-26 23:16 +0000
Re: MSI interrupts scott@slp53.sl.home (Scott Lurndal) - 2025-03-26 21:35 +0000
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-26 23:22 +0000
Re: MSI interrupts scott@slp53.sl.home (Scott Lurndal) - 2025-03-26 23:48 +0000
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-27 00:18 +0000
Re: MSI interrupts cross@spitfire.i.gajendra.net (Dan Cross) - 2025-03-27 12:46 +0000
Re: MSI interrupts cross@spitfire.i.gajendra.net (Dan Cross) - 2025-03-27 17:19 +0000
Re: MSI interrupts Stefan Monnier <monnier@iro.umontreal.ca> - 2025-03-27 14:27 -0400
Re: MSI interrupts cross@spitfire.i.gajendra.net (Dan Cross) - 2025-03-27 18:34 +0000
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-27 19:18 +0000
Re: MSI interrupts cross@spitfire.i.gajendra.net (Dan Cross) - 2025-03-28 01:01 +0000
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-28 01:35 +0000
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-28 01:47 +0000
Re: MSI interrupts Stefan Monnier <monnier@iro.umontreal.ca> - 2025-03-27 22:02 -0400
Re: MSI interrupts "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> - 2025-03-27 19:14 -0700
Re: MSI interrupts Stefan Monnier <monnier@iro.umontreal.ca> - 2025-03-29 10:31 -0400
Re: MSI interrupts cross@spitfire.i.gajendra.net (Dan Cross) - 2025-03-28 02:56 +0000
Re: MSI interrupts Stefan Monnier <monnier@iro.umontreal.ca> - 2025-03-27 21:52 -0400
Re: MSI interrupts cross@spitfire.i.gajendra.net (Dan Cross) - 2025-03-28 02:58 +0000
Re: MSI interrupts "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> - 2025-03-27 19:09 -0700
Re: MSI interrupts "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> - 2025-03-27 19:10 -0700
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-27 21:47 +0000
Re: MSI interrupts cross@spitfire.i.gajendra.net (Dan Cross) - 2025-03-28 02:53 +0000
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-28 15:24 +0000
Re: MSI interrupts Stefan Monnier <monnier@iro.umontreal.ca> - 2025-03-28 11:41 -0400
Re: MSI interrupts "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> - 2025-03-28 09:51 -0700
Re: MSI interrupts cross@spitfire.i.gajendra.net (Dan Cross) - 2025-03-29 14:28 +0000
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-29 15:23 +0000
Re: MSI interrupts cross@spitfire.i.gajendra.net (Dan Cross) - 2025-03-29 20:58 +0000
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-29 22:04 +0000
Re: MSI interrupts EricP <ThatWouldBeTelling@thevillage.com> - 2025-03-25 10:54 -0400
Re: MSI interrupts scott@slp53.sl.home (Scott Lurndal) - 2025-03-25 16:31 +0000
Re: MSI interrupts cross@spitfire.i.gajendra.net (Dan Cross) - 2025-03-25 17:44 +0000
Re: MSI interrupts EricP <ThatWouldBeTelling@thevillage.com> - 2025-03-28 14:40 -0400
Re: MSI interrupts cross@spitfire.i.gajendra.net (Dan Cross) - 2025-03-29 20:55 +0000
Re: MSI interrupts EricP <ThatWouldBeTelling@thevillage.com> - 2025-03-30 16:34 -0400
Re: MSI interrupts cross@spitfire.i.gajendra.net (Dan Cross) - 2025-03-31 03:33 +0000
Re: MSI interrupts Robert Finch <robfi680@gmail.com> - 2025-03-31 01:30 -0400
Software synchronization (was: Re: MSI interrupts) cross@spitfire.i.gajendra.net (Dan Cross) - 2025-03-31 14:42 +0000
Threads (was Re: MSI interrupts) scott@slp53.sl.home (Scott Lurndal) - 2025-03-31 15:28 +0000
Re: Threads (was Re: MSI interrupts) Terje Mathisen <terje.mathisen@tmsw.no> - 2025-04-01 16:44 +0200
Re: Threads (was Re: MSI interrupts) scott@slp53.sl.home (Scott Lurndal) - 2025-04-01 15:31 +0000
Re: Threads (was Re: MSI interrupts) "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> - 2025-04-01 12:49 -0700
Re: Threads (was Re: MSI interrupts) scott@slp53.sl.home (Scott Lurndal) - 2025-04-01 20:01 +0000
Re: Threads (was Re: MSI interrupts) "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> - 2025-04-01 13:17 -0700
Re: Threads (was Re: MSI interrupts) scott@slp53.sl.home (Scott Lurndal) - 2025-04-01 22:04 +0000
Re: Threads (was Re: MSI interrupts) anton@mips.complang.tuwien.ac.at (Anton Ertl) - 2025-04-02 06:25 +0000
Re: Threads (was Re: MSI interrupts) cross@spitfire.i.gajendra.net (Dan Cross) - 2025-04-02 13:04 +0000
Re: Threads (was Re: MSI interrupts) Terje Mathisen <terje.mathisen@tmsw.no> - 2025-04-03 16:45 +0200
Re: Threads (was Re: MSI interrupts) scott@slp53.sl.home (Scott Lurndal) - 2025-04-03 15:00 +0000
Re: Threads (was Re: MSI interrupts) cross@spitfire.i.gajendra.net (Dan Cross) - 2025-04-03 17:16 +0000
Re: Threads (was Re: MSI interrupts) scott@slp53.sl.home (Scott Lurndal) - 2025-04-03 17:24 +0000
Re: Threads (was Re: MSI interrupts) cross@spitfire.i.gajendra.net (Dan Cross) - 2025-04-04 01:44 +0000
Centronics (was Re: Threads (was Re: MSI interrupts)) George Neuner <gneuner2@comcast.net> - 2025-04-03 22:22 -0400
Re: Centronics (was Re: Threads (was Re: MSI interrupts)) scott@slp53.sl.home (Scott Lurndal) - 2025-04-04 02:29 +0000
Re: Threads (was Re: MSI interrupts) George Neuner <gneuner2@comcast.net> - 2025-04-03 01:38 -0400
Re: Threads (was Re: MSI interrupts) cross@spitfire.i.gajendra.net (Dan Cross) - 2025-04-03 12:15 +0000
Re: Threads (was Re: MSI interrupts) Robert Swindells <rjs@fdy2.co.uk> - 2025-04-04 10:41 +0000
Re: Threads (was Re: MSI interrupts) George Neuner <gneuner2@comcast.net> - 2025-04-04 23:10 -0400
Re: MSI interrupts EricP <ThatWouldBeTelling@thevillage.com> - 2025-04-01 10:07 -0400
Re: MSI interrupts cross@spitfire.i.gajendra.net (Dan Cross) - 2025-04-01 17:39 +0000
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-19 23:28 +0000
Re: MSI interrupts cross@spitfire.i.gajendra.net (Dan Cross) - 2025-03-20 12:50 +0000
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-20 22:28 +0000
Re: MSI interrupts EricP <ThatWouldBeTelling@thevillage.com> - 2025-03-23 12:16 -0400
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-23 16:48 +0000
Re: MSI interrupts EricP <ThatWouldBeTelling@thevillage.com> - 2025-03-24 11:55 -0400
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-24 18:56 +0000
Re: MSI interrupts "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> - 2025-03-23 17:48 -0700
Re: MSI interrupts cross@spitfire.i.gajendra.net (Dan Cross) - 2025-03-24 18:39 +0000
Re: MSI interrupts EricP <ThatWouldBeTelling@thevillage.com> - 2025-03-20 12:03 -0400
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-20 19:52 +0000
Re: MSI interrupts EricP <ThatWouldBeTelling@thevillage.com> - 2025-03-19 13:07 -0400
Re: MSI interrupts Robert Finch <robfi680@gmail.com> - 2025-03-19 15:47 -0400
Re: MSI interrupts - Instructions Robert Finch <robfi680@gmail.com> - 2025-03-26 06:56 -0400
Re: MSI interrupts - Instructions EricP <ThatWouldBeTelling@thevillage.com> - 2025-03-26 10:33 -0400
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-18 22:49 +0000
Re: MSI interrupts Robert Finch <robfi680@gmail.com> - 2025-03-18 20:39 -0400
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-19 01:22 +0000
Re: MSI interrupts Robert Finch <robfi680@gmail.com> - 2025-03-18 23:22 -0400
Re: MSI interrupts scott@slp53.sl.home (Scott Lurndal) - 2025-03-19 13:54 +0000
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-19 15:56 +0000
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-17 21:49 +0000
Re: MSI interrupts scott@slp53.sl.home (Scott Lurndal) - 2025-03-18 01:34 +0000
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-18 18:51 +0000
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-18 19:19 +0000
Re: MSI interrupts Stefan Monnier <monnier@iro.umontreal.ca> - 2025-03-19 11:20 -0400
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-19 16:00 +0000
Re: MSI interrupts Robert Finch <robfi680@gmail.com> - 2025-03-19 13:27 -0400
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-19 19:17 +0000
Re: MSI interrupts Robert Finch <robfi680@gmail.com> - 2025-03-17 10:12 -0400
Re: MSI interrupts scott@slp53.sl.home (Scott Lurndal) - 2025-03-17 15:53 +0000
Re: MSI interrupts mitchalsup@aol.com (MitchAlsup1) - 2025-03-17 19:06 +0000
Re: MSI interrupts Robert Finch <robfi680@gmail.com> - 2025-03-17 17:33 -0400
Page 7 of 11 — ← Prev page 1 … 5 6 [7] 8 9 … 11 Next page →
| From | scott@slp53.sl.home (Scott Lurndal) |
|---|---|
| Date | 2025-03-26 23:48 +0000 |
| Message-ID | <5n0FP.78185$qx73.45402@fx01.iad> |
| In reply to | #111338 |
mitchalsup@aol.com (MitchAlsup1) writes: >On Wed, 26 Mar 2025 21:35:01 +0000, Scott Lurndal wrote: > >> mitchalsup@aol.com (MitchAlsup1) writes: >>>On Wed, 26 Mar 2025 4:08:57 +0000, Dan Cross wrote: >>> >>>> In article <b1c74762d01f71cc1b8ac838dcf6d4fa@www.novabbs.org>, >> >>> >>>> Earlier you wrote, >>>> "esmINTERFERENCE is a query on the state of th event and whether >>>> a 2nd party has performed a memory action that kills the event." >>>> Ok, I take this to mean _at the time of the store_, which >>>> appears later in the program. If it were simply a query at the >>>> time it was written in the code, then it seems like it opens a >>>> TOCTOU bug (what if something interfers after the check, but >>>> before the the locked store at the end?). Or perhaps it signals >>>> to the architecture that this is the point to which the >>>> processor should rollback, if anything subsequently fails; sort >>>> of a `setjmp` kind of thing that conceptually returns twice. >>> >>>The "if( esmINTERFERENCE() )" not only queries the state of the >>>participating cache lines, it sets up a guarding-shadow over >>>the STs to follow such that if killing-interference is detected >>>none of the STs are performed AND control is transferred to the >>>branch label (outside of the event). So not only does it query >>>the state at it execution, it monitors that over the rest of >>>the event. >>> >>>BUT ALSO: it initializes the ability to NAK SNOOPs. The thought >>>is that if we have reached the manifestation phase of the event >>>(that is ready to do all the stores) then we should be able to >>>drive the event forward by not sending the data in response to >>>the SNOOP. We send a Nak (based on priority or the data) >> >> If you can NAK a snoop, are you then assuming that all DMA is >> non-coherent? That will really piss off the kernel folks. > >Not at all. DMA is as coherent as the I/O MMU's translating >PTE prescribes. > >But; why would any sane coder be performing an ATOMIC event on >memory that can be blindly overwritten by DMA ??? Or for that >mater, critical section data being overwritten (at some unknown >and unpredictable time) by DMA ?? Polling on a completion status which the device DMAs (often with an allocate hint) is not uncommon. A 32-bit word in memory updated by the device - the allocate hint might update the LLC directly and invalidate L1/L2 or may update the target cache line that the driver is polling directly.
[toc] | [prev] | [next] | [standalone]
| From | mitchalsup@aol.com (MitchAlsup1) |
|---|---|
| Date | 2025-03-27 00:18 +0000 |
| Message-ID | <efd7a61f2d4995d42225ecf999b6786f@www.novabbs.org> |
| In reply to | #111339 |
On Wed, 26 Mar 2025 23:48:17 +0000, Scott Lurndal wrote: > mitchalsup@aol.com (MitchAlsup1) writes: >>On Wed, 26 Mar 2025 21:35:01 +0000, Scott Lurndal wrote: >> >>> mitchalsup@aol.com (MitchAlsup1) writes: >>>>On Wed, 26 Mar 2025 4:08:57 +0000, Dan Cross wrote: >>>> >>>>> In article <b1c74762d01f71cc1b8ac838dcf6d4fa@www.novabbs.org>, >>> >>>> >>>>> Earlier you wrote, >>>>> "esmINTERFERENCE is a query on the state of th event and whether >>>>> a 2nd party has performed a memory action that kills the event." >>>>> Ok, I take this to mean _at the time of the store_, which >>>>> appears later in the program. If it were simply a query at the >>>>> time it was written in the code, then it seems like it opens a >>>>> TOCTOU bug (what if something interfers after the check, but >>>>> before the the locked store at the end?). Or perhaps it signals >>>>> to the architecture that this is the point to which the >>>>> processor should rollback, if anything subsequently fails; sort >>>>> of a `setjmp` kind of thing that conceptually returns twice. >>>> >>>>The "if( esmINTERFERENCE() )" not only queries the state of the >>>>participating cache lines, it sets up a guarding-shadow over >>>>the STs to follow such that if killing-interference is detected >>>>none of the STs are performed AND control is transferred to the >>>>branch label (outside of the event). So not only does it query >>>>the state at it execution, it monitors that over the rest of >>>>the event. >>>> >>>>BUT ALSO: it initializes the ability to NAK SNOOPs. The thought >>>>is that if we have reached the manifestation phase of the event >>>>(that is ready to do all the stores) then we should be able to >>>>drive the event forward by not sending the data in response to >>>>the SNOOP. We send a Nak (based on priority or the data) >>> >>> If you can NAK a snoop, are you then assuming that all DMA is >>> non-coherent? That will really piss off the kernel folks. >> >>Not at all. DMA is as coherent as the I/O MMU's translating >>PTE prescribes. >> >>But; why would any sane coder be performing an ATOMIC event on >>memory that can be blindly overwritten by DMA ??? Or for that >>mater, critical section data being overwritten (at some unknown >>and unpredictable time) by DMA ?? > > Polling on a completion status which the device DMAs > (often with an allocate hint) is not uncommon. A 32-bit > word in memory updated by the device - the allocate hint > might update the LLC directly and invalidate L1/L2 or > may update the target cache line that the driver is > polling directly. Why do this ATOMICally ?? why not just read it ?? Or is this a grab (read and modify so someone else doesn't get it.) ??
[toc] | [prev] | [next] | [standalone]
| From | cross@spitfire.i.gajendra.net (Dan Cross) |
|---|---|
| Date | 2025-03-27 12:46 +0000 |
| Message-ID | <vs3hae$ahk$1@reader1.panix.com> |
| In reply to | #111340 |
In article <efd7a61f2d4995d42225ecf999b6786f@www.novabbs.org>, MitchAlsup1 <mitchalsup@aol.com> wrote: >On Wed, 26 Mar 2025 23:48:17 +0000, Scott Lurndal wrote: >> mitchalsup@aol.com (MitchAlsup1) writes: >>>On Wed, 26 Mar 2025 21:35:01 +0000, Scott Lurndal wrote: >>>> [snip] >>>> If you can NAK a snoop, are you then assuming that all DMA is >>>> non-coherent? That will really piss off the kernel folks. >>> >>>Not at all. DMA is as coherent as the I/O MMU's translating >>>PTE prescribes. >>> >>>But; why would any sane coder be performing an ATOMIC event on >>>memory that can be blindly overwritten by DMA ??? Or for that >>>mater, critical section data being overwritten (at some unknown >>>and unpredictable time) by DMA ?? >> >> Polling on a completion status which the device DMAs >> (often with an allocate hint) is not uncommon. A 32-bit >> word in memory updated by the device - the allocate hint >> might update the LLC directly and invalidate L1/L2 or >> may update the target cache line that the driver is >> polling directly. > >Why do this ATOMICally ?? why not just read it ?? Or is this a >grab (read and modify so someone else doesn't get it.) ?? Consider multiword DMA descriptors in a ring buffer in DRAM, shared with a device. Manipulating these races against hardware that's consuming the ring, and on many devices you need to do a multiword write to transfer ownership of a descriptor to the device for processing (say, setting both a DMA address and an ownership bit in two consecutive words). A few particular models of ethernet controllers in Cortex-M7 series parts from ST come to mind as examples of devices that do this. Of course proper maintenance of ring tail pointers in the device coupled with a barrier can mitigate this, but it would be nice to do both the descriptor and tail pointer updates atomically; instead of maintaining a separate tail pointer into the ring, you could just initiate a start operation by writing an arbitrary value into a register on the device. Indeed, for the aforementioned ST part, many drivers have gotten this stuff wrong. See, for example, https://github.com/oxidecomputer/hubris/pull/1493 and https://github.com/embassy-rs/embassy/pull/2155 - Dan C.
[toc] | [prev] | [next] | [standalone]
| From | cross@spitfire.i.gajendra.net (Dan Cross) |
|---|---|
| Date | 2025-03-27 17:19 +0000 |
| Message-ID | <vs41ap$n43$1@reader1.panix.com> |
| In reply to | #111331 |
In article <7a093bbb356e3bda3782c15ca27e98a7@www.novabbs.org>,
MitchAlsup1 <mitchalsup@aol.com> wrote:
>On Wed, 26 Mar 2025 4:08:57 +0000, Dan Cross wrote:
>> [snip]
>> 1. `from` may point to a cache line
>> 2. `to` may point to a different cache line.
>> 3. Does your architecture use a stack?
>
>No
You make multiple references to a stack just below. I
understand stack references are excluded from the atomic event,
but I asked whether the architecture uses a stack. Does it, or
doesn't it?
>> What sort of alignment criteria do you impose?
>
>On the participating data none. On the cache line--cache line.
I meant for data on the stack. The point is moot, though, since
as you said stack data does not participate in the atomic event.
>> At first blush, it seems to me that
>> the pointers `from_next`, `from_prev`, and `to_next` could be
>> on the stack and if so, will be on at least one cache line,
>> and possibly 2, if the call frame for `MoveElement` spans
>> more than one.
>
>The illustrated code is using 6 participating cache lines.
>Where local variables are kept (stack, registers) does not
>count against the tally of participating cache lines.
Huh. How would this handle something like an MCS lock, where
the lock node may be on a stack, though accessible globally in
some virtual address space?
>> Of course, it's impossible to say when this
>> is compiled, but maybe you require that the stack is always
>> aligned to a cache line boundary or something.
>
>Data stack is DoubleWord aligned at all times.
Ok. So just for context, the point of this question was to
understand whether local data participating in one of these
atomic events could span multiple cache lines. Since they
don't participate it doesn't matter vis the current discussion.
>> these are all in registers and it's fine.
>
>Local variables are not participating in the event--they are just
>around to guide the event towards conclusion.
Ok.
>> 4. Depending on the size and alignment criteria of the the
>> `Element` structure and the placement of the `next` and
>> `prev` elements in the struct (unknowable from this example),
>> `to->next` may be on another cache line.
>> 5. Similarly, `from->next`
>> 6. And `from->prev`
>> 7. And `from_prev->next`
>> 8. And `from_next->prev`
>> 9. And `to_next->prev`.
>>
>> So potentially this code _could_ touch 9 or 10 cache lines, more
>> than you said are supported for a single atomic event. What
>> happens in that case? Does the code just always return false?
>> The programmer best make sure the optimizer is keeping those
>> temporary pointers off the stack; good luck with a debug build.
>>
>> Anyway, let's assume that's not the case, and we don't hit the
>> pathological case with cache lines, and we're not dealing with
>> edge cases at the front or end of the lists (so various next or
>> prev pointers are not NULL). Why don't I try to explain what I
>> think this code is doing, and you tell me whether I'm right or
>> wrong.
>>
>> If I understand this correctly, the `esmLOCKload`s are simply
>> loads, but they make sure that the compiler emits instructions
>> that tie into your atomic support framework.
>
>So far so good.
>
>> Earlier you wrote,
>> "esmINTERFERENCE is a query on the state of th event and whether
>> a 2nd party has performed a memory action that kills the event."
>> Ok, I take this to mean _at the time of the store_, which
>> appears later in the program. If it were simply a query at the
>> time it was written in the code, then it seems like it opens a
>> TOCTOU bug (what if something interfers after the check, but
>> before the the locked store at the end?). Or perhaps it signals
>> to the architecture that this is the point to which the
>> processor should rollback, if anything subsequently fails; sort
>> of a `setjmp` kind of thing that conceptually returns twice.
>
>The "if( esmINTERFERENCE() )" not only queries the state of the
>participating cache lines, it sets up a guarding-shadow over
>the STs to follow such that if killing-interference is detected
>none of the STs are performed AND control is transferred to the
>branch label (outside of the event). So not only does it query
>the state at it execution, it monitors that over the rest of
>the event.
>
>BUT ALSO: it initializes the ability to NAK SNOOPs. The thought
>is that if we have reached the manifestation phase of the event
>(that is ready to do all the stores) then we should be able to
>drive the event forward by not sending the data in response to
>the SNOOP. We send a Nak (based on priority or the data)
>
>If the SNOOPing core receives a NaK and it is not attempting an
>event the request is simply reissued. If it is attempting, then
>its event fails. So a core benignly accessing data that another
>core is attempting an event upon, is delayed by an interconnect
>round trip--but if it is attempting, it knows its event will
>fail and it will either retry from before the event started,
>or it will goto the interferrence-label where other facilities
>\are at its disposal.
Ok.
>> Assuming nothing has poisoned the event, the various pointer
>> stores inside the conditional are conditionally performed; the
>> `esmLOCKstore` to `from->next` commits the event, at which
>> point all side effects are exposed and durable.
>
>From a SW perspective, that is accurate enough.
>
>From a HW perspective, the participating STs are performed and
>held in a buffer, and when it is known the event has succeeded,
>a bit is flipped and these writes to memory all appear to have
>been performed in the same cycle. One cycle earlier and no
>interested 2nd party has seen any of them, the subsequent cycle
>any interested 2nd party will see them all.
>
>I should also note: ESM works even when there are no caches in
>the system. ...
Ok, good to go.
>> Is that correct? Let's suppose that it is.
>>
>> But later you wrote,
>>
>>>So, if you want the property whereby the lock disappears on any
>>>control transfer out of the event {exception, interrupt, SVC, SVR, ...};
>>>then you want to use my ATOMIC stuff; otherwise, you can use the
>>>normal ATOMIC primitives everyone and his brother provide.
>>
>> Well, what precisely do you mean here when you say, "if you want
>> the property whereby the lock disappears on any control transfer
>> out of the event"?
>
>If you want that property--you use the tools at hand.
>If you don't just use them as primitive generators.
I wasn't clear enough here. I'm asking what, exactly, you mean
by this _property_, not what you mean when you write that one
can use these atomic events if one wants the property. That is,
I'm asking you to precisely define what it means for a lock to
"disappear".
It seems clear enough _now_ that once the event concludes
successfully the lock value is set to whatever it was set to in
during the event, but that wasn't clear to me earlier and I
wanted confirmation.
In particular, it seems odd to me that one would bother with a
lock of some kind during an event if it didn't remain set after
the event. If you do all of your critical section stuff inside
of the event, and setting the lock in the event is not visible
until the event concludes, why bother? If on the other hand you
use the event to set the lock, why bother doing additional work
inside the event itself, but in this case I definitely don't
want something else to come along and just unset the lock on its
own, higher priority or not.
>> What I _hope_ you mean is that if you transfer "out of the
>> event" before the `esmLOCKstore` then anything that's been done
>> since the "event" started is rolled back, but NOT if control
>> transfer happens _after_ the `esmLOCKstore`. If THAT were the
>> case, then this entire mechanism is useless.
>
>One can use a esmLOCKstore( *SP, 0 ) to abandon an event. HW
>detects that *SP is not participating in the event, and uses
>the lock bit in the instruction to know it is time to leave
>the event in the failed manner. No ST is performed to the
>non-participating cache line. {{I guess I could wrap the
>abandonment into a macro ...}}
That's not what I was asking. I'm not interested in what
happens if I transfer control out of the event myself, as it is
obviously my responsibility to ensure whatever contract the
hardware requires in that case is fulfilled. I'm interested in
what happens control is usurped by some higher priority thing.
>> Perhaps you would clarify?
>>
>> But for the time being, let's assume that `esmLOCKstore` is
>> conceptually a commit that ends the atomic event, and the store
>> is durable and observable after that. Let's see if we can
>> construct a simple spinlock from these primitives.
>
>Notice that only the last ST has the esmLOCKstore() applied.
>HW knows that STs to participating lines have the ATOMIC
>property. The final ST uses the LOCK bit to denote the end
>of the event.
Again, not what I was asking. I'm trying to understand what you
meant when you talked about disappearing locks.
Again, I think you meant that, if an atomic event that sets a
lock variable fails, whether it got far enough along to set the
variable inside the event or not is not observable if the event
fails. But if the event succeeds, then the variable is set and
that's that. IF that's the case, then great! But if not, then
disappearing locks are bad. This is especially murky to me
because there seem to be bits of OS primitives, like threads and
priority, that are known directly to the hardware.
>> typedef struct Spinlock Spinlock;
>> struct Spinlock {
>> uint32_t lock;
>> };
>>
>> uint32_t
>> xcas(volatile Spinlock *lk, uint32_t expected, uint32_t val)
>> {
>> uint32_t v = esmLOCKload(&lk->lock);
>> if (v == expected && !esmINTERFERENCE()) {
>> esmLOCKstore(&lk->lock, val)
>> }
>> return v;
>> }
>
>If the interference check is desired at all:: (see below)
>I think you should move interference check in front of the
>expected check, but that is a first impression--have to think
>about it more.
Probably, given that it sets up a guard over the event, as you
wrote above.
>> void
>> spin_acquire(volatile Spinlock *lk)
>> {
>> splhi();
>> while (xcas(lk, 0, 1) != 0)
>> relax(); // Spin loop intrinsic.
>> }
>>
>> void
>> spin_release(volatile Spinlock *lk)
>> {
>> xcas(lk, 1, 0);
>> // Maybe we should `spllo` here, maybe not. Either choice
>> // reflects lots of assumptions about whether or not we might
>> // take and hold multiple locks concurrently, etc.
>> spllo();
>> }
>>
>> Does this seem correct? It is not immediately clear to me
>> whether `esmINTERFERENCE` is actually needed here, but
>> avoiding the store seemed fine.
>
>There are MANY cases where the interference query is not needed.
>Without the interference query, deadly interference backs the
>core up to the first instruction of the event. With the interference
>check, if deadly interference happens before the query, control
>reverts to before the event, and afterwards, control is transferred
>to the interference-label.
>
>> One could imagine shorting
>> `xcas` to simply:
>>
>> uint32_t
>> xcas(volatile Spinlock *lk, uint32_t expected, uint32_t val)
>> {
>> uint32_t v = esmLOCKload(&lk->lock);
>> if (v == expected) esmLOCKstore(&lk->lock, val);
>> return v;
>> }
>
>Yes, unless you want the alternate exit strategy, this works fine.
>In fact test and set is written::
>
>boolean char TestAndSet( type char *p )
>{
> boolean t = esmLOCKload ( *p );
> esmLOCKstore( *p, 1 );
> return t;
>}
>
>Should interference transpire between load and store control
>reverts back to the LD again, so the return value is the LD
>which did not encounter interference.
Ok, so perhaps this is a reasonable implementation of spinlock
acquire, as I wrote to Stefan yesterday:
void
spinlock_acquire(volatile Spinlock *lk)
{
while (esmLOCKload(&lk->lock) != 0)
cpu_spin_hint();
esmLOCKstore(&lk->lock, 1);
}
Yes? If this is preempted, or another CPU wins the race on the
lock, the hardware will back this up and start over in the while
loop, right?
>>>For the sizes of CSs this covers--it prevent priority inversion.
>>
>> Except you've admitted that this is not a general solution, as
>> it cannot "cover" arbitrary software-defined critical sections:
>> the window isn't big enough and we've seen how even a fairly
>> small problem may exceed the number of cache lines supported by
>> one of these events.
>
>Remember only the lines participating in the event count. For
>example, say one had sprintf() inside the event. None of the
>lines touched by springt() are participating in the event,
>and are not counted against the 8 cache lines available.
This doesn't change the point, though. Anything that touches
more than 8 cache lines is not suitable for an atomic event.
Insertion into a doubly linked list is already up to 6 lines,
recall that my original hypothetical also involved popping off a
queue, which adds another (the queue head). If I threw up
incrementing a couple of counters I can't do it. And this seems
like a perfectly reasonable real-world scenario; consider a work
stealing thread scheduler that takes a task from one core to
another and keeps some stats on how things move.
Or maybe even just swapping places between two elements in a
linked list. For example:
void
swap_places(Node **head, Node *a, Node *b)
{
Node *an, *ap, *bn, *bp;
assert(head != NULL);
assert(a != NULL);
assert(b != NULL);
if (a == b)
return;
if (*head == a)
*head = b;
else if (*head == b)
*head = a;
an = a->next;
ap = a->prev;
bn = b->next;
bp = b->prev;
b->next = an;
if (an != NULL)
an->prev = b;
b->prev = ap;
if (ap != NULL)
ap->next = b;
a->next = bn;
if (bn != NULL)
bn->prev = a;
a->prev = bp;
if (bp != NULL)
bp->next = a;
}
Perhaps we might wrap this in an atomic event by annotating it
like so:
void
swap_places(Node **head, Node *a, Node *b)
{
Node *hp, *an, *ap, *bn, *bp;
assert(head != NULL);
assert(a != NULL);
assert(b != NULL);
if (a == b)
return;
esmLOCKprefetch(*head);
if (*head == a)
*head = b;
else if (*head == b)
*head = a;
an = esmLOCKload(a->next);
ap = esmLOCKload(a->prev);
bn = esmLOCKload(b->next);
bp = esmLOCKload(b->prev);
b->next = an;
if (an != NULL) {
esmLOCKprefetch(an->prev);
an->prev = b;
}
b->prev = ap;
if (ap != NULL) {
esmLOCKprefetch(ap->next);
ap->next = b;
}
a->next = bn;
if (bn != NULL) {
esmLOCKprefetch(bn->prev);
bn->prev = a;
}
if (bp != NULL) {
esmLOCKprefetch(bp->next);
bp->next = a;
}
esmLOCKstore(a->prev, bp);
}
By my count, that could be touching 9 cache lines, even with
all of the various structures well-aligned; indeed, if the Node
type were annotated with a constraint that it be aligned on a
cacheline boundary, that would be maximally pessimistic in this
regard.
Anyway, I'd like to understand what the hardware does when a
program attempts an event where the conditions for success
cannot be met.
>---------------------
>> So now I, as a systems software person, have to ask design
>> question around how I want to use these architectural features,
>> if at all, weighing various pros and cons. This is where I'm
>> struggling with your proposal: given that I'm going to have to
>> implement more general support for the things that the
>> architecture gives me anyway, what do I gain by also supporting
>> the specialized mechanisms?
>
>All I can say about this, was that when I presented ASF (ESM's
>predecessor) to Microsoft and spend a couple hours explaining
>to them how it works, MS wanted it implemented as rapidly as
>AMD could pull it off. Dave Cutler was particularly impressed.
I'm not sure that a Dave C. ref is the flex you are making it
out to be.
>>>And here I thought you had an imagination .....
>>
>> You admitted you're not an OS person and asked other people to
>> poke holes in your proposal. I'm sorry if you find the holes
>> I'm poking upsetting.
>
>I enjoy your poking of holes--I really do
>
>But you seem like the calculus student that keeps asking what
>problem can I solve now that I can differentiate.
Huh. Well, I don't know what to tell ya about that, guy.
Look, here's the deal. This all sounds neat. And my point is
not to bore you with a bunch of impudent challenges. You've
obviously had a long and distinguished career, and I can't
imagine what purpose that would serve, other than being a jerk.
I'm not interested in that.
But there isn't a lot of publicly available documentation for
any of this, at least none that I could find, and from the way
it's been described thus far it seems like there are a number of
edge cases that it is not clear how one is supposed to address.
So I want to understand the boundaries of where these hardware
facilities are applicable, given architectural limitations. I
think this thread has shown that those boundaries exist. The
universe of interesting and useful software is big, and someone,
somewhere, is going to trip over those limitations sooner or
later. Once we accept that, we must ask, "Then what?" It seems
obvious that programmers are going to have to find techniques to
work around those limitations, and that the mechanisms chosen
are unknowable to hardware.
But now we're back in the same boat of software having to deal
with all the thorny problems that these mechanisms were designed
to address in the first place, like priority inversion: take the
spinlock from earlier, the hardware has no semantic knowledge
that the lock is used as a mutex, and can't know; it's just some
bit in memory somewhere that was set during an atomic event.
Maybe ESM helps in a race when trying to acquire the spinlock,
but once it's acquired, it's acquired, and because the spinlock
is not a primitive understood by the hardware, the hardware has
no way of incorporating the spinlock's semantics into its view
of priority, threading, etc.
So while an atomic event might help me in the implementation of
a generic mutex of some kind, so that I can protect critical
sections that can't be handled with the atomic event framework
directly because of their size, that's the extent of the help it
can give me, because once the mutex is acquired the hardware has
no insight into the fact that it's a mutex. So if some higher
priority thing subsequently comes along and preempts the thread
holding that lock and tries to take that lock itself, then the
hardware can't help me out anymore, and I'm back to a
priority-inversion deadlock that has to be dealt with by
software.
The point is, even if the hardware gives me a cool
mechanism for helping prevent priority inversion problems,
programmers are inevitably going to have to handle those
situations themselves anyway.
This is the sort of thing I'm trying to communicate, it's really
not me sitting here tossing out barbs like, "oh yeah?! Well,
how about THIS one, smart guy?"
To use your mathematics analogy, you are describing a technique
for differeniating real single variable continuous functions.
I'm pointing out that this is not applicable to all functions,
because discontinuous functions and higher order spaces exist,
and giving examples of such functions that are important parts
of real-world problems, not just abstract distractions.
I can't help feel like you're telling me to just stick to
polynomials over R because calculus is useful. Calculus may be
useful, but sometimes you really do need analysis.
- Dan C.
[toc] | [prev] | [next] | [standalone]
| From | Stefan Monnier <monnier@iro.umontreal.ca> |
|---|---|
| Date | 2025-03-27 14:27 -0400 |
| Message-ID | <jwv7c4ajq11.fsf-monnier+comp.arch@gnu.org> |
| In reply to | #111345 |
>>The illustrated code is using 6 participating cache lines.
>>Where local variables are kept (stack, registers) does not
>>count against the tally of participating cache lines.
>
> Huh. How would this handle something like an MCS lock, where
> the lock node may be on a stack, though accessible globally in
> some virtual address space?
ESM doesn't distinguish the stack from other places. It just doesn't
pay attention to memory accesses that aren't marked specially. So, if
you lock is on the stack it will still work, as long as you use
`esmLOCKload` to read it.
> Again, not what I was asking. I'm trying to understand what you
> meant when you talked about disappearing locks.
He was talking about the conceptual lock used internally by the CPU
to implement an ESM event, that gives their name to `esmLOCK*`.
I.e. the same kind of lock used in other CPUs to implement LL/SC and
that gives its name to "load locked" (tho it's also sometimes named
"load linked").
Stefan
[toc] | [prev] | [next] | [standalone]
| From | cross@spitfire.i.gajendra.net (Dan Cross) |
|---|---|
| Date | 2025-03-27 18:34 +0000 |
| Message-ID | <vs45mo$b1g$1@reader1.panix.com> |
| In reply to | #111346 |
In article <jwv7c4ajq11.fsf-monnier+comp.arch@gnu.org>, Stefan Monnier <monnier@iro.umontreal.ca> wrote: >>>The illustrated code is using 6 participating cache lines. >>>Where local variables are kept (stack, registers) does not >>>count against the tally of participating cache lines. >> >> Huh. How would this handle something like an MCS lock, where >> the lock node may be on a stack, though accessible globally in >> some virtual address space? > >ESM doesn't distinguish the stack from other places. It just doesn't >pay attention to memory accesses that aren't marked specially. So, if >you lock is on the stack it will still work, as long as you use >`esmLOCKload` to read it. Ah, I see. Ok, that makes more sense. >> Again, not what I was asking. I'm trying to understand what you >> meant when you talked about disappearing locks. > >He was talking about the conceptual lock used internally by the CPU >to implement an ESM event, that gives their name to `esmLOCK*`. >I.e. the same kind of lock used in other CPUs to implement LL/SC and >that gives its name to "load locked" (tho it's also sometimes named >"load linked"). Ok. Thanks, Stefan. - Dan C.
[toc] | [prev] | [next] | [standalone]
| From | mitchalsup@aol.com (MitchAlsup1) |
|---|---|
| Date | 2025-03-27 19:18 +0000 |
| Message-ID | <601781e2c91d42a73526562b419fdf02@www.novabbs.org> |
| In reply to | #111345 |
On Thu, 27 Mar 2025 17:19:21 +0000, Dan Cross wrote:
> In article <7a093bbb356e3bda3782c15ca27e98a7@www.novabbs.org>,
> MitchAlsup1 <mitchalsup@aol.com> wrote:
>>On Wed, 26 Mar 2025 4:08:57 +0000, Dan Cross wrote:
>>> [snip]
>>> 1. `from` may point to a cache line
>>> 2. `to` may point to a different cache line.
>>> 3. Does your architecture use a stack?
>>
>>No
>
> You make multiple references to a stack just below. I
> understand stack references are excluded from the atomic event,
> but I asked whether the architecture uses a stack. Does it, or
> doesn't it?
>
>>> What sort of alignment criteria do you impose?
>>
>>On the participating data none. On the cache line--cache line.
>
> I meant for data on the stack. The point is moot, though, since
> as you said stack data does not participate in the atomic event.
Stack does not NECESSARILY participate. However, if you perform
a esmLOCKload( SP[23] ) then that line on the stack is participating.
>>> At first blush, it seems to me that
>>> the pointers `from_next`, `from_prev`, and `to_next` could be
>>> on the stack and if so, will be on at least one cache line,
>>> and possibly 2, if the call frame for `MoveElement` spans
>>> more than one.
>>
>>The illustrated code is using 6 participating cache lines.
>>Where local variables are kept (stack, registers) does not
>>count against the tally of participating cache lines.
>
> Huh.
There are 6 esmLOCKxxxxx() so there are 6 participating lines--
and these are absolutely independent from where the variables
are located. The variables carry information but do not part-
ticipate they carry data between operations on the particupating
lines.
> How would this handle something like an MCS lock, where
> the lock node may be on a stack, though accessible globally in
> some virtual address space?
As illustrated above esmLOCKload( SP[23]) will cause a line on
the stack to participate in the event.
----------------
>>> But later you wrote,
>>>
>>>>So, if you want the property whereby the lock disappears on any
>>>>control transfer out of the event {exception, interrupt, SVC, SVR, ...};
>>>>then you want to use my ATOMIC stuff; otherwise, you can use the
>>>>normal ATOMIC primitives everyone and his brother provide.
>>>
>>> Well, what precisely do you mean here when you say, "if you want
>>> the property whereby the lock disappears on any control transfer
>>> out of the event"?
>>
>>If you want that property--you use the tools at hand.
>>If you don't just use them as primitive generators.
>
> I wasn't clear enough here. I'm asking what, exactly, you mean
> by this _property_,
The property that if an interrupt (or exception or SVC or SVR)
prevents executing the event to its conclusion, HW makes the
event look like it never started. So that when/if control returns
we have another chance to execute the event as-if ATOMICally.
> not what you mean when you write that one
> can use these atomic events if one wants the property. That is,
> I'm asking you to precisely define what it means for a lock to
> "disappear".
>
> It seems clear enough _now_ that once the event concludes
> successfully the lock value is set to whatever it was set to in
> during the event, but that wasn't clear to me earlier and I
> wanted confirmation.
Not "the lock value" but "all participating values" are set
and become visible system-wide in a single instant. No 3rd
party sees some set and others remain unset. Everybody sees
all of then change value between this-clock and its successor.
> In particular, it seems odd to me that one would bother with a
> lock of some kind during an event if it didn't remain set after
> the event.
Consider CompareTripleSwapFour() as an atomic primitive. How
would you program this such that nobody nowhere could ever
tell that the four updated locations changes in any order
then simultaneously ??? Without using a LOCK to guard the
event ??? And without demanding that all four updates are
to the same cache line ???
{{Nobody nowhere included ATOMIC-blind DMA requests}}
> If you do all of your critical section stuff inside
> of the event, and setting the lock in the event is not visible
> until the event concludes, why bother? If on the other hand you
> use the event to set the lock, why bother doing additional work
> inside the event itself, but in this case I definitely don't
> want something else to come along and just unset the lock on its
> own, higher priority or not.
It is clear you do not understand the trouble HW takes to implement
even DCADS where between any 2 clocks, one (or more) SNOOPs can
take the lines you are DCADSing. It is this property of cache
coherence that gets in the way of multi-operation ATOMIC events.
In 2004 Fred Weber came to me and ask: "Why can't we (AMD) give
Microsoft the DCADS they want. I dug into it, and ASF was my
solution in x86 ISA, ESM is my solution in RISC-based ISA. It
removes the need to add ATOMIC primitives to ISA over generations.
It also provides the ability to do other things--which apparently
you will not use--sort of like complaining that your new car has
auto-park a feature you will never use.
>>> What I _hope_ you mean is that if you transfer "out of the
>>> event" before the `esmLOCKstore` then anything that's been done
>>> since the "event" started is rolled back, but NOT if control
>>> transfer happens _after_ the `esmLOCKstore`. If THAT were the
>>> case, then this entire mechanism is useless.
>>
>>One can use a esmLOCKstore( *SP, 0 ) to abandon an event. HW
>>detects that *SP is not participating in the event, and uses
>>the lock bit in the instruction to know it is time to leave
>>the event in the failed manner. No ST is performed to the
>>non-participating cache line. {{I guess I could wrap the
>>abandonment into a macro ...}}
>
>
> Again, not what I was asking. I'm trying to understand what you
> meant when you talked about disappearing locks.
If the event cannot be completed due to interrupt or exception or
SCVC or SVR, the HW backs up the event such that is looks like
it never started. SO when control return, the ATOMIC event runs
in its entirety ATOMICally !! So, the illusion of ATOMICITY is
preserved.
> Again, I think you meant that, if an atomic event that sets a
> lock variable fails,
The event has NO lock variable--it has up to 8 cache line address
monitors.
> whether it got far enough along to set the
> variable inside the event or not is not observable if the event
> fails.
Non-participating cache lines can be used to leak information
out of the event (like for debug purposes). Participating data
does not leak.
> But if the event succeeds, then the variable is set and
> that's that. IF that's the case, then great! But if not, then
> disappearing locks are bad.
It is NOT a FRIGGEN LOCK--it is a set of address monitors which
are used to determine the success of the event -- the monitors
remain invisible to SW just like miss buffers remain invisible
to SW. The monitors are not preservable across privilege changes.
> This is especially murky to me
> because there seem to be bits of OS primitives, like threads and
> priority, that are known directly to the hardware.
Just its instantaneous privilege level--which is congruent to
its register file, Root Pointer, ASID, and operating modes.
------------------
>
> Ok, so perhaps this is a reasonable implementation of spinlock
> acquire, as I wrote to Stefan yesterday:
>
> void
> spinlock_acquire(volatile Spinlock *lk)
> {
> while (esmLOCKload(&lk->lock) != 0)
> cpu_spin_hint();
> esmLOCKstore(&lk->lock, 1);
> }
>
> Yes? If this is preempted, or another CPU wins the race on the
> lock, the hardware will back this up and start over in the while
> loop, right?
Yes.
------------
>>Remember only the lines participating in the event count. For
>>example, say one had sprintf() inside the event. None of the
>>lines touched by springt() are participating in the event,
>>and are not counted against the 8 cache lines available.
>
> This doesn't change the point, though. Anything that touches
> more than 8 cache lines is not suitable for an atomic event.
I see what you are saying, but the event is allowed to have an
unbounded amount of touches to non-participating lies.
> Insertion into a doubly linked list is already up to 6 lines,
I moved a doubly linked element from a CDS and inserted it
somewhere else doubly-linked in same CDS in 6 cache lines.
You need to refine how you count.
BOOLEAN InsertElement( Element *el, Element *to )
{
tn = esmLOCKload( to->next );
esmLOCKprefetch( el );
esmLOCKprefetch( tn );
if( !esmINTERFERENCE() )
{
el->next = tn;
el->prev = to;
to->next = el;
esmLOCKstore( tn->prev, el );
return TRUE;
}
return FALSE;
}
> recall that my original hypothetical also involved popping off a
> queue, which adds another (the queue head). If I threw up
> incrementing a couple of counters I can't do it. And this seems
> like a perfectly reasonable real-world scenario; consider a work
> stealing thread scheduler that takes a task from one core to
> another and keeps some stats on how things move.
>
> Or maybe even just swapping places between two elements in a
> linked list. For example:
The example has 3 structures each requiring 3 participating lines.
{themself, the struct their next pointer points at and the struct
their prev pointer points at}
3×3 = 9 and there is no getting around it.
-------------------------
>
> But there isn't a lot of publicly available documentation for
> any of this, at least none that I could find, and from the way
> it's been described thus far it seems like there are a number of
> edge cases that it is not clear how one is supposed to address.
> So I want to understand the boundaries of where these hardware
> facilities are applicable, given architectural limitations. I
> think this thread has shown that those boundaries exist. The
> universe of interesting and useful software is big, and someone,
> somewhere, is going to trip over those limitations sooner or
> later. Once we accept that, we must ask, "Then what?" It seems
> obvious that programmers are going to have to find techniques to
> work around those limitations, and that the mechanisms chosen
> are unknowable to hardware.
Fair enough.
> But now we're back in the same boat of software having to deal
> with all the thorny problems that these mechanisms were designed
> to address in the first place, like priority inversion: take the
> spinlock from earlier, the hardware has no semantic knowledge
> that the lock is used as a mutex, and can't know; it's just some
> bit in memory somewhere that was set during an atomic event.
It is not a FRIGGEN lock, it is a set of address monitors which
guard the instructions in the event. A Lock is a SW concept
which requires a value change to announce to others that "you
got it".
> Maybe ESM helps in a race when trying to acquire the spinlock,
> but once it's acquired, it's acquired, and because the spinlock
> is not a primitive understood by the hardware, the hardware has
> no way of incorporating the spinlock's semantics into its view
> of priority, threading, etc.
Consider a 1024 core system and the time-quantum goes off and
every core wants to rotate the thread it is running; putting
it at the back of the execution queue and taking off the one on
the front to run. Nick McLaren timed a big SUN server (100 core)
on this and it could take up to 6 seconds !! for something that
should take a few milliseconds. This is because of the BigO( n^3 )
nature of bus traffic wrt SNOOPing on that system.
One can use ESM to convert that particular case to BigO( 3 )
or in the general case of random insert and removal times:
BigO( ln( n ) ).
> So while an atomic event might help me in the implementation of
> a generic mutex of some kind, so that I can protect critical
> sections that can't be handled with the atomic event framework
> directly because of their size, that's the extent of the help it
> can give me, because once the mutex is acquired the hardware has
> no insight into the fact that it's a mutex.
Exactly:: once you convert address-based mutual exclusion to
data-based mutual exclusion, HW no longer sees it at all.
> So if some higher
> priority thing subsequently comes along and preempts the thread
> holding that lock and tries to take that lock itself, then the
> hardware can't help me out anymore, and I'm back to a
> priority-inversion deadlock that has to be dealt with by
> software.
HW can only "take" address based locking, not data-based locking.
> The point is, even if the hardware gives me a cool
> mechanism for helping prevent priority inversion problems,
> programmers are inevitably going to have to handle those
> situations themselves anyway.
Never promised you a rose garden.
> This is the sort of thing I'm trying to communicate, it's really
> not me sitting here tossing out barbs like, "oh yeah?! Well,
> how about THIS one, smart guy?"
Look at it this way:: over the decade from 2000-2010 x86 added
a new ATOMIC instruction every iteration.
Using ASF or ESM; my architecture would never have to add one.
SW has the tools to build what it wants/needs.
>
> - Dan C.
[toc] | [prev] | [next] | [standalone]
| From | cross@spitfire.i.gajendra.net (Dan Cross) |
|---|---|
| Date | 2025-03-28 01:01 +0000 |
| Message-ID | <vs4se6$hvb$1@reader1.panix.com> |
| In reply to | #111348 |
In article <601781e2c91d42a73526562b419fdf02@www.novabbs.org>,
MitchAlsup1 <mitchalsup@aol.com> wrote:
>On Thu, 27 Mar 2025 17:19:21 +0000, Dan Cross wrote:
>> In article <7a093bbb356e3bda3782c15ca27e98a7@www.novabbs.org>,
>> MitchAlsup1 <mitchalsup@aol.com> wrote:
>>>On Wed, 26 Mar 2025 4:08:57 +0000, Dan Cross wrote:
>>>> [snip]
>>>> 1. `from` may point to a cache line
>>>> 2. `to` may point to a different cache line.
>>>> 3. Does your architecture use a stack?
>>>
>>>No
>>
>> You make multiple references to a stack just below. I
>> understand stack references are excluded from the atomic event,
>> but I asked whether the architecture uses a stack. Does it, or
>> doesn't it?
>>
>>>> What sort of alignment criteria do you impose?
>>>
>>>On the participating data none. On the cache line--cache line.
>>
>> I meant for data on the stack. The point is moot, though, since
>> as you said stack data does not participate in the atomic event.
>
>Stack does not NECESSARILY participate. However, if you perform
>a esmLOCKload( SP[23] ) then that line on the stack is participating.
Very good.
>>>> At first blush, it seems to me that
>>>> the pointers `from_next`, `from_prev`, and `to_next` could be
>>>> on the stack and if so, will be on at least one cache line,
>>>> and possibly 2, if the call frame for `MoveElement` spans
>>>> more than one.
>>>
>>>The illustrated code is using 6 participating cache lines.
>>>Where local variables are kept (stack, registers) does not
>>>count against the tally of participating cache lines.
>>
>> Huh.
>
>There are 6 esmLOCKxxxxx() so there are 6 participating lines--
>and these are absolutely independent from where the variables
>are located. The variables carry information but do not part-
>ticipate they carry data between operations on the particupating
>lines.
That was clearly in the code. That was not what the "Huh" was
in resposne to.
>> How would this handle something like an MCS lock, where
>> the lock node may be on a stack, though accessible globally in
>> some virtual address space?
>
>As illustrated above esmLOCKload( SP[23]) will cause a line on
>the stack to participate in the event.
Ok, good to know.
>----------------
>>>> But later you wrote,
>>>>
>>>>>So, if you want the property whereby the lock disappears on any
>>>>>control transfer out of the event {exception, interrupt, SVC, SVR, ...};
>>>>>then you want to use my ATOMIC stuff; otherwise, you can use the
>>>>>normal ATOMIC primitives everyone and his brother provide.
>>>>
>>>> Well, what precisely do you mean here when you say, "if you want
>>>> the property whereby the lock disappears on any control transfer
>>>> out of the event"?
>>>
>>>If you want that property--you use the tools at hand.
>>>If you don't just use them as primitive generators.
>>
>> I wasn't clear enough here. I'm asking what, exactly, you mean
>> by this _property_,
>
>The property that if an interrupt (or exception or SVC or SVR)
>prevents executing the event to its conclusion, HW makes the
>event look like it never started. So that when/if control returns
>we have another chance to execute the event as-if ATOMICally.
Ok. So let's be crystal clear here: after the event concludes
successfully, a subsequent exception, interrupt, or whatever,
will not magically roll back any of the values set and made
visible system-wide as a result of the successful conclusion of
the atomic event. Correct? That's what I've been asking.
>> not what you mean when you write that one
>> can use these atomic events if one wants the property. That is,
>> I'm asking you to precisely define what it means for a lock to
>> "disappear".
>>
>> It seems clear enough _now_ that once the event concludes
>> successfully the lock value is set to whatever it was set to in
>> during the event, but that wasn't clear to me earlier and I
>> wanted confirmation.
>
>Not "the lock value" but "all participating values" are set
>and become visible system-wide in a single instant.
Sure. But there was only one participating value in the
spinlock example. ;-}
>No 3rd
>party sees some set and others remain unset. Everybody sees
>all of then change value between this-clock and its successor.
Sure. You've said this all along, and I've never disputed it.
>> In particular, it seems odd to me that one would bother with a
>> lock of some kind during an event if it didn't remain set after
>> the event.
>
>Consider CompareTripleSwapFour() as an atomic primitive. How
>would you program this such that nobody nowhere could ever
>tell that the four updated locations changes in any order
>then simultaneously ??? Without using a LOCK to guard the
>event ??? And without demanding that all four updates are
>to the same cache line ???
>
>{{Nobody nowhere included ATOMIC-blind DMA requests}}
The fundamental issue here is that, when I refer to a "lock" I
have been talking about a software mutual exclusion primitive,
and you are referring to something else entirely.
I would have thought my meaning was clear from context;
evidently not.
>> If you do all of your critical section stuff inside
>> of the event, and setting the lock in the event is not visible
>> until the event concludes, why bother? If on the other hand you
>> use the event to set the lock, why bother doing additional work
>> inside the event itself, but in this case I definitely don't
>> want something else to come along and just unset the lock on its
>> own, higher priority or not.
>
>It is clear you do not understand the trouble HW takes to implement
>even DCADS where between any 2 clocks, one (or more) SNOOPs can
>take the lines you are DCADSing. It is this property of cache
>coherence that gets in the way of multi-operation ATOMIC events.
As above, when I refer to "setting the lock" I am referring to
acquiring a mutex; a software primitive. I am not referring to
what the hardware does internally. Consider what I wrote above,
the section that you quoted, in that context.
>In 2004 Fred Weber came to me and ask: "Why can't we (AMD) give
>Microsoft the DCADS they want. I dug into it, and ASF was my
>solution in x86 ISA, ESM is my solution in RISC-based ISA. It
>removes the need to add ATOMIC primitives to ISA over generations.
>
>It also provides the ability to do other things--which apparently
>you will not use--sort of like complaining that your new car has
>auto-park a feature you will never use.
"It is clear you do not understand the trouble SW takes to
implement even simple things that are beyond the capability of
hardware."
(Back at you. I think I've been pretty polite here; I'm getting
tired of the barbs.)
>>>> What I _hope_ you mean is that if you transfer "out of the
>>>> event" before the `esmLOCKstore` then anything that's been done
>>>> since the "event" started is rolled back, but NOT if control
>>>> transfer happens _after_ the `esmLOCKstore`. If THAT were the
>>>> case, then this entire mechanism is useless.
>>>
>>>One can use a esmLOCKstore( *SP, 0 ) to abandon an event. HW
>>>detects that *SP is not participating in the event, and uses
>>>the lock bit in the instruction to know it is time to leave
>>>the event in the failed manner. No ST is performed to the
>>>non-participating cache line. {{I guess I could wrap the
>>>abandonment into a macro ...}}
>>
>> Again, not what I was asking. I'm trying to understand what you
>> meant when you talked about disappearing locks.
>
>If the event cannot be completed due to interrupt or exception or
>SCVC or SVR, the HW backs up the event such that is looks like
>it never started. SO when control return, the ATOMIC event runs
>in its entirety ATOMICally !! So, the illusion of ATOMICITY is
>preserved.
Sigh. You've said this many times over now. I get it. I'm not
disputing it, nor do I believe I ever have.
Again, when I said "lock" earlier, I am referring to a mutux
from the perspective of software. That is, the value of the
structure member that I called "lock" in the version of the
`Spinlock` struct that I posted however-many responses ago, and
that I keep referring to in example code. I am not asking about
what the hardware does internally. I'm not talking about locked
bus cycles or whatever. I'm not referring to cache line address
monitors or any of these other things that you keep referring
to. I'm talking the the value of the structure member called
`lock` in the `Spinlock` C structure I posted earlier.
What I have been asking, over and over again, is whether that
_value_ "disappears" _after_ an atomic event successfully
concludes. That is, the value that was written into the memory,
not whatever internal marker you use in hardware to monitor and
guard those stores until the atomic event completes or is
otherwise exited, aborted, cancelled, interrupted, or however
you prefer to refer to it not completing successfully. That is
a totally separate thing and I not presume to speak of how its
done, though in fairness to you you've made it reasonably clear.
Does that clarify what I have been asking about? I honestly
thought this would be obvious from context.
>> Again, I think you meant that, if an atomic event that sets a
>> lock variable fails,
>
>The event has NO lock variable--it has up to 8 cache line address
>monitors.
The structure member is literally called `lock` and the context
is software spinlocks. Note, spinlocks because they're simple
and easy to discuss, not because they're great mutex primtives.
As frustrating as this evidently is to you, it strikes me that
it actually reinforces my point: hardware has no insight into
what the software is _doing_ at a higher semantic level.
>> whether it got far enough along to set the
>> variable inside the event or not is not observable if the event
>> fails.
>
>Non-participating cache lines can be used to leak information
>out of the event (like for debug purposes). Participating data
>does not leak.
>
>> But if the event succeeds, then the variable is set and
>> that's that. IF that's the case, then great! But if not, then
>> disappearing locks are bad.
>
>It is NOT a FRIGGEN LOCK--
I am talking about the spinlock. That is a software object, and
in that context, it is very much called a "lock".
I understand when you say that it is not a lock, it is not a
lock in the sense relevant to your world in hardware, but that's
not what _I'm_ talking about.
I'm trying to square what you meant when you talked about
"disappearing locks" with what happens to those software objects
_after_ an atomic event modifying them successfully completes.
Evidently you were talking about something else entirely, and
that's fine. I gather it has nothing to do with _my_ notion of
locks after they've been successfully acquired; I was just
trying to confirm that.
>it is a set of address monitors which
>are used to determine the success of the event -- the monitors
>remain invisible to SW just like miss buffers remain invisible
>to SW. The monitors are not preservable across privilege changes.
See above. Tell me what you want me to call it and I'll use
that terminology instead.
>> This is especially murky to me
>> because there seem to be bits of OS primitives, like threads and
>> priority, that are known directly to the hardware.
>
>Just its instantaneous privilege level--which is congruent to
>its register file, Root Pointer, ASID, and operating modes.
What is "it" here?
>------------------
>>
>> Ok, so perhaps this is a reasonable implementation of spinlock
>> acquire, as I wrote to Stefan yesterday:
>>
>> void
>> spinlock_acquire(volatile Spinlock *lk)
>> {
>> while (esmLOCKload(&lk->lock) != 0)
>> cpu_spin_hint();
>> esmLOCKstore(&lk->lock, 1);
>> }
>>
>> Yes? If this is preempted, or another CPU wins the race on the
>> lock, the hardware will back this up and start over in the while
>> loop, right?
>
>Yes.
>------------
>>>Remember only the lines participating in the event count. For
>>>example, say one had sprintf() inside the event. None of the
>>>lines touched by springt() are participating in the event,
>>>and are not counted against the 8 cache lines available.
>>
>> This doesn't change the point, though. Anything that touches
>> more than 8 cache lines is not suitable for an atomic event.
>
>I see what you are saying, but the event is allowed to have an
>unbounded amount of touches to non-participating lies.
I don't see how that's helpful if I need to touch data in more
than 8 cache lines atomically.
>> Insertion into a doubly linked list is already up to 6 lines,
>
>I moved a doubly linked element from a CDS and inserted it
>somewhere else doubly-linked in same CDS in 6 cache lines.
>You need to refine how you count.
<Tongue-in-cheek mode on>
Ah, yes. The program that would dump core if you tried to move
from the head of a list or the rear and that couldn't insert at
the head pointer was, in fact, called `MoveElement`. My
mistake, even if it was clear that that's what I was talking
about, given that quoted where I referred to it in the context
of work stealing later on. ;-P
>BOOLEAN InsertElement( Element *el, Element *to )
>{
> tn = esmLOCKload( to->next );
> esmLOCKprefetch( el );
> esmLOCKprefetch( tn );
> if( !esmINTERFERENCE() )
> {
> el->next = tn;
> el->prev = to;
> to->next = el;
> esmLOCKstore( tn->prev, el );
> return TRUE;
> }
> return FALSE;
>}
If you're going to be pendatic when I slip up and call "move"
"insert", then let's be pendantic.
This code won't compile, because of the obvious error of not
defining `tn`. Even if you fix that, it's not correct if you
try to insert at the end of the list. It's not clear now you
would insert at the head of a list, or into an empty list, but
that depends on how you represent the list itself. ie, the
head may just be a pointer to the first element, or it might be
an otherwise unused node; something like this:
typedef struct List List;
struct List {
Element head;
};
where the first element in the list is always `head.next`.
It would help if you used standard types, like `bool`, which has
been available standard for more than 25 years now since
introduced in C99 as `_Bool` with accompanying typedefs and
macros in `stdbool.h` for `bool`, `true` and `false`.
And doesn't this presume that `el->next`, `el->prev` and `el`
are all on the same cache line? Similarly with `tn` and
`tn->prev`?
You need to refine how you write programs. ;-}
Here's a better version, that assumes the list representation
above.
#include <stddef.h>
#include <stdbool.h>
typedef struct Element Element;
struct Element {
Element *next;
Element *prev;
};
typedef struct List List;
struct List {
Element head;
};
Element *
ListBegin(List *list)
{
return list->head.next;
}
bool
InsertElement(Element *el, Element *to)
{
Element *tn = esmLOCKload(to->next);
esmLOCKprefetch(el->next);
esmLOCKprefetch(el->prev);
if (tn != NULL)
esmLOCKprefetch(tn->prev);
if (!esmINTERFERENCE()) {
el->next = tn;
el->prev = to;
to->next = el;
esmLOCKstore(tn->prev, el);
return true;
}
return false;
}
Inserting at the head, or if the list is empty, are both easy
now: `InsertElement(&list.head, el)`. But note, again, that the
first element of the list is really `list.head.next`, hence the
`ListBegin` function.
Here I count up to three, but possibly up to 4, cache lines.
Note that if you did Not represent the list this way, in order
to support generalized insertions anywhere in the list, you'd
either need a circular representation (which is annoying to use
in practice), or you'd most likely have to support separate
head and tail pointers and two operations, one to insert before,
and another after, the 'to' pointer. But then you'd have to be
prepared to update either the head or tail pointers (or both if
the list is empty); either way it's potentially two more cache
lines to account for, though if the list is empty you don't have
to touch `tn->prev`.
>> recall that my original hypothetical also involved popping off a
>> queue, which adds another (the queue head). If I threw up
>> incrementing a couple of counters I can't do it. And this seems
>> like a perfectly reasonable real-world scenario; consider a work
>> stealing thread scheduler that takes a task from one core to
>> another and keeps some stats on how things move.
>>
>> Or maybe even just swapping places between two elements in a
>> linked list. For example:
>
>The example has 3 structures each requiring 3 participating lines.
>{themself, the struct their next pointer points at and the struct
>their prev pointer points at}
>
>3×3 = 9 and there is no getting around it.
Yup. So what does the hardware do if I try it?
>-------------------------
>>
>> But there isn't a lot of publicly available documentation for
>> any of this, at least none that I could find, and from the way
>> it's been described thus far it seems like there are a number of
>> edge cases that it is not clear how one is supposed to address.
>> So I want to understand the boundaries of where these hardware
>> facilities are applicable, given architectural limitations. I
>> think this thread has shown that those boundaries exist. The
>> universe of interesting and useful software is big, and someone,
>> somewhere, is going to trip over those limitations sooner or
>> later. Once we accept that, we must ask, "Then what?" It seems
>> obvious that programmers are going to have to find techniques to
>> work around those limitations, and that the mechanisms chosen
>> are unknowable to hardware.
>
>Fair enough.
>
>> But now we're back in the same boat of software having to deal
>> with all the thorny problems that these mechanisms were designed
>> to address in the first place, like priority inversion: take the
>> spinlock from earlier, the hardware has no semantic knowledge
>> that the lock is used as a mutex, and can't know; it's just some
>> bit in memory somewhere that was set during an atomic event.
>
>It is not a FRIGGEN lock, it is a set of address monitors which
>guard the instructions in the event.
Now I have to wonder if you're deliberatly ignoring the context.
I called it a "lock" in the same sentence I said it was a
"spinlock". Are you seriously going to tell me that you could
not figure out that the "lock" I was referring to was the
spinlock? You know that that's a software thing; you yourself
said that.
Note the words, "it's just some bit in memory that _was set
during an atomic event._" Is it not clear that the event is
over? It's done. It was successful. All of the address
monitors guarding instructions in the event are no longer
relevant; they've been retired or whatever.
Now, some (software) thread holds a (software) (spin)lock.
I've tried over and over to make this clear, yet you insist on
going back to talking about what happens _during_ an atomic
event. I'm talking about _after_ the event completed
_succesfully_.
>A Lock is a SW concept
>which requires a value change to announce to others that "you
>got it".
That's exactly what `spinlock_acquire` from above _does_, isn't
it?
You cannot seriously expect me to accept that you did not
understand this.
>> Maybe ESM helps in a race when trying to acquire the spinlock,
>> but once it's acquired, it's acquired, and because the spinlock
>> is not a primitive understood by the hardware, the hardware has
>> no way of incorporating the spinlock's semantics into its view
>> of priority, threading, etc.
>
>Consider a 1024 core system and the time-quantum goes off and
>every core wants to rotate the thread it is running; putting
>it at the back of the execution queue and taking off the one on
>the front to run. Nick McLaren timed a big SUN server (100 core)
>on this and it could take up to 6 seconds !! for something that
>should take a few milliseconds. This is because of the BigO( n^3 )
>nature of bus traffic wrt SNOOPing on that system.
This was solved with per-core run queues.
You're senior enough that I would think it clear that spinlocks
are just a convenient illustrative example here because they're
so simple, but their pitfalls as core counts increase are well
studied and understood. In the real world, mutex structures
like MCS locks or CLH locks scale approximately linearly with
core-count under contention. I posted a reference to this
earlier, but here it is again. You may have a hard time
accepting it because they call the things that they refer to,
"locks". ;-}
https://pdos.csail.mit.edu/papers/linux:lock.pdf
>One can use ESM to convert that particular case to BigO( 3 )
>or in the general case of random insert and removal times:
>BigO( ln( n ) ).
Modern scheduling algorithms already run in amortized constant
time on conventional hardware.
>> So while an atomic event might help me in the implementation of
>> a generic mutex of some kind, so that I can protect critical
>> sections that can't be handled with the atomic event framework
>> directly because of their size, that's the extent of the help it
>> can give me, because once the mutex is acquired the hardware has
>> no insight into the fact that it's a mutex.
>
>Exactly:: once you convert address-based mutual exclusion to
>data-based mutual exclusion, HW no longer sees it at all.
That's what I've been saying all along.
>> So if some higher
>> priority thing subsequently comes along and preempts the thread
>> holding that lock and tries to take that lock itself, then the
>> hardware can't help me out anymore, and I'm back to a
>> priority-inversion deadlock that has to be dealt with by
>> software.
>
>HW can only "take" address based locking, not data-based locking.
Precisely. So it can't help me with priority inversion
generally.
>> The point is, even if the hardware gives me a cool
>> mechanism for helping prevent priority inversion problems,
>> programmers are inevitably going to have to handle those
>> situations themselves anyway.
>
>Never promised you a rose garden.
It's ok; I've already been through Parris Island. And OCS at
Quantico, AFG deployment, la la la. But that's neither here nor
there. Point is, I don't need your roses, and I probably don't
want your threads or anything more advanced than things that let
me build my own synchronization primitives, threads, priority
abstractions, and so on.
>> This is the sort of thing I'm trying to communicate, it's really
>> not me sitting here tossing out barbs like, "oh yeah?! Well,
>> how about THIS one, smart guy?"
>
>Look at it this way:: over the decade from 2000-2010 x86 added
>a new ATOMIC instruction every iteration.
>
>Using ASF or ESM; my architecture would never have to add one.
>SW has the tools to build what it wants/needs.
Ok, now we're getting somewhere useful. That's handy.
Look, while don't you put the documentation of this stuff
somewhere publicly accessible? It'd probably save people like
me a lot of time, frustration, and dad-level insults. Probably
save you some frustration as well.
- Dan C.
[toc] | [prev] | [next] | [standalone]
| From | mitchalsup@aol.com (MitchAlsup1) |
|---|---|
| Date | 2025-03-28 01:35 +0000 |
| Message-ID | <71ce5497488c10e3f4ade7b451acee08@www.novabbs.org> |
| In reply to | #111351 |
On Fri, 28 Mar 2025 1:01:59 +0000, Dan Cross wrote:
> In article <601781e2c91d42a73526562b419fdf02@www.novabbs.org>,
> MitchAlsup1 <mitchalsup@aol.com> wrote:
>>On Thu, 27 Mar 2025 17:19:21 +0000, Dan Cross wrote:
>>> In article <7a093bbb356e3bda3782c15ca27e98a7@www.novabbs.org>,
>>> MitchAlsup1 <mitchalsup@aol.com> wrote:
>>>>On Wed, 26 Mar 2025 4:08:57 +0000, Dan Cross wrote:
>>>>> [snip]
>>>>> 1. `from` may point to a cache line
>>>>> 2. `to` may point to a different cache line.
>>>>> 3. Does your architecture use a stack?
>>>>
>>>>No
>>>
>>> You make multiple references to a stack just below. I
>>> understand stack references are excluded from the atomic event,
>>> but I asked whether the architecture uses a stack. Does it, or
>>> doesn't it?
>>>
>>>>> What sort of alignment criteria do you impose?
>>>>
>>>>On the participating data none. On the cache line--cache line.
>>>
>>> I meant for data on the stack. The point is moot, though, since
>>> as you said stack data does not participate in the atomic event.
>>
>>Stack does not NECESSARILY participate. However, if you perform
>>a esmLOCKload( SP[23] ) then that line on the stack is participating.
>
> Very good.
>
>>>>> At first blush, it seems to me that
>>>>> the pointers `from_next`, `from_prev`, and `to_next` could be
>>>>> on the stack and if so, will be on at least one cache line,
>>>>> and possibly 2, if the call frame for `MoveElement` spans
>>>>> more than one.
>>>>
>>>>The illustrated code is using 6 participating cache lines.
>>>>Where local variables are kept (stack, registers) does not
>>>>count against the tally of participating cache lines.
>>>
>>> Huh.
>>
>>There are 6 esmLOCKxxxxx() so there are 6 participating lines--
>>and these are absolutely independent from where the variables
>>are located. The variables carry information but do not part-
>>ticipate they carry data between operations on the particupating
>>lines.
>
> That was clearly in the code. That was not what the "Huh" was
> in resposne to.
>
>>> How would this handle something like an MCS lock, where
>>> the lock node may be on a stack, though accessible globally in
>>> some virtual address space?
>>
>>As illustrated above esmLOCKload( SP[23]) will cause a line on
>>the stack to participate in the event.
>
> Ok, good to know.
>
>>----------------
>>>>> But later you wrote,
>>>>>
>>>>>>So, if you want the property whereby the lock disappears on any
>>>>>>control transfer out of the event {exception, interrupt, SVC, SVR, ...};
>>>>>>then you want to use my ATOMIC stuff; otherwise, you can use the
>>>>>>normal ATOMIC primitives everyone and his brother provide.
>>>>>
>>>>> Well, what precisely do you mean here when you say, "if you want
>>>>> the property whereby the lock disappears on any control transfer
>>>>> out of the event"?
>>>>
>>>>If you want that property--you use the tools at hand.
>>>>If you don't just use them as primitive generators.
>>>
>>> I wasn't clear enough here. I'm asking what, exactly, you mean
>>> by this _property_,
>>
>>The property that if an interrupt (or exception or SVC or SVR)
>>prevents executing the event to its conclusion, HW makes the
>>event look like it never started. So that when/if control returns
>>we have another chance to execute the event as-if ATOMICally.
>
> Ok. So let's be crystal clear here: after the event concludes
> successfully, a subsequent exception, interrupt, or whatever,
> will not magically roll back any of the values set and made
> visible system-wide as a result of the successful conclusion of
> the atomic event. Correct? That's what I've been asking.
>
>>> not what you mean when you write that one
>>> can use these atomic events if one wants the property. That is,
>>> I'm asking you to precisely define what it means for a lock to
>>> "disappear".
>>>
>>> It seems clear enough _now_ that once the event concludes
>>> successfully the lock value is set to whatever it was set to in
>>> during the event, but that wasn't clear to me earlier and I
>>> wanted confirmation.
>>
>>Not "the lock value" but "all participating values" are set
>>and become visible system-wide in a single instant.
>
> Sure. But there was only one participating value in the
> spinlock example. ;-}
>
>>No 3rd
>>party sees some set and others remain unset. Everybody sees
>>all of then change value between this-clock and its successor.
>
> Sure. You've said this all along, and I've never disputed it.
>
>>> In particular, it seems odd to me that one would bother with a
>>> lock of some kind during an event if it didn't remain set after
>>> the event.
>>
>>Consider CompareTripleSwapFour() as an atomic primitive. How
>>would you program this such that nobody nowhere could ever
>>tell that the four updated locations changes in any order
>>then simultaneously ??? Without using a LOCK to guard the
>>event ??? And without demanding that all four updates are
>>to the same cache line ???
>>
>>{{Nobody nowhere included ATOMIC-blind DMA requests}}
>
> The fundamental issue here is that, when I refer to a "lock" I
> have been talking about a software mutual exclusion primitive,
> and you are referring to something else entirely.
>
> I would have thought my meaning was clear from context;
> evidently not.
>
>>> If you do all of your critical section stuff inside
>>> of the event, and setting the lock in the event is not visible
>>> until the event concludes, why bother? If on the other hand you
>>> use the event to set the lock, why bother doing additional work
>>> inside the event itself, but in this case I definitely don't
>>> want something else to come along and just unset the lock on its
>>> own, higher priority or not.
>>
>>It is clear you do not understand the trouble HW takes to implement
>>even DCADS where between any 2 clocks, one (or more) SNOOPs can
>>take the lines you are DCADSing. It is this property of cache
>>coherence that gets in the way of multi-operation ATOMIC events.
>
> As above, when I refer to "setting the lock" I am referring to
> acquiring a mutex; a software primitive. I am not referring to
> what the hardware does internally. Consider what I wrote above,
> the section that you quoted, in that context.
>
>>In 2004 Fred Weber came to me and ask: "Why can't we (AMD) give
>>Microsoft the DCADS they want. I dug into it, and ASF was my
>>solution in x86 ISA, ESM is my solution in RISC-based ISA. It
>>removes the need to add ATOMIC primitives to ISA over generations.
>>
>>It also provides the ability to do other things--which apparently
>>you will not use--sort of like complaining that your new car has
>>auto-park a feature you will never use.
>
> "It is clear you do not understand the trouble SW takes to
> implement even simple things that are beyond the capability of
> hardware."
>
> (Back at you. I think I've been pretty polite here; I'm getting
> tired of the barbs.)
>
>>>>> What I _hope_ you mean is that if you transfer "out of the
>>>>> event" before the `esmLOCKstore` then anything that's been done
>>>>> since the "event" started is rolled back, but NOT if control
>>>>> transfer happens _after_ the `esmLOCKstore`. If THAT were the
>>>>> case, then this entire mechanism is useless.
>>>>
>>>>One can use a esmLOCKstore( *SP, 0 ) to abandon an event. HW
>>>>detects that *SP is not participating in the event, and uses
>>>>the lock bit in the instruction to know it is time to leave
>>>>the event in the failed manner. No ST is performed to the
>>>>non-participating cache line. {{I guess I could wrap the
>>>>abandonment into a macro ...}}
>>>
>>> Again, not what I was asking. I'm trying to understand what you
>>> meant when you talked about disappearing locks.
>>
>>If the event cannot be completed due to interrupt or exception or
>>SCVC or SVR, the HW backs up the event such that is looks like
>>it never started. SO when control return, the ATOMIC event runs
>>in its entirety ATOMICally !! So, the illusion of ATOMICITY is
>>preserved.
>
> Sigh. You've said this many times over now. I get it. I'm not
> disputing it, nor do I believe I ever have.
>
> Again, when I said "lock" earlier, I am referring to a mutux
> from the perspective of software. That is, the value of the
> structure member that I called "lock" in the version of the
> `Spinlock` struct that I posted however-many responses ago, and
> that I keep referring to in example code. I am not asking about
> what the hardware does internally. I'm not talking about locked
> bus cycles or whatever. I'm not referring to cache line address
> monitors or any of these other things that you keep referring
> to. I'm talking the the value of the structure member called
> `lock` in the `Spinlock` C structure I posted earlier.
>
> What I have been asking, over and over again, is whether that
> _value_ "disappears" _after_ an atomic event successfully
> concludes. That is, the value that was written into the memory,
> not whatever internal marker you use in hardware to monitor and
> guard those stores until the atomic event completes or is
> otherwise exited, aborted, cancelled, interrupted, or however
> you prefer to refer to it not completing successfully. That is
> a totally separate thing and I not presume to speak of how its
> done, though in fairness to you you've made it reasonably clear.
>
> Does that clarify what I have been asking about? I honestly
> thought this would be obvious from context.
>
>>> Again, I think you meant that, if an atomic event that sets a
>>> lock variable fails,
>>
>>The event has NO lock variable--it has up to 8 cache line address
>>monitors.
>
> The structure member is literally called `lock` and the context
> is software spinlocks. Note, spinlocks because they're simple
> and easy to discuss, not because they're great mutex primtives.
>
> As frustrating as this evidently is to you, it strikes me that
> it actually reinforces my point: hardware has no insight into
> what the software is _doing_ at a higher semantic level.
>
>>> whether it got far enough along to set the
>>> variable inside the event or not is not observable if the event
>>> fails.
>>
>>Non-participating cache lines can be used to leak information
>>out of the event (like for debug purposes). Participating data
>>does not leak.
>>
>>> But if the event succeeds, then the variable is set and
>>> that's that. IF that's the case, then great! But if not, then
>>> disappearing locks are bad.
>>
>>It is NOT a FRIGGEN LOCK--
>
> I am talking about the spinlock. That is a software object, and
> in that context, it is very much called a "lock".
>
> I understand when you say that it is not a lock, it is not a
> lock in the sense relevant to your world in hardware, but that's
> not what _I'm_ talking about.
>
> I'm trying to square what you meant when you talked about
> "disappearing locks" with what happens to those software objects
> _after_ an atomic event modifying them successfully completes.
>
> Evidently you were talking about something else entirely, and
> that's fine. I gather it has nothing to do with _my_ notion of
> locks after they've been successfully acquired; I was just
> trying to confirm that.
>
>>it is a set of address monitors which
>>are used to determine the success of the event -- the monitors
>>remain invisible to SW just like miss buffers remain invisible
>>to SW. The monitors are not preservable across privilege changes.
>
> See above. Tell me what you want me to call it and I'll use
> that terminology instead.
>
>>> This is especially murky to me
>>> because there seem to be bits of OS primitives, like threads and
>>> priority, that are known directly to the hardware.
>>
>>Just its instantaneous privilege level--which is congruent to
>>its register file, Root Pointer, ASID, and operating modes.
>
> What is "it" here?
>
>>------------------
>>>
>>> Ok, so perhaps this is a reasonable implementation of spinlock
>>> acquire, as I wrote to Stefan yesterday:
>>>
>>> void
>>> spinlock_acquire(volatile Spinlock *lk)
>>> {
>>> while (esmLOCKload(&lk->lock) != 0)
>>> cpu_spin_hint();
>>> esmLOCKstore(&lk->lock, 1);
>>> }
>>>
>>> Yes? If this is preempted, or another CPU wins the race on the
>>> lock, the hardware will back this up and start over in the while
>>> loop, right?
>>
>>Yes.
>>------------
>>>>Remember only the lines participating in the event count. For
>>>>example, say one had sprintf() inside the event. None of the
>>>>lines touched by springt() are participating in the event,
>>>>and are not counted against the 8 cache lines available.
>>>
>>> This doesn't change the point, though. Anything that touches
>>> more than 8 cache lines is not suitable for an atomic event.
>>
>>I see what you are saying, but the event is allowed to have an
>>unbounded amount of touches to non-participating lies.
>
> I don't see how that's helpful if I need to touch data in more
> than 8 cache lines atomically.
>
>>> Insertion into a doubly linked list is already up to 6 lines,
>>
>>I moved a doubly linked element from a CDS and inserted it
>>somewhere else doubly-linked in same CDS in 6 cache lines.
>>You need to refine how you count.
>
> <Tongue-in-cheek mode on>
> Ah, yes. The program that would dump core if you tried to move
> from the head of a list or the rear and that couldn't insert at
> the head pointer was, in fact, called `MoveElement`. My
> mistake, even if it was clear that that's what I was talking
> about, given that quoted where I referred to it in the context
> of work stealing later on. ;-P
>
>>BOOLEAN InsertElement( Element *el, Element *to )
>>{
>> tn = esmLOCKload( to->next );
>> esmLOCKprefetch( el );
>> esmLOCKprefetch( tn );
>> if( !esmINTERFERENCE() )
>> {
>> el->next = tn;
>> el->prev = to;
>> to->next = el;
>> esmLOCKstore( tn->prev, el );
>> return TRUE;
>> }
>> return FALSE;
>>}
>
> If you're going to be pendatic when I slip up and call "move"
> "insert", then let's be pendantic.
>
> This code won't compile, because of the obvious error of not
> defining `tn`. Even if you fix that, it's not correct if you
> try to insert at the end of the list. It's not clear now you
> would insert at the head of a list, or into an empty list, but
> that depends on how you represent the list itself. ie, the
> head may just be a pointer to the first element, or it might be
> an otherwise unused node; something like this:
>
> typedef struct List List;
> struct List {
> Element head;
> };
>
> where the first element in the list is always `head.next`.
>
> It would help if you used standard types, like `bool`, which has
> been available standard for more than 25 years now since
> introduced in C99 as `_Bool` with accompanying typedefs and
> macros in `stdbool.h` for `bool`, `true` and `false`.
>
> And doesn't this presume that `el->next`, `el->prev` and `el`
> are all on the same cache line? Similarly with `tn` and
> `tn->prev`?
>
> You need to refine how you write programs. ;-}
>
> Here's a better version, that assumes the list representation
> above.
>
> #include <stddef.h>
> #include <stdbool.h>
>
> typedef struct Element Element;
> struct Element {
> Element *next;
> Element *prev;
> };
>
> typedef struct List List;
> struct List {
> Element head;
> };
>
> Element *
> ListBegin(List *list)
> {
> return list->head.next;
> }
>
> bool
> InsertElement(Element *el, Element *to)
> {
> Element *tn = esmLOCKload(to->next);
> esmLOCKprefetch(el->next);
> esmLOCKprefetch(el->prev);
> if (tn != NULL)
> esmLOCKprefetch(tn->prev);
>
> if (!esmINTERFERENCE()) {
> el->next = tn;
> el->prev = to;
> to->next = el;
> esmLOCKstore(tn->prev, el);
> return true;
> }
> return false;
> }
>
> Inserting at the head, or if the list is empty, are both easy
> now: `InsertElement(&list.head, el)`. But note, again, that the
> first element of the list is really `list.head.next`, hence the
> `ListBegin` function.
>
> Here I count up to three, but possibly up to 4, cache lines.
>
> Note that if you did Not represent the list this way, in order
> to support generalized insertions anywhere in the list, you'd
> either need a circular representation (which is annoying to use
> in practice), or you'd most likely have to support separate
> head and tail pointers and two operations, one to insert before,
> and another after, the 'to' pointer. But then you'd have to be
> prepared to update either the head or tail pointers (or both if
> the list is empty); either way it's potentially two more cache
> lines to account for, though if the list is empty you don't have
> to touch `tn->prev`.
>
>>> recall that my original hypothetical also involved popping off a
>>> queue, which adds another (the queue head). If I threw up
>>> incrementing a couple of counters I can't do it. And this seems
>>> like a perfectly reasonable real-world scenario; consider a work
>>> stealing thread scheduler that takes a task from one core to
>>> another and keeps some stats on how things move.
>>>
>>> Or maybe even just swapping places between two elements in a
>>> linked list. For example:
>>
>>The example has 3 structures each requiring 3 participating lines.
>>{themself, the struct their next pointer points at and the struct
>>their prev pointer points at}
>>
>>3×3 = 9 and there is no getting around it.
>
> Yup. So what does the hardware do if I try it?
>
>>-------------------------
>>>
>>> But there isn't a lot of publicly available documentation for
>>> any of this, at least none that I could find, and from the way
>>> it's been described thus far it seems like there are a number of
>>> edge cases that it is not clear how one is supposed to address.
>>> So I want to understand the boundaries of where these hardware
>>> facilities are applicable, given architectural limitations. I
>>> think this thread has shown that those boundaries exist. The
>>> universe of interesting and useful software is big, and someone,
>>> somewhere, is going to trip over those limitations sooner or
>>> later. Once we accept that, we must ask, "Then what?" It seems
>>> obvious that programmers are going to have to find techniques to
>>> work around those limitations, and that the mechanisms chosen
>>> are unknowable to hardware.
>>
>>Fair enough.
>>
>>> But now we're back in the same boat of software having to deal
>>> with all the thorny problems that these mechanisms were designed
>>> to address in the first place, like priority inversion: take the
>>> spinlock from earlier, the hardware has no semantic knowledge
>>> that the lock is used as a mutex, and can't know; it's just some
>>> bit in memory somewhere that was set during an atomic event.
>>
>>It is not a FRIGGEN lock, it is a set of address monitors which
>>guard the instructions in the event.
>
> Now I have to wonder if you're deliberatly ignoring the context.
>
> I called it a "lock" in the same sentence I said it was a
> "spinlock". Are you seriously going to tell me that you could
> not figure out that the "lock" I was referring to was the
> spinlock? You know that that's a software thing; you yourself
> said that.
>
> Note the words, "it's just some bit in memory that _was set
> during an atomic event._" Is it not clear that the event is
> over? It's done. It was successful. All of the address
> monitors guarding instructions in the event are no longer
> relevant; they've been retired or whatever.
>
> Now, some (software) thread holds a (software) (spin)lock.
> I've tried over and over to make this clear, yet you insist on
> going back to talking about what happens _during_ an atomic
> event. I'm talking about _after_ the event completed
> _succesfully_.
>
>>A Lock is a SW concept
>>which requires a value change to announce to others that "you
>>got it".
>
> That's exactly what `spinlock_acquire` from above _does_, isn't
> it?
>
> You cannot seriously expect me to accept that you did not
> understand this.
>
>>> Maybe ESM helps in a race when trying to acquire the spinlock,
>>> but once it's acquired, it's acquired, and because the spinlock
>>> is not a primitive understood by the hardware, the hardware has
>>> no way of incorporating the spinlock's semantics into its view
>>> of priority, threading, etc.
>>
>>Consider a 1024 core system and the time-quantum goes off and
>>every core wants to rotate the thread it is running; putting
>>it at the back of the execution queue and taking off the one on
>>the front to run. Nick McLaren timed a big SUN server (100 core)
>>on this and it could take up to 6 seconds !! for something that
>>should take a few milliseconds. This is because of the BigO( n^3 )
>>nature of bus traffic wrt SNOOPing on that system.
>
> This was solved with per-core run queues.
>
> You're senior enough that I would think it clear that spinlocks
> are just a convenient illustrative example here because they're
> so simple, but their pitfalls as core counts increase are well
> studied and understood. In the real world, mutex structures
> like MCS locks or CLH locks scale approximately linearly with
> core-count under contention. I posted a reference to this
> earlier, but here it is again. You may have a hard time
> accepting it because they call the things that they refer to,
> "locks". ;-}
> https://pdos.csail.mit.edu/papers/linux:lock.pdf
>
>>One can use ESM to convert that particular case to BigO( 3 )
>>or in the general case of random insert and removal times:
>>BigO( ln( n ) ).
>
> Modern scheduling algorithms already run in amortized constant
> time on conventional hardware.
>
>>> So while an atomic event might help me in the implementation of
>>> a generic mutex of some kind, so that I can protect critical
>>> sections that can't be handled with the atomic event framework
>>> directly because of their size, that's the extent of the help it
>>> can give me, because once the mutex is acquired the hardware has
>>> no insight into the fact that it's a mutex.
>>
>>Exactly:: once you convert address-based mutual exclusion to
>>data-based mutual exclusion, HW no longer sees it at all.
>
> That's what I've been saying all along.
>
>>> So if some higher
>>> priority thing subsequently comes along and preempts the thread
>>> holding that lock and tries to take that lock itself, then the
>>> hardware can't help me out anymore, and I'm back to a
>>> priority-inversion deadlock that has to be dealt with by
>>> software.
>>
>>HW can only "take" address based locking, not data-based locking.
>
> Precisely. So it can't help me with priority inversion
> generally.
>
>>> The point is, even if the hardware gives me a cool
>>> mechanism for helping prevent priority inversion problems,
>>> programmers are inevitably going to have to handle those
>>> situations themselves anyway.
>>
>>Never promised you a rose garden.
>
> It's ok; I've already been through Parris Island. And OCS at
> Quantico, AFG deployment, la la la. But that's neither here nor
> there. Point is, I don't need your roses, and I probably don't
> want your threads or anything more advanced than things that let
> me build my own synchronization primitives, threads, priority
> abstractions, and so on.
>
>>> This is the sort of thing I'm trying to communicate, it's really
>>> not me sitting here tossing out barbs like, "oh yeah?! Well,
>>> how about THIS one, smart guy?"
>>
>>Look at it this way:: over the decade from 2000-2010 x86 added
>>a new ATOMIC instruction every iteration.
>>
>>Using ASF or ESM; my architecture would never have to add one.
>>SW has the tools to build what it wants/needs.
>
> Ok, now we're getting somewhere useful. That's handy.
>
> Look, while don't you put the documentation of this stuff
> somewhere publicly accessible? It'd probably save people like
> me a lot of time, frustration, and dad-level insults. Probably
> save you some frustration as well.
>
> - Dan C.
[toc] | [prev] | [next] | [standalone]
| From | mitchalsup@aol.com (MitchAlsup1) |
|---|---|
| Date | 2025-03-28 01:47 +0000 |
| Message-ID | <7b53bc22cfbacd27f9e735cc42ce908c@www.novabbs.org> |
| In reply to | #111351 |
Somehow the post was pushed before being edited. There seems to be some control sequence that finds and pushes the Post-button--this in not the only web site this happens to me on. On Fri, 28 Mar 2025 1:01:59 +0000, Dan Cross wrote: > In article <601781e2c91d42a73526562b419fdf02@www.novabbs.org>, > MitchAlsup1 <mitchalsup@aol.com> wrote: ----------------------- >> >>The property that if an interrupt (or exception or SVC or SVR) >>prevents executing the event to its conclusion, HW makes the >>event look like it never started. So that when/if control returns >>we have another chance to execute the event as-if ATOMICally. > > Ok. So let's be crystal clear here: after the event concludes > successfully, a subsequent exception, interrupt, or whatever, > will not magically roll back any of the values set and made > visible system-wide as a result of the successful conclusion of > the atomic event. Correct? That's what I've been asking. If the esmLOCKstore() retires, the participating memory becomes visible at that instant and cannot be subsequently undone. If the esmLOCKabandon( ) retires, participating memory reverts to its pre-event state in that instant as if the event never happened. > > - Dan C.
[toc] | [prev] | [next] | [standalone]
| From | Stefan Monnier <monnier@iro.umontreal.ca> |
|---|---|
| Date | 2025-03-27 22:02 -0400 |
| Message-ID | <jwv1puhj4r9.fsf-monnier+comp.arch@gnu.org> |
| In reply to | #111353 |
> If the esmLOCKstore() retires, the participating memory becomes visible
> at that instant and cannot be subsequently undone.
>
> If the esmLOCKabandon( ) retires, participating memory reverts to its
> pre-event state in that instant as if the event never happened.
Thinking about Haskell's STM (Software Transactional Memory) library:
do you happen to offer an `esmLOCKretry` which would cause the event to
be aborted and started over? If you do, does it start over right away
or can it *wait* until a snoop request comes to invalidate one of the
participating lines?
[ This last functionality is how STM's `retry` works: it watches all
the memory locations that were consulted during the failed run, knowing
that as long as they're not modified it's pointless to run the
transaction again because it would fail in the same way. ]
Stefan
[toc] | [prev] | [next] | [standalone]
| From | "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> |
|---|---|
| Date | 2025-03-27 19:14 -0700 |
| Message-ID | <vs50lt$1jq5h$4@dont-email.me> |
| In reply to | #111355 |
On 3/27/2025 7:02 PM, Stefan Monnier wrote: >> If the esmLOCKstore() retires, the participating memory becomes visible >> at that instant and cannot be subsequently undone. >> >> If the esmLOCKabandon( ) retires, participating memory reverts to its >> pre-event state in that instant as if the event never happened. > > Thinking about Haskell's STM (Software Transactional Memory) library: Oh shit. I have had some bad experiences with STM. Way back. Damn live lock scenarios, ect... Issues with LL/SC live lock wrt false sharing on the reservation granules, ect... > do you happen to offer an `esmLOCKretry` which would cause the event to > be aborted and started over? If you do, does it start over right away > or can it *wait* until a snoop request comes to invalidate one of the > participating lines? > [ This last functionality is how STM's `retry` works: it watches all > the memory locations that were consulted during the failed run, knowing > that as long as they're not modified it's pointless to run the > transaction again because it would fail in the same way. ] >
[toc] | [prev] | [next] | [standalone]
| From | Stefan Monnier <monnier@iro.umontreal.ca> |
|---|---|
| Date | 2025-03-29 10:31 -0400 |
| Message-ID | <jwva593hq0k.fsf-monnier+comp.arch@gnu.org> |
| In reply to | #111358 |
Chris M. Thomasson [2025-03-27 19:14:21] wrote:
> On 3/27/2025 7:02 PM, Stefan Monnier wrote:
>> Thinking about Haskell's STM (Software Transactional Memory) library:
> Oh shit. I have had some bad experiences with STM. Way back. Damn live lock
> scenarios, ect... Issues with LL/SC live lock wrt false sharing on the
> reservation granules, ect...
No wonder: STM, like Haskell, is an attempt to push the boundaries of
abstraction (or call it programmer convenience), not performance.
Stefan
[toc] | [prev] | [next] | [standalone]
| From | cross@spitfire.i.gajendra.net (Dan Cross) |
|---|---|
| Date | 2025-03-28 02:56 +0000 |
| Message-ID | <vs534q$rd4$2@reader1.panix.com> |
| In reply to | #111353 |
In article <7b53bc22cfbacd27f9e735cc42ce908c@www.novabbs.org>, MitchAlsup1 <mitchalsup@aol.com> wrote: >Somehow the post was pushed before being edited. There seems to be >some control sequence that finds and pushes the Post-button--this >in not the only web site this happens to me on. Clearly it wasn't part of some ESM atomic event. (Jokes aside, it happens. No worries.) >On Fri, 28 Mar 2025 1:01:59 +0000, Dan Cross wrote: > >> In article <601781e2c91d42a73526562b419fdf02@www.novabbs.org>, >> MitchAlsup1 <mitchalsup@aol.com> wrote: >----------------------- >>> >>>The property that if an interrupt (or exception or SVC or SVR) >>>prevents executing the event to its conclusion, HW makes the >>>event look like it never started. So that when/if control returns >>>we have another chance to execute the event as-if ATOMICally. >> >> Ok. So let's be crystal clear here: after the event concludes >> successfully, a subsequent exception, interrupt, or whatever, >> will not magically roll back any of the values set and made >> visible system-wide as a result of the successful conclusion of >> the atomic event. Correct? That's what I've been asking. > >If the esmLOCKstore() retires, the participating memory becomes visible >at that instant and cannot be subsequently undone. Ok. That's what I was asking. >If the esmLOCKabandon( ) retires, participating memory reverts to its >pre-event state in that instant as if the event never happened. - Dan C.
[toc] | [prev] | [next] | [standalone]
| From | Stefan Monnier <monnier@iro.umontreal.ca> |
|---|---|
| Date | 2025-03-27 21:52 -0400 |
| Message-ID | <jwv7c49j540.fsf-monnier+comp.arch@gnu.org> |
| In reply to | #111351 |
> Ok. So let's be crystal clear here: after the event concludes
> successfully, a subsequent exception, interrupt, or whatever,
> will not magically roll back any of the values set and made
> visible system-wide as a result of the successful conclusion of
> the atomic event. Correct? That's what I've been asking.
I don't really understand how that can be a question. It would
completely invalidate the notion of the chunk of code being executed
*atomically*.
If a value has been made visible system-wide, it's like sending an
email: you can't undo that.
Stefan
[toc] | [prev] | [next] | [standalone]
| From | cross@spitfire.i.gajendra.net (Dan Cross) |
|---|---|
| Date | 2025-03-28 02:58 +0000 |
| Message-ID | <vs538l$rd4$3@reader1.panix.com> |
| In reply to | #111354 |
In article <jwv7c49j540.fsf-monnier+comp.arch@gnu.org>, Stefan Monnier <monnier@iro.umontreal.ca> wrote: >> Ok. So let's be crystal clear here: after the event concludes >> successfully, a subsequent exception, interrupt, or whatever, >> will not magically roll back any of the values set and made >> visible system-wide as a result of the successful conclusion of >> the atomic event. Correct? That's what I've been asking. > >I don't really understand how that can be a question. It would >completely invalidate the notion of the chunk of code being executed >*atomically*. > >If a value has been made visible system-wide, it's like sending an >email: you can't undo that. Yup, it sure would. But when Mitch was talking about locks disappearing, it wasn't clear (to me anyway) that he was talking about something in the hardware, and not the mutexes I was trying to build in software; hence why I was asking. - Dan C.
[toc] | [prev] | [next] | [standalone]
| From | "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> |
|---|---|
| Date | 2025-03-27 19:09 -0700 |
| Message-ID | <vs50cb$1jq5h$2@dont-email.me> |
| In reply to | #111351 |
On 3/27/2025 6:01 PM, Dan Cross wrote:
> In article <601781e2c91d42a73526562b419fdf02@www.novabbs.org>,
> MitchAlsup1 <mitchalsup@aol.com> wrote:
>> On Thu, 27 Mar 2025 17:19:21 +0000, Dan Cross wrote:
>>> In article <7a093bbb356e3bda3782c15ca27e98a7@www.novabbs.org>,
>>> MitchAlsup1 <mitchalsup@aol.com> wrote:
>>>> On Wed, 26 Mar 2025 4:08:57 +0000, Dan Cross wrote:
>>>>> [snip]
>>>>> 1. `from` may point to a cache line
>>>>> 2. `to` may point to a different cache line.
>>>>> 3. Does your architecture use a stack?
>>>>
>>>> No
>>>
>>> You make multiple references to a stack just below. I
>>> understand stack references are excluded from the atomic event,
>>> but I asked whether the architecture uses a stack. Does it, or
>>> doesn't it?
>>>
>>>>> What sort of alignment criteria do you impose?
>>>>
>>>> On the participating data none. On the cache line--cache line.
>>>
>>> I meant for data on the stack. The point is moot, though, since
>>> as you said stack data does not participate in the atomic event.
>>
>> Stack does not NECESSARILY participate. However, if you perform
>> a esmLOCKload( SP[23] ) then that line on the stack is participating.
>
> Very good.
>
>>>>> At first blush, it seems to me that
>>>>> the pointers `from_next`, `from_prev`, and `to_next` could be
>>>>> on the stack and if so, will be on at least one cache line,
>>>>> and possibly 2, if the call frame for `MoveElement` spans
>>>>> more than one.
>>>>
>>>> The illustrated code is using 6 participating cache lines.
>>>> Where local variables are kept (stack, registers) does not
>>>> count against the tally of participating cache lines.
>>>
>>> Huh.
>>
>> There are 6 esmLOCKxxxxx() so there are 6 participating lines--
>> and these are absolutely independent from where the variables
>> are located. The variables carry information but do not part-
>> ticipate they carry data between operations on the particupating
>> lines.
>
> That was clearly in the code. That was not what the "Huh" was
> in resposne to.
>
>>> How would this handle something like an MCS lock, where
>>> the lock node may be on a stack, though accessible globally in
>>> some virtual address space?
>>
>> As illustrated above esmLOCKload( SP[23]) will cause a line on
>> the stack to participate in the event.
>
> Ok, good to know.
>
>> ----------------
>>>>> But later you wrote,
>>>>>
>>>>>> So, if you want the property whereby the lock disappears on any
>>>>>> control transfer out of the event {exception, interrupt, SVC, SVR, ...};
>>>>>> then you want to use my ATOMIC stuff; otherwise, you can use the
>>>>>> normal ATOMIC primitives everyone and his brother provide.
>>>>>
>>>>> Well, what precisely do you mean here when you say, "if you want
>>>>> the property whereby the lock disappears on any control transfer
>>>>> out of the event"?
>>>>
>>>> If you want that property--you use the tools at hand.
>>>> If you don't just use them as primitive generators.
>>>
>>> I wasn't clear enough here. I'm asking what, exactly, you mean
>>> by this _property_,
>>
>> The property that if an interrupt (or exception or SVC or SVR)
>> prevents executing the event to its conclusion, HW makes the
>> event look like it never started. So that when/if control returns
>> we have another chance to execute the event as-if ATOMICally.
>
> Ok. So let's be crystal clear here: after the event concludes
> successfully, a subsequent exception, interrupt, or whatever,
> will not magically roll back any of the values set and made
> visible system-wide as a result of the successful conclusion of
> the atomic event. Correct? That's what I've been asking.
>
>>> not what you mean when you write that one
>>> can use these atomic events if one wants the property. That is,
>>> I'm asking you to precisely define what it means for a lock to
>>> "disappear".
>>>
>>> It seems clear enough _now_ that once the event concludes
>>> successfully the lock value is set to whatever it was set to in
>>> during the event, but that wasn't clear to me earlier and I
>>> wanted confirmation.
>>
>> Not "the lock value" but "all participating values" are set
>> and become visible system-wide in a single instant.
>
> Sure. But there was only one participating value in the
> spinlock example. ;-}
>
>> No 3rd
>> party sees some set and others remain unset. Everybody sees
>> all of then change value between this-clock and its successor.
>
> Sure. You've said this all along, and I've never disputed it.
>
>>> In particular, it seems odd to me that one would bother with a
>>> lock of some kind during an event if it didn't remain set after
>>> the event.
>>
>> Consider CompareTripleSwapFour() as an atomic primitive. How
>> would you program this such that nobody nowhere could ever
>> tell that the four updated locations changes in any order
>> then simultaneously ??? Without using a LOCK to guard the
>> event ??? And without demanding that all four updates are
>> to the same cache line ???
>>
>> {{Nobody nowhere included ATOMIC-blind DMA requests}}
>
> The fundamental issue here is that, when I refer to a "lock" I
> have been talking about a software mutual exclusion primitive,
> and you are referring to something else entirely.
>
> I would have thought my meaning was clear from context;
> evidently not.
>
>>> If you do all of your critical section stuff inside
>>> of the event, and setting the lock in the event is not visible
>>> until the event concludes, why bother? If on the other hand you
>>> use the event to set the lock, why bother doing additional work
>>> inside the event itself, but in this case I definitely don't
>>> want something else to come along and just unset the lock on its
>>> own, higher priority or not.
>>
>> It is clear you do not understand the trouble HW takes to implement
>> even DCADS where between any 2 clocks, one (or more) SNOOPs can
>> take the lines you are DCADSing. It is this property of cache
>> coherence that gets in the way of multi-operation ATOMIC events.
>
> As above, when I refer to "setting the lock" I am referring to
> acquiring a mutex; a software primitive. I am not referring to
> what the hardware does internally. Consider what I wrote above,
> the section that you quoted, in that context.
>
>> In 2004 Fred Weber came to me and ask: "Why can't we (AMD) give
>> Microsoft the DCADS they want. I dug into it, and ASF was my
>> solution in x86 ISA, ESM is my solution in RISC-based ISA. It
>> removes the need to add ATOMIC primitives to ISA over generations.
>>
>> It also provides the ability to do other things--which apparently
>> you will not use--sort of like complaining that your new car has
>> auto-park a feature you will never use.
>
> "It is clear you do not understand the trouble SW takes to
> implement even simple things that are beyond the capability of
> hardware."
>
> (Back at you. I think I've been pretty polite here; I'm getting
> tired of the barbs.)
>
>>>>> What I _hope_ you mean is that if you transfer "out of the
>>>>> event" before the `esmLOCKstore` then anything that's been done
>>>>> since the "event" started is rolled back, but NOT if control
>>>>> transfer happens _after_ the `esmLOCKstore`. If THAT were the
>>>>> case, then this entire mechanism is useless.
>>>>
>>>> One can use a esmLOCKstore( *SP, 0 ) to abandon an event. HW
>>>> detects that *SP is not participating in the event, and uses
>>>> the lock bit in the instruction to know it is time to leave
>>>> the event in the failed manner. No ST is performed to the
>>>> non-participating cache line. {{I guess I could wrap the
>>>> abandonment into a macro ...}}
>>>
>>> Again, not what I was asking. I'm trying to understand what you
>>> meant when you talked about disappearing locks.
>>
>> If the event cannot be completed due to interrupt or exception or
>> SCVC or SVR, the HW backs up the event such that is looks like
>> it never started. SO when control return, the ATOMIC event runs
>> in its entirety ATOMICally !! So, the illusion of ATOMICITY is
>> preserved.
>
> Sigh. You've said this many times over now. I get it. I'm not
> disputing it, nor do I believe I ever have.
>
> Again, when I said "lock" earlier, I am referring to a mutux
> from the perspective of software. That is, the value of the
> structure member that I called "lock" in the version of the
> `Spinlock` struct that I posted however-many responses ago, and
> that I keep referring to in example code. I am not asking about
> what the hardware does internally. I'm not talking about locked
> bus cycles or whatever. I'm not referring to cache line address
> monitors or any of these other things that you keep referring
> to. I'm talking the the value of the structure member called
> `lock` in the `Spinlock` C structure I posted earlier.
>
> What I have been asking, over and over again, is whether that
> _value_ "disappears" _after_ an atomic event successfully
> concludes. That is, the value that was written into the memory,
> not whatever internal marker you use in hardware to monitor and
> guard those stores until the atomic event completes or is
> otherwise exited, aborted, cancelled, interrupted, or however
> you prefer to refer to it not completing successfully. That is
> a totally separate thing and I not presume to speak of how its
> done, though in fairness to you you've made it reasonably clear.
>
> Does that clarify what I have been asking about? I honestly
> thought this would be obvious from context.
>
>>> Again, I think you meant that, if an atomic event that sets a
>>> lock variable fails,
>>
>> The event has NO lock variable--it has up to 8 cache line address
>> monitors.
>
> The structure member is literally called `lock` and the context
> is software spinlocks. Note, spinlocks because they're simple
> and easy to discuss, not because they're great mutex primtives.
>
> As frustrating as this evidently is to you, it strikes me that
> it actually reinforces my point: hardware has no insight into
> what the software is _doing_ at a higher semantic level.
>
>>> whether it got far enough along to set the
>>> variable inside the event or not is not observable if the event
>>> fails.
>>
>> Non-participating cache lines can be used to leak information
>> out of the event (like for debug purposes). Participating data
>> does not leak.
>>
>>> But if the event succeeds, then the variable is set and
>>> that's that. IF that's the case, then great! But if not, then
>>> disappearing locks are bad.
>>
>> It is NOT a FRIGGEN LOCK--
>
> I am talking about the spinlock. That is a software object, and
> in that context, it is very much called a "lock".
>
> I understand when you say that it is not a lock, it is not a
> lock in the sense relevant to your world in hardware, but that's
> not what _I'm_ talking about.
>
> I'm trying to square what you meant when you talked about
> "disappearing locks" with what happens to those software objects
> _after_ an atomic event modifying them successfully completes.
>
> Evidently you were talking about something else entirely, and
> that's fine. I gather it has nothing to do with _my_ notion of
> locks after they've been successfully acquired; I was just
> trying to confirm that.
>
>> it is a set of address monitors which
>> are used to determine the success of the event -- the monitors
>> remain invisible to SW just like miss buffers remain invisible
>> to SW. The monitors are not preservable across privilege changes.
>
> See above. Tell me what you want me to call it and I'll use
> that terminology instead.
>
>>> This is especially murky to me
>>> because there seem to be bits of OS primitives, like threads and
>>> priority, that are known directly to the hardware.
>>
>> Just its instantaneous privilege level--which is congruent to
>> its register file, Root Pointer, ASID, and operating modes.
>
> What is "it" here?
>
>> ------------------
>>>
>>> Ok, so perhaps this is a reasonable implementation of spinlock
>>> acquire, as I wrote to Stefan yesterday:
>>>
>>> void
>>> spinlock_acquire(volatile Spinlock *lk)
>>> {
>>> while (esmLOCKload(&lk->lock) != 0)
>>> cpu_spin_hint();
>>> esmLOCKstore(&lk->lock, 1);
>>> }
>>>
>>> Yes? If this is preempted, or another CPU wins the race on the
>>> lock, the hardware will back this up and start over in the while
>>> loop, right?
>>
>> Yes.
>> ------------
>>>> Remember only the lines participating in the event count. For
>>>> example, say one had sprintf() inside the event. None of the
>>>> lines touched by springt() are participating in the event,
>>>> and are not counted against the 8 cache lines available.
>>>
>>> This doesn't change the point, though. Anything that touches
>>> more than 8 cache lines is not suitable for an atomic event.
>>
>> I see what you are saying, but the event is allowed to have an
>> unbounded amount of touches to non-participating lies.
>
> I don't see how that's helpful if I need to touch data in more
> than 8 cache lines atomically.
>
>>> Insertion into a doubly linked list is already up to 6 lines,
>>
>> I moved a doubly linked element from a CDS and inserted it
>> somewhere else doubly-linked in same CDS in 6 cache lines.
>> You need to refine how you count.
>
> <Tongue-in-cheek mode on>
> Ah, yes. The program that would dump core if you tried to move
> from the head of a list or the rear and that couldn't insert at
> the head pointer was, in fact, called `MoveElement`. My
> mistake, even if it was clear that that's what I was talking
> about, given that quoted where I referred to it in the context
> of work stealing later on. ;-P
>
>> BOOLEAN InsertElement( Element *el, Element *to )
>> {
>> tn = esmLOCKload( to->next );
>> esmLOCKprefetch( el );
>> esmLOCKprefetch( tn );
>> if( !esmINTERFERENCE() )
>> {
>> el->next = tn;
>> el->prev = to;
>> to->next = el;
>> esmLOCKstore( tn->prev, el );
>> return TRUE;
>> }
>> return FALSE;
>> }
>
> If you're going to be pendatic when I slip up and call "move"
> "insert", then let's be pendantic.
>
> This code won't compile, because of the obvious error of not
> defining `tn`. Even if you fix that, it's not correct if you
> try to insert at the end of the list. It's not clear now you
> would insert at the head of a list, or into an empty list, but
> that depends on how you represent the list itself. ie, the
> head may just be a pointer to the first element, or it might be
> an otherwise unused node; something like this:
>
> typedef struct List List;
> struct List {
> Element head;
> };
>
> where the first element in the list is always `head.next`.
>
> It would help if you used standard types, like `bool`, which has
> been available standard for more than 25 years now since
> introduced in C99 as `_Bool` with accompanying typedefs and
> macros in `stdbool.h` for `bool`, `true` and `false`.
>
> And doesn't this presume that `el->next`, `el->prev` and `el`
> are all on the same cache line? Similarly with `tn` and
> `tn->prev`?
>
> You need to refine how you write programs. ;-}
>
> Here's a better version, that assumes the list representation
> above.
>
> #include <stddef.h>
> #include <stdbool.h>
>
> typedef struct Element Element;
> struct Element {
> Element *next;
> Element *prev;
> };
>
> typedef struct List List;
> struct List {
> Element head;
> };
>
> Element *
> ListBegin(List *list)
> {
> return list->head.next;
> }
>
> bool
> InsertElement(Element *el, Element *to)
> {
> Element *tn = esmLOCKload(to->next);
> esmLOCKprefetch(el->next);
> esmLOCKprefetch(el->prev);
> if (tn != NULL)
> esmLOCKprefetch(tn->prev);
>
> if (!esmINTERFERENCE()) {
> el->next = tn;
> el->prev = to;
> to->next = el;
> esmLOCKstore(tn->prev, el);
> return true;
> }
> return false;
> }
>
> Inserting at the head, or if the list is empty, are both easy
> now: `InsertElement(&list.head, el)`. But note, again, that the
> first element of the list is really `list.head.next`, hence the
> `ListBegin` function.
>
> Here I count up to three, but possibly up to 4, cache lines.
>
> Note that if you did Not represent the list this way, in order
> to support generalized insertions anywhere in the list, you'd
> either need a circular representation (which is annoying to use
> in practice), or you'd most likely have to support separate
> head and tail pointers and two operations, one to insert before,
> and another after, the 'to' pointer. But then you'd have to be
> prepared to update either the head or tail pointers (or both if
> the list is empty); either way it's potentially two more cache
> lines to account for, though if the list is empty you don't have
> to touch `tn->prev`.
>
>>> recall that my original hypothetical also involved popping off a
>>> queue, which adds another (the queue head). If I threw up
>>> incrementing a couple of counters I can't do it. And this seems
>>> like a perfectly reasonable real-world scenario; consider a work
>>> stealing thread scheduler that takes a task from one core to
>>> another and keeps some stats on how things move.
>>>
>>> Or maybe even just swapping places between two elements in a
>>> linked list. For example:
>>
>> The example has 3 structures each requiring 3 participating lines.
>> {themself, the struct their next pointer points at and the struct
>> their prev pointer points at}
>>
>> 3×3 = 9 and there is no getting around it.
>
> Yup. So what does the hardware do if I try it?
>
>> -------------------------
>>>
>>> But there isn't a lot of publicly available documentation for
>>> any of this, at least none that I could find, and from the way
>>> it's been described thus far it seems like there are a number of
>>> edge cases that it is not clear how one is supposed to address.
>>> So I want to understand the boundaries of where these hardware
>>> facilities are applicable, given architectural limitations. I
>>> think this thread has shown that those boundaries exist. The
>>> universe of interesting and useful software is big, and someone,
>>> somewhere, is going to trip over those limitations sooner or
>>> later. Once we accept that, we must ask, "Then what?" It seems
>>> obvious that programmers are going to have to find techniques to
>>> work around those limitations, and that the mechanisms chosen
>>> are unknowable to hardware.
>>
>> Fair enough.
>>
>>> But now we're back in the same boat of software having to deal
>>> with all the thorny problems that these mechanisms were designed
>>> to address in the first place, like priority inversion: take the
>>> spinlock from earlier, the hardware has no semantic knowledge
>>> that the lock is used as a mutex, and can't know; it's just some
>>> bit in memory somewhere that was set during an atomic event.
>>
>> It is not a FRIGGEN lock, it is a set of address monitors which
>> guard the instructions in the event.
>
> Now I have to wonder if you're deliberatly ignoring the context.
>
> I called it a "lock" in the same sentence I said it was a
> "spinlock". Are you seriously going to tell me that you could
> not figure out that the "lock" I was referring to was the
> spinlock? You know that that's a software thing; you yourself
> said that.
>
> Note the words, "it's just some bit in memory that _was set
> during an atomic event._" Is it not clear that the event is
> over? It's done. It was successful. All of the address
> monitors guarding instructions in the event are no longer
> relevant; they've been retired or whatever.
>
> Now, some (software) thread holds a (software) (spin)lock.
> I've tried over and over to make this clear, yet you insist on
> going back to talking about what happens _during_ an atomic
> event. I'm talking about _after_ the event completed
> _succesfully_.
>
>> A Lock is a SW concept
>> which requires a value change to announce to others that "you
>> got it".
>
> That's exactly what `spinlock_acquire` from above _does_, isn't
> it?
>
> You cannot seriously expect me to accept that you did not
> understand this.
>
>>> Maybe ESM helps in a race when trying to acquire the spinlock,
>>> but once it's acquired, it's acquired, and because the spinlock
>>> is not a primitive understood by the hardware, the hardware has
>>> no way of incorporating the spinlock's semantics into its view
>>> of priority, threading, etc.
>>
>> Consider a 1024 core system and the time-quantum goes off and
>> every core wants to rotate the thread it is running; putting
>> it at the back of the execution queue and taking off the one on
>> the front to run. Nick McLaren timed a big SUN server (100 core)
>> on this and it could take up to 6 seconds !! for something that
>> should take a few milliseconds. This is because of the BigO( n^3 )
>> nature of bus traffic wrt SNOOPing on that system.
>
> This was solved with per-core run queues.
>
> You're senior enough that I would think it clear that spinlocks
> are just a convenient illustrative example here because they're
> so simple, but their pitfalls as core counts increase are well
> studied and understood. In the real world, mutex structures
> like MCS locks or CLH locks scale approximately linearly with
> core-count under contention. I posted a reference to this
> earlier, but here it is again. You may have a hard time
> accepting it because they call the things that they refer to,
> "locks". ;-}
> https://pdos.csail.mit.edu/papers/linux:lock.pdf
>
>> One can use ESM to convert that particular case to BigO( 3 )
>> or in the general case of random insert and removal times:
>> BigO( ln( n ) ).
>
> Modern scheduling algorithms already run in amortized constant
> time on conventional hardware.
>
>>> So while an atomic event might help me in the implementation of
>>> a generic mutex of some kind, so that I can protect critical
>>> sections that can't be handled with the atomic event framework
>>> directly because of their size, that's the extent of the help it
>>> can give me, because once the mutex is acquired the hardware has
>>> no insight into the fact that it's a mutex.
>>
>> Exactly:: once you convert address-based mutual exclusion to
>> data-based mutual exclusion, HW no longer sees it at all.
>
> That's what I've been saying all along.
>
>>> So if some higher
>>> priority thing subsequently comes along and preempts the thread
>>> holding that lock and tries to take that lock itself, then the
>>> hardware can't help me out anymore, and I'm back to a
>>> priority-inversion deadlock that has to be dealt with by
>>> software.
>>
>> HW can only "take" address based locking, not data-based locking.
>
> Precisely. So it can't help me with priority inversion
> generally.
>
>>> The point is, even if the hardware gives me a cool
>>> mechanism for helping prevent priority inversion problems,
>>> programmers are inevitably going to have to handle those
>>> situations themselves anyway.
>>
>> Never promised you a rose garden.
>
> It's ok; I've already been through Parris Island. And OCS at
> Quantico, AFG deployment, la la la. But that's neither here nor
> there. Point is, I don't need your roses, and I probably don't
> want your threads or anything more advanced than things that let
> me build my own synchronization primitives, threads, priority
> abstractions, and so on.
>
>>> This is the sort of thing I'm trying to communicate, it's really
>>> not me sitting here tossing out barbs like, "oh yeah?! Well,
>>> how about THIS one, smart guy?"
>>
>> Look at it this way:: over the decade from 2000-2010 x86 added
>> a new ATOMIC instruction every iteration.
>>
>> Using ASF or ESM; my architecture would never have to add one.
>> SW has the tools to build what it wants/needs.
>
> Ok, now we're getting somewhere useful. That's handy.
>
> Look, while don't you put the documentation of this stuff
> somewhere publicly accessible? It'd probably save people like
> me a lot of time, frustration, and dad-level insults. Probably
> save you some frustration as well.
Just wondering, do you happen to be familiar with and/or studied KCSS
before? K-Compare Single Swap? For some damn reason you made me think of
it. From many years ago:
https://groups.google.com/g/comp.arch/c/shshLdF1uqs/m/VLmZSCBGDTkJ
Damn, back in 2005. Ahhh shit. I am getting older! ;^o
[toc] | [prev] | [next] | [standalone]
| From | "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> |
|---|---|
| Date | 2025-03-27 19:10 -0700 |
| Message-ID | <vs50eq$1jq5h$3@dont-email.me> |
| In reply to | #111356 |
On 3/27/2025 7:09 PM, Chris M. Thomasson wrote: > On 3/27/2025 6:01 PM, Dan Cross wrote: >> In article <601781e2c91d42a73526562b419fdf02@www.novabbs.org>, >> MitchAlsup1 <mitchalsup@aol.com> wrote: >>> On Thu, 27 Mar 2025 17:19:21 +0000, Dan Cross wrote: >>>> In article <7a093bbb356e3bda3782c15ca27e98a7@www.novabbs.org>, >>>> MitchAlsup1 <mitchalsup@aol.com> wrote: >>>>> On Wed, 26 Mar 2025 4:08:57 +0000, Dan Cross wrote: [...] >> Look, while don't you put the documentation of this stuff >> somewhere publicly accessible? It'd probably save people like >> me a lot of time, frustration, and dad-level insults. Probably >> save you some frustration as well. > > Just wondering, do you happen to be familiar with and/or studied KCSS > before? K-Compare Single Swap? For some damn reason you made me think of > it. From many years ago: > > https://groups.google.com/g/comp.arch/c/shshLdF1uqs/m/VLmZSCBGDTkJ > > Damn, back in 2005. Ahhh shit. I am getting older! ;^o > > Ahhh shit again, I forgot to snip the lengthy aspects. Sorry about that.
[toc] | [prev] | [next] | [standalone]
| From | mitchalsup@aol.com (MitchAlsup1) |
|---|---|
| Date | 2025-03-27 21:47 +0000 |
| Message-ID | <34434320650f5844b18b1c0b684acf43@www.novabbs.org> |
| In reply to | #111345 |
On Thu, 27 Mar 2025 17:19:21 +0000, Dan Cross wrote:
> In article <7a093bbb356e3bda3782c15ca27e98a7@www.novabbs.org>,
> MitchAlsup1 <mitchalsup@aol.com> wrote:
-------------------
> Or maybe even just swapping places between two elements in a
> linked list. For example:
>
> void
> swap_places(Node **head, Node *a, Node *b)
> {
> Node *hp, *an, *ap, *bn, *bp;
>
> assert(head != NULL);
> assert(a != NULL);
> assert(b != NULL);
>
> if (a == b)
> return;
>
> esmLOCKprefetch(*head);
This should be a load not prefetch--you want the value of *head
> if (*head == a) // see !
> *head = b;
> else if (*head == b)
> *head = a;
There is a ESM rule that states:: all participating cache lines
must be touched before any participating cache lines can be
modified.
Also note: Participating cache lines are checked for write permission
at touch time, and on cache miss, read with intent to modify.
>
> an = esmLOCKload(a->next);
> ap = esmLOCKload(a->prev);
> bn = esmLOCKload(b->next);
> bp = esmLOCKload(b->prev);
>
> b->next = an;
> if (an != NULL) {
> esmLOCKprefetch(an->prev);
> an->prev = b;
> }
> b->prev = ap;
> if (ap != NULL) {
> esmLOCKprefetch(ap->next);
> ap->next = b;
> }
>
> a->next = bn;
> if (bn != NULL) {
> esmLOCKprefetch(bn->prev);
> bn->prev = a;
> }
> if (bp != NULL) {
> esmLOCKprefetch(bp->next);
> bp->next = a;
> }
> esmLOCKstore(a->prev, bp);
> }
>
>
> - Dan C.
What I think you want:: (ignoring the 9 participants limit)
> void
> swap_places(Node **head, Node *a, Node *b)
> {
> Node *hp, *an, *ap, *bn, *bp;
>
> assert(head != NULL);
> assert(a != NULL);
> assert(b != NULL);
>
> if (a == b)
> return;
top_of_ATOMIC_event:
// this is the recovery point is you don't use esmINTERFERENCE()
// the very next instruction begins the event.
> esmLOCKprefetch( an = esmLOCKload(a->next) );
> esmLOCKprefetch( ap = esmLOCKload(a->prev) );
> esmLOCKprefetch( bn = esmLOCKload(b->next) );
> esmLOCKprefetch( bp = esmLOCKload(b->prev) );
Node *Ehead = esmLOCKload(*head);
// by placing all the the touching before any manifestation, you put
// all the touch latency* in the code before it has tried to damage any
// participating memory location. (*) and TLB latency and 2nd party
// observation of your event.
// this would be the point where you would insert if( esmINTERFERENCE(
))
// if you wanted control at a known failure point rather than at the
// top of the event on failure.
> if (Ehead == a)
> *head = b;
> else if (Ehead == b)
> *head = a;
>
> b->next = an;
> if (an != NULL) {
> an->prev = b;
> }
> b->prev = ap;
> if (ap != NULL) {
> ap->next = b;
> }
>
> a->next = bn;
> if (bn != NULL) {
> bn->prev = a;
> }
> if (bp != NULL) {
> bp->next = a;
> }
> esmLOCKstore(a->prev, bp);
> }
// now manifestation has lowest possible latency (as seen by this core
alone)
[toc] | [prev] | [next] | [standalone]
| From | cross@spitfire.i.gajendra.net (Dan Cross) |
|---|---|
| Date | 2025-03-28 02:53 +0000 |
| Message-ID | <vs5305$rd4$1@reader1.panix.com> |
| In reply to | #111350 |
In article <34434320650f5844b18b1c0b684acf43@www.novabbs.org>,
MitchAlsup1 <mitchalsup@aol.com> wrote:
>On Thu, 27 Mar 2025 17:19:21 +0000, Dan Cross wrote:
>> In article <7a093bbb356e3bda3782c15ca27e98a7@www.novabbs.org>,
>> MitchAlsup1 <mitchalsup@aol.com> wrote:
>-------------------
>> Or maybe even just swapping places between two elements in a
>> linked list. For example:
>>
>> void
>> swap_places(Node **head, Node *a, Node *b)
>> {
>> Node *hp, *an, *ap, *bn, *bp;
>>
>> assert(head != NULL);
>> assert(a != NULL);
>> assert(b != NULL);
>>
>> if (a == b)
>> return;
>>
>> esmLOCKprefetch(*head);
>
>This should be a load not prefetch--you want the value of *head
I wondered about that. I ASSumed that `esmLOCKprefetch` was
enough to set up a cache line address monitor on `*head`, but if
you tell me I need `esmLOCKload`, then I need `esmLOCKload`:
hp = esmLOCKload(*head);
if (hp == a)
*head = b; // Note I need to set whatever head points to, not hp.
else if (hp == b)
*head = a;
(You can see a vestige of me trying that earlier by the `hp`
temporary that I left in when I posted that initially, but I
didn't like how it looked, so I changed it to what was posted.)
>> if (*head == a) // see !
>> *head = b;
>> else if (*head == b)
>> *head = a;
>
>There is a ESM rule that states:: all participating cache lines
>must be touched before any participating cache lines can be
>modified.
Are these rules in a reference somewhere I can see? I kinda
reckoned `esmLOCKprefetch` would be enough to "touch" the line,
but I fully admit I was just guessing. Looks like `prefetch` is
mostly for the case where you'll store, but don't need to read?
>Also note: Participating cache lines are checked for write permission
>at touch time, and on cache miss, read with intent to modify.
Ok.
>> an = esmLOCKload(a->next);
>> ap = esmLOCKload(a->prev);
>> bn = esmLOCKload(b->next);
>> bp = esmLOCKload(b->prev);
>>
>> b->next = an;
>> if (an != NULL) {
>> esmLOCKprefetch(an->prev);
>> an->prev = b;
>> }
>> b->prev = ap;
>> if (ap != NULL) {
>> esmLOCKprefetch(ap->next);
>> ap->next = b;
>> }
>>
>> a->next = bn;
>> if (bn != NULL) {
>> esmLOCKprefetch(bn->prev);
>> bn->prev = a;
>> }
>> if (bp != NULL) {
>> esmLOCKprefetch(bp->next);
>> bp->next = a;
>> }
>> esmLOCKstore(a->prev, bp);
>> }
>
>What I think you want:: (ignoring the 9 participants limit)
You mean 8 participants limit? That counting thing again. ;-}
What I really wanted was an example that exceeded that limit as
an expository vehicle for understanding what happens when the
limit is exceeded. What does the hardware do in that case?
>> void
>> swap_places(Node **head, Node *a, Node *b)
>> {
>> Node *hp, *an, *ap, *bn, *bp;
>>
>> assert(head != NULL);
>> assert(a != NULL);
>> assert(b != NULL);
>>
>> if (a == b)
>> return;
>
>top_of_ATOMIC_event:
>
>// this is the recovery point is you don't use esmINTERFERENCE()
>// the very next instruction begins the event.
Yup. Tracking.
>> esmLOCKprefetch( an = esmLOCKload(a->next) );
>> esmLOCKprefetch( ap = esmLOCKload(a->prev) );
>> esmLOCKprefetch( bn = esmLOCKload(b->next) );
>> esmLOCKprefetch( bp = esmLOCKload(b->prev) );
What exactly does this do? This "touches" whatever lines the
things that `an`, `ap`, `bn`, and `bp` are all in, as well as
the lines that the `a` and `b` prev and next pointers point
into as well, right?
> Node *Ehead = esmLOCKload(*head);
You shoulda just used the `hp` local I left for you for exactly
this. :-D
>// by placing all the the touching before any manifestation, you put
>// all the touch latency* in the code before it has tried to damage any
>// participating memory location. (*) and TLB latency and 2nd party
>// observation of your event.
>
>// this would be the point where you would insert if( esmINTERFERENCE(
>))
>// if you wanted control at a known failure point rather than at the
>// top of the event on failure.
>
>> if (Ehead == a)
>> *head = b;
>> else if (Ehead == b)
>> *head = a;
>>
>> b->next = an;
>> if (an != NULL) {
>> an->prev = b;
>> }
>> b->prev = ap;
>> if (ap != NULL) {
>> ap->next = b;
>> }
>>
>> a->next = bn;
>> if (bn != NULL) {
>> bn->prev = a;
>> }
>> if (bp != NULL) {
>> bp->next = a;
>> }
>> esmLOCKstore(a->prev, bp);
>> }
>
>// now manifestation has lowest possible latency (as seen by this core
>alone)
Doesn't this code now assume that `an->prev` is in the same
cache line as `an`, and similarly that `bn` is in the same line
as `bn->prev`? Absent an alignment constraint on the `Node`
type, that's not guaranteed given the definition posted earlier.
That may be an odd and ill-advised thing for a programmer to do
if they want their list type to work with atomic events, but it
is possible.
The node pointers and their respective `next` pointers are okay,
so I wonder if perhaps this might have been written as:
void
swap_places(Node **head, Node *a, Node *b)
{
Node *hp, *an, *ap, *bn, *bp;
assert(head != NULL);
assert(a != NULL);
assert(b != NULL);
if (a == b)
return;
hp = esmLOCKload(*head);
esmLOCKprefetch(an = esmLOCKload(a->next));
ap = esmLOCKload(a->prev);
esmLOCKprefetch(bn = esmLOCKload(b->next));
bp = esmLOCKload(b->prev);
if (an != NULL)
esmLOCKprefetch(an->prev);
if (bn != NULL) {
esmLOCKprefetch(bn->prev);
bn->prev = a;
}
if (hp == a)
*head = b;
else if (hp == b)
*head = a;
b->next = an;
if (an != NULL)
an->prev = b;
b->prev = ap;
if (ap != NULL)
ap->next = b;
a->next = bn;
if (bp != NULL)
bp->next = a;
esmLOCKstore(a->prev, bp);
}
But now the conditional testing whether or not `an` is `NULL` is
repeated. Is the additional branch overhead worth it here?
- Dan C.
[toc] | [prev] | [next] | [standalone]
Page 7 of 11 — ← Prev page 1 … 5 6 [7] 8 9 … 11 Next page →
Back to top | Article view | comp.arch
csiph-web