Re: [patch 00/29] lockup_detector: Cure hotplug deadlocks and replace duct tape

From	Nicholas Piggin <npiggin@gmail.com>
Newsgroups	linux.kernel
Subject	Re: [patch 00/29] lockup_detector: Cure hotplug deadlocks and replace duct tape
Date	2017-09-01 06:50 +0200
Message-ID	<ukM41-7iP-9@gated-at.bofh.it> (permalink)
References	<uksf0-2Jn-3@gated-at.bofh.it> <ukFYB-3cT-7@gated-at.bofh.it>
Organization	IBM

Show all headers | View raw

On Thu, 31 Aug 2017 18:10:14 -0400
Don Zickus <dzickus@redhat.com> wrote:

> On Thu, Aug 31, 2017 at 09:15:58AM +0200, Thomas Gleixner wrote:
> > The lockup detector is broken is several ways:
> > 
> >     - It's deadlock prone vs. CPU hotplug in various ways. Some of these
> >       are due to recursive cpus_read_lock() others are due to
> >       cpus_read_lock() from CPU hotplug callbacks which immediately lock
> >       the machine because cpus are write locked.
> > 
> >     - The handling of the cpu hotplug threads happens sideways to the
> >       smpboot thread infrastructure, which is racy and pointless
> > 
> >     - The handling of the user space sysctl interface is a complete
> >       trainwreck as it fiddles directly with variables which can be
> >       modified or evaluated by the running watchdogs.
> > 
> >     - The perf event initialization is a steaming pile of duct tape as it
> >       idiotically tries to create perf events over and over even if perf is
> >       not functional (no hardware, ....). To avoid excessive dmesg spam it
> >       contains magic printk ratelimiting along with either wrong or useless
> >       messages.
> > 
> >     - The code structure is horrible as ifdef sections are scattered all
> >       over the place which makes it unreadable
> > 
> >     - There is more wreckage, but see the changelogs for the ugly details.
> > 
> > Before I get utterly grumpy, I just pretend that I don't give a sh*t!
> > 
> > The following series sanitizes the facility and addresses the problems.  
> 
> Hi Thomas,
> 
> Thanks for the patchset.  I agree with most your issues you complained
> about, just wasn't smart enough to figure out the right way to solve them.
> Despite your aggressive comments, I will review the code to see if it covers
> the scenarios that have popped up over the years and run some testing on my
> side.  Probably need a few days to do that.

The powerpc bits look fine, there's no real changes pending there,
so just take them through your tree if you like.

I had a glance throught the series, no comments yet. The powerpc watchdog
already duplicates the proc tunables rather than using them directly, so
in theory it did not need the 2 stage reconfigure. In practice, it has a
brown paper bag bug because it does not stop the watchdog before changing
its internal variables :P 2 stage is probably safer and clearer way to go
though.

Thanks,
Nick

Thread

[patch 00/29] lockup_detector: Cure hotplug deadlocks and replace  duct tape Thomas Gleixner <tglx@linutronix.de> - 2017-08-31 09:40 +0200
  [patch 02/29] perf/x86/intel: Sanitize PMU HT bug workaround Thomas Gleixner <tglx@linutronix.de> - 2017-08-31 09:40 +0200
  [patch 25/29] lockup_detector: Implement init time detection of perf Thomas Gleixner <tglx@linutronix.de> - 2017-08-31 09:40 +0200
  [patch 19/29] lockup_detector: Cleanup header mess Thomas Gleixner <tglx@linutronix.de> - 2017-08-31 09:40 +0200
  [patch 26/29] lockup_detector/perf: Implement CPU enable replacement Thomas Gleixner <tglx@linutronix.de> - 2017-08-31 09:40 +0200
  [patch 24/29] lockup_detector/perf: Implement init time perf  validation Thomas Gleixner <tglx@linutronix.de> - 2017-08-31 09:40 +0200
    Re: [patch 24/29] lockup_detector/perf: Implement init time perf  validation Don Zickus <dzickus@redhat.com> - 2017-09-07 18:00 +0200
  [patch 10/29] lockup_detector/perf: Prevent cpu hotplug deadlock Thomas Gleixner <tglx@linutronix.de> - 2017-08-31 09:40 +0200
    Re: [patch 10/29] lockup_detector/perf: Prevent cpu hotplug deadlock Don Zickus <dzickus@redhat.com> - 2017-09-01 21:10 +0200
      Re: [patch 10/29] lockup_detector/perf: Prevent cpu hotplug  deadlock Thomas Gleixner <tglx@linutronix.de> - 2017-09-01 21:30 +0200
        Re: [patch 10/29] lockup_detector/perf: Prevent cpu hotplug deadlock Don Zickus <dzickus@redhat.com> - 2017-09-05 17:00 +0200
  [patch 21/29] lockup_detector: Cleanup sysctl variable name space Thomas Gleixner <tglx@linutronix.de> - 2017-08-31 09:40 +0200
  [patch 18/29] lockup_detector: Further simplify sysctl handling Thomas Gleixner <tglx@linutronix.de> - 2017-08-31 09:40 +0200
  [patch 23/29] lockup_detector: Get rid of the racy update loop Thomas Gleixner <tglx@linutronix.de> - 2017-08-31 09:40 +0200
  [patch 27/29] lockup_detector: Use new perf CPU enable mechanism Thomas Gleixner <tglx@linutronix.de> - 2017-08-31 09:40 +0200
  [patch 13/29] lockup_detector: Cleanup the ifdef maze Thomas Gleixner <tglx@linutronix.de> - 2017-08-31 09:40 +0200
  [patch 06/29] lockup_detector: Rework cpu hotplug locking Thomas Gleixner <tglx@linutronix.de> - 2017-08-31 09:40 +0200
  [patch 14/29] lockup_detector: Split out cpumask write function Thomas Gleixner <tglx@linutronix.de> - 2017-08-31 09:40 +0200
  [patch 11/29] lockup_detector: Remove park_in_progress hackery Thomas Gleixner <tglx@linutronix.de> - 2017-08-31 09:40 +0200
    Re: [patch 11/29] lockup_detector: Remove park_in_progress hackery Peter Zijlstra <peterz@infradead.org> - 2017-09-04 14:20 +0200
      Re: [patch 11/29] lockup_detector: Remove park_in_progress hackery Don Zickus <dzickus@redhat.com> - 2017-09-05 17:20 +0200
        Re: [patch 11/29] lockup_detector: Remove park_in_progress hackery Thomas Gleixner <tglx@linutronix.de> - 2017-09-05 17:50 +0200
    Re: [patch 11/29] lockup_detector: Remove park_in_progress hackery Thomas Gleixner <tglx@linutronix.de> - 2017-09-05 16:00 +0200
      [patch V2 11/29] lockup_detector: Remove park_in_progress  obfuscation Thomas Gleixner <tglx@linutronix.de> - 2017-09-05 21:20 +0200
  [patch 04/29] parisc: Use lockup_detector_stop() Thomas Gleixner <tglx@linutronix.de> - 2017-08-31 09:40 +0200
  [patch 29/29] lockup_detector: Cleanup hotplug locking mess Thomas Gleixner <tglx@linutronix.de> - 2017-08-31 09:40 +0200
  [patch 12/29] lockup_detector: Cleanup stub functions Thomas Gleixner <tglx@linutronix.de> - 2017-08-31 09:40 +0200
  [patch 03/29] lockup_detector: Provide interface to stop from  poweroff() Thomas Gleixner <tglx@linutronix.de> - 2017-08-31 09:40 +0200
  [patch 20/29] lockup_detector/sysctl: Get rid of the ifdeffery Thomas Gleixner <tglx@linutronix.de> - 2017-08-31 09:40 +0200
  [patch 08/29] lockup_detector: Mark hardlockup_detector_disable()  __init Thomas Gleixner <tglx@linutronix.de> - 2017-08-31 09:40 +0200
  [patch 17/29] lockup_detector: Get rid of the thread teardown/setup  dance Thomas Gleixner <tglx@linutronix.de> - 2017-08-31 09:40 +0200
    Re: [patch 17/29] lockup_detector: Get rid of the thread  teardown/setup dance Don Zickus <dzickus@redhat.com> - 2017-09-01 21:10 +0200
      Re: [patch 17/29] lockup_detector: Get rid of the thread teardown/setup  dance Thomas Gleixner <tglx@linutronix.de> - 2017-09-01 21:50 +0200
  [patch 15/29] smpboot/threads: Avoid runtime allocation Thomas Gleixner <tglx@linutronix.de> - 2017-08-31 09:40 +0200
  [patch 07/29] lockup_detector: Rename watchdog_proc_mutex Thomas Gleixner <tglx@linutronix.de> - 2017-08-31 09:40 +0200
  [patch 16/29] lockup_detector: Create new thread handling  infrastructure Thomas Gleixner <tglx@linutronix.de> - 2017-08-31 09:40 +0200
  [patch 22/29] lockup_detector: Make watchdog_nmi_reconfigure() two  stage Thomas Gleixner <tglx@linutronix.de> - 2017-08-31 09:40 +0200
  [patch 01/29] hardlockup_detector: Provide interface to stop/restart  perf events Thomas Gleixner <tglx@linutronix.de> - 2017-08-31 09:40 +0200
    Re: [patch 01/29] hardlockup_detector: Provide interface to  stop/restart perf events Borislav Petkov <bp@alien8.de> - 2017-09-06 18:20 +0200
  [patch 05/29] lockup_detector: Remove broken suspend/resume interfaces Thomas Gleixner <tglx@linutronix.de> - 2017-08-31 09:50 +0200
  Re: [patch 00/29] lockup_detector: Cure hotplug deadlocks and  replace duct tape Don Zickus <dzickus@redhat.com> - 2017-09-01 00:20 +0200
    Re: [patch 00/29] lockup_detector: Cure hotplug deadlocks and  replace duct tape Nicholas Piggin <npiggin@gmail.com> - 2017-09-01 06:50 +0200
    Re: [patch 00/29] lockup_detector: Cure hotplug deadlocks and replace  duct tape Thomas Gleixner <tglx@linutronix.de> - 2017-09-01 11:20 +0200
  Re: [patch 00/29] lockup_detector: Cure hotplug deadlocks and  replace duct tape Don Zickus <dzickus@redhat.com> - 2017-09-07 18:10 +0200

csiph-web