Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1229290 > unrolled thread

[RFC v2 00/18] kthread: Use kthread worker API more widely

Back to article view | Back to linux.kernel

Contents

  [RFC v2 00/18] kthread: Use kthread worker API more widely Petr Mladek <pmladek@suse.com> - 2015-09-21 15:10 +0200
    [RFC v2 02/18] kthread: Add create_kthread_worker*() Petr Mladek <pmladek@suse.com> - 2015-09-21 15:20 +0200
    [RFC v2 05/18] kthread: Add pending flag to kthread work Petr Mladek <pmladek@suse.com> - 2015-09-21 15:20 +0200

#1229290 — [RFC v2 00/18] kthread: Use kthread worker API more widely

My intention is to make it easier to manipulate kthreads. This RFC tries
to use the kthread worker API. It is based on comments from the
first attempt. See https://lkml.org/lkml/2015/7/28/648 and
the list of changes below.

1st..8th patches: improve the existing kthread worker API

9th, 12th, 17th patches: convert three kthreads into the new API,
     namely: khugepaged, ring buffer benchmark, RCU gp kthreads[*]

10th, 11th patches: fix potential problems in the ring buffer
      benchmark; also sent separately

13th patch: small fix for RCU kthread; also sent separately;
     being tested by Paul

14th..16th patches: preparation steps for the RCU threads
     conversion; they are needed _only_ if we split GP start
     and QS handling into separate works[*]

18th patch: does a possible improvement of the kthread worker API;
     it adds an extra parameter to the create*() functions, so I
     rather put it into this draft
     

[*] IMPORTANT: I tried to split RCU GP start and GS state handling
    into separate works this time. But there is a problem with
    a race in rcu_gp_kthread_worker_poke(). It might queue
    the wrong work. It can be detected and fixed by the work
    itself but it is a bit ugly. Alternative solution is to
    do both operations in one work. But then we sleep too much
    in the work which is ugly as well. Any idea is appreciated.
    

Changes against v1:

+ remove wrappers to manipulate the scheduling policy and priority

+ remove questionable wakeup_and_destroy_kthread_worker() variant

+ do not check for chained work when draining the queue

+ allocate struct kthread worker in create_kthread_work() and
  use more simple checks for running worker

+ add support for delayed kthread works and use them instead
  of waiting inside the works

+ rework the "unrelated" fixes for the ring buffer benchmark
  as discussed in the 1st RFC; also sent separately

+ convert also the consumer in the ring buffer benchmark


I have tested this patch set against the stable Linus tree
for 4.3-rc2.

Petr Mladek (18):
  kthread: Allow to call __kthread_create_on_node() with va_list args
  kthread: Add create_kthread_worker*()
  kthread: Add drain_kthread_worker()
  kthread: Add destroy_kthread_worker()
  kthread: Add pending flag to kthread work
  kthread: Initial support for delayed kthread work
  kthread: Allow to cancel kthread work
  kthread: Allow to modify delayed kthread work
  mm/huge_page: Convert khugepaged() into kthread worker API
  ring_buffer: Do no not complete benchmark reader too early
  ring_buffer: Fix more races when terminating the producer in the
    benchmark
  ring_buffer: Convert benchmark kthreads into kthread worker API
  rcu: Finish folding ->fqs_state into ->gp_state
  rcu: Store first_gp_fqs into struct rcu_state
  rcu: Clean up timeouts for forcing the quiescent state
  rcu: Check actual RCU_GP_FLAG_FQS when handling quiescent state
  rcu: Convert RCU gp kthreads into kthread worker API
  kthread: Better support freezable kthread workers

 include/linux/kthread.h              |  67 +++++
 kernel/kthread.c                     | 544 ++++++++++++++++++++++++++++++++---
 kernel/rcu/tree.c                    | 407 ++++++++++++++++----------
 kernel/rcu/tree.h                    |  24 +-
 kernel/rcu/tree_plugin.h             |  16 +-
 kernel/rcu/tree_trace.c              |   2 +-
 kernel/trace/ring_buffer_benchmark.c | 194 ++++++-------
 mm/huge_memory.c                     | 116 ++++----
 8 files changed, 1017 insertions(+), 353 deletions(-)

-- 
1.8.5.6

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [next] | [standalone]

#1229293 — [RFC v2 02/18] kthread: Add create_kthread_worker*()

Kthread workers are currently created using the classic kthread API,
namely kthread_run(). kthread_worker_fn() is passed as the @threadfn
parameter.

This patch defines create_kthread_worker_on_node() and
create_kthread_worker() functions that hide implementation details.

It enforces using kthread_worker_fn() for the main thread. But I doubt
that there are any plans to create any alternative. In fact, I think
that we do not want any alternative main thread because it would be
hard to support consistency with the rest of the kthread worker API.

The naming and function is inspired by the workqueues API like
the rest of the kthread worker API.

This patch does _not_ convert existing kthread workers. The kthread worker
API need more improvements first, e.g. a function to destroy the worker.

Signed-off-by: Petr Mladek <pmladek@suse.com>
---
 include/linux/kthread.h |  7 +++++++
 kernel/kthread.c        | 51 ++++++++++++++++++++++++++++++++++++++++++++++++-
 2 files changed, 57 insertions(+), 1 deletion(-)

diff --git a/include/linux/kthread.h b/include/linux/kthread.h
index e691b6a23f72..e390069a3f68 100644
--- a/include/linux/kthread.h
+++ b/include/linux/kthread.h
@@ -124,6 +124,13 @@ extern void __init_kthread_worker(struct kthread_worker *worker,
 
 int kthread_worker_fn(void *worker_ptr);
 
+__printf(2, 3)
+struct kthread_worker *
+create_kthread_worker_on_node(int node, const char namefmt[], ...);
+
+#define create_kthread_worker(namefmt, arg...)				\
+	create_kthread_worker_on_node(-1, namefmt, ##arg)
+
 bool queue_kthread_work(struct kthread_worker *worker,
 			struct kthread_work *work);
 void flush_kthread_work(struct kthread_work *work);
diff --git a/kernel/kthread.c b/kernel/kthread.c
index 1d29e675fe44..8f8813b42632 100644
--- a/kernel/kthread.c
+++ b/kernel/kthread.c
@@ -579,7 +579,11 @@ int kthread_worker_fn(void *worker_ptr)
 	struct kthread_worker *worker = worker_ptr;
 	struct kthread_work *work;
 
-	WARN_ON(worker->task);
+	/*
+	 * FIXME: Update the check and remove the assignment when all kthread
+	 * worker users are created using create_kthread_worker*() functions.
+	 */
+	WARN_ON(worker->task && worker->task != current);
 	worker->task = current;
 repeat:
 	set_current_state(TASK_INTERRUPTIBLE);	/* mb paired w/ kthread_stop */
@@ -613,6 +617,51 @@ repeat:
 }
 EXPORT_SYMBOL_GPL(kthread_worker_fn);
 
+/**
+ * create_kthread_worker_on_node - create a kthread worker.
+ * @node: memory node number.
+ * @namefmt: printf-style name for the kthread worker (task).
+ *
+ * If the worker is going to be bound on a particular CPU, give its node
+ * in @node, to get NUMA affinity for kthread stack, or else give -1.
+ *
+ * Returns pointer to allocated worker on success, ERR_PTR(-ENOMEM) when
+ * the needed structures could not get allocated, and ERR_PTR(-EINTR)
+ * when the worker was SIGKILLed.
+ */
+struct kthread_worker *
+create_kthread_worker_on_node(int node, const char namefmt[], ...)
+{
+	struct kthread_worker *worker;
+	struct task_struct *task;
+	va_list args;
+
+	worker = kzalloc(sizeof(*worker), GFP_KERNEL);
+	if (!worker)
+		return ERR_PTR(-ENOMEM);
+
+	init_kthread_worker(worker);
+
+	va_start(args, namefmt);
+	task = __kthread_create_on_node(kthread_worker_fn, worker, node,
+					namefmt, args);
+	va_end(args);
+
+	if (IS_ERR(task))
+		goto fail_task;
+
+	worker->task = task;
+	wake_up_process(task);
+
+	return worker;
+
+fail_task:
+	kfree(worker);
+	return ERR_CAST(task);
+
+}
+EXPORT_SYMBOL(create_kthread_worker_on_node);
+
 /* insert @work before @pos in @worker */
 static void insert_kthread_work(struct kthread_worker *worker,
 			       struct kthread_work *work,
-- 
1.8.5.6

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1229304 — [RFC v2 05/18] kthread: Add pending flag to kthread work

This is a preparation step for delayed kthread works. It will use
a timer to queue the work with the requested delay. We need to
somehow mark the work in the meantime.

The implementation is inspired by workqueues. It adds a flag that
is manipulated using bit operations. If the flag is set, it means
that the work is going to be queued and any new attempts to queue
the work should fail. As a side effect, queue_kthread_work() could
test pending work even without the lock.

In compare with workqueues, the flag is stored in a separate bitmap
instead of sharing with the worker pointer. Kthread worker does not
use pools of kthreads and the handling is much easier here. I did
not fix a situation where we would need to manipulate both the flag
and the worker pointer atomically.

Signed-off-by: Petr Mladek <pmladek@suse.com>
---
 include/linux/kthread.h |  6 ++++++
 kernel/kthread.c        | 30 ++++++++++++++++++++++++++----
 2 files changed, 32 insertions(+), 4 deletions(-)

diff --git a/include/linux/kthread.h b/include/linux/kthread.h
index bef97e06d2b6..aabb105d3d4b 100644
--- a/include/linux/kthread.h
+++ b/include/linux/kthread.h
@@ -71,7 +71,13 @@ struct kthread_worker {
 	struct kthread_work	*current_work;
 };
 
+enum {
+	/* work item is pending execution */
+	KTHREAD_WORK_PENDING_BIT	= 0,
+};
+
 struct kthread_work {
+	DECLARE_BITMAP(flags, 8);
 	struct list_head	node;
 	kthread_work_func_t	func;
 	struct kthread_worker	*worker;
diff --git a/kernel/kthread.c b/kernel/kthread.c
index 65c263336b8b..fe1510e7ad04 100644
--- a/kernel/kthread.c
+++ b/kernel/kthread.c
@@ -602,6 +602,7 @@ repeat:
 		work = list_first_entry(&worker->work_list,
 					struct kthread_work, node);
 		list_del_init(&work->node);
+		clear_bit(KTHREAD_WORK_PENDING_BIT, work->flags);
 	}
 	worker->current_work = work;
 	spin_unlock_irq(&worker->lock);
@@ -675,6 +676,27 @@ static void insert_kthread_work(struct kthread_worker *worker,
 		wake_up_process(worker->task);
 }
 
+/*
+ * Queue @work without the check for the pending flag.
+ * Must be called with IRQs disabled.
+ */
+static void __queue_kthread_work(struct kthread_worker *worker,
+			  struct kthread_work *work)
+{
+	/*
+	 * While a work item is PENDING && off queue, a task trying to
+	 * steal the PENDING will busy-loop waiting for it to either get
+	 * queued or lose PENDING.  Grabbing PENDING and queuing should
+	 * happen with IRQ disabled.
+	 */
+	WARN_ON_ONCE(!irqs_disabled());
+	WARN_ON_ONCE(!list_empty(&work->node));
+
+	spin_lock(&worker->lock);
+	insert_kthread_work(worker, work, &worker->work_list);
+	spin_unlock(&worker->lock);
+}
+
 /**
  * queue_kthread_work - queue a kthread_work
  * @worker: target kthread_worker
@@ -690,12 +712,12 @@ bool queue_kthread_work(struct kthread_worker *worker,
 	bool ret = false;
 	unsigned long flags;
 
-	spin_lock_irqsave(&worker->lock, flags);
-	if (list_empty(&work->node)) {
-		insert_kthread_work(worker, work, &worker->work_list);
+	local_irq_save(flags);
+	if (!test_and_set_bit(KTHREAD_WORK_PENDING_BIT, work->flags)) {
+		__queue_kthread_work(worker, work);
 		ret = true;
 	}
-	spin_unlock_irqrestore(&worker->lock, flags);
+	local_irq_restore(flags);
 	return ret;
 }
 EXPORT_SYMBOL_GPL(queue_kthread_work);
-- 
1.8.5.6

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web