Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1620792 > unrolled thread

Re: [WARNING] kernel/workqueue.c:2041 process_one_work (when cpu goes offline)

Back to article view | Back to linux.kernel

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: [WARNING] kernel/workqueue.c:2041 process_one_work (when cpu  goes offline) Tejun Heo <tj@kernel.org> - 2017-04-11 02:10 +0200
    Re: [WARNING] kernel/workqueue.c:2041 process_one_work (when cpu  goes offline) Tejun Heo <tj@kernel.org> - 2017-04-18 10:20 +0200

#1620792 — Re: [WARNING] kernel/workqueue.c:2041 process_one_work (when cpu goes offline)

Hello, Steven.

On Wed, Apr 05, 2017 at 03:16:28PM -0400, Steven Rostedt wrote:
> My tests have started to recently trigger this warning quite often,
> which causes my tests to fail. The test that triggers this is running
> the mmiotracer which forces all but one CPU offline.
> 
> ------------[ cut here ]------------
> WARNING: CPU: 0 PID: 6 at /work/autotest/nobackup/linux-test.git/kernel/workqueue.c:2041 process_one_work+0x90/0x485
> Modules linked in: ppdev parport_pc parport [last unloaded: trace_events_sample]
> CPU: 0 PID: 6 Comm: vmstat Not tainted 4.11.0-rc5-test+ #3
> Hardware name: MSI MS-7823/CSM-H87M-G43 (MS-7823), BIOS V1.6 02/22/2014
> Call Trace:
>  dump_stack+0x68/0x92
>  __warn+0xc2/0xdd
>  warn_slowpath_null+0x1d/0x1f
>  process_one_work+0x90/0x485
>  process_scheduled_works+0x2c/0x33
>  rescuer_thread+0x19c/0x295
>  ? process_scheduled_works+0x33/0x33
>  kthread+0xf4/0xf9
>  ? __list_del_entry+0x22/0x22
>  ret_from_fork+0x2e/0x40
> ---[ end trace ed53fc9d3ce10aa8 ]---
> 
> #ifdef CONFIG_LOCKDEP
> 	/*
> 	 * It is permissible to free the struct work_struct from
> 	 * inside the function that is called from it, this we need to
> 	 * take into account for lockdep too.  To avoid bogus "held
> 	 * lock freed" warnings as well as problems when looking into
> 	 * work->lockdep_map, make a copy and use that here.
> 	 */
> 	struct lockdep_map lockdep_map;
> 
> 	lockdep_copy_map(&lockdep_map, &work->lockdep_map);
> #endif
> 	/* ensure we're on the correct CPU */
> 	WARN_ON_ONCE(!(pool->flags & POOL_DISASSOCIATED) &&   <<--- line 2041
> 		     raw_smp_processor_id() != pool->cpu);
> 
> 	/*
> 	 * A single work shouldn't be executed concurrently by
> 	 * multiple workers on a single cpu.  Check whether anyone is
> 	 * already processing the work.  If so, defer the work to the
> 	 * currently executing one.
> 	 */
> 
> 
> I'm assuming that this thread was migrated due to the CPU offlining and
> causes pool->cpu not to equal raw_smp_processor_id(). Or should that
> not be happening?

If this happens while CPU is going donw, the pool should have
POOL_DISASSOCIATED set by that point and the actual affinity shouldn't
matter.  Maybe I messed up the rescuer part of it.  I'll look into it.

Thanks.

-- 
tejun

[toc] | [next] | [standalone]

#1625119

Hello,

On Tue, Apr 11, 2017 at 09:08:37AM +0900, Tejun Heo wrote:
> On Wed, Apr 05, 2017 at 03:16:28PM -0400, Steven Rostedt wrote:
> > My tests have started to recently trigger this warning quite often,
> > which causes my tests to fail. The test that triggers this is running
> > the mmiotracer which forces all but one CPU offline.

So, the rescuer handling seems fine and nothing really changed from
workqueue side.  Any chance cpuset is involved?  If so, there was a
recent race condition fix 77f88796cee8 ("cgroup, kthread: close race
window where new kthreads can be migrated to non-root cgroups").  If
that's not it, it'd be great if you can explain the test case so that
I can repro the problem.

Thanks.

-- 
tejun

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web