Deadlock when running DevStack on latest pull of net tree

Path	csiph.com!aioe.org!gothmog.csi.it!bofh.it!news.nic.it!robomod
From	Alexander Duyck <alexander.duyck@gmail.com>
Newsgroups	linux.kernel
Subject	Deadlock when running DevStack on latest pull of net tree
Date	Fri, 27 May 2016 20:30:02 +0200
Message-ID	<rDuGe-6SR-13@gated-at.bofh.it> (permalink)
Dkim-Signature	v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to; bh=IXVxSTjcd/dCHWu3QEUtJfyoaeZqo5nsjl4N3VzI/ig=; b=JX4vrqIlhq/qUfy0wAtcioS2ze9V70wD5CXhR1G90a/7CNZ1Jqtgw5f9z9lcLsEKJ/ sOI+RvXRf10gyYan+VJvRErJ15CK1OFSdRCT5RyNR/ERfkPIxt/F149IN05noSmdKhwO XW6EOaThT82NrRWoiQdI2L5LZS4icx0Nc7FVIpivjO2mCnzwFkU7xxXzTpg6IA3iLeFM Y/VzpIx0MgkRTt0GqOy6vciUZis3G+DvPAl/HntPGcHrpEJvuxA2PMQiGUgL9U7F1AVs lBIB/ny1ZkqTcZ8mEtdsAsDvCTzHHniJQJC6gaix9W3frQ3RwTYOM4BJ5jo/SN1hha99 FSXQ==
X-Google-Dkim-Signature	v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:date:message-id:subject:from:to; bh=IXVxSTjcd/dCHWu3QEUtJfyoaeZqo5nsjl4N3VzI/ig=; b=BC1uQaZf0PbX6oF+0VSjPKFlRYmItEn24WbeqnPHpWLEdoOiGb8L+BXQkEsHeebiLK SYNCznUSIffoz3TvhnTv72xHi4J0RHITuBWSNWaBiUL6jpA4sV7eOSQsr5wNrbVixSsH BNRTN25MzdFtGXArorjrndaaH4es63Eet4uAheheN5iYDij2v1sbc46niG0eZTVw9Og+ lg0hROf6ux1Jn5Jq4ZuXyFkJi+oVlil0wRTl92Tv1gOG3cMrJ2OWN4UjbRU69DCxJsQG yC6ZgMa7j7QgzCgQB47IzbjdPd19aKqfPgLNCeF1U9CxhQT10DfnJ4QfELTY6IFsk6ov ErwA==
X-Gm-Message-State	ALyK8tJphsNQUWl1ZkafiSpf+N114M40lZ8R91G82rJXF0a/h7cTklzw+wr0b3qPQVC+6S39Pv9Uor46PL6QQQ==
MIME-Version	1.0
X-Received	by 10.50.60.195 with SMTP id j3mr30546igr.4.1464373311124; Fri, 27 May 2016 11:21:51 -0700 (PDT)
Content-Type	text/plain; charset=UTF-8
Sender	robomod@news.nic.it
List-ID	<linux-kernel.vger.kernel.org>
X-Mailing-List	linux-kernel@vger.kernel.org
Approved	robomod@news.nic.it
Lines	63
Organization	linux.* mail to news gateway
X-Original-Date	Fri, 27 May 2016 11:21:50 -0700
X-Original-Message-ID	<CAKgT0Uc=kpE1UArBpbtFu3eUVzW75zU+XD5hsc39k2CFE7PtNA@mail.gmail.com>
X-Original-Sender	linux-kernel-owner@vger.kernel.org
Xref	csiph.com linux.kernel:1408233

Show key headers only | View raw

I started out this morning by trying to run DevStack on the latest
"net' kernel and it looks like I am hanging on some sort of locking
problem with RabbitMQ.  Specifically I am seeing one CPU jump to 100%
with perf showing that I am spinning on a lock.

I'm working to bisect it now, but just thought I would put it out
there if anybody had already root caused this issue.  Below is a few
traces to the spin lock call on the spinning CPU:
-   18.81%    18.81%  beam.smp  [kernel.vmlinux]  [k] _raw_spin_lock_irqsave
   - _raw_spin_lock_irqsave
      - 32.86% add_wait_queue
           n_tty_write
           tty_write
           do_loop_readv_writev
           do_readv_writev
           vfs_writev
           do_writev
           sys_writev
           do_syscall_64
           return_from_SYSCALL_64
           __libc_writev
      - 32.49% remove_wait_queue
           n_tty_write
           tty_write
           do_loop_readv_writev
           do_readv_writev
           vfs_writev
           do_writev
           sys_writev
           do_syscall_64
           return_from_SYSCALL_64
           __libc_writev
      - 31.71% __wake_up
           tty_write_unlock
           tty_write
           do_loop_readv_writev
           do_readv_writev
           vfs_writev
           do_writev
           sys_writev
           do_syscall_64
           return_from_SYSCALL_64
           __libc_writev
      - 1.96% n_tty_write
           tty_write
           do_loop_readv_writev
           do_readv_writev
           vfs_writev
           do_writev
           sys_writev
           do_syscall_64
           return_from_SYSCALL_64
           __libc_writev
      - 0.87% tty_write_unlock
           tty_write
           do_loop_readv_writev
           do_readv_writev
           vfs_writev
           do_writev
           sys_writev
           do_syscall_64
           return_from_SYSCALL_64
           __libc_writev

Back to linux.kernel | Previous | Next | Find similar | Unroll thread

Thread

Deadlock when running DevStack on latest pull of net tree Alexander Duyck <alexander.duyck@gmail.com> - 2016-05-27 20:30 +0200

csiph-web