Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1444215 > unrolled thread

Re: Linux 4.1.28

Started bySasha Levin <sasha.levin@oracle.com>
First post2016-07-15 14:00 +0200
Last post2016-07-20 14:20 +0200
Articles 6 — 4 participants

Back to article view | Back to linux.kernel

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: Linux 4.1.28 Sasha Levin <sasha.levin@oracle.com> - 2016-07-15 14:00 +0200
    Re: Linux 4.1.28 Thomas Voegtle <tv@lio96.de> - 2016-07-15 15:10 +0200
      Re: Linux 4.1.28 Arkadiusz Miskiewicz <a.miskiewicz@gmail.com> - 2016-07-20 08:50 +0200
        Re: Linux 4.1.28 Arkadiusz Miskiewicz <a.miskiewicz@gmail.com> - 2016-07-20 09:10 +0200
      Re: Linux 4.1.28 Michal Kubecek <mkubecek@suse.cz> - 2016-07-20 10:30 +0200
        Re: Linux 4.1.28 Thomas Voegtle <tv@lio96.de> - 2016-07-20 14:20 +0200

#1444215 — Re: Linux 4.1.28

FromSasha Levin <sasha.levin@oracle.com>
Date2016-07-15 14:00 +0200
SubjectRe: Linux 4.1.28
Message-ID<rV9WG-6re-9@gated-at.bofh.it>
On 07/15/2016 07:38 AM, Thomas Voegtle wrote:
> On Wed, 13 Jul 2016, Sasha Levin wrote:
> 
>> I'm announcing the release of the 4.1.28 kernel.
> 
> I have a serious memleak with 4.1.28 (like 20mb/s)
> I stripped down my kernel config and started a bisect, which came to:
> 
> # first bad commit: [c5ad33184354260be6d05de57e46a5498692f6d6] mm/swap.c:
> flush lru pvecs on compound page arrival
> =>
> commit c5ad33184354260be6d05de57e46a5498692f6d6
> Author: Lukasz Odzioba <lukasz.odzioba@intel.com>
> Date:   Fri Jun 24 14:50:01 2016 -0700
> 
>     mm/swap.c: flush lru pvecs on compound page arrival
> 
> 
> Reverting this on top 4.1.28 helps. Config attached.

Yup, this was reported and a fix is already queued for 4.1.29.


Thanks,
Sasha

[toc] | [next] | [standalone]

#1444304

FromThomas Voegtle <tv@lio96.de>
Date2016-07-15 15:10 +0200
Message-ID<rVb2r-7js-43@gated-at.bofh.it>
In reply to#1444215
On Fri, 15 Jul 2016, Sasha Levin wrote:

> On 07/15/2016 07:38 AM, Thomas Voegtle wrote:
>> On Wed, 13 Jul 2016, Sasha Levin wrote:
>>
>>> I'm announcing the release of the 4.1.28 kernel.
>>
>> I have a serious memleak with 4.1.28 (like 20mb/s)
>> I stripped down my kernel config and started a bisect, which came to:
>>
>> # first bad commit: [c5ad33184354260be6d05de57e46a5498692f6d6] mm/swap.c:
>> flush lru pvecs on compound page arrival
>> =>
>> commit c5ad33184354260be6d05de57e46a5498692f6d6
>> Author: Lukasz Odzioba <lukasz.odzioba@intel.com>
>> Date:   Fri Jun 24 14:50:01 2016 -0700
>>
>>     mm/swap.c: flush lru pvecs on compound page arrival
>>
>>
>> Reverting this on top 4.1.28 helps. Config attached.
>
> Yup, this was reported and a fix is already queued for 4.1.29.

And that one?
Happens while trying to start a firewall script with iptables-restore.


[  180.071999] NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! 
[iptables-restor:2338]
[  180.072000] Modules linked in: xt_CT xt_addrtype xt_set iptable_raw 
iptable_mangle iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 
iptable_filter ip_tables nf_conntrack_netbios_ns nf_conntrack_broadcast 
nf_nat_tftp nf_conntrack_tftp nf_nat_pptp nf_nat_proto_gre 
nf_conntrack_pptp nf_conntrack_proto_gre nf_nat_irc nf_conntrack_irc 
nf_nat_sip nf_conntrack_sip nf_nat_ftp nf_nat nf_conntrack_ftp 
nfnetlink_log nfsd lockd grace nfs_acl rpcsec_gss_krb5 auth_rpcgss sunrpc 
oid_registry nf_conntrack_netlink nf_conntrack autofs4 msr xfrm_user ipv6 
dummy ip_set_hash_netiface ip_set_list_set ip_set_hash_net ip_set 
vhost_net macvtap macvlan vhost tun capi kernelcapi ipmi_devintf 
ipmi_msghandler sg st snd_hda_codec_realtek snd_hda_codec_generic 
snd_hda_intel snd_hda_controller snd_hda_codec snd_hda_core
[  180.072000]  i915 snd_pcm coretemp snd_timer kvm_intel snd_hwdep 
drm_kms_helper snd kvm soundcore drm e1000e acpi_cpufreq firewire_ohci 
video processor e100 i2c_i801 i2c_algo_bit tpm_tis ptp i2c_core intel_agp 
intel_gtt firewire_core agpgart ppdev thermal_sys parport_pc tpm lpc_ich 
evdev microcode 8250_fintek parport button hwmon mfd_core pps_core mii
[  180.072000] CPU: 1 PID: 2338 Comm: iptables-restor Not tainted 
4.1.28-cx #1
[  180.072000] Hardware name:                  /DG33BU, BIOS 
DPP3510J.86A.0572.2009.0715.2346 07/15/2009
[  180.072000] task: ffff8800df0a60d0 ti: ffff8800c6d50000 task.ti: 
ffff8800c6d50000
[  180.072000] RIP: 0010:[<ffffffffc05bdac6>]  [<ffffffffc05bdac6>] 
translate_table+0x3a6/0x6bd [ip_tables]
[  180.072000] RSP: 0018:ffff8800c6d53c98  EFLAGS: 00000283
[  180.072000] RAX: 0000000000000e38 RBX: ffff88021bfd4b18 RCX: 
ffff8801f34e9988
[  180.072000] RDX: 0000000000000000 RSI: ffff8801f34ea908 RDI: 
ffff8800df14ae38
[  180.072000] RBP: ffff8800c6d53d88 R08: 0000000000000001 R09: 
0000000000000009
[  180.072000] R10: ffff8801f34e9988 R11: 0000000000000000 R12: 
0000000100000000
[  180.072000] R13: ffff88021bfd3b50 R14: ffff88021bfd4b00 R15: 
ffff8800c6d53ce0
[  180.072000] FS:  00007f31ba21a700(0000) GS:ffff88021bc80000(0000) 
knlGS:0000000000000000
[  180.072000] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  180.072000] CR2: 00000000015e1b28 CR3: 00000000ca20e000 CR4: 
00000000000406e0
[  180.072000] Stack:
[  180.072000]  ffff8800c6d53d08 ffff8801f34e9988 ffff880200000001 
0000000000000009
[  180.072000]  0000000000000000 0000000000000000 ffff8800df14a108 
ffffffff81ce2c40
[  180.072000]  ffff8800c6d53dcc ffff8800c6d53de0 0000000900000003 
0000000000000000
[  180.072000] Call Trace:
[  180.072000]  [<ffffffff8111a923>] ? __kmalloc_node+0x26/0x2d
[  180.072000]  [<ffffffff81667495>] ? cpumask_next+0x1b/0x1d
[  180.072000]  [<ffffffff81668ec3>] ? xt_alloc_table_info+0xc2/0xdf
[  180.072000]  [<ffffffffc05be6c3>] do_ipt_set_ctl+0xfd/0x193 [ip_tables]
[  180.072000]  [<ffffffff81665181>] nf_sockopt+0x64/0x7e
[  180.072000]  [<ffffffff816651cc>] nf_setsockopt+0x1c/0x20
[  180.072000]  [<ffffffff81674dd1>] ip_setsockopt+0x6e/0x8e
[  180.072000]  [<ffffffff8168fe47>] raw_setsockopt+0x11/0x4c
[  180.072000]  [<ffffffff816296ab>] sock_common_setsockopt+0xf/0x11
[  180.072000]  [<ffffffff81628ab2>] SyS_setsockopt+0x85/0xa5
[  180.072000]  [<ffffffff816e7dee>] system_call_fastpath+0x12/0x71
[  180.072000] Code: 02 00 8b 46 20 75 3b 89 c6 83 ee 00 78 34 8b 33 48 63 
f8 48 8d 4e 90 48 39 cf 0f 87 df 02 00 00 4c 01 ef 4c 89 d1 4c 01 d6 eb 0d 
<48> 39 f9 74 22 44 0f b7 59 5a 4c 01 d9 48 39 f1 72 ee e9 bd 02

[toc] | [prev] | [next] | [standalone]

#1447019

FromArkadiusz Miskiewicz <a.miskiewicz@gmail.com>
Date2016-07-20 08:50 +0200
Message-ID<rWTup-6ma-1@gated-at.bofh.it>
In reply to#1444304
On Friday 15 of July 2016, Thomas Voegtle wrote:
> On Fri, 15 Jul 2016, Sasha Levin wrote:
> > On 07/15/2016 07:38 AM, Thomas Voegtle wrote:
> >> On Wed, 13 Jul 2016, Sasha Levin wrote:
> >>> I'm announcing the release of the 4.1.28 kernel.
> >> 
> >> I have a serious memleak with 4.1.28 (like 20mb/s)
> >> I stripped down my kernel config and started a bisect, which came to:
> >> 
> >> # first bad commit: [c5ad33184354260be6d05de57e46a5498692f6d6]
> >> mm/swap.c: flush lru pvecs on compound page arrival
> >> =>
> >> commit c5ad33184354260be6d05de57e46a5498692f6d6
> >> Author: Lukasz Odzioba <lukasz.odzioba@intel.com>
> >> Date:   Fri Jun 24 14:50:01 2016 -0700
> >> 
> >>     mm/swap.c: flush lru pvecs on compound page arrival
> >> 
> >> Reverting this on top 4.1.28 helps. Config attached.
> > 
> > Yup, this was reported and a fix is already queued for 4.1.29.
> 
> And that one?
> Happens while trying to start a firewall script with iptables-restore.
> 
> 
> [  180.071999] NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s!
> [iptables-restor:2338]


Same here but I don't have actuall trace

Applying iptables firewall rules...................................[   
42.297704] NMI watchdog: BUG: soft lockup - CPU#7 stuck for 22s! [iptables-
restor:900]
[   70.287999] NMI watchdog: BUG: soft lockup - CPU#7 stuck for 22s! 
[iptables-restor:900]
[  102.276912] NMI watchdog: BUG: soft lockup - CPU#7 stuck for 23s! 
[iptables-restor:900]

-- 
Arkadiusz Miśkiewicz, arekm / ( maven.pl | pld-linux.org )

[toc] | [prev] | [next] | [standalone]

#1447032

FromArkadiusz Miskiewicz <a.miskiewicz@gmail.com>
Date2016-07-20 09:10 +0200
Message-ID<rWTNM-6HG-15@gated-at.bofh.it>
In reply to#1447019
On Wednesday 20 of July 2016, Arkadiusz Miskiewicz wrote:
> On Friday 15 of July 2016, Thomas Voegtle wrote:
> > On Fri, 15 Jul 2016, Sasha Levin wrote:
> > > On 07/15/2016 07:38 AM, Thomas Voegtle wrote:
> > >> On Wed, 13 Jul 2016, Sasha Levin wrote:
> > >>> I'm announcing the release of the 4.1.28 kernel.
> > >> 
> > >> I have a serious memleak with 4.1.28 (like 20mb/s)
> > >> I stripped down my kernel config and started a bisect, which came to:
> > >> 
> > >> # first bad commit: [c5ad33184354260be6d05de57e46a5498692f6d6]
> > >> mm/swap.c: flush lru pvecs on compound page arrival
> > >> =>
> > >> commit c5ad33184354260be6d05de57e46a5498692f6d6
> > >> Author: Lukasz Odzioba <lukasz.odzioba@intel.com>
> > >> Date:   Fri Jun 24 14:50:01 2016 -0700
> > >> 
> > >>     mm/swap.c: flush lru pvecs on compound page arrival
> > >> 
> > >> Reverting this on top 4.1.28 helps. Config attached.
> > > 
> > > Yup, this was reported and a fix is already queued for 4.1.29.
> > 
> > And that one?
> > Happens while trying to start a firewall script with iptables-restore.
> > 
> > 
> > [  180.071999] NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s!
> > [iptables-restor:2338]
> 
> Same here but I don't have actuall trace
> 
> Applying iptables firewall rules...................................[
> 42.297704] NMI watchdog: BUG: soft lockup - CPU#7 stuck for 22s! [iptables-
> restor:900]
> [   70.287999] NMI watchdog: BUG: soft lockup - CPU#7 stuck for 22s!
> [iptables-restor:900]
> [  102.276912] NMI watchdog: BUG: soft lockup - CPU#7 stuck for 23s!
> [iptables-restor:900]

Hm, maintainer no longer at oracle?

Delivery to the following recipient failed permanently:
550 5.1.1 Unknown oracle.com recipient.

-- 
Arkadiusz Miśkiewicz, arekm / ( maven.pl | pld-linux.org )

[toc] | [prev] | [next] | [standalone]

#1447079

FromMichal Kubecek <mkubecek@suse.cz>
Date2016-07-20 10:30 +0200
Message-ID<rWV3b-7pU-9@gated-at.bofh.it>
In reply to#1444304
On Fri, Jul 15, 2016 at 03:04:48PM +0200, Thomas Voegtle wrote:
> And that one?
> Happens while trying to start a firewall script with iptables-restore.
> 
> [  180.071999] NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s!
> [iptables-restor:2338]
...
> [  180.072000] Call Trace:
> [  180.072000]  [<ffffffff8111a923>] ? __kmalloc_node+0x26/0x2d
> [  180.072000]  [<ffffffff81667495>] ? cpumask_next+0x1b/0x1d
> [  180.072000]  [<ffffffff81668ec3>] ? xt_alloc_table_info+0xc2/0xdf
> [  180.072000]  [<ffffffffc05be6c3>] do_ipt_set_ctl+0xfd/0x193 [ip_tables]
> [  180.072000]  [<ffffffff81665181>] nf_sockopt+0x64/0x7e
> [  180.072000]  [<ffffffff816651cc>] nf_setsockopt+0x1c/0x20
> [  180.072000]  [<ffffffff81674dd1>] ip_setsockopt+0x6e/0x8e
> [  180.072000]  [<ffffffff8168fe47>] raw_setsockopt+0x11/0x4c
> [  180.072000]  [<ffffffff816296ab>] sock_common_setsockopt+0xf/0x11
> [  180.072000]  [<ffffffff81628ab2>] SyS_setsockopt+0x85/0xa5
> [  180.072000]  [<ffffffff816e7dee>] system_call_fastpath+0x12/0x71

This should be the problem addressed by

   http://article.gmane.org/gmane.linux.network/422215

Michal Kubecek

[toc] | [prev] | [next] | [standalone]

#1447201

FromThomas Voegtle <tv@lio96.de>
Date2016-07-20 14:20 +0200
Message-ID<rWYDL-1iY-11@gated-at.bofh.it>
In reply to#1447079
On Wed, 20 Jul 2016, Michal Kubecek wrote:

> On Fri, Jul 15, 2016 at 03:04:48PM +0200, Thomas Voegtle wrote:
>> And that one?
>> Happens while trying to start a firewall script with iptables-restore.
>>
>> [  180.071999] NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s!
>> [iptables-restor:2338]
> ...
>> [  180.072000] Call Trace:
>> [  180.072000]  [<ffffffff8111a923>] ? __kmalloc_node+0x26/0x2d
>> [  180.072000]  [<ffffffff81667495>] ? cpumask_next+0x1b/0x1d
>> [  180.072000]  [<ffffffff81668ec3>] ? xt_alloc_table_info+0xc2/0xdf
>> [  180.072000]  [<ffffffffc05be6c3>] do_ipt_set_ctl+0xfd/0x193 [ip_tables]
>> [  180.072000]  [<ffffffff81665181>] nf_sockopt+0x64/0x7e
>> [  180.072000]  [<ffffffff816651cc>] nf_setsockopt+0x1c/0x20
>> [  180.072000]  [<ffffffff81674dd1>] ip_setsockopt+0x6e/0x8e
>> [  180.072000]  [<ffffffff8168fe47>] raw_setsockopt+0x11/0x4c
>> [  180.072000]  [<ffffffff816296ab>] sock_common_setsockopt+0xf/0x11
>> [  180.072000]  [<ffffffff81628ab2>] SyS_setsockopt+0x85/0xa5
>> [  180.072000]  [<ffffffff816e7dee>] system_call_fastpath+0x12/0x71
>
> This should be the problem addressed by
>
>   http://article.gmane.org/gmane.linux.network/422215
>
> Michal Kubecek


Mh, doesn't fix the problem for me.
Looks like the same, see below.

In the original commit 482cfc318559 you mentioned, are changes in 
netfilter/x_tables.c aswell, aren't these missing here? 
(xt_alloc_table_info)

Actually I'm just guessing here, haven't a clue.

thanks,

     Thomas



[  288.072000] NMI watchdog: BUG: soft lockup - CPU#0 stuck for 23s! 
[iptables-restor:2874]
[  288.072000] Modules linked in: xt_CT xt_addrtype xt_set iptable_raw 
iptable_mangle iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 
nf_nat_tftp nf_conntrack_tftp nf_nat_pptp nf_nat_proto_gre 
nf_conntrack_pptp nf_conntrack_proto_gre nf_nat_irc nf_conntrack_irc 
nf_nat_sip nf_conntrack_sip nf_nat_ftp nf_nat nf_conntrack_ftp nfsd lockd 
grace nfs_acl auth_rpcgss oid_registry sunrpc nfnetlink_log msr 
iptable_filter ip_tables nf_conntrack_netlink nf_conntrack autofs4 arc4 
ecb ppp_mppe xfrm_user pppoe pppox ppp_generic slhc ipv6 dummy 
ip_set_hash_netiface ip_set_list_set ip_set_hash_net ip_set ipmi_devintf 
ipmi_msghandler lcd_module(O) sg st snd_hda_codec_realtek 
snd_hda_codec_generic snd_hda_intel snd_hda_controller snd_hda_codec 
snd_hda_core i915 snd_pcm coretemp acpi_cpufreq snd_timer snd_hwdep
[  288.072000]  drm_kms_helper kvm_intel divas(O) drm kvm processor video 
snd e1000e thermal_sys divadidd(O) tpm_tis firewire_ohci i2c_algo_bit 
intel_agp evdev i2c_i801 firewire_core microcode e100 ptp intel_gtt ppdev 
parport_pc tpm pps_core lpc_ich soundcore agpgart 8250_fintek hwmon button 
parport i2c_core mii shpchp mfd_core
[  288.072000] CPU: 0 PID: 2874 Comm: iptables-restor Tainted: G 
O    4.1.28-cx #1
[  288.072000] Hardware name:                  /DG33BU, BIOS 
DPP3510J.86A.0572.2009.0715.2346 07/15/2009
[  288.072000] task: ffff8801c188a090 ti: ffff8801c1814000 task.ti: 
ffff8801c1814000
[  288.072000] RIP: 0010:[<ffffffffc04f8ad3>]  [<ffffffffc04f8ad3>] 
translate_table+0x3b3/0x6bd [ip_tables]
[  288.072000] RSP: 0018:ffff8801c1817c98  EFLAGS: 00000286
[  288.072000] RAX: 0000000000000f68 RBX: 8000000000000163 RCX: 
ffff8801c185da88
[  288.072000] RDX: 0000000000000000 RSI: ffff8801c185eb38 RDI: 
ffffc900010c6f68
[  288.072000] RBP: ffff8801c1817d88 R08: 0000000000000001 R09: 
0000000000000009
[  288.072000] R10: ffff8801c185da88 R11: 0000000000000000 R12: 
ffffc900010d0fff
[  288.072000] R13: ffffc900010d0fff R14: ffffffff81c0bc90 R15: 
0000000000000000
[  288.072000] FS:  00007f0713a61700(0000) GS:ffff88021bc00000(0000) 
knlGS:0000000000000000
[  288.072000] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  288.072000] CR2: 0000000002478e28 CR3: 00000001c182e000 CR4: 
00000000000406f0
[  288.072000] Stack:
[  288.072000]  ffff8801c1817cf8 ffff8801c185da88 ffff880200000001 
ffffffff00000009
[  288.072000]  0000000000000000 0000000000000000 ffffc900010c6108 
ffffffff81ce2ec0
[  288.072000]  ffff8801c1817dcc ffff8801c1817de0 0000000900011030 
0000000000000000
[  288.072000] Call Trace:
[  288.072000]  [<ffffffff811079ce>] ? __vmalloc_node+0x39/0x3b
[  288.072000]  [<ffffffff8166b0dc>] ? xt_alloc_table_info+0x97/0xdf
[  288.072000]  [<ffffffff8123a32f>] ? find_next_bit+0x18/0x1a
[  288.072000]  [<ffffffff816696d9>] ? cpumask_next+0x1b/0x1d
[  288.072000]  [<ffffffff8166b107>] ? xt_alloc_table_info+0xc2/0xdf
[  288.072000]  [<ffffffffc04f96c3>] do_ipt_set_ctl+0xfd/0x193 [ip_tables]
[  288.072000]  [<ffffffff816673c5>] nf_sockopt+0x64/0x7e
[  288.072000]  [<ffffffff81667410>] nf_setsockopt+0x1c/0x20
[  288.072000]  [<ffffffff81677045>] ip_setsockopt+0x6e/0x8e
[  288.072000]  [<ffffffff816920bb>] raw_setsockopt+0x11/0x4c
[  288.072000]  [<ffffffff8162b84b>] sock_common_setsockopt+0xf/0x11
[  288.072000]  [<ffffffff8162ac53>] SyS_setsockopt+0x85/0xa5
[  288.072000]  [<ffffffff816ea0ee>] system_call_fastpath+0x12/0x71
[  288.072000] Code: 34 8b 33 48 63 f8 48 8d 4e 90 48 39 cf 0f 87 df 02 00 
00 4c 01 ef 4c 89 d1 4c 01 d6 eb 0d 48 39 f9 74 22 44 0f b7 59 5a 4c 01 d9 
<48> 39 f1 72 ee e9 bd 02 00 00 41 0f b7 47 5a 44 01 e0 3b 03 0f

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web