Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > linux.kernel > #1422753 > unrolled thread
| Started by | Sergey Senozhatsky <sergey.senozhatsky.work@gmail.com> |
|---|---|
| First post | 2016-06-15 10:00 +0200 |
| Last post | 2016-06-17 09:30 +0200 |
| Articles | 11 — 3 participants |
Back to article view | Back to linux.kernel
This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by
below is the oldest one visible, not the original post.
Re: [PATCH v7 00/12] Support non-lru page migration Sergey Senozhatsky <sergey.senozhatsky.work@gmail.com> - 2016-06-15 10:00 +0200
Re: [PATCH v7 00/12] Support non-lru page migration Minchan Kim <minchan@kernel.org> - 2016-06-16 01:20 +0200
Re: [PATCH v7 00/12] Support non-lru page migration Sergey Senozhatsky <sergey.senozhatsky.work@gmail.com> - 2016-06-16 04:50 +0200
Re: [PATCH v7 00/12] Support non-lru page migration Minchan Kim <minchan@kernel.org> - 2016-06-16 05:00 +0200
Re: [PATCH v7 00/12] Support non-lru page migration Sergey Senozhatsky <sergey.senozhatsky.work@gmail.com> - 2016-06-16 06:30 +0200
Re: [PATCH v7 00/12] Support non-lru page migration Minchan Kim <minchan@kernel.org> - 2016-06-16 06:50 +0200
Re: [PATCH v7 00/12] Support non-lru page migration Sergey Senozhatsky <sergey.senozhatsky.work@gmail.com> - 2016-06-16 07:30 +0200
Re: [PATCH v7 00/12] Support non-lru page migration Minchan Kim <minchan@kernel.org> - 2016-06-16 08:50 +0200
Re: [PATCH v7 00/12] Support non-lru page migration Sergey Senozhatsky <sergey.senozhatsky.work@gmail.com> - 2016-06-16 10:50 +0200
Re: [PATCH v7 00/12] Support non-lru page migration Minchan Kim <minchan@kernel.org> - 2016-06-16 12:10 +0200
Re: [PATCH v7 00/12] Support non-lru page migration Joonsoo Kim <iamjoonsoo.kim@lge.com> - 2016-06-17 09:30 +0200
| From | Sergey Senozhatsky <sergey.senozhatsky.work@gmail.com> |
|---|---|
| Date | 2016-06-15 10:00 +0200 |
| Subject | Re: [PATCH v7 00/12] Support non-lru page migration |
| Message-ID | <rKdTX-88q-1@gated-at.bofh.it> |
Hello Minchan, -next 4.7.0-rc3-next-20160614 [ 315.146533] kasan: CONFIG_KASAN_INLINE enabled [ 315.146538] kasan: GPF could be caused by NULL-ptr deref or user memory access [ 315.146546] general protection fault: 0000 [#1] PREEMPT SMP KASAN [ 315.146576] Modules linked in: lzo zram zsmalloc mousedev coretemp hwmon crc32c_intel r8169 i2c_i801 mii snd_hda_codec_realtek snd_hda_codec_generic snd_hda_intel snd_hda_codec snd_hda_core acpi_cpufreq snd_pcm snd_timer snd soundcore lpc_ich mfd_core processor sch_fq_codel sd_mod hid_generic usbhid hid ahci libahci libata ehci_pci ehci_hcd scsi_mod usbcore usb_common [ 315.146785] CPU: 3 PID: 38 Comm: khugepaged Not tainted 4.7.0-rc3-next-20160614-dbg-00004-ga1c2cbc-dirty #488 [ 315.146841] task: ffff8800bfaf2900 ti: ffff880112468000 task.ti: ffff880112468000 [ 315.146859] RIP: 0010:[<ffffffffa02c413d>] [<ffffffffa02c413d>] zs_page_migrate+0x355/0xaa0 [zsmalloc] [ 315.146892] RSP: 0000:ffff88011246f138 EFLAGS: 00010293 [ 315.146906] RAX: 736761742d6f6e2c RBX: ffff880017ad9a80 RCX: 0000000000000000 [ 315.146924] RDX: 1ffffffff064d704 RSI: ffff88000511469a RDI: ffffffff8326ba20 [ 315.146942] RBP: ffff88011246f328 R08: 0000000000000001 R09: 0000000000000000 [ 315.146959] R10: ffff88011246f0a8 R11: ffff8800bfc07fff R12: ffff88011246f300 [ 315.146977] R13: ffffed0015523e6f R14: ffff8800aa91f378 R15: ffffea0000144500 [ 315.146995] FS: 0000000000000000(0000) GS:ffff880113780000(0000) knlGS:0000000000000000 [ 315.147015] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 315.147030] CR2: 00007f3f97911000 CR3: 0000000002209000 CR4: 00000000000006e0 [ 315.147046] Stack: [ 315.147052] 1ffff10015523e0f ffff88011246f240 ffff880005116800 00017f80e0000000 [ 315.147083] ffff880017ad9aa8 736761742d6f6e2c 1ffff1002248de34 ffff880017ad9a90 [ 315.147113] 0000069a1246f660 000000000000069a ffff880005114000 ffffea0002ff0180 [ 315.147143] Call Trace: [ 315.147154] [<ffffffffa02c3de8>] ? obj_to_head+0x9d/0x9d [zsmalloc] [ 315.147175] [<ffffffff81d31dbc>] ? _raw_spin_unlock_irqrestore+0x47/0x5c [ 315.147195] [<ffffffff812275b1>] ? isolate_freepages_block+0x2f9/0x5a6 [ 315.147213] [<ffffffff8127f15c>] ? kasan_poison_shadow+0x2f/0x31 [ 315.147230] [<ffffffff8127f66a>] ? kasan_alloc_pages+0x39/0x3b [ 315.147246] [<ffffffff812267e6>] ? map_pages+0x1f3/0x3ad [ 315.147262] [<ffffffff812265f3>] ? update_pageblock_skip+0x18d/0x18d [ 315.147280] [<ffffffff81115972>] ? up_read+0x1a/0x30 [ 315.147296] [<ffffffff8111ec7e>] ? debug_check_no_locks_freed+0x150/0x22b [ 315.147315] [<ffffffff812842d1>] move_to_new_page+0x4dd/0x615 [ 315.147332] [<ffffffff81283df4>] ? migrate_page+0x75/0x75 [ 315.147347] [<ffffffff8122785e>] ? isolate_freepages_block+0x5a6/0x5a6 [ 315.147366] [<ffffffff812851c1>] migrate_pages+0xadd/0x131a [ 315.147382] [<ffffffff8122785e>] ? isolate_freepages_block+0x5a6/0x5a6 [ 315.147399] [<ffffffff81226375>] ? kzfree+0x2b/0x2b [ 315.147414] [<ffffffff812846e4>] ? buffer_migrate_page+0x2db/0x2db [ 315.147431] [<ffffffff8122a6cf>] compact_zone+0xcdb/0x1155 [ 315.147448] [<ffffffff812299f4>] ? compaction_suitable+0x76/0x76 [ 315.147465] [<ffffffff8122ac29>] compact_zone_order+0xe0/0x167 [ 315.147481] [<ffffffff8111f0ac>] ? debug_show_all_locks+0x226/0x226 [ 315.147499] [<ffffffff8122ab49>] ? compact_zone+0x1155/0x1155 [ 315.147515] [<ffffffff810d58d1>] ? finish_task_switch+0x3de/0x484 [ 315.147533] [<ffffffff8122bcff>] try_to_compact_pages+0x2f1/0x648 [ 315.147550] [<ffffffff8122bcff>] ? try_to_compact_pages+0x2f1/0x648 [ 315.147568] [<ffffffff8122ba0e>] ? compaction_zonelist_suitable+0x3a6/0x3a6 [ 315.147589] [<ffffffff811ee129>] ? get_page_from_freelist+0x2c0/0x129a [ 315.147608] [<ffffffff811ef1ed>] __alloc_pages_direct_compact+0xea/0x30d [ 315.147626] [<ffffffff811ef103>] ? get_page_from_freelist+0x129a/0x129a [ 315.147645] [<ffffffff811f0422>] __alloc_pages_nodemask+0x840/0x16b6 [ 315.147663] [<ffffffff810dba27>] ? try_to_wake_up+0x696/0x6c8 [ 315.149147] [<ffffffff811efbe2>] ? warn_alloc_failed+0x226/0x226 [ 315.150615] [<ffffffff810dba69>] ? wake_up_process+0x10/0x12 [ 315.152078] [<ffffffff810dbaf4>] ? wake_up_q+0x89/0xa7 [ 315.153539] [<ffffffff81128b6f>] ? rwsem_wake+0x131/0x15c [ 315.155007] [<ffffffff812922e7>] ? khugepaged+0x4072/0x484f [ 315.156471] [<ffffffff8128e449>] khugepaged+0x1d4/0x484f [ 315.157940] [<ffffffff8128e275>] ? hugepage_vma_revalidate+0xef/0xef [ 315.159402] [<ffffffff810d58d1>] ? finish_task_switch+0x3de/0x484 [ 315.160870] [<ffffffff81d31df8>] ? _raw_spin_unlock_irq+0x27/0x45 [ 315.162341] [<ffffffff8111cde6>] ? trace_hardirqs_on_caller+0x3d2/0x492 [ 315.163814] [<ffffffff8111112e>] ? prepare_to_wait_event+0x3f7/0x3f7 [ 315.165295] [<ffffffff81d27ad5>] ? __schedule+0xa4d/0xd16 [ 315.166763] [<ffffffff810ccde3>] kthread+0x252/0x261 [ 315.168214] [<ffffffff8128e275>] ? hugepage_vma_revalidate+0xef/0xef [ 315.169646] [<ffffffff810ccb91>] ? kthread_create_on_node+0x377/0x377 [ 315.171056] [<ffffffff81d3277f>] ret_from_fork+0x1f/0x40 [ 315.172462] [<ffffffff810ccb91>] ? kthread_create_on_node+0x377/0x377 [ 315.173869] Code: 03 b5 60 fe ff ff e8 2e fc ff ff a8 01 74 4c 48 83 e0 fe bf 01 00 00 00 48 89 85 38 fe ff ff e8 41 18 e1 e0 48 8b 85 38 fe ff ff <f0> 0f ba 28 00 73 29 bf 01 00 00 00 41 bc f5 ff ff ff e8 ea 27 [ 315.175573] RIP [<ffffffffa02c413d>] zs_page_migrate+0x355/0xaa0 [zsmalloc] [ 315.177084] RSP <ffff88011246f138> [ 315.186572] ---[ end trace 0962b8ee48c98bbc ]--- [ 315.186577] BUG: sleeping function called from invalid context at include/linux/sched.h:2960 [ 315.186580] in_atomic(): 1, irqs_disabled(): 0, pid: 38, name: khugepaged [ 315.186581] INFO: lockdep is turned off. [ 315.186583] Preemption disabled at:[<ffffffffa02c3f1d>] zs_page_migrate+0x135/0xaa0 [zsmalloc] [ 315.186594] CPU: 3 PID: 38 Comm: khugepaged Tainted: G D 4.7.0-rc3-next-20160614-dbg-00004-ga1c2cbc-dirty #488 [ 315.186599] 0000000000000000 ffff88011246ed58 ffffffff814d56bf ffff8800bfaf2900 [ 315.186604] 0000000000000004 ffff88011246ed98 ffffffff810d5e6a 0000000000000000 [ 315.186609] ffff8800bfaf2900 ffffffff81e39820 0000000000000b90 0000000000000000 [ 315.186614] Call Trace: [ 315.186618] [<ffffffff814d56bf>] dump_stack+0x68/0x92 [ 315.186622] [<ffffffff810d5e6a>] ___might_sleep+0x3bd/0x3c9 [ 315.186625] [<ffffffff810d5fd1>] __might_sleep+0x15b/0x167 [ 315.186630] [<ffffffff810ac4c1>] exit_signals+0x7a/0x34f [ 315.186633] [<ffffffff810ac447>] ? get_signal+0xd9b/0xd9b [ 315.186636] [<ffffffff811aee21>] ? irq_work_queue+0x101/0x11c [ 315.186640] [<ffffffff8111f0ac>] ? debug_show_all_locks+0x226/0x226 [ 315.186645] [<ffffffff81096357>] do_exit+0x34d/0x1b4e [ 315.186648] [<ffffffff81130e16>] ? vprintk_emit+0x4b1/0x4d3 [ 315.186652] [<ffffffff8109600a>] ? is_current_pgrp_orphaned+0x8c/0x8c [ 315.186655] [<ffffffff81122c56>] ? lock_acquire+0xec/0x147 [ 315.186658] [<ffffffff811321ef>] ? kmsg_dump+0x12/0x27a [ 315.186662] [<ffffffff81132448>] ? kmsg_dump+0x26b/0x27a [ 315.186666] [<ffffffff81036507>] oops_end+0x9d/0xa4 [ 315.186669] [<ffffffff8103662c>] die+0x55/0x5e [ 315.186672] [<ffffffff81032aa0>] do_general_protection+0x16c/0x337 [ 315.186676] [<ffffffff81d33abf>] general_protection+0x1f/0x30 [ 315.186681] [<ffffffffa02c413d>] ? zs_page_migrate+0x355/0xaa0 [zsmalloc] [ 315.186686] [<ffffffffa02c4136>] ? zs_page_migrate+0x34e/0xaa0 [zsmalloc] [ 315.186691] [<ffffffffa02c3de8>] ? obj_to_head+0x9d/0x9d [zsmalloc] [ 315.186695] [<ffffffff81d31dbc>] ? _raw_spin_unlock_irqrestore+0x47/0x5c [ 315.186699] [<ffffffff812275b1>] ? isolate_freepages_block+0x2f9/0x5a6 [ 315.186702] [<ffffffff8127f15c>] ? kasan_poison_shadow+0x2f/0x31 [ 315.186706] [<ffffffff8127f66a>] ? kasan_alloc_pages+0x39/0x3b [ 315.186709] [<ffffffff812267e6>] ? map_pages+0x1f3/0x3ad [ 315.186712] [<ffffffff812265f3>] ? update_pageblock_skip+0x18d/0x18d [ 315.186716] [<ffffffff81115972>] ? up_read+0x1a/0x30 [ 315.186719] [<ffffffff8111ec7e>] ? debug_check_no_locks_freed+0x150/0x22b [ 315.186723] [<ffffffff812842d1>] move_to_new_page+0x4dd/0x615 [ 315.186726] [<ffffffff81283df4>] ? migrate_page+0x75/0x75 [ 315.186730] [<ffffffff8122785e>] ? isolate_freepages_block+0x5a6/0x5a6 [ 315.186733] [<ffffffff812851c1>] migrate_pages+0xadd/0x131a [ 315.186737] [<ffffffff8122785e>] ? isolate_freepages_block+0x5a6/0x5a6 [ 315.186740] [<ffffffff81226375>] ? kzfree+0x2b/0x2b [ 315.186743] [<ffffffff812846e4>] ? buffer_migrate_page+0x2db/0x2db [ 315.186747] [<ffffffff8122a6cf>] compact_zone+0xcdb/0x1155 [ 315.186751] [<ffffffff812299f4>] ? compaction_suitable+0x76/0x76 [ 315.186754] [<ffffffff8122ac29>] compact_zone_order+0xe0/0x167 [ 315.186757] [<ffffffff8111f0ac>] ? debug_show_all_locks+0x226/0x226 [ 315.186761] [<ffffffff8122ab49>] ? compact_zone+0x1155/0x1155 [ 315.186764] [<ffffffff810d58d1>] ? finish_task_switch+0x3de/0x484 [ 315.186768] [<ffffffff8122bcff>] try_to_compact_pages+0x2f1/0x648 [ 315.186771] [<ffffffff8122bcff>] ? try_to_compact_pages+0x2f1/0x648 [ 315.186775] [<ffffffff8122ba0e>] ? compaction_zonelist_suitable+0x3a6/0x3a6 [ 315.186780] [<ffffffff811ee129>] ? get_page_from_freelist+0x2c0/0x129a [ 315.186783] [<ffffffff811ef1ed>] __alloc_pages_direct_compact+0xea/0x30d [ 315.186787] [<ffffffff811ef103>] ? get_page_from_freelist+0x129a/0x129a [ 315.186791] [<ffffffff811f0422>] __alloc_pages_nodemask+0x840/0x16b6 [ 315.186794] [<ffffffff810dba27>] ? try_to_wake_up+0x696/0x6c8 [ 315.186798] [<ffffffff811efbe2>] ? warn_alloc_failed+0x226/0x226 [ 315.186801] [<ffffffff810dba69>] ? wake_up_process+0x10/0x12 [ 315.186804] [<ffffffff810dbaf4>] ? wake_up_q+0x89/0xa7 [ 315.186807] [<ffffffff81128b6f>] ? rwsem_wake+0x131/0x15c [ 315.186811] [<ffffffff812922e7>] ? khugepaged+0x4072/0x484f [ 315.186815] [<ffffffff8128e449>] khugepaged+0x1d4/0x484f [ 315.186819] [<ffffffff8128e275>] ? hugepage_vma_revalidate+0xef/0xef [ 315.186822] [<ffffffff810d58d1>] ? finish_task_switch+0x3de/0x484 [ 315.186826] [<ffffffff81d31df8>] ? _raw_spin_unlock_irq+0x27/0x45 [ 315.186829] [<ffffffff8111cde6>] ? trace_hardirqs_on_caller+0x3d2/0x492 [ 315.186832] [<ffffffff8111112e>] ? prepare_to_wait_event+0x3f7/0x3f7 [ 315.186836] [<ffffffff81d27ad5>] ? __schedule+0xa4d/0xd16 [ 315.186840] [<ffffffff810ccde3>] kthread+0x252/0x261 [ 315.186843] [<ffffffff8128e275>] ? hugepage_vma_revalidate+0xef/0xef [ 315.186846] [<ffffffff810ccb91>] ? kthread_create_on_node+0x377/0x377 [ 315.186851] [<ffffffff81d3277f>] ret_from_fork+0x1f/0x40 [ 315.186854] [<ffffffff810ccb91>] ? kthread_create_on_node+0x377/0x377 [ 315.186869] note: khugepaged[38] exited with preempt_count 4 [ 340.319852] NMI watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [jbd2/zram0-8:405] [ 340.319856] Modules linked in: lzo zram zsmalloc mousedev coretemp hwmon crc32c_intel r8169 i2c_i801 mii snd_hda_codec_realtek snd_hda_codec_generic snd_hda_intel snd_hda_codec snd_hda_core acpi_cpufreq snd_pcm snd_timer snd soundcore lpc_ich mfd_core processor sch_fq_codel sd_mod hid_generic usbhid hid ahci libahci libata ehci_pci ehci_hcd scsi_mod usbcore usb_common [ 340.319900] irq event stamp: 834296 [ 340.319902] hardirqs last enabled at (834295): [<ffffffff81280b07>] quarantine_put+0xa1/0xe6 [ 340.319911] hardirqs last disabled at (834296): [<ffffffff81d31e68>] _raw_write_lock_irqsave+0x13/0x4c [ 340.319917] softirqs last enabled at (833836): [<ffffffff81d3455e>] __do_softirq+0x406/0x48f [ 340.319922] softirqs last disabled at (833831): [<ffffffff8109914a>] irq_exit+0x6a/0x113 [ 340.319929] CPU: 2 PID: 405 Comm: jbd2/zram0-8 Tainted: G D 4.7.0-rc3-next-20160614-dbg-00004-ga1c2cbc-dirty #488 [ 340.319935] task: ffff8800bb512900 ti: ffff8800a69c0000 task.ti: ffff8800a69c0000 [ 340.319937] RIP: 0010:[<ffffffff814ed772>] [<ffffffff814ed772>] delay_tsc+0x0/0xa4 [ 340.319943] RSP: 0018:ffff8800a69c70f8 EFLAGS: 00000206 [ 340.319945] RAX: 0000000000000001 RBX: ffff8800aa91f300 RCX: 0000000000000000 [ 340.319947] RDX: 0000000000000003 RSI: ffffffff81ed2840 RDI: 0000000000000001 [ 340.319949] RBP: ffff8800a69c7100 R08: 0000000000000001 R09: 0000000000000000 [ 340.319951] R10: ffff8800a69c70e8 R11: 000000007e7516b9 R12: ffff8800aa91f310 [ 340.319954] R13: ffff8800aa91f308 R14: 000000001f3306fa R15: 0000000000000000 [ 340.319956] FS: 0000000000000000(0000) GS:ffff880113700000(0000) knlGS:0000000000000000 [ 340.319959] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 340.319961] CR2: 00007fc99caba080 CR3: 00000000b9796000 CR4: 00000000000006e0 [ 340.319963] Stack: [ 340.319964] ffffffff814ed89c ffff8800a69c7148 ffffffff8112795d ffffed0015523e60 [ 340.319970] 000000009e857390 ffff8800aa91f300 ffff8800bbe21cc0 ffff8800047d6f80 [ 340.319975] ffff8800a69c72b0 ffff8800aa91f300 ffff8800a69c7168 ffffffff81d31bed [ 340.319980] Call Trace: [ 340.319983] [<ffffffff814ed89c>] ? __delay+0xa/0xc [ 340.319988] [<ffffffff8112795d>] do_raw_spin_lock+0x197/0x257 [ 340.319991] [<ffffffff81d31bed>] _raw_spin_lock+0x35/0x3c [ 340.319998] [<ffffffffa02c6062>] ? zs_free+0x191/0x27a [zsmalloc] [ 340.320003] [<ffffffffa02c6062>] zs_free+0x191/0x27a [zsmalloc] [ 340.320008] [<ffffffffa02c5ed1>] ? free_zspage+0xe8/0xe8 [zsmalloc] [ 340.320012] [<ffffffff810d58d1>] ? finish_task_switch+0x3de/0x484 [ 340.320015] [<ffffffff810d58a6>] ? finish_task_switch+0x3b3/0x484 [ 340.320021] [<ffffffff81d27ad5>] ? __schedule+0xa4d/0xd16 [ 340.320024] [<ffffffff81d28086>] ? preempt_schedule+0x1f/0x21 [ 340.320028] [<ffffffff81d27ff9>] ? preempt_schedule_common+0xb7/0xe8 [ 340.320034] [<ffffffffa02d3f0e>] zram_free_page+0x112/0x1f6 [zram] [ 340.320039] [<ffffffffa02d5e6c>] zram_make_request+0x45d/0x89f [zram] [ 340.320045] [<ffffffffa02d5a0f>] ? zram_rw_page+0x21d/0x21d [zram] [ 340.320048] [<ffffffff81493657>] ? blk_exit_rl+0x39/0x39 [ 340.320053] [<ffffffff8148fe3f>] ? handle_bad_sector+0x192/0x192 [ 340.320056] [<ffffffff8127f83e>] ? kasan_slab_alloc+0x12/0x14 [ 340.320059] [<ffffffff8127ca68>] ? kmem_cache_alloc+0xf3/0x101 [ 340.320062] [<ffffffff81494e37>] generic_make_request+0x2bc/0x496 [ 340.320066] [<ffffffff81494b7b>] ? blk_plug_queued_count+0x103/0x103 [ 340.320069] [<ffffffff8111ec7e>] ? debug_check_no_locks_freed+0x150/0x22b [ 340.320072] [<ffffffff81495309>] submit_bio+0x2f8/0x324 [ 340.320075] [<ffffffff81495011>] ? generic_make_request+0x496/0x496 [ 340.320078] [<ffffffff811190fc>] ? lockdep_init_map+0x1ef/0x4b0 [ 340.320082] [<ffffffff814880a4>] submit_bio_wait+0xff/0x138 [ 340.320085] [<ffffffff81487fa5>] ? bio_add_page+0x292/0x292 [ 340.320090] [<ffffffff814ab82c>] blkdev_issue_discard+0xee/0x148 [ 340.320093] [<ffffffff814ab73e>] ? __blkdev_issue_discard+0x399/0x399 [ 340.320097] [<ffffffff8111f0ac>] ? debug_show_all_locks+0x226/0x226 [ 340.320101] [<ffffffff81404de8>] ext4_free_data_callback+0x2cc/0x8bc [ 340.320104] [<ffffffff81404de8>] ? ext4_free_data_callback+0x2cc/0x8bc [ 340.320107] [<ffffffff81404b1c>] ? ext4_mb_release_context+0x10aa/0x10aa [ 340.320111] [<ffffffff81122c56>] ? lock_acquire+0xec/0x147 [ 340.320115] [<ffffffff813c8a6a>] ? ext4_journal_commit_callback+0x203/0x220 [ 340.320119] [<ffffffff813c8a61>] ext4_journal_commit_callback+0x1fa/0x220 [ 340.320124] [<ffffffff81438bf5>] jbd2_journal_commit_transaction+0x3753/0x3c20 [ 340.320128] [<ffffffff814354a2>] ? journal_submit_commit_record+0x777/0x777 [ 340.320132] [<ffffffff8111f0ac>] ? debug_show_all_locks+0x226/0x226 [ 340.320135] [<ffffffff811205a5>] ? __lock_acquire+0x14f9/0x33b8 [ 340.320139] [<ffffffff81d31db0>] ? _raw_spin_unlock_irqrestore+0x3b/0x5c [ 340.320143] [<ffffffff8111cde6>] ? trace_hardirqs_on_caller+0x3d2/0x492 [ 340.320146] [<ffffffff81d31dbc>] ? _raw_spin_unlock_irqrestore+0x47/0x5c [ 340.320151] [<ffffffff81156945>] ? try_to_del_timer_sync+0xa5/0xce [ 340.320154] [<ffffffff8111cde6>] ? trace_hardirqs_on_caller+0x3d2/0x492 [ 340.320157] [<ffffffff8143febd>] kjournald2+0x246/0x6e1 [ 340.320160] [<ffffffff8143febd>] ? kjournald2+0x246/0x6e1 [ 340.320163] [<ffffffff8143fc77>] ? commit_timeout+0xb/0xb [ 340.320167] [<ffffffff8111112e>] ? prepare_to_wait_event+0x3f7/0x3f7 [ 340.320171] [<ffffffff810ccde3>] kthread+0x252/0x261 [ 340.320174] [<ffffffff8143fc77>] ? commit_timeout+0xb/0xb [ 340.320177] [<ffffffff810ccb91>] ? kthread_create_on_node+0x377/0x377 [ 340.320181] [<ffffffff81d3277f>] ret_from_fork+0x1f/0x40 [ 340.320185] [<ffffffff810ccb91>] ? kthread_create_on_node+0x377/0x377 [ 340.320186] Code: 5c 5d c3 55 48 8d 04 bd 00 00 00 00 65 48 8b 15 8d 59 b2 7e 48 69 d2 fa 00 00 00 48 89 e5 f7 e2 48 8d 7a 01 e8 22 01 00 00 5d c3 <55> 48 89 e5 41 56 41 55 41 54 53 49 89 fd bf 01 00 00 00 e8 ed -ss
[toc] | [next] | [standalone]
| From | Minchan Kim <minchan@kernel.org> |
|---|---|
| Date | 2016-06-16 01:20 +0200 |
| Message-ID | <rKsgi-wr-11@gated-at.bofh.it> |
| In reply to | #1422753 |
Hi Sergey, On Wed, Jun 15, 2016 at 04:59:09PM +0900, Sergey Senozhatsky wrote: > Hello Minchan, > > -next 4.7.0-rc3-next-20160614 > > > [ 315.146533] kasan: CONFIG_KASAN_INLINE enabled > [ 315.146538] kasan: GPF could be caused by NULL-ptr deref or user memory access > [ 315.146546] general protection fault: 0000 [#1] PREEMPT SMP KASAN > [ 315.146576] Modules linked in: lzo zram zsmalloc mousedev coretemp hwmon crc32c_intel r8169 i2c_i801 mii snd_hda_codec_realtek snd_hda_codec_generic snd_hda_intel snd_hda_codec snd_hda_core acpi_cpufreq snd_pcm snd_timer snd soundcore lpc_ich mfd_core processor sch_fq_codel sd_mod hid_generic usbhid hid ahci libahci libata ehci_pci ehci_hcd scsi_mod usbcore usb_common > [ 315.146785] CPU: 3 PID: 38 Comm: khugepaged Not tainted 4.7.0-rc3-next-20160614-dbg-00004-ga1c2cbc-dirty #488 > [ 315.146841] task: ffff8800bfaf2900 ti: ffff880112468000 task.ti: ffff880112468000 > [ 315.146859] RIP: 0010:[<ffffffffa02c413d>] [<ffffffffa02c413d>] zs_page_migrate+0x355/0xaa0 [zsmalloc] Thanks for the report! zs_page_migrate+0x355? Could you tell me what line is it? It seems to be related to obj_to_head. Could you test with [zsmalloc: keep first object offset in struct page] in mmotm? > [ 315.146892] RSP: 0000:ffff88011246f138 EFLAGS: 00010293 > [ 315.146906] RAX: 736761742d6f6e2c RBX: ffff880017ad9a80 RCX: 0000000000000000 > [ 315.146924] RDX: 1ffffffff064d704 RSI: ffff88000511469a RDI: ffffffff8326ba20 > [ 315.146942] RBP: ffff88011246f328 R08: 0000000000000001 R09: 0000000000000000 > [ 315.146959] R10: ffff88011246f0a8 R11: ffff8800bfc07fff R12: ffff88011246f300 > [ 315.146977] R13: ffffed0015523e6f R14: ffff8800aa91f378 R15: ffffea0000144500 > [ 315.146995] FS: 0000000000000000(0000) GS:ffff880113780000(0000) knlGS:0000000000000000 > [ 315.147015] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > [ 315.147030] CR2: 00007f3f97911000 CR3: 0000000002209000 CR4: 00000000000006e0 > [ 315.147046] Stack: > [ 315.147052] 1ffff10015523e0f ffff88011246f240 ffff880005116800 00017f80e0000000 > [ 315.147083] ffff880017ad9aa8 736761742d6f6e2c 1ffff1002248de34 ffff880017ad9a90 > [ 315.147113] 0000069a1246f660 000000000000069a ffff880005114000 ffffea0002ff0180 > [ 315.147143] Call Trace: > [ 315.147154] [<ffffffffa02c3de8>] ? obj_to_head+0x9d/0x9d [zsmalloc] > [ 315.147175] [<ffffffff81d31dbc>] ? _raw_spin_unlock_irqrestore+0x47/0x5c > [ 315.147195] [<ffffffff812275b1>] ? isolate_freepages_block+0x2f9/0x5a6 > [ 315.147213] [<ffffffff8127f15c>] ? kasan_poison_shadow+0x2f/0x31 > [ 315.147230] [<ffffffff8127f66a>] ? kasan_alloc_pages+0x39/0x3b > [ 315.147246] [<ffffffff812267e6>] ? map_pages+0x1f3/0x3ad > [ 315.147262] [<ffffffff812265f3>] ? update_pageblock_skip+0x18d/0x18d > [ 315.147280] [<ffffffff81115972>] ? up_read+0x1a/0x30 > [ 315.147296] [<ffffffff8111ec7e>] ? debug_check_no_locks_freed+0x150/0x22b > [ 315.147315] [<ffffffff812842d1>] move_to_new_page+0x4dd/0x615 > [ 315.147332] [<ffffffff81283df4>] ? migrate_page+0x75/0x75 > [ 315.147347] [<ffffffff8122785e>] ? isolate_freepages_block+0x5a6/0x5a6 > [ 315.147366] [<ffffffff812851c1>] migrate_pages+0xadd/0x131a > [ 315.147382] [<ffffffff8122785e>] ? isolate_freepages_block+0x5a6/0x5a6 > [ 315.147399] [<ffffffff81226375>] ? kzfree+0x2b/0x2b > [ 315.147414] [<ffffffff812846e4>] ? buffer_migrate_page+0x2db/0x2db > [ 315.147431] [<ffffffff8122a6cf>] compact_zone+0xcdb/0x1155 > [ 315.147448] [<ffffffff812299f4>] ? compaction_suitable+0x76/0x76 > [ 315.147465] [<ffffffff8122ac29>] compact_zone_order+0xe0/0x167 > [ 315.147481] [<ffffffff8111f0ac>] ? debug_show_all_locks+0x226/0x226 > [ 315.147499] [<ffffffff8122ab49>] ? compact_zone+0x1155/0x1155 > [ 315.147515] [<ffffffff810d58d1>] ? finish_task_switch+0x3de/0x484 > [ 315.147533] [<ffffffff8122bcff>] try_to_compact_pages+0x2f1/0x648 > [ 315.147550] [<ffffffff8122bcff>] ? try_to_compact_pages+0x2f1/0x648 > [ 315.147568] [<ffffffff8122ba0e>] ? compaction_zonelist_suitable+0x3a6/0x3a6 > [ 315.147589] [<ffffffff811ee129>] ? get_page_from_freelist+0x2c0/0x129a > [ 315.147608] [<ffffffff811ef1ed>] __alloc_pages_direct_compact+0xea/0x30d > [ 315.147626] [<ffffffff811ef103>] ? get_page_from_freelist+0x129a/0x129a > [ 315.147645] [<ffffffff811f0422>] __alloc_pages_nodemask+0x840/0x16b6 > [ 315.147663] [<ffffffff810dba27>] ? try_to_wake_up+0x696/0x6c8 > [ 315.149147] [<ffffffff811efbe2>] ? warn_alloc_failed+0x226/0x226 > [ 315.150615] [<ffffffff810dba69>] ? wake_up_process+0x10/0x12 > [ 315.152078] [<ffffffff810dbaf4>] ? wake_up_q+0x89/0xa7 > [ 315.153539] [<ffffffff81128b6f>] ? rwsem_wake+0x131/0x15c > [ 315.155007] [<ffffffff812922e7>] ? khugepaged+0x4072/0x484f > [ 315.156471] [<ffffffff8128e449>] khugepaged+0x1d4/0x484f > [ 315.157940] [<ffffffff8128e275>] ? hugepage_vma_revalidate+0xef/0xef > [ 315.159402] [<ffffffff810d58d1>] ? finish_task_switch+0x3de/0x484 > [ 315.160870] [<ffffffff81d31df8>] ? _raw_spin_unlock_irq+0x27/0x45 > [ 315.162341] [<ffffffff8111cde6>] ? trace_hardirqs_on_caller+0x3d2/0x492 > [ 315.163814] [<ffffffff8111112e>] ? prepare_to_wait_event+0x3f7/0x3f7 > [ 315.165295] [<ffffffff81d27ad5>] ? __schedule+0xa4d/0xd16 > [ 315.166763] [<ffffffff810ccde3>] kthread+0x252/0x261 > [ 315.168214] [<ffffffff8128e275>] ? hugepage_vma_revalidate+0xef/0xef > [ 315.169646] [<ffffffff810ccb91>] ? kthread_create_on_node+0x377/0x377 > [ 315.171056] [<ffffffff81d3277f>] ret_from_fork+0x1f/0x40 > [ 315.172462] [<ffffffff810ccb91>] ? kthread_create_on_node+0x377/0x377 > [ 315.173869] Code: 03 b5 60 fe ff ff e8 2e fc ff ff a8 01 74 4c 48 83 e0 fe bf 01 00 00 00 48 89 85 38 fe ff ff e8 41 18 e1 e0 48 8b 85 38 fe ff ff <f0> 0f ba 28 00 73 29 bf 01 00 00 00 41 bc f5 ff ff ff e8 ea 27 > [ 315.175573] RIP [<ffffffffa02c413d>] zs_page_migrate+0x355/0xaa0 [zsmalloc] > [ 315.177084] RSP <ffff88011246f138> > [ 315.186572] ---[ end trace 0962b8ee48c98bbc ]--- > > > > > [ 315.186577] BUG: sleeping function called from invalid context at include/linux/sched.h:2960 > [ 315.186580] in_atomic(): 1, irqs_disabled(): 0, pid: 38, name: khugepaged > [ 315.186581] INFO: lockdep is turned off. > [ 315.186583] Preemption disabled at:[<ffffffffa02c3f1d>] zs_page_migrate+0x135/0xaa0 [zsmalloc] > > [ 315.186594] CPU: 3 PID: 38 Comm: khugepaged Tainted: G D 4.7.0-rc3-next-20160614-dbg-00004-ga1c2cbc-dirty #488 > [ 315.186599] 0000000000000000 ffff88011246ed58 ffffffff814d56bf ffff8800bfaf2900 > [ 315.186604] 0000000000000004 ffff88011246ed98 ffffffff810d5e6a 0000000000000000 > [ 315.186609] ffff8800bfaf2900 ffffffff81e39820 0000000000000b90 0000000000000000 > [ 315.186614] Call Trace: > [ 315.186618] [<ffffffff814d56bf>] dump_stack+0x68/0x92 > [ 315.186622] [<ffffffff810d5e6a>] ___might_sleep+0x3bd/0x3c9 > [ 315.186625] [<ffffffff810d5fd1>] __might_sleep+0x15b/0x167 > [ 315.186630] [<ffffffff810ac4c1>] exit_signals+0x7a/0x34f > [ 315.186633] [<ffffffff810ac447>] ? get_signal+0xd9b/0xd9b > [ 315.186636] [<ffffffff811aee21>] ? irq_work_queue+0x101/0x11c > [ 315.186640] [<ffffffff8111f0ac>] ? debug_show_all_locks+0x226/0x226 > [ 315.186645] [<ffffffff81096357>] do_exit+0x34d/0x1b4e > [ 315.186648] [<ffffffff81130e16>] ? vprintk_emit+0x4b1/0x4d3 > [ 315.186652] [<ffffffff8109600a>] ? is_current_pgrp_orphaned+0x8c/0x8c > [ 315.186655] [<ffffffff81122c56>] ? lock_acquire+0xec/0x147 > [ 315.186658] [<ffffffff811321ef>] ? kmsg_dump+0x12/0x27a > [ 315.186662] [<ffffffff81132448>] ? kmsg_dump+0x26b/0x27a > [ 315.186666] [<ffffffff81036507>] oops_end+0x9d/0xa4 > [ 315.186669] [<ffffffff8103662c>] die+0x55/0x5e > [ 315.186672] [<ffffffff81032aa0>] do_general_protection+0x16c/0x337 > [ 315.186676] [<ffffffff81d33abf>] general_protection+0x1f/0x30 > [ 315.186681] [<ffffffffa02c413d>] ? zs_page_migrate+0x355/0xaa0 [zsmalloc] > [ 315.186686] [<ffffffffa02c4136>] ? zs_page_migrate+0x34e/0xaa0 [zsmalloc] > [ 315.186691] [<ffffffffa02c3de8>] ? obj_to_head+0x9d/0x9d [zsmalloc] > [ 315.186695] [<ffffffff81d31dbc>] ? _raw_spin_unlock_irqrestore+0x47/0x5c > [ 315.186699] [<ffffffff812275b1>] ? isolate_freepages_block+0x2f9/0x5a6 > [ 315.186702] [<ffffffff8127f15c>] ? kasan_poison_shadow+0x2f/0x31 > [ 315.186706] [<ffffffff8127f66a>] ? kasan_alloc_pages+0x39/0x3b > [ 315.186709] [<ffffffff812267e6>] ? map_pages+0x1f3/0x3ad > [ 315.186712] [<ffffffff812265f3>] ? update_pageblock_skip+0x18d/0x18d > [ 315.186716] [<ffffffff81115972>] ? up_read+0x1a/0x30 > [ 315.186719] [<ffffffff8111ec7e>] ? debug_check_no_locks_freed+0x150/0x22b > [ 315.186723] [<ffffffff812842d1>] move_to_new_page+0x4dd/0x615 > [ 315.186726] [<ffffffff81283df4>] ? migrate_page+0x75/0x75 > [ 315.186730] [<ffffffff8122785e>] ? isolate_freepages_block+0x5a6/0x5a6 > [ 315.186733] [<ffffffff812851c1>] migrate_pages+0xadd/0x131a > [ 315.186737] [<ffffffff8122785e>] ? isolate_freepages_block+0x5a6/0x5a6 > [ 315.186740] [<ffffffff81226375>] ? kzfree+0x2b/0x2b > [ 315.186743] [<ffffffff812846e4>] ? buffer_migrate_page+0x2db/0x2db > [ 315.186747] [<ffffffff8122a6cf>] compact_zone+0xcdb/0x1155 > [ 315.186751] [<ffffffff812299f4>] ? compaction_suitable+0x76/0x76 > [ 315.186754] [<ffffffff8122ac29>] compact_zone_order+0xe0/0x167 > [ 315.186757] [<ffffffff8111f0ac>] ? debug_show_all_locks+0x226/0x226 > [ 315.186761] [<ffffffff8122ab49>] ? compact_zone+0x1155/0x1155 > [ 315.186764] [<ffffffff810d58d1>] ? finish_task_switch+0x3de/0x484 > [ 315.186768] [<ffffffff8122bcff>] try_to_compact_pages+0x2f1/0x648 > [ 315.186771] [<ffffffff8122bcff>] ? try_to_compact_pages+0x2f1/0x648 > [ 315.186775] [<ffffffff8122ba0e>] ? compaction_zonelist_suitable+0x3a6/0x3a6 > [ 315.186780] [<ffffffff811ee129>] ? get_page_from_freelist+0x2c0/0x129a > [ 315.186783] [<ffffffff811ef1ed>] __alloc_pages_direct_compact+0xea/0x30d > [ 315.186787] [<ffffffff811ef103>] ? get_page_from_freelist+0x129a/0x129a > [ 315.186791] [<ffffffff811f0422>] __alloc_pages_nodemask+0x840/0x16b6 > [ 315.186794] [<ffffffff810dba27>] ? try_to_wake_up+0x696/0x6c8 > [ 315.186798] [<ffffffff811efbe2>] ? warn_alloc_failed+0x226/0x226 > [ 315.186801] [<ffffffff810dba69>] ? wake_up_process+0x10/0x12 > [ 315.186804] [<ffffffff810dbaf4>] ? wake_up_q+0x89/0xa7 > [ 315.186807] [<ffffffff81128b6f>] ? rwsem_wake+0x131/0x15c > [ 315.186811] [<ffffffff812922e7>] ? khugepaged+0x4072/0x484f > [ 315.186815] [<ffffffff8128e449>] khugepaged+0x1d4/0x484f > [ 315.186819] [<ffffffff8128e275>] ? hugepage_vma_revalidate+0xef/0xef > [ 315.186822] [<ffffffff810d58d1>] ? finish_task_switch+0x3de/0x484 > [ 315.186826] [<ffffffff81d31df8>] ? _raw_spin_unlock_irq+0x27/0x45 > [ 315.186829] [<ffffffff8111cde6>] ? trace_hardirqs_on_caller+0x3d2/0x492 > [ 315.186832] [<ffffffff8111112e>] ? prepare_to_wait_event+0x3f7/0x3f7 > [ 315.186836] [<ffffffff81d27ad5>] ? __schedule+0xa4d/0xd16 > [ 315.186840] [<ffffffff810ccde3>] kthread+0x252/0x261 > [ 315.186843] [<ffffffff8128e275>] ? hugepage_vma_revalidate+0xef/0xef > [ 315.186846] [<ffffffff810ccb91>] ? kthread_create_on_node+0x377/0x377 > [ 315.186851] [<ffffffff81d3277f>] ret_from_fork+0x1f/0x40 > [ 315.186854] [<ffffffff810ccb91>] ? kthread_create_on_node+0x377/0x377 > [ 315.186869] note: khugepaged[38] exited with preempt_count 4 > > > > [ 340.319852] NMI watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [jbd2/zram0-8:405] > [ 340.319856] Modules linked in: lzo zram zsmalloc mousedev coretemp hwmon crc32c_intel r8169 i2c_i801 mii snd_hda_codec_realtek snd_hda_codec_generic snd_hda_intel snd_hda_codec snd_hda_core acpi_cpufreq snd_pcm snd_timer snd soundcore lpc_ich mfd_core processor sch_fq_codel sd_mod hid_generic usbhid hid ahci libahci libata ehci_pci ehci_hcd scsi_mod usbcore usb_common > [ 340.319900] irq event stamp: 834296 > [ 340.319902] hardirqs last enabled at (834295): [<ffffffff81280b07>] quarantine_put+0xa1/0xe6 > [ 340.319911] hardirqs last disabled at (834296): [<ffffffff81d31e68>] _raw_write_lock_irqsave+0x13/0x4c > [ 340.319917] softirqs last enabled at (833836): [<ffffffff81d3455e>] __do_softirq+0x406/0x48f > [ 340.319922] softirqs last disabled at (833831): [<ffffffff8109914a>] irq_exit+0x6a/0x113 > [ 340.319929] CPU: 2 PID: 405 Comm: jbd2/zram0-8 Tainted: G D 4.7.0-rc3-next-20160614-dbg-00004-ga1c2cbc-dirty #488 > [ 340.319935] task: ffff8800bb512900 ti: ffff8800a69c0000 task.ti: ffff8800a69c0000 > [ 340.319937] RIP: 0010:[<ffffffff814ed772>] [<ffffffff814ed772>] delay_tsc+0x0/0xa4 > [ 340.319943] RSP: 0018:ffff8800a69c70f8 EFLAGS: 00000206 > [ 340.319945] RAX: 0000000000000001 RBX: ffff8800aa91f300 RCX: 0000000000000000 > [ 340.319947] RDX: 0000000000000003 RSI: ffffffff81ed2840 RDI: 0000000000000001 > [ 340.319949] RBP: ffff8800a69c7100 R08: 0000000000000001 R09: 0000000000000000 > [ 340.319951] R10: ffff8800a69c70e8 R11: 000000007e7516b9 R12: ffff8800aa91f310 > [ 340.319954] R13: ffff8800aa91f308 R14: 000000001f3306fa R15: 0000000000000000 > [ 340.319956] FS: 0000000000000000(0000) GS:ffff880113700000(0000) knlGS:0000000000000000 > [ 340.319959] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > [ 340.319961] CR2: 00007fc99caba080 CR3: 00000000b9796000 CR4: 00000000000006e0 > [ 340.319963] Stack: > [ 340.319964] ffffffff814ed89c ffff8800a69c7148 ffffffff8112795d ffffed0015523e60 > [ 340.319970] 000000009e857390 ffff8800aa91f300 ffff8800bbe21cc0 ffff8800047d6f80 > [ 340.319975] ffff8800a69c72b0 ffff8800aa91f300 ffff8800a69c7168 ffffffff81d31bed > [ 340.319980] Call Trace: > [ 340.319983] [<ffffffff814ed89c>] ? __delay+0xa/0xc > [ 340.319988] [<ffffffff8112795d>] do_raw_spin_lock+0x197/0x257 > [ 340.319991] [<ffffffff81d31bed>] _raw_spin_lock+0x35/0x3c > [ 340.319998] [<ffffffffa02c6062>] ? zs_free+0x191/0x27a [zsmalloc] > [ 340.320003] [<ffffffffa02c6062>] zs_free+0x191/0x27a [zsmalloc] > [ 340.320008] [<ffffffffa02c5ed1>] ? free_zspage+0xe8/0xe8 [zsmalloc] > [ 340.320012] [<ffffffff810d58d1>] ? finish_task_switch+0x3de/0x484 > [ 340.320015] [<ffffffff810d58a6>] ? finish_task_switch+0x3b3/0x484 > [ 340.320021] [<ffffffff81d27ad5>] ? __schedule+0xa4d/0xd16 > [ 340.320024] [<ffffffff81d28086>] ? preempt_schedule+0x1f/0x21 > [ 340.320028] [<ffffffff81d27ff9>] ? preempt_schedule_common+0xb7/0xe8 > [ 340.320034] [<ffffffffa02d3f0e>] zram_free_page+0x112/0x1f6 [zram] > [ 340.320039] [<ffffffffa02d5e6c>] zram_make_request+0x45d/0x89f [zram] > [ 340.320045] [<ffffffffa02d5a0f>] ? zram_rw_page+0x21d/0x21d [zram] > [ 340.320048] [<ffffffff81493657>] ? blk_exit_rl+0x39/0x39 > [ 340.320053] [<ffffffff8148fe3f>] ? handle_bad_sector+0x192/0x192 > [ 340.320056] [<ffffffff8127f83e>] ? kasan_slab_alloc+0x12/0x14 > [ 340.320059] [<ffffffff8127ca68>] ? kmem_cache_alloc+0xf3/0x101 > [ 340.320062] [<ffffffff81494e37>] generic_make_request+0x2bc/0x496 > [ 340.320066] [<ffffffff81494b7b>] ? blk_plug_queued_count+0x103/0x103 > [ 340.320069] [<ffffffff8111ec7e>] ? debug_check_no_locks_freed+0x150/0x22b > [ 340.320072] [<ffffffff81495309>] submit_bio+0x2f8/0x324 > [ 340.320075] [<ffffffff81495011>] ? generic_make_request+0x496/0x496 > [ 340.320078] [<ffffffff811190fc>] ? lockdep_init_map+0x1ef/0x4b0 > [ 340.320082] [<ffffffff814880a4>] submit_bio_wait+0xff/0x138 > [ 340.320085] [<ffffffff81487fa5>] ? bio_add_page+0x292/0x292 > [ 340.320090] [<ffffffff814ab82c>] blkdev_issue_discard+0xee/0x148 > [ 340.320093] [<ffffffff814ab73e>] ? __blkdev_issue_discard+0x399/0x399 > [ 340.320097] [<ffffffff8111f0ac>] ? debug_show_all_locks+0x226/0x226 > [ 340.320101] [<ffffffff81404de8>] ext4_free_data_callback+0x2cc/0x8bc > [ 340.320104] [<ffffffff81404de8>] ? ext4_free_data_callback+0x2cc/0x8bc > [ 340.320107] [<ffffffff81404b1c>] ? ext4_mb_release_context+0x10aa/0x10aa > [ 340.320111] [<ffffffff81122c56>] ? lock_acquire+0xec/0x147 > [ 340.320115] [<ffffffff813c8a6a>] ? ext4_journal_commit_callback+0x203/0x220 > [ 340.320119] [<ffffffff813c8a61>] ext4_journal_commit_callback+0x1fa/0x220 > [ 340.320124] [<ffffffff81438bf5>] jbd2_journal_commit_transaction+0x3753/0x3c20 > [ 340.320128] [<ffffffff814354a2>] ? journal_submit_commit_record+0x777/0x777 > [ 340.320132] [<ffffffff8111f0ac>] ? debug_show_all_locks+0x226/0x226 > [ 340.320135] [<ffffffff811205a5>] ? __lock_acquire+0x14f9/0x33b8 > [ 340.320139] [<ffffffff81d31db0>] ? _raw_spin_unlock_irqrestore+0x3b/0x5c > [ 340.320143] [<ffffffff8111cde6>] ? trace_hardirqs_on_caller+0x3d2/0x492 > [ 340.320146] [<ffffffff81d31dbc>] ? _raw_spin_unlock_irqrestore+0x47/0x5c > [ 340.320151] [<ffffffff81156945>] ? try_to_del_timer_sync+0xa5/0xce > [ 340.320154] [<ffffffff8111cde6>] ? trace_hardirqs_on_caller+0x3d2/0x492 > [ 340.320157] [<ffffffff8143febd>] kjournald2+0x246/0x6e1 > [ 340.320160] [<ffffffff8143febd>] ? kjournald2+0x246/0x6e1 > [ 340.320163] [<ffffffff8143fc77>] ? commit_timeout+0xb/0xb > [ 340.320167] [<ffffffff8111112e>] ? prepare_to_wait_event+0x3f7/0x3f7 > [ 340.320171] [<ffffffff810ccde3>] kthread+0x252/0x261 > [ 340.320174] [<ffffffff8143fc77>] ? commit_timeout+0xb/0xb > [ 340.320177] [<ffffffff810ccb91>] ? kthread_create_on_node+0x377/0x377 > [ 340.320181] [<ffffffff81d3277f>] ret_from_fork+0x1f/0x40 > [ 340.320185] [<ffffffff810ccb91>] ? kthread_create_on_node+0x377/0x377 > [ 340.320186] Code: 5c 5d c3 55 48 8d 04 bd 00 00 00 00 65 48 8b 15 8d 59 b2 7e 48 69 d2 fa 00 00 00 48 89 e5 f7 e2 48 8d 7a 01 e8 22 01 00 00 5d c3 <55> 48 89 e5 41 56 41 55 41 54 53 49 89 fd bf 01 00 00 00 e8 ed > > -ss
[toc] | [prev] | [next] | [standalone]
| From | Sergey Senozhatsky <sergey.senozhatsky.work@gmail.com> |
|---|---|
| Date | 2016-06-16 04:50 +0200 |
| Message-ID | <rKvxv-2y1-1@gated-at.bofh.it> |
| In reply to | #1423574 |
Hi,
On (06/16/16 08:12), Minchan Kim wrote:
> > [ 315.146533] kasan: CONFIG_KASAN_INLINE enabled
> > [ 315.146538] kasan: GPF could be caused by NULL-ptr deref or user memory access
> > [ 315.146546] general protection fault: 0000 [#1] PREEMPT SMP KASAN
> > [ 315.146576] Modules linked in: lzo zram zsmalloc mousedev coretemp hwmon crc32c_intel r8169 i2c_i801 mii snd_hda_codec_realtek snd_hda_codec_generic snd_hda_intel snd_hda_codec snd_hda_core acpi_cpufreq snd_pcm snd_timer snd soundcore lpc_ich mfd_core processor sch_fq_codel sd_mod hid_generic usbhid hid ahci libahci libata ehci_pci ehci_hcd scsi_mod usbcore usb_common
> > [ 315.146785] CPU: 3 PID: 38 Comm: khugepaged Not tainted 4.7.0-rc3-next-20160614-dbg-00004-ga1c2cbc-dirty #488
> > [ 315.146841] task: ffff8800bfaf2900 ti: ffff880112468000 task.ti: ffff880112468000
> > [ 315.146859] RIP: 0010:[<ffffffffa02c413d>] [<ffffffffa02c413d>] zs_page_migrate+0x355/0xaa0 [zsmalloc]
>
> Thanks for the report!
>
> zs_page_migrate+0x355? Could you tell me what line is it?
>
> It seems to be related to obj_to_head.
reproduced. a bit different call stack this time. but the problem is
still the same.
zs_compact()
...
6371: e8 00 00 00 00 callq 6376 <zs_compact+0x22b>
6376: 0f 0b ud2
6378: 48 8b 95 a8 fe ff ff mov -0x158(%rbp),%rdx
637f: 4d 8d 74 24 78 lea 0x78(%r12),%r14
6384: 4c 89 ee mov %r13,%rsi
6387: 4c 89 e7 mov %r12,%rdi
638a: e8 86 c7 ff ff callq 2b15 <get_first_obj_offset>
638f: 41 89 c5 mov %eax,%r13d
6392: 4c 89 f0 mov %r14,%rax
6395: 48 c1 e8 03 shr $0x3,%rax
6399: 8a 04 18 mov (%rax,%rbx,1),%al
639c: 84 c0 test %al,%al
639e: 0f 85 f2 02 00 00 jne 6696 <zs_compact+0x54b>
63a4: 41 8b 44 24 78 mov 0x78(%r12),%eax
63a9: 41 0f af c7 imul %r15d,%eax
63ad: 41 01 c5 add %eax,%r13d
63b0: 4c 89 f0 mov %r14,%rax
63b3: 48 c1 e8 03 shr $0x3,%rax
63b7: 48 01 d8 add %rbx,%rax
63ba: 48 89 85 88 fe ff ff mov %rax,-0x178(%rbp)
63c1: 41 81 fd ff 0f 00 00 cmp $0xfff,%r13d
63c8: 0f 87 1a 03 00 00 ja 66e8 <zs_compact+0x59d>
63ce: 49 63 f5 movslq %r13d,%rsi
63d1: 48 03 b5 98 fe ff ff add -0x168(%rbp),%rsi
63d8: 48 8b bd a8 fe ff ff mov -0x158(%rbp),%rdi
63df: e8 67 d9 ff ff callq 3d4b <obj_to_head>
63e4: a8 01 test $0x1,%al
63e6: 0f 84 d9 02 00 00 je 66c5 <zs_compact+0x57a>
63ec: 48 83 e0 fe and $0xfffffffffffffffe,%rax
63f0: bf 01 00 00 00 mov $0x1,%edi
63f5: 48 89 85 b0 fe ff ff mov %rax,-0x150(%rbp)
63fc: e8 00 00 00 00 callq 6401 <zs_compact+0x2b6>
6401: 48 8b 85 b0 fe ff ff mov -0x150(%rbp),%rax
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
6408: f0 0f ba 28 00 lock btsl $0x0,(%rax)
640d: 0f 82 98 02 00 00 jb 66ab <zs_compact+0x560>
6413: 48 8b 85 10 fe ff ff mov -0x1f0(%rbp),%rax
641a: 48 8d b8 48 10 00 00 lea 0x1048(%rax),%rdi
6421: 48 89 f8 mov %rdi,%rax
6424: 48 c1 e8 03 shr $0x3,%rax
6428: 8a 04 18 mov (%rax,%rbx,1),%al
642b: 84 c0 test %al,%al
642d: 0f 85 c5 02 00 00 jne 66f8 <zs_compact+0x5ad>
6433: 48 8b 85 10 fe ff ff mov -0x1f0(%rbp),%rax
643a: 65 4c 8b 2c 25 00 00 mov %gs:0x0,%r13
6441: 00 00
6443: 49 8d bd 48 10 00 00 lea 0x1048(%r13),%rdi
644a: ff 88 48 10 00 00 decl 0x1048(%rax)
6450: 48 89 f8 mov %rdi,%rax
6453: 48 c1 e8 03 shr $0x3,%rax
6457: 8a 04 18 mov (%rax,%rbx,1),%al
645a: 84 c0 test %al,%al
645c: 0f 85 a8 02 00 00 jne 670a <zs_compact+0x5bf>
6462: 41 83 bd 48 10 00 00 cmpl $0x0,0x1048(%r13)
which is
_next/./arch/x86/include/asm/bitops.h:206
_next/./arch/x86/include/asm/bitops.h:219
_next/include/linux/bit_spinlock.h:44
_next/mm/zsmalloc.c:950
_next/mm/zsmalloc.c:1774
_next/mm/zsmalloc.c:1809
_next/mm/zsmalloc.c:2306
_next/mm/zsmalloc.c:2346
smells like race conditon.
backtraces:
[ 319.363646] kasan: CONFIG_KASAN_INLINE enabled
[ 319.363650] kasan: GPF could be caused by NULL-ptr deref or user memory access
[ 319.363658] general protection fault: 0000 [#1] PREEMPT SMP KASAN
[ 319.363688] Modules linked in: lzo zram zsmalloc mousedev coretemp hwmon crc32c_intel snd_hda_codec_realtek snd_hda_codec_generic r8169 mii i2c_i801 snd_hda_intel snd_hda_codec snd_hda_core snd_pcm snd_timer acpi_cpufreq snd lpc_ich soundcore mfd_core processor sch_fq_codel sd_mod hid_generic usbhid hid ahci libahci ehci_pci libata ehci_hcd usbcore scsi_mod usb_common
[ 319.363895] CPU: 0 PID: 45 Comm: kswapd0 Not tainted 4.7.0-rc3-next-20160615-dbg-00004-g550dc8a-dirty #490
[ 319.363950] task: ffff8800bfb93d80 ti: ffff880112200000 task.ti: ffff880112200000
[ 319.363968] RIP: 0010:[<ffffffffa03ce408>] [<ffffffffa03ce408>] zs_compact+0x2bd/0xf22 [zsmalloc]
[ 319.364000] RSP: 0018:ffff8801122077f8 EFLAGS: 00010293
[ 319.364014] RAX: 2065676162726166 RBX: dffffc0000000000 RCX: 0000000000000000
[ 319.364032] RDX: 1ffffffff064c504 RSI: ffff88003217c770 RDI: ffffffff83262ae0
[ 319.364049] RBP: ffff880112207a18 R08: 0000000000000001 R09: 0000000000000000
[ 319.364067] R10: ffff880112207768 R11: 00000000a19f2c26 R12: ffff8800a7caab00
[ 319.364085] R13: 0000000000000770 R14: ffff8800a7caab78 R15: 0000000000000000
[ 319.364103] FS: 0000000000000000(0000) GS:ffff880113600000(0000) knlGS:0000000000000000
[ 319.364123] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 319.364138] CR2: 00007fa154633d70 CR3: 00000000b183d000 CR4: 00000000000006f0
[ 319.364154] Stack:
[ 319.364160] ffffed00163d6a81 1ffff10017f729b9 ffff8800bfb944a0 ffffed0017f729b9
[ 319.364191] ffff8800bfb93d80 ffff8800b1eb5408 ffff8800bfb93d80 ffff8800bfb94dc8
[ 319.364222] ffff8800bfb944f8 ffff880000000001 1ffff10022440f1a 0000000041b58ab3
[ 319.364252] Call Trace:
[ 319.364264] [<ffffffff8111f405>] ? debug_show_all_locks+0x226/0x226
[ 319.364284] [<ffffffffa03ce14b>] ? zs_free+0x27a/0x27a [zsmalloc]
[ 319.364303] [<ffffffff812303e3>] ? list_lru_count_one+0x65/0x6d
[ 319.364320] [<ffffffff81122faf>] ? lock_acquire+0xec/0x147
[ 319.364336] [<ffffffff812303b7>] ? list_lru_count_one+0x39/0x6d
[ 319.364353] [<ffffffff81d32e4f>] ? _raw_spin_unlock+0x2c/0x3f
[ 319.364371] [<ffffffffa03cf0a8>] zs_shrinker_scan+0x3b/0x4e [zsmalloc]
[ 319.364391] [<ffffffff81204eef>] shrink_slab.part.5.constprop.17+0x2e4/0x432
[ 319.364411] [<ffffffff81204c0b>] ? cpu_callback+0xb0/0xb0
[ 319.364426] [<ffffffff8120bfbc>] shrink_zone+0x19b/0x416
[ 319.364442] [<ffffffff8120be21>] ? shrink_zone_memcg.isra.14+0xd08/0xd08
[ 319.364461] [<ffffffff811f0b10>] ? zone_watermark_ok_safe+0x1e9/0x1f8
[ 319.364478] [<ffffffff81205fd7>] ? zone_reclaimable+0x14b/0x170
[ 319.364495] [<ffffffff8120d2fb>] kswapd+0xaad/0xcee
[ 319.364510] [<ffffffff8120c84e>] ? try_to_free_pages+0x617/0x617
[ 319.364527] [<ffffffff8111d13f>] ? trace_hardirqs_on_caller+0x3d2/0x492
[ 319.364545] [<ffffffff81111487>] ? prepare_to_wait_event+0x3f7/0x3f7
[ 319.364564] [<ffffffff810cd0de>] kthread+0x252/0x261
[ 319.364578] [<ffffffff8120c84e>] ? try_to_free_pages+0x617/0x617
[ 319.364595] [<ffffffff810cce8c>] ? kthread_create_on_node+0x377/0x377
[ 319.364614] [<ffffffff81d3387f>] ret_from_fork+0x1f/0x40
[ 319.364629] [<ffffffff810cce8c>] ? kthread_create_on_node+0x377/0x377
[ 319.364645] Code: ff ff e8 67 d9 ff ff a8 01 0f 84 d9 02 00 00 48 83 e0 fe bf 01 00 00 00 48 89 85 b0 fe ff ff e8 71 78 d0 e0 48 8b 85 b0 fe ff ff <f0> 0f ba 28 00 0f 82 98 02 00 00 48 8b 85 10 fe ff ff 48 8d b8
[ 319.364913] RIP [<ffffffffa03ce408>] zs_compact+0x2bd/0xf22 [zsmalloc]
[ 319.364937] RSP <ffff8801122077f8>
[ 319.372870] ---[ end trace bcefd5a456f6b462 ]---
[ 319.372875] BUG: sleeping function called from invalid context at include/linux/sched.h:2960
[ 319.372877] in_atomic(): 1, irqs_disabled(): 0, pid: 45, name: kswapd0
[ 319.372879] INFO: lockdep is turned off.
[ 319.372880] Preemption disabled at:[<ffffffffa03ce2c3>] zs_compact+0x178/0xf22 [zsmalloc]
[ 319.372891] CPU: 0 PID: 45 Comm: kswapd0 Tainted: G D 4.7.0-rc3-next-20160615-dbg-00004-g550dc8a-dirty #490
[ 319.372895] 0000000000000000 ffff880112207418 ffffffff814d69b0 ffff8800bfb93d80
[ 319.372901] 0000000000000003 ffff880112207458 ffffffff810d6165 0000000000000000
[ 319.372906] ffff8800bfb93d80 ffffffff81e39860 0000000000000b90 0000000000000000
[ 319.372911] Call Trace:
[ 319.372915] [<ffffffff814d69b0>] dump_stack+0x68/0x92
[ 319.372919] [<ffffffff810d6165>] ___might_sleep+0x3bd/0x3c9
[ 319.372922] [<ffffffff810d62cc>] __might_sleep+0x15b/0x167
[ 319.372927] [<ffffffff810ac7bf>] exit_signals+0x7a/0x34f
[ 319.372931] [<ffffffff810ac745>] ? get_signal+0xd9b/0xd9b
[ 319.372934] [<ffffffff811af758>] ? irq_work_queue+0x101/0x11c
[ 319.372938] [<ffffffff8111f405>] ? debug_show_all_locks+0x226/0x226
[ 319.372943] [<ffffffff81096655>] do_exit+0x34d/0x1b4e
[ 319.372947] [<ffffffff8113119f>] ? vprintk_emit+0x4b1/0x4d3
[ 319.372951] [<ffffffff81096308>] ? is_current_pgrp_orphaned+0x8c/0x8c
[ 319.372954] [<ffffffff81122faf>] ? lock_acquire+0xec/0x147
[ 319.372957] [<ffffffff81132578>] ? kmsg_dump+0x12/0x27a
[ 319.372961] [<ffffffff811327d1>] ? kmsg_dump+0x26b/0x27a
[ 319.372965] [<ffffffff81036507>] oops_end+0x9d/0xa4
[ 319.372968] [<ffffffff81036641>] die+0x55/0x5e
[ 319.372971] [<ffffffff81032aa0>] do_general_protection+0x16c/0x337
[ 319.372975] [<ffffffff81d34bbf>] general_protection+0x1f/0x30
[ 319.372981] [<ffffffffa03ce408>] ? zs_compact+0x2bd/0xf22 [zsmalloc]
[ 319.372986] [<ffffffffa03ce401>] ? zs_compact+0x2b6/0xf22 [zsmalloc]
[ 319.372989] [<ffffffff8111f405>] ? debug_show_all_locks+0x226/0x226
[ 319.372995] [<ffffffffa03ce14b>] ? zs_free+0x27a/0x27a [zsmalloc]
[ 319.372999] [<ffffffff812303e3>] ? list_lru_count_one+0x65/0x6d
[ 319.373002] [<ffffffff81122faf>] ? lock_acquire+0xec/0x147
[ 319.373005] [<ffffffff812303b7>] ? list_lru_count_one+0x39/0x6d
[ 319.373009] [<ffffffff81d32e4f>] ? _raw_spin_unlock+0x2c/0x3f
[ 319.373014] [<ffffffffa03cf0a8>] zs_shrinker_scan+0x3b/0x4e [zsmalloc]
[ 319.373018] [<ffffffff81204eef>] shrink_slab.part.5.constprop.17+0x2e4/0x432
[ 319.373022] [<ffffffff81204c0b>] ? cpu_callback+0xb0/0xb0
[ 319.373025] [<ffffffff8120bfbc>] shrink_zone+0x19b/0x416
[ 319.373029] [<ffffffff8120be21>] ? shrink_zone_memcg.isra.14+0xd08/0xd08
[ 319.373032] [<ffffffff811f0b10>] ? zone_watermark_ok_safe+0x1e9/0x1f8
[ 319.373036] [<ffffffff81205fd7>] ? zone_reclaimable+0x14b/0x170
[ 319.373039] [<ffffffff8120d2fb>] kswapd+0xaad/0xcee
[ 319.373043] [<ffffffff8120c84e>] ? try_to_free_pages+0x617/0x617
[ 319.373046] [<ffffffff8111d13f>] ? trace_hardirqs_on_caller+0x3d2/0x492
[ 319.373050] [<ffffffff81111487>] ? prepare_to_wait_event+0x3f7/0x3f7
[ 319.373054] [<ffffffff810cd0de>] kthread+0x252/0x261
[ 319.373057] [<ffffffff8120c84e>] ? try_to_free_pages+0x617/0x617
[ 319.373060] [<ffffffff810cce8c>] ? kthread_create_on_node+0x377/0x377
[ 319.373064] [<ffffffff81d3387f>] ret_from_fork+0x1f/0x40
[ 319.373068] [<ffffffff810cce8c>] ? kthread_create_on_node+0x377/0x377
[ 319.373071] note: kswapd0[45] exited with preempt_count 3
[ 322.891083] kmemleak: Cannot allocate a kmemleak_object structure
[ 322.891091] kmemleak: Kernel memory leak detector disabled
[ 322.891194] kmemleak: Automatic memory scanning thread ended
[ 344.264076] NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [kworker/u8:3:108]
[ 344.264080] Modules linked in: lzo zram zsmalloc mousedev coretemp hwmon crc32c_intel snd_hda_codec_realtek snd_hda_codec_generic r8169 mii i2c_i801 snd_hda_intel snd_hda_codec snd_hda_core snd_pcm snd_timer acpi_cpufreq snd lpc_ich soundcore mfd_core processor sch_fq_codel sd_mod hid_generic usbhid hid ahci libahci ehci_pci libata ehci_hcd usbcore scsi_mod usb_common
[ 344.264118] irq event stamp: 13848655
[ 344.264119] hardirqs last enabled at (13848655): [<ffffffff8127dbd8>] __slab_alloc.isra.18.constprop.23+0x53/0x61
[ 344.264127] hardirqs last disabled at (13848654): [<ffffffff8127db9e>] __slab_alloc.isra.18.constprop.23+0x19/0x61
[ 344.264131] softirqs last enabled at (13848614): [<ffffffff81d3565e>] __do_softirq+0x406/0x48f
[ 344.264136] softirqs last disabled at (13848593): [<ffffffff81099448>] irq_exit+0x6a/0x113
[ 344.264143] CPU: 1 PID: 108 Comm: kworker/u8:3 Tainted: G D 4.7.0-rc3-next-20160615-dbg-00004-g550dc8a-dirty #490
[ 344.264151] Workqueue: writeback wb_workfn (flush-254:0)
[ 344.264155] task: ffff8800ba1c2900 ti: ffff8801122a0000 task.ti: ffff8801122a0000
[ 344.264157] RIP: 0010:[<ffffffff814eeae3>] [<ffffffff814eeae3>] delay_tsc+0x81/0xa4
[ 344.264162] RSP: 0018:ffff8801122a70d0 EFLAGS: 00000206
[ 344.264164] RAX: 000000000000001c RBX: 000000dc3a548e47 RCX: 0000000000000000
[ 344.264166] RDX: 000000dc3a548e63 RSI: ffffffff81ed2e80 RDI: ffffffff81ed2ec0
[ 344.264168] RBP: ffff8801122a70f0 R08: 0000000000000001 R09: 0000000000000000
[ 344.264170] R10: ffff8801122a70e8 R11: 0000000045cb5d4f R12: 000000dc3a548e63
[ 344.264172] R13: 0000000000000001 R14: 0000000000000001 R15: 0000000000000000
[ 344.264175] FS: 0000000000000000(0000) GS:ffff880113680000(0000) knlGS:0000000000000000
[ 344.264177] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 344.264179] CR2: 00007fa26a978978 CR3: 0000000002209000 CR4: 00000000000006e0
[ 344.264180] Stack:
[ 344.264181] ffff8800a7caab00 ffff8800a7caab10 ffff8800a7caab08 0000000022af534e
[ 344.264186] ffff8801122a7100 ffffffff814eeb8c ffff8801122a7148 ffffffff81127ce6
[ 344.264191] ffffed0014f95560 000000009e85cd68 ffff8800a7caab00 ffff8800a7caab58
[ 344.264196] Call Trace:
[ 344.264199] [<ffffffff814eeb8c>] __delay+0xa/0xc
[ 344.264203] [<ffffffff81127ce6>] do_raw_spin_lock+0x197/0x257
[ 344.264206] [<ffffffff81d32d0d>] _raw_spin_lock+0x35/0x3c
[ 344.264212] [<ffffffffa03ccd78>] ? zs_malloc+0x17e/0xb71 [zsmalloc]
[ 344.264217] [<ffffffffa03ccd78>] zs_malloc+0x17e/0xb71 [zsmalloc]
[ 344.264220] [<ffffffffa0190204>] ? lzo_decompress+0x11d/0x11d [lzo]
[ 344.264223] [<ffffffff81122faf>] ? lock_acquire+0xec/0x147
[ 344.264228] [<ffffffffa03ccbfa>] ? obj_malloc+0x372/0x372 [zsmalloc]
[ 344.264233] [<ffffffff81472ff9>] ? crypto_compress+0x87/0x93
[ 344.264238] [<ffffffffa041522d>] zram_bvec_rw+0x1073/0x1638 [zram]
[ 344.264243] [<ffffffffa04141ba>] ? zram_slot_free_notify+0x1c8/0x1c8 [zram]
[ 344.264247] [<ffffffff812fc37b>] ? wb_writeback+0x316/0x44c
[ 344.264251] [<ffffffffa0416104>] zram_make_request+0x6f5/0x89f [zram]
[ 344.264255] [<ffffffff81111ef0>] ? woken_wake_function+0x51/0x51
[ 344.264260] [<ffffffffa0415a0f>] ? zram_rw_page+0x21d/0x21d [zram]
[ 344.264263] [<ffffffff81494948>] ? blk_exit_rl+0x39/0x39
[ 344.264267] [<ffffffff81491130>] ? handle_bad_sector+0x192/0x192
[ 344.264271] [<ffffffff811506a1>] ? call_rcu+0x12/0x14
[ 344.264274] [<ffffffff8129a684>] ? put_object+0x58/0x5b
[ 344.264277] [<ffffffff81496128>] generic_make_request+0x2bc/0x496
[ 344.264280] [<ffffffff81495e6c>] ? blk_plug_queued_count+0x103/0x103
[ 344.264283] [<ffffffff814965fa>] submit_bio+0x2f8/0x324
[ 344.264286] [<ffffffff81496302>] ? generic_make_request+0x496/0x496
[ 344.264289] [<ffffffff813aa993>] ? ext4_reserve_inode_write+0x101/0x101
[ 344.264292] [<ffffffff813b44e8>] ext4_io_submit+0x12d/0x15d
[ 344.264295] [<ffffffff813ac54d>] ext4_writepages+0x15f9/0x1660
[ 344.264298] [<ffffffff813aaf54>] ? ext4_mark_inode_dirty+0x5c1/0x5c1
[ 344.264301] [<ffffffff8111f405>] ? debug_show_all_locks+0x226/0x226
[ 344.264304] [<ffffffff8111f405>] ? debug_show_all_locks+0x226/0x226
[ 344.264307] [<ffffffff8111f9a4>] ? __lock_acquire+0x59f/0x33b8
[ 344.264311] [<ffffffff811fa6ea>] do_writepages+0x93/0xa1
[ 344.264315] [<ffffffff812fb7a0>] ? writeback_sb_inodes+0x270/0x85e
[ 344.264317] [<ffffffff811fa6ea>] ? do_writepages+0x93/0xa1
[ 344.264321] [<ffffffff812fb287>] __writeback_single_inode+0x8b/0x334
[ 344.264324] [<ffffffff812fb9c9>] writeback_sb_inodes+0x499/0x85e
[ 344.264327] [<ffffffff812fb530>] ? __writeback_single_inode+0x334/0x334
[ 344.264331] [<ffffffff81115e1c>] ? down_read_trylock+0x53/0xaf
[ 344.264335] [<ffffffff812a7398>] ? trylock_super+0x16/0xaf
[ 344.264338] [<ffffffff812fbe95>] __writeback_inodes_wb+0x107/0x17d
[ 344.264341] [<ffffffff812fc37b>] wb_writeback+0x316/0x44c
[ 344.264345] [<ffffffff812fc065>] ? writeback_inodes_wb.constprop.10+0x15a/0x15a
[ 344.264348] [<ffffffff811f837f>] ? wb_over_bg_thresh+0x110/0x194
[ 344.264351] [<ffffffff811f826f>] ? balance_dirty_pages_ratelimited+0x14f5/0x14f5
[ 344.264354] [<ffffffff812fce5d>] ? wb_workfn+0x296/0x6d6
[ 344.264357] [<ffffffff812fced4>] wb_workfn+0x30d/0x6d6
[ 344.264360] [<ffffffff812fced4>] ? wb_workfn+0x30d/0x6d6
[ 344.264364] [<ffffffff812fcbc7>] ? inode_wait_for_writeback+0x2e/0x2e
[ 344.264368] [<ffffffff810be6d0>] process_one_work+0x6f4/0xb2c
[ 344.264371] [<ffffffff810bdfdc>] ? pwq_dec_nr_in_flight+0x22b/0x22b
[ 344.264375] [<ffffffff810c0de0>] worker_thread+0x5bb/0x88e
[ 344.264378] [<ffffffff810cd0de>] kthread+0x252/0x261
[ 344.264381] [<ffffffff810c0825>] ? rescuer_thread+0x879/0x879
[ 344.264383] [<ffffffff810cce8c>] ? kthread_create_on_node+0x377/0x377
[ 344.264387] [<ffffffff81d3387f>] ret_from_fork+0x1f/0x40
[ 344.264390] [<ffffffff810cce8c>] ? kthread_create_on_node+0x377/0x377
[ 344.264392] Code: 14 6a b2 7e 85 c0 75 05 e8 8b 35 b1 ff f3 90 bf 01 00 00 00 e8 a1 71 be ff e8 e6 f3 01 00 44 39 f0 74 b6 4c 29 e3 49 01 dd eb 97 <bf> 01 00 00 00 e8 4c 81 be ff 65 8b 05 dc 69 b2 7e 85 c0 75 05
> Could you test with [zsmalloc: keep first object offset in struct page]
> in mmotm?
sure, I can. will it help, tho? we have a race condition here I think.
-ss
[toc] | [prev] | [next] | [standalone]
| From | Minchan Kim <minchan@kernel.org> |
|---|---|
| Date | 2016-06-16 05:00 +0200 |
| Message-ID | <rKvHb-2Bm-5@gated-at.bofh.it> |
| In reply to | #1423646 |
On Thu, Jun 16, 2016 at 11:48:27AM +0900, Sergey Senozhatsky wrote: > Hi, > > On (06/16/16 08:12), Minchan Kim wrote: > > > [ 315.146533] kasan: CONFIG_KASAN_INLINE enabled > > > [ 315.146538] kasan: GPF could be caused by NULL-ptr deref or user memory access > > > [ 315.146546] general protection fault: 0000 [#1] PREEMPT SMP KASAN > > > [ 315.146576] Modules linked in: lzo zram zsmalloc mousedev coretemp hwmon crc32c_intel r8169 i2c_i801 mii snd_hda_codec_realtek snd_hda_codec_generic snd_hda_intel snd_hda_codec snd_hda_core acpi_cpufreq snd_pcm snd_timer snd soundcore lpc_ich mfd_core processor sch_fq_codel sd_mod hid_generic usbhid hid ahci libahci libata ehci_pci ehci_hcd scsi_mod usbcore usb_common > > > [ 315.146785] CPU: 3 PID: 38 Comm: khugepaged Not tainted 4.7.0-rc3-next-20160614-dbg-00004-ga1c2cbc-dirty #488 > > > [ 315.146841] task: ffff8800bfaf2900 ti: ffff880112468000 task.ti: ffff880112468000 > > > [ 315.146859] RIP: 0010:[<ffffffffa02c413d>] [<ffffffffa02c413d>] zs_page_migrate+0x355/0xaa0 [zsmalloc] > > > > Thanks for the report! > > > > zs_page_migrate+0x355? Could you tell me what line is it? > > > > It seems to be related to obj_to_head. > > reproduced. a bit different call stack this time. but the problem is > still the same. > > zs_compact() > ... > 6371: e8 00 00 00 00 callq 6376 <zs_compact+0x22b> > 6376: 0f 0b ud2 > 6378: 48 8b 95 a8 fe ff ff mov -0x158(%rbp),%rdx > 637f: 4d 8d 74 24 78 lea 0x78(%r12),%r14 > 6384: 4c 89 ee mov %r13,%rsi > 6387: 4c 89 e7 mov %r12,%rdi > 638a: e8 86 c7 ff ff callq 2b15 <get_first_obj_offset> > 638f: 41 89 c5 mov %eax,%r13d > 6392: 4c 89 f0 mov %r14,%rax > 6395: 48 c1 e8 03 shr $0x3,%rax > 6399: 8a 04 18 mov (%rax,%rbx,1),%al > 639c: 84 c0 test %al,%al > 639e: 0f 85 f2 02 00 00 jne 6696 <zs_compact+0x54b> > 63a4: 41 8b 44 24 78 mov 0x78(%r12),%eax > 63a9: 41 0f af c7 imul %r15d,%eax > 63ad: 41 01 c5 add %eax,%r13d > 63b0: 4c 89 f0 mov %r14,%rax > 63b3: 48 c1 e8 03 shr $0x3,%rax > 63b7: 48 01 d8 add %rbx,%rax > 63ba: 48 89 85 88 fe ff ff mov %rax,-0x178(%rbp) > 63c1: 41 81 fd ff 0f 00 00 cmp $0xfff,%r13d > 63c8: 0f 87 1a 03 00 00 ja 66e8 <zs_compact+0x59d> > 63ce: 49 63 f5 movslq %r13d,%rsi > 63d1: 48 03 b5 98 fe ff ff add -0x168(%rbp),%rsi > 63d8: 48 8b bd a8 fe ff ff mov -0x158(%rbp),%rdi > 63df: e8 67 d9 ff ff callq 3d4b <obj_to_head> > 63e4: a8 01 test $0x1,%al > 63e6: 0f 84 d9 02 00 00 je 66c5 <zs_compact+0x57a> > 63ec: 48 83 e0 fe and $0xfffffffffffffffe,%rax > 63f0: bf 01 00 00 00 mov $0x1,%edi > 63f5: 48 89 85 b0 fe ff ff mov %rax,-0x150(%rbp) > 63fc: e8 00 00 00 00 callq 6401 <zs_compact+0x2b6> > 6401: 48 8b 85 b0 fe ff ff mov -0x150(%rbp),%rax RAX: 2065676162726166 so rax is totally garbage, I think. It means obj_to_head returns garbage because get_first_obj_offset is utter crab because (page_idx / class->pages_per_zspage) was totally wrong. > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > 6408: f0 0f ba 28 00 lock btsl $0x0,(%rax) <snip> > > Could you test with [zsmalloc: keep first object offset in struct page] > > in mmotm? > > sure, I can. will it help, tho? we have a race condition here I think. I guess root cause is caused by get_first_obj_offset. Please test with it. Thanks!
[toc] | [prev] | [next] | [standalone]
| From | Sergey Senozhatsky <sergey.senozhatsky.work@gmail.com> |
|---|---|
| Date | 2016-06-16 06:30 +0200 |
| Message-ID | <rKx6h-3Bp-21@gated-at.bofh.it> |
| In reply to | #1423650 |
On (06/16/16 11:58), Minchan Kim wrote: [..] > RAX: 2065676162726166 so rax is totally garbage, I think. > It means obj_to_head returns garbage because get_first_obj_offset is > utter crab because (page_idx / class->pages_per_zspage) was totally > wrong. > > > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > > 6408: f0 0f ba 28 00 lock btsl $0x0,(%rax) > > <snip> > > > > Could you test with [zsmalloc: keep first object offset in struct page] > > > in mmotm? > > > > sure, I can. will it help, tho? we have a race condition here I think. > > I guess root cause is caused by get_first_obj_offset. sounds reasonable. > Please test with it. this is what I'm getting with the [zsmalloc: keep first object offset in struct page] applied: "count:0 mapcount:-127". which may be not related to zsmalloc at this point. kernel: BUG: Bad page state in process khugepaged pfn:101db8 kernel: page:ffffea0004076e00 count:0 mapcount:-127 mapping: (null) index:0x1 kernel: flags: 0x8000000000000000() kernel: page dumped because: nonzero mapcount kernel: Modules linked in: lzo zram zsmalloc mousedev coretemp hwmon crc32c_intel snd_hda_codec_realtek i2c_i801 snd_hda_codec_generic r8169 mii snd_hda_intel snd_hda_codec snd_hda_core acpi_cpufreq snd_pcm snd_timer snd soundcore lpc_ich processor mfd_core sch_fq_codel sd_mod hid_generic usb kernel: CPU: 3 PID: 38 Comm: khugepaged Not tainted 4.7.0-rc3-next-20160615-dbg-00005-gfd11984-dirty #491 kernel: 0000000000000000 ffff8801124c73f8 ffffffff814d69b0 ffffea0004076e00 kernel: ffffffff81e658a0 ffff8801124c7420 ffffffff811e9b63 0000000000000000 kernel: ffffea0004076e00 ffffffff81e658a0 ffff8801124c7440 ffffffff811e9ca9 kernel: Call Trace: kernel: [<ffffffff814d69b0>] dump_stack+0x68/0x92 kernel: [<ffffffff811e9b63>] bad_page+0x158/0x1a2 kernel: [<ffffffff811e9ca9>] free_pages_check_bad+0xfc/0x101 kernel: [<ffffffff811ee516>] free_hot_cold_page+0x135/0x5de kernel: [<ffffffff811eea26>] __free_pages+0x67/0x72 kernel: [<ffffffff81227c63>] release_freepages+0x13a/0x191 kernel: [<ffffffff8122b3c2>] compact_zone+0x845/0x1155 kernel: [<ffffffff8122ab7d>] ? compaction_suitable+0x76/0x76 kernel: [<ffffffff8122bdb2>] compact_zone_order+0xe0/0x167 kernel: [<ffffffff8122bcd2>] ? compact_zone+0x1155/0x1155 kernel: [<ffffffff8122ce88>] try_to_compact_pages+0x2f1/0x648 kernel: [<ffffffff8122ce88>] ? try_to_compact_pages+0x2f1/0x648 kernel: [<ffffffff8122cb97>] ? compaction_zonelist_suitable+0x3a6/0x3a6 kernel: [<ffffffff811ef1ea>] ? get_page_from_freelist+0x2c0/0x133c kernel: [<ffffffff811f0350>] __alloc_pages_direct_compact+0xea/0x30d kernel: [<ffffffff811f0266>] ? get_page_from_freelist+0x133c/0x133c kernel: [<ffffffff811ee3b2>] ? drain_all_pages+0x1d6/0x205 kernel: [<ffffffff811f21a8>] __alloc_pages_nodemask+0x143d/0x16b6 kernel: [<ffffffff8111f405>] ? debug_show_all_locks+0x226/0x226 kernel: [<ffffffff811f0d6b>] ? warn_alloc_failed+0x24c/0x24c kernel: [<ffffffff81110ffc>] ? finish_wait+0x1a4/0x1b0 kernel: [<ffffffff81122faf>] ? lock_acquire+0xec/0x147 kernel: [<ffffffff81d32ed0>] ? _raw_spin_unlock_irqrestore+0x3b/0x5c kernel: [<ffffffff81d32edc>] ? _raw_spin_unlock_irqrestore+0x47/0x5c kernel: [<ffffffff81110ffc>] ? finish_wait+0x1a4/0x1b0 kernel: [<ffffffff8128f73a>] khugepaged+0x1d4/0x484f kernel: [<ffffffff8128f566>] ? hugepage_vma_revalidate+0xef/0xef kernel: [<ffffffff810d5bcc>] ? finish_task_switch+0x3de/0x484 kernel: [<ffffffff81d32f18>] ? _raw_spin_unlock_irq+0x27/0x45 kernel: [<ffffffff8111d13f>] ? trace_hardirqs_on_caller+0x3d2/0x492 kernel: [<ffffffff81111487>] ? prepare_to_wait_event+0x3f7/0x3f7 kernel: [<ffffffff81d28bf5>] ? __schedule+0xa4d/0xd16 kernel: [<ffffffff810cd0de>] kthread+0x252/0x261 kernel: [<ffffffff8128f566>] ? hugepage_vma_revalidate+0xef/0xef kernel: [<ffffffff810cce8c>] ? kthread_create_on_node+0x377/0x377 kernel: [<ffffffff81d3387f>] ret_from_fork+0x1f/0x40 kernel: [<ffffffff810cce8c>] ? kthread_create_on_node+0x377/0x377 -- Reboot -- -ss
[toc] | [prev] | [next] | [standalone]
| From | Minchan Kim <minchan@kernel.org> |
|---|---|
| Date | 2016-06-16 06:50 +0200 |
| Message-ID | <rKxpD-3Jz-5@gated-at.bofh.it> |
| In reply to | #1423672 |
On Thu, Jun 16, 2016 at 01:23:43PM +0900, Sergey Senozhatsky wrote: > On (06/16/16 11:58), Minchan Kim wrote: > [..] > > RAX: 2065676162726166 so rax is totally garbage, I think. > > It means obj_to_head returns garbage because get_first_obj_offset is > > utter crab because (page_idx / class->pages_per_zspage) was totally > > wrong. > > > > > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > > > 6408: f0 0f ba 28 00 lock btsl $0x0,(%rax) > > > > <snip> > > > > > > Could you test with [zsmalloc: keep first object offset in struct page] > > > > in mmotm? > > > > > > sure, I can. will it help, tho? we have a race condition here I think. > > > > I guess root cause is caused by get_first_obj_offset. > > sounds reasonable. > > > Please test with it. > > > this is what I'm getting with the [zsmalloc: keep first object offset in struct page] > applied: "count:0 mapcount:-127". which may be not related to zsmalloc at this point. > > kernel: BUG: Bad page state in process khugepaged pfn:101db8 > kernel: page:ffffea0004076e00 count:0 mapcount:-127 mapping: (null) index:0x1 Hm, it seems double free. It doen't happen if you disable zram? IOW, it seems to be related zsmalloc migration? How easy can you reprodcue it? Could you bisect it? > kernel: flags: 0x8000000000000000() > kernel: page dumped because: nonzero mapcount > kernel: Modules linked in: lzo zram zsmalloc mousedev coretemp hwmon crc32c_intel snd_hda_codec_realtek i2c_i801 snd_hda_codec_generic r8169 mii snd_hda_intel snd_hda_codec snd_hda_core acpi_cpufreq snd_pcm snd_timer snd soundcore lpc_ich processor mfd_core sch_fq_codel sd_mod hid_generic usb > kernel: CPU: 3 PID: 38 Comm: khugepaged Not tainted 4.7.0-rc3-next-20160615-dbg-00005-gfd11984-dirty #491 > kernel: 0000000000000000 ffff8801124c73f8 ffffffff814d69b0 ffffea0004076e00 > kernel: ffffffff81e658a0 ffff8801124c7420 ffffffff811e9b63 0000000000000000 > kernel: ffffea0004076e00 ffffffff81e658a0 ffff8801124c7440 ffffffff811e9ca9 > kernel: Call Trace: > kernel: [<ffffffff814d69b0>] dump_stack+0x68/0x92 > kernel: [<ffffffff811e9b63>] bad_page+0x158/0x1a2 > kernel: [<ffffffff811e9ca9>] free_pages_check_bad+0xfc/0x101 > kernel: [<ffffffff811ee516>] free_hot_cold_page+0x135/0x5de > kernel: [<ffffffff811eea26>] __free_pages+0x67/0x72 > kernel: [<ffffffff81227c63>] release_freepages+0x13a/0x191 > kernel: [<ffffffff8122b3c2>] compact_zone+0x845/0x1155 > kernel: [<ffffffff8122ab7d>] ? compaction_suitable+0x76/0x76 > kernel: [<ffffffff8122bdb2>] compact_zone_order+0xe0/0x167 > kernel: [<ffffffff8122bcd2>] ? compact_zone+0x1155/0x1155 > kernel: [<ffffffff8122ce88>] try_to_compact_pages+0x2f1/0x648 > kernel: [<ffffffff8122ce88>] ? try_to_compact_pages+0x2f1/0x648 > kernel: [<ffffffff8122cb97>] ? compaction_zonelist_suitable+0x3a6/0x3a6 > kernel: [<ffffffff811ef1ea>] ? get_page_from_freelist+0x2c0/0x133c > kernel: [<ffffffff811f0350>] __alloc_pages_direct_compact+0xea/0x30d > kernel: [<ffffffff811f0266>] ? get_page_from_freelist+0x133c/0x133c > kernel: [<ffffffff811ee3b2>] ? drain_all_pages+0x1d6/0x205 > kernel: [<ffffffff811f21a8>] __alloc_pages_nodemask+0x143d/0x16b6 > kernel: [<ffffffff8111f405>] ? debug_show_all_locks+0x226/0x226 > kernel: [<ffffffff811f0d6b>] ? warn_alloc_failed+0x24c/0x24c > kernel: [<ffffffff81110ffc>] ? finish_wait+0x1a4/0x1b0 > kernel: [<ffffffff81122faf>] ? lock_acquire+0xec/0x147 > kernel: [<ffffffff81d32ed0>] ? _raw_spin_unlock_irqrestore+0x3b/0x5c > kernel: [<ffffffff81d32edc>] ? _raw_spin_unlock_irqrestore+0x47/0x5c > kernel: [<ffffffff81110ffc>] ? finish_wait+0x1a4/0x1b0 > kernel: [<ffffffff8128f73a>] khugepaged+0x1d4/0x484f > kernel: [<ffffffff8128f566>] ? hugepage_vma_revalidate+0xef/0xef > kernel: [<ffffffff810d5bcc>] ? finish_task_switch+0x3de/0x484 > kernel: [<ffffffff81d32f18>] ? _raw_spin_unlock_irq+0x27/0x45 > kernel: [<ffffffff8111d13f>] ? trace_hardirqs_on_caller+0x3d2/0x492 > kernel: [<ffffffff81111487>] ? prepare_to_wait_event+0x3f7/0x3f7 > kernel: [<ffffffff81d28bf5>] ? __schedule+0xa4d/0xd16 > kernel: [<ffffffff810cd0de>] kthread+0x252/0x261 > kernel: [<ffffffff8128f566>] ? hugepage_vma_revalidate+0xef/0xef > kernel: [<ffffffff810cce8c>] ? kthread_create_on_node+0x377/0x377 > kernel: [<ffffffff81d3387f>] ret_from_fork+0x1f/0x40 > kernel: [<ffffffff810cce8c>] ? kthread_create_on_node+0x377/0x377 > -- Reboot -- > > -ss
[toc] | [prev] | [next] | [standalone]
| From | Sergey Senozhatsky <sergey.senozhatsky.work@gmail.com> |
|---|---|
| Date | 2016-06-16 07:30 +0200 |
| Message-ID | <rKy2l-4dk-7@gated-at.bofh.it> |
| In reply to | #1423676 |
On (06/16/16 13:47), Minchan Kim wrote: [..] > > this is what I'm getting with the [zsmalloc: keep first object offset in struct page] > > applied: "count:0 mapcount:-127". which may be not related to zsmalloc at this point. > > > > kernel: BUG: Bad page state in process khugepaged pfn:101db8 > > kernel: page:ffffea0004076e00 count:0 mapcount:-127 mapping: (null) index:0x1 > > Hm, it seems double free. > > It doen't happen if you disable zram? IOW, it seems to be related > zsmalloc migration? need to test more, can't confidently answer now. > How easy can you reprodcue it? Could you bisect it? it takes some (um.. random) time to trigger the bug. I'll try to come up with more details. -ss > > kernel: flags: 0x8000000000000000() > > kernel: page dumped because: nonzero mapcount > > kernel: Modules linked in: lzo zram zsmalloc mousedev coretemp hwmon crc32c_intel snd_hda_codec_realtek i2c_i801 snd_hda_codec_generic r8169 mii snd_hda_intel snd_hda_codec snd_hda_core acpi_cpufreq snd_pcm snd_timer snd soundcore lpc_ich processor mfd_core sch_fq_codel sd_mod hid_generic usb > > kernel: CPU: 3 PID: 38 Comm: khugepaged Not tainted 4.7.0-rc3-next-20160615-dbg-00005-gfd11984-dirty #491 > > kernel: 0000000000000000 ffff8801124c73f8 ffffffff814d69b0 ffffea0004076e00 > > kernel: ffffffff81e658a0 ffff8801124c7420 ffffffff811e9b63 0000000000000000 > > kernel: ffffea0004076e00 ffffffff81e658a0 ffff8801124c7440 ffffffff811e9ca9 > > kernel: Call Trace: > > kernel: [<ffffffff814d69b0>] dump_stack+0x68/0x92 > > kernel: [<ffffffff811e9b63>] bad_page+0x158/0x1a2 > > kernel: [<ffffffff811e9ca9>] free_pages_check_bad+0xfc/0x101 > > kernel: [<ffffffff811ee516>] free_hot_cold_page+0x135/0x5de > > kernel: [<ffffffff811eea26>] __free_pages+0x67/0x72 > > kernel: [<ffffffff81227c63>] release_freepages+0x13a/0x191 > > kernel: [<ffffffff8122b3c2>] compact_zone+0x845/0x1155 > > kernel: [<ffffffff8122ab7d>] ? compaction_suitable+0x76/0x76 > > kernel: [<ffffffff8122bdb2>] compact_zone_order+0xe0/0x167 > > kernel: [<ffffffff8122bcd2>] ? compact_zone+0x1155/0x1155 > > kernel: [<ffffffff8122ce88>] try_to_compact_pages+0x2f1/0x648 > > kernel: [<ffffffff8122ce88>] ? try_to_compact_pages+0x2f1/0x648 > > kernel: [<ffffffff8122cb97>] ? compaction_zonelist_suitable+0x3a6/0x3a6 > > kernel: [<ffffffff811ef1ea>] ? get_page_from_freelist+0x2c0/0x133c > > kernel: [<ffffffff811f0350>] __alloc_pages_direct_compact+0xea/0x30d > > kernel: [<ffffffff811f0266>] ? get_page_from_freelist+0x133c/0x133c > > kernel: [<ffffffff811ee3b2>] ? drain_all_pages+0x1d6/0x205 > > kernel: [<ffffffff811f21a8>] __alloc_pages_nodemask+0x143d/0x16b6 > > kernel: [<ffffffff8111f405>] ? debug_show_all_locks+0x226/0x226 > > kernel: [<ffffffff811f0d6b>] ? warn_alloc_failed+0x24c/0x24c > > kernel: [<ffffffff81110ffc>] ? finish_wait+0x1a4/0x1b0 > > kernel: [<ffffffff81122faf>] ? lock_acquire+0xec/0x147 > > kernel: [<ffffffff81d32ed0>] ? _raw_spin_unlock_irqrestore+0x3b/0x5c > > kernel: [<ffffffff81d32edc>] ? _raw_spin_unlock_irqrestore+0x47/0x5c > > kernel: [<ffffffff81110ffc>] ? finish_wait+0x1a4/0x1b0 > > kernel: [<ffffffff8128f73a>] khugepaged+0x1d4/0x484f > > kernel: [<ffffffff8128f566>] ? hugepage_vma_revalidate+0xef/0xef > > kernel: [<ffffffff810d5bcc>] ? finish_task_switch+0x3de/0x484 > > kernel: [<ffffffff81d32f18>] ? _raw_spin_unlock_irq+0x27/0x45 > > kernel: [<ffffffff8111d13f>] ? trace_hardirqs_on_caller+0x3d2/0x492 > > kernel: [<ffffffff81111487>] ? prepare_to_wait_event+0x3f7/0x3f7 > > kernel: [<ffffffff81d28bf5>] ? __schedule+0xa4d/0xd16 > > kernel: [<ffffffff810cd0de>] kthread+0x252/0x261 > > kernel: [<ffffffff8128f566>] ? hugepage_vma_revalidate+0xef/0xef > > kernel: [<ffffffff810cce8c>] ? kthread_create_on_node+0x377/0x377 > > kernel: [<ffffffff81d3387f>] ret_from_fork+0x1f/0x40 > > kernel: [<ffffffff810cce8c>] ? kthread_create_on_node+0x377/0x377 > > -- Reboot --
[toc] | [prev] | [next] | [standalone]
| From | Minchan Kim <minchan@kernel.org> |
|---|---|
| Date | 2016-06-16 08:50 +0200 |
| Message-ID | <rKzhL-4S6-3@gated-at.bofh.it> |
| In reply to | #1423683 |
On Thu, Jun 16, 2016 at 02:22:09PM +0900, Sergey Senozhatsky wrote: > On (06/16/16 13:47), Minchan Kim wrote: > [..] > > > this is what I'm getting with the [zsmalloc: keep first object offset in struct page] > > > applied: "count:0 mapcount:-127". which may be not related to zsmalloc at this point. > > > > > > kernel: BUG: Bad page state in process khugepaged pfn:101db8 > > > kernel: page:ffffea0004076e00 count:0 mapcount:-127 mapping: (null) index:0x1 > > > > Hm, it seems double free. > > > > It doen't happen if you disable zram? IOW, it seems to be related > > zsmalloc migration? > > need to test more, can't confidently answer now. > > > How easy can you reprodcue it? Could you bisect it? > > it takes some (um.. random) time to trigger the bug. > I'll try to come up with more details. Could you revert [1] and retest? [1] mm/compaction: split freepages without holding the zone lock > > -ss > > > > kernel: flags: 0x8000000000000000() > > > kernel: page dumped because: nonzero mapcount > > > kernel: Modules linked in: lzo zram zsmalloc mousedev coretemp hwmon crc32c_intel snd_hda_codec_realtek i2c_i801 snd_hda_codec_generic r8169 mii snd_hda_intel snd_hda_codec snd_hda_core acpi_cpufreq snd_pcm snd_timer snd soundcore lpc_ich processor mfd_core sch_fq_codel sd_mod hid_generic usb > > > kernel: CPU: 3 PID: 38 Comm: khugepaged Not tainted 4.7.0-rc3-next-20160615-dbg-00005-gfd11984-dirty #491 > > > kernel: 0000000000000000 ffff8801124c73f8 ffffffff814d69b0 ffffea0004076e00 > > > kernel: ffffffff81e658a0 ffff8801124c7420 ffffffff811e9b63 0000000000000000 > > > kernel: ffffea0004076e00 ffffffff81e658a0 ffff8801124c7440 ffffffff811e9ca9 > > > kernel: Call Trace: > > > kernel: [<ffffffff814d69b0>] dump_stack+0x68/0x92 > > > kernel: [<ffffffff811e9b63>] bad_page+0x158/0x1a2 > > > kernel: [<ffffffff811e9ca9>] free_pages_check_bad+0xfc/0x101 > > > kernel: [<ffffffff811ee516>] free_hot_cold_page+0x135/0x5de > > > kernel: [<ffffffff811eea26>] __free_pages+0x67/0x72 > > > kernel: [<ffffffff81227c63>] release_freepages+0x13a/0x191 > > > kernel: [<ffffffff8122b3c2>] compact_zone+0x845/0x1155 > > > kernel: [<ffffffff8122ab7d>] ? compaction_suitable+0x76/0x76 > > > kernel: [<ffffffff8122bdb2>] compact_zone_order+0xe0/0x167 > > > kernel: [<ffffffff8122bcd2>] ? compact_zone+0x1155/0x1155 > > > kernel: [<ffffffff8122ce88>] try_to_compact_pages+0x2f1/0x648 > > > kernel: [<ffffffff8122ce88>] ? try_to_compact_pages+0x2f1/0x648 > > > kernel: [<ffffffff8122cb97>] ? compaction_zonelist_suitable+0x3a6/0x3a6 > > > kernel: [<ffffffff811ef1ea>] ? get_page_from_freelist+0x2c0/0x133c > > > kernel: [<ffffffff811f0350>] __alloc_pages_direct_compact+0xea/0x30d > > > kernel: [<ffffffff811f0266>] ? get_page_from_freelist+0x133c/0x133c > > > kernel: [<ffffffff811ee3b2>] ? drain_all_pages+0x1d6/0x205 > > > kernel: [<ffffffff811f21a8>] __alloc_pages_nodemask+0x143d/0x16b6 > > > kernel: [<ffffffff8111f405>] ? debug_show_all_locks+0x226/0x226 > > > kernel: [<ffffffff811f0d6b>] ? warn_alloc_failed+0x24c/0x24c > > > kernel: [<ffffffff81110ffc>] ? finish_wait+0x1a4/0x1b0 > > > kernel: [<ffffffff81122faf>] ? lock_acquire+0xec/0x147 > > > kernel: [<ffffffff81d32ed0>] ? _raw_spin_unlock_irqrestore+0x3b/0x5c > > > kernel: [<ffffffff81d32edc>] ? _raw_spin_unlock_irqrestore+0x47/0x5c > > > kernel: [<ffffffff81110ffc>] ? finish_wait+0x1a4/0x1b0 > > > kernel: [<ffffffff8128f73a>] khugepaged+0x1d4/0x484f > > > kernel: [<ffffffff8128f566>] ? hugepage_vma_revalidate+0xef/0xef > > > kernel: [<ffffffff810d5bcc>] ? finish_task_switch+0x3de/0x484 > > > kernel: [<ffffffff81d32f18>] ? _raw_spin_unlock_irq+0x27/0x45 > > > kernel: [<ffffffff8111d13f>] ? trace_hardirqs_on_caller+0x3d2/0x492 > > > kernel: [<ffffffff81111487>] ? prepare_to_wait_event+0x3f7/0x3f7 > > > kernel: [<ffffffff81d28bf5>] ? __schedule+0xa4d/0xd16 > > > kernel: [<ffffffff810cd0de>] kthread+0x252/0x261 > > > kernel: [<ffffffff8128f566>] ? hugepage_vma_revalidate+0xef/0xef > > > kernel: [<ffffffff810cce8c>] ? kthread_create_on_node+0x377/0x377 > > > kernel: [<ffffffff81d3387f>] ret_from_fork+0x1f/0x40 > > > kernel: [<ffffffff810cce8c>] ? kthread_create_on_node+0x377/0x377 > > > -- Reboot --
[toc] | [prev] | [next] | [standalone]
| From | Sergey Senozhatsky <sergey.senozhatsky.work@gmail.com> |
|---|---|
| Date | 2016-06-16 10:50 +0200 |
| Message-ID | <rKB9T-62Y-9@gated-at.bofh.it> |
| In reply to | #1423718 |
On (06/16/16 15:47), Minchan Kim wrote: > > [..] > > > > this is what I'm getting with the [zsmalloc: keep first object offset in struct page] > > > > applied: "count:0 mapcount:-127". which may be not related to zsmalloc at this point. > > > > > > > > kernel: BUG: Bad page state in process khugepaged pfn:101db8 > > > > kernel: page:ffffea0004076e00 count:0 mapcount:-127 mapping: (null) index:0x1 > > > > > > Hm, it seems double free. > > > > > > It doen't happen if you disable zram? IOW, it seems to be related > > > zsmalloc migration? > > > > need to test more, can't confidently answer now. > > > > > How easy can you reprodcue it? Could you bisect it? > > > > it takes some (um.. random) time to trigger the bug. > > I'll try to come up with more details. > > Could you revert [1] and retest? > > [1] mm/compaction: split freepages without holding the zone lock ok, so this is not related to zsmalloc. finally manged to reproduce it. will fork a separate thread. -ss
[toc] | [prev] | [next] | [standalone]
| From | Minchan Kim <minchan@kernel.org> |
|---|---|
| Date | 2016-06-16 12:10 +0200 |
| Message-ID | <rKCpj-72x-5@gated-at.bofh.it> |
| In reply to | #1423812 |
On Thu, Jun 16, 2016 at 05:42:11PM +0900, Sergey Senozhatsky wrote:
> On (06/16/16 15:47), Minchan Kim wrote:
> > > [..]
> > > > > this is what I'm getting with the [zsmalloc: keep first object offset in struct page]
> > > > > applied: "count:0 mapcount:-127". which may be not related to zsmalloc at this point.
> > > > >
> > > > > kernel: BUG: Bad page state in process khugepaged pfn:101db8
> > > > > kernel: page:ffffea0004076e00 count:0 mapcount:-127 mapping: (null) index:0x1
> > > >
> > > > Hm, it seems double free.
> > > >
> > > > It doen't happen if you disable zram? IOW, it seems to be related
> > > > zsmalloc migration?
> > >
> > > need to test more, can't confidently answer now.
> > >
> > > > How easy can you reprodcue it? Could you bisect it?
> > >
> > > it takes some (um.. random) time to trigger the bug.
> > > I'll try to come up with more details.
> >
> > Could you revert [1] and retest?
> >
> > [1] mm/compaction: split freepages without holding the zone lock
>
> ok, so this is not related to zsmalloc. finally manged to reproduce
> it. will fork a separate thread.
The reason I mentioned [1] is that it seems to have a bug.
isolate_freepages_block
__isolate_free_page
if(!zone_watermark_ok())
return 0;
list_add_tail(&page->lru, freelist);
However, the page is not isolated.
Joonsoo?
[toc] | [prev] | [next] | [standalone]
| From | Joonsoo Kim <iamjoonsoo.kim@lge.com> |
|---|---|
| Date | 2016-06-17 09:30 +0200 |
| Message-ID | <rKWo1-3vB-13@gated-at.bofh.it> |
| In reply to | #1423890 |
On Thu, Jun 16, 2016 at 07:09:32PM +0900, Minchan Kim wrote: > On Thu, Jun 16, 2016 at 05:42:11PM +0900, Sergey Senozhatsky wrote: > > On (06/16/16 15:47), Minchan Kim wrote: > > > > [..] > > > > > > this is what I'm getting with the [zsmalloc: keep first object offset in struct page] > > > > > > applied: "count:0 mapcount:-127". which may be not related to zsmalloc at this point. > > > > > > > > > > > > kernel: BUG: Bad page state in process khugepaged pfn:101db8 > > > > > > kernel: page:ffffea0004076e00 count:0 mapcount:-127 mapping: (null) index:0x1 > > > > > > > > > > Hm, it seems double free. > > > > > > > > > > It doen't happen if you disable zram? IOW, it seems to be related > > > > > zsmalloc migration? > > > > > > > > need to test more, can't confidently answer now. > > > > > > > > > How easy can you reprodcue it? Could you bisect it? > > > > > > > > it takes some (um.. random) time to trigger the bug. > > > > I'll try to come up with more details. > > > > > > Could you revert [1] and retest? > > > > > > [1] mm/compaction: split freepages without holding the zone lock > > > > ok, so this is not related to zsmalloc. finally manged to reproduce > > it. will fork a separate thread. > > The reason I mentioned [1] is that it seems to have a bug. > > isolate_freepages_block > __isolate_free_page > if(!zone_watermark_ok()) > return 0; > list_add_tail(&page->lru, freelist); > > However, the page is not isolated. > Joonsoo? Good job! I will fix it soon. Thanks.
[toc] | [prev] | [standalone]
Back to top | Article view | linux.kernel
csiph-web