Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > linux.kernel > #1211968
| From | Jiri Slaby <jslaby@suse.cz> |
|---|---|
| Newsgroups | linux.kernel |
| Subject | [PATCH 3.12 72/82] mm, vmscan: Do not wait for page writeback for GFP_NOFS allocations |
| Date | 2015-08-24 11:20 +0200 |
| Message-ID | <q0W57-3om-79@gated-at.bofh.it> (permalink) |
| References | <q0VVn-3cH-1@gated-at.bofh.it> |
| Organization | linux.* mail to news gateway |
From: Michal Hocko <mhocko@suse.cz>
3.12-stable review patch. If anyone has any objections, please let me know.
===============
commit ecf5fc6e9654cd7a268c782a523f072b2f1959f9 upstream.
Nikolay has reported a hang when a memcg reclaim got stuck with the
following backtrace:
PID: 18308 TASK: ffff883d7c9b0a30 CPU: 1 COMMAND: "rsync"
#0 __schedule at ffffffff815ab152
#1 schedule at ffffffff815ab76e
#2 schedule_timeout at ffffffff815ae5e5
#3 io_schedule_timeout at ffffffff815aad6a
#4 bit_wait_io at ffffffff815abfc6
#5 __wait_on_bit at ffffffff815abda5
#6 wait_on_page_bit at ffffffff8111fd4f
#7 shrink_page_list at ffffffff81135445
#8 shrink_inactive_list at ffffffff81135845
#9 shrink_lruvec at ffffffff81135ead
#10 shrink_zone at ffffffff811360c3
#11 shrink_zones at ffffffff81136eff
#12 do_try_to_free_pages at ffffffff8113712f
#13 try_to_free_mem_cgroup_pages at ffffffff811372be
#14 try_charge at ffffffff81189423
#15 mem_cgroup_try_charge at ffffffff8118c6f5
#16 __add_to_page_cache_locked at ffffffff8112137d
#17 add_to_page_cache_lru at ffffffff81121618
#18 pagecache_get_page at ffffffff8112170b
#19 grow_dev_page at ffffffff811c8297
#20 __getblk_slow at ffffffff811c91d6
#21 __getblk_gfp at ffffffff811c92c1
#22 ext4_ext_grow_indepth at ffffffff8124565c
#23 ext4_ext_create_new_leaf at ffffffff81246ca8
#24 ext4_ext_insert_extent at ffffffff81246f09
#25 ext4_ext_map_blocks at ffffffff8124a848
#26 ext4_map_blocks at ffffffff8121a5b7
#27 mpage_map_one_extent at ffffffff8121b1fa
#28 mpage_map_and_submit_extent at ffffffff8121f07b
#29 ext4_writepages at ffffffff8121f6d5
#30 do_writepages at ffffffff8112c490
#31 __filemap_fdatawrite_range at ffffffff81120199
#32 filemap_flush at ffffffff8112041c
#33 ext4_alloc_da_blocks at ffffffff81219da1
#34 ext4_rename at ffffffff81229b91
#35 ext4_rename2 at ffffffff81229e32
#36 vfs_rename at ffffffff811a08a5
#37 SYSC_renameat2 at ffffffff811a3ffc
#38 sys_renameat2 at ffffffff811a408e
#39 sys_rename at ffffffff8119e51e
#40 system_call_fastpath at ffffffff815afa89
Dave Chinner has properly pointed out that this is a deadlock in the
reclaim code because ext4 doesn't submit pages which are marked by
PG_writeback right away.
The heuristic was introduced by commit e62e384e9da8 ("memcg: prevent OOM
with too many dirty pages") and it was applied only when may_enter_fs
was specified. The code has been changed by c3b94f44fcb0 ("memcg:
further prevent OOM with too many dirty pages") which has removed the
__GFP_FS restriction with a reasoning that we do not get into the fs
code. But this is not sufficient apparently because the fs doesn't
necessarily submit pages marked PG_writeback for IO right away.
ext4_bio_write_page calls io_submit_add_bh but that doesn't necessarily
submit the bio. Instead it tries to map more pages into the bio and
mpage_map_one_extent might trigger memcg charge which might end up
waiting on a page which is marked PG_writeback but hasn't been submitted
yet so we would end up waiting for something that never finishes.
Fix this issue by replacing __GFP_IO by may_enter_fs check (for case 2)
before we go to wait on the writeback. The page fault path, which is
the only path that triggers memcg oom killer since 3.12, shouldn't
require GFP_NOFS and so we shouldn't reintroduce the premature OOM
killer issue which was originally addressed by the heuristic.
As per David Chinner the xfs is doing similar thing since 2.6.15 already
so ext4 is not the only affected filesystem. Moreover he notes:
: For example: IO completion might require unwritten extent conversion
: which executes filesystem transactions and GFP_NOFS allocations. The
: writeback flag on the pages can not be cleared until unwritten
: extent conversion completes. Hence memory reclaim cannot wait on
: page writeback to complete in GFP_NOFS context because it is not
: safe to do so, memcg reclaim or otherwise.
[tytso@mit.edu: corrected the control flow]
Fixes: c3b94f44fcb0 ("memcg: further prevent OOM with too many dirty pages")
Reported-by: Nikolay Borisov <kernel@kyup.com>
Signed-off-by: Michal Hocko <mhocko@suse.cz>
Signed-off-by: Hugh Dickins <hughd@google.com>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Jiri Slaby <jslaby@suse.cz>
---
mm/vmscan.c | 14 +++++---------
1 file changed, 5 insertions(+), 9 deletions(-)
diff --git a/mm/vmscan.c b/mm/vmscan.c
index ee8363f73cab..04c33d5fb079 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -871,21 +871,17 @@ static unsigned long shrink_page_list(struct list_head *page_list,
*
* 2) Global reclaim encounters a page, memcg encounters a
* page that is not marked for immediate reclaim or
- * the caller does not have __GFP_IO. In this case mark
+ * the caller does not have __GFP_FS (or __GFP_IO if it's
+ * simply going to swap, not to fs). In this case mark
* the page for immediate reclaim and continue scanning.
*
- * __GFP_IO is checked because a loop driver thread might
+ * Require may_enter_fs because we would wait on fs, which
+ * may not have submitted IO yet. And the loop driver might
* enter reclaim, and deadlock if it waits on a page for
* which it is needed to do the write (loop masks off
* __GFP_IO|__GFP_FS for this reason); but more thought
* would probably show more reasons.
*
- * Don't require __GFP_FS, since we're not going into the
- * FS, just waiting on its writeback completion. Worryingly,
- * ext4 gfs2 and xfs allocate pages with
- * grab_cache_page_write_begin(,,AOP_FLAG_NOFS), so testing
- * may_enter_fs here is liable to OOM on them.
- *
* 3) memcg encounters a page that is not already marked
* PageReclaim. memcg does not have any dirty pages
* throttling so we could easily OOM just because too many
@@ -902,7 +898,7 @@ static unsigned long shrink_page_list(struct list_head *page_list,
/* Case 2 above */
} else if (global_reclaim(sc) ||
- !PageReclaim(page) || !(sc->gfp_mask & __GFP_IO)) {
+ !PageReclaim(page) || !may_enter_fs) {
/*
* This is slightly racy - end_page_writeback()
* might have just cleared PageReclaim, then
--
2.5.0
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Back to linux.kernel | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread
[PATCH 3.12 01/82] efi: fix 32bit kernel boot failed problem using efi Jiri Slaby <jslaby@suse.cz> - 2015-08-24 11:10 +0200
[PATCH 3.12 67/82] signalfd: fix information leak in signalfd_copyinfo Jiri Slaby <jslaby@suse.cz> - 2015-08-24 11:20 +0200
[PATCH 3.12 54/82] ipc: modify message queue accounting to not take kernel data structures into account Jiri Slaby <jslaby@suse.cz> - 2015-08-24 11:20 +0200
[PATCH 3.12 75/82] mm/hwpoison: fix page refcount of unknown non LRU page Jiri Slaby <jslaby@suse.cz> - 2015-08-24 11:20 +0200
[PATCH 3.12 64/82] x86/ldt: Correct LDT access in single stepping logic Jiri Slaby <jslaby@suse.cz> - 2015-08-24 11:20 +0200
[PATCH 3.12 44/82] ipmi: fix timeout calculation when bmc is disconnected Jiri Slaby <jslaby@suse.cz> - 2015-08-24 11:20 +0200
[PATCH 3.12 07/82] ARC: make sure instruction_pointer() returns unsigned value Jiri Slaby <jslaby@suse.cz> - 2015-08-24 11:20 +0200
[PATCH 3.12 59/82] x86/nmi/64: Switch stacks on userspace NMI entry Jiri Slaby <jslaby@suse.cz> - 2015-08-24 11:20 +0200
[PATCH 3.12 62/82] rcu: Move lockless_dereference() out of rcupdate.h Jiri Slaby <jslaby@suse.cz> - 2015-08-24 11:20 +0200
[PATCH 3.12 55/82] ocfs2: fix BUG in ocfs2_downconvert_thread_do_work() Jiri Slaby <jslaby@suse.cz> - 2015-08-24 11:20 +0200
[PATCH 3.12 60/82] arch: Introduce smp_load_acquire(), smp_store_release() Jiri Slaby <jslaby@suse.cz> - 2015-08-24 11:20 +0200
[PATCH 3.12 57/82] x86/nmi: Enable nested do_nmi() handling for 64-bit kernels Jiri Slaby <jslaby@suse.cz> - 2015-08-24 11:20 +0200
[PATCH 3.12 71/82] md/bitmap: return an error when bitmap superblock is corrupt. Jiri Slaby <jslaby@suse.cz> - 2015-08-24 11:20 +0200
[PATCH 3.12 76/82] xen-blkfront: don't add indirect pages to list when !feature_persistent Jiri Slaby <jslaby@suse.cz> - 2015-08-24 11:20 +0200
[PATCH 3.12 45/82] sparc64: Fix userspace FPU register corruptions. Jiri Slaby <jslaby@suse.cz> - 2015-08-24 11:20 +0200
[PATCH 3.12 74/82] ipc,sem: fix use after free on IPC_RMID after a task using same semaphore set exits Jiri Slaby <jslaby@suse.cz> - 2015-08-24 11:20 +0200
[PATCH 3.12 05/82] freeing unlinked file indefinitely delayed Jiri Slaby <jslaby@suse.cz> - 2015-08-24 11:20 +0200
[PATCH 3.12 66/82] x86/ldt: Further fix FPU emulation Jiri Slaby <jslaby@suse.cz> - 2015-08-24 11:20 +0200
[PATCH 3.12 69/82] signal: fix information leak in copy_siginfo_from_user32 Jiri Slaby <jslaby@suse.cz> - 2015-08-24 11:20 +0200
[PATCH 3.12 80/82] EDAC, ppc4xx: Access mci->csrows array elements properly Jiri Slaby <jslaby@suse.cz> - 2015-08-24 11:20 +0200
[PATCH 3.12 56/82] md/raid1: extend spinlock to protect raid1_end_read_request against inconsistencies Jiri Slaby <jslaby@suse.cz> - 2015-08-24 11:20 +0200
[PATCH 3.12 77/82] perf: Fix fasync handling on inherited events Jiri Slaby <jslaby@suse.cz> - 2015-08-24 11:20 +0200
[PATCH 3.12 63/82] x86/ldt: Make modify_ldt synchronous Jiri Slaby <jslaby@suse.cz> - 2015-08-24 11:20 +0200
[PATCH 3.12 82/82] rbd: fix copyup completion race Jiri Slaby <jslaby@suse.cz> - 2015-08-24 11:20 +0200
[PATCH 3.12 78/82] dm thin metadata: delete btrees when releasing metadata snapshot Jiri Slaby <jslaby@suse.cz> - 2015-08-24 11:20 +0200
[PATCH 3.12 73/82] ipc/sem.c: update/correct memory barriers Jiri Slaby <jslaby@suse.cz> - 2015-08-24 11:20 +0200
[PATCH 3.12 65/82] x86/ldt: Correct FPU emulation access to LDT Jiri Slaby <jslaby@suse.cz> - 2015-08-24 11:20 +0200
[PATCH 3.12 68/82] signal: fix information leak in copy_siginfo_to_user Jiri Slaby <jslaby@suse.cz> - 2015-08-24 11:20 +0200
[PATCH 3.12 81/82] drm/radeon: add new OLAND pci id Jiri Slaby <jslaby@suse.cz> - 2015-08-24 11:20 +0200
[PATCH 3.12 58/82] x86/nmi/64: Remove asm code that saves CR2 Jiri Slaby <jslaby@suse.cz> - 2015-08-24 11:20 +0200
[PATCH 3.12 79/82] localmodconfig: Use Kbuild files too Jiri Slaby <jslaby@suse.cz> - 2015-08-24 11:20 +0200
[PATCH 3.12 61/82] rcu: Provide counterpart to rcu_dereference() for non-RCU situations Jiri Slaby <jslaby@suse.cz> - 2015-08-24 11:20 +0200
[PATCH 3.12 72/82] mm, vmscan: Do not wait for page writeback for GFP_NOFS allocations Jiri Slaby <jslaby@suse.cz> - 2015-08-24 11:20 +0200
[PATCH 3.12 46/82] md: use kzalloc() when bitmap is disabled Jiri Slaby <jslaby@suse.cz> - 2015-08-24 11:20 +0200
[PATCH 3.12 70/82] path_openat(): fix double fput() Jiri Slaby <jslaby@suse.cz> - 2015-08-24 11:20 +0200
[PATCH 3.12 39/82] ipr: Fix invalid array indexing for HRRQ Jiri Slaby <jslaby@suse.cz> - 2015-08-24 11:30 +0200
[PATCH 3.12 38/82] ipr: Fix incorrect trace indexing Jiri Slaby <jslaby@suse.cz> - 2015-08-24 11:30 +0200
[PATCH 3.12 51/82] ARM: OMAP2+: hwmod: Fix _wait_target_ready() for hwmods without sysc Jiri Slaby <jslaby@suse.cz> - 2015-08-24 11:30 +0200
[PATCH 3.12 32/82] ARM: realview: fix sparsemem build Jiri Slaby <jslaby@suse.cz> - 2015-08-24 11:30 +0200
[PATCH 3.12 31/82] hwrng: via-rng - Mark device ID table as __maybe_unused Jiri Slaby <jslaby@suse.cz> - 2015-08-24 11:30 +0200
[PATCH 3.12 10/82] ALSA: usb-audio: add dB range mapping for some devices Jiri Slaby <jslaby@suse.cz> - 2015-08-24 11:30 +0200
[PATCH 3.12 33/82] MIPS: Fix sched_getaffinity with MT FPAFF enabled Jiri Slaby <jslaby@suse.cz> - 2015-08-24 11:30 +0200
[PATCH 3.12 43/82] ima: extend "mask" policy matching support Jiri Slaby <jslaby@suse.cz> - 2015-08-24 11:30 +0200
[PATCH 3.12 41/82] USB: sierra: add 1199:68AB device ID Jiri Slaby <jslaby@suse.cz> - 2015-08-24 11:30 +0200
[PATCH 3.12 30/82] 3w-xxxx: fix mis-aligned struct accesses Jiri Slaby <jslaby@suse.cz> - 2015-08-24 11:30 +0200
[PATCH 3.12 40/82] xhci: fix off by one error in TRB DMA address boundary check Jiri Slaby <jslaby@suse.cz> - 2015-08-24 11:30 +0200
[PATCH 3.12 53/82] ALSA: hda - fix cs4210_spdif_automute() Jiri Slaby <jslaby@suse.cz> - 2015-08-24 11:30 +0200
[PATCH 3.12 08/82] genirq: Prevent resend to interrupts marked IRQ_NESTED_THREAD Jiri Slaby <jslaby@suse.cz> - 2015-08-24 11:30 +0200
[PATCH 3.12 52/82] iscsi-target: Fix iscsit_start_kthreads failure OOPs Jiri Slaby <jslaby@suse.cz> - 2015-08-24 11:30 +0200
[PATCH 3.12 27/82] vhost: actually track log eventfd file Jiri Slaby <jslaby@suse.cz> - 2015-08-24 11:30 +0200
[PATCH 3.12 35/82] fsnotify: fix oops in fsnotify_clear_marks_by_group_flags() Jiri Slaby <jslaby@suse.cz> - 2015-08-24 11:30 +0200
[PATCH 3.12 37/82] ipr: Fix locking for unit attention handling Jiri Slaby <jslaby@suse.cz> - 2015-08-24 11:30 +0200
[PATCH 3.12 36/82] drm/radeon/combios: add some validation of lvds values Jiri Slaby <jslaby@suse.cz> - 2015-08-24 11:30 +0200
[PATCH 3.12 47/82] ASoC: pcm1681: Fix setting de-emphasis sampling rate selection Jiri Slaby <jslaby@suse.cz> - 2015-08-24 11:30 +0200
[PATCH 3.12 50/82] crypto: ixp4xx - Remove bogus BUG_ON on scattered dst buffer Jiri Slaby <jslaby@suse.cz> - 2015-08-24 11:30 +0200
[PATCH 3.12 29/82] iscsi-target: Fix iser explicit logout TX kthread leak Jiri Slaby <jslaby@suse.cz> - 2015-08-24 11:30 +0200
[PATCH 3.12 48/82] x86/xen: Probe target addresses in set_aliased_prot() before the hypercall Jiri Slaby <jslaby@suse.cz> - 2015-08-24 11:30 +0200
[PATCH 3.12 49/82] xen/gntdevt: Fix race condition in gntdev_release() Jiri Slaby <jslaby@suse.cz> - 2015-08-24 11:30 +0200
Re: [PATCH 3.12 49/82] xen/gntdevt: Fix race condition in gntdev_release() Luis Henriques <luis.henriques@canonical.com> - 2015-08-25 13:40 +0200
Re: [PATCH 3.12 49/82] xen/gntdevt: Fix race condition in gntdev_release() Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com> - 2015-08-25 14:00 +0200
Re: [PATCH 3.12 49/82] xen/gntdevt: Fix race condition in gntdev_release() Jiri Slaby <jslaby@suse.cz> - 2015-08-25 15:20 +0200
Re: [PATCH 3.12 49/82] xen/gntdevt: Fix race condition in gntdev_release() Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com> - 2015-08-25 16:10 +0200
Re: [PATCH 3.12 49/82] xen/gntdevt: Fix race condition in gntdev_release() Jiri Slaby <jslaby@suse.cz> - 2015-08-27 10:10 +0200
[PATCH 3.12 42/82] ima: add support for new "euid" policy condition Jiri Slaby <jslaby@suse.cz> - 2015-08-24 11:30 +0200
[PATCH 3.12 34/82] MIPS: Make set_pte() SMP safe. Jiri Slaby <jslaby@suse.cz> - 2015-08-24 11:30 +0200
[PATCH 3.12 02/82] futex: Fix a race condition between REQUEUE_PI and task death Jiri Slaby <jslaby@suse.cz> - 2015-08-24 11:40 +0200
[PATCH 3.12 28/82] iscsi-target: Fix use-after-free during TPG session shutdown Jiri Slaby <jslaby@suse.cz> - 2015-08-24 11:40 +0200
[PATCH 3.12 09/82] ALSA: usb-audio: Add MIDI support for Steinberg MI2/MI4 Jiri Slaby <jslaby@suse.cz> - 2015-08-24 11:40 +0200
[PATCH 3.12 15/82] mmc: sdhci-pxav3: fix platform_data is not initialized Jiri Slaby <jslaby@suse.cz> - 2015-08-24 11:40 +0200
[PATCH 3.12 19/82] blkcg: fix gendisk reference leak in blkg_conf_prep() Jiri Slaby <jslaby@suse.cz> - 2015-08-24 11:40 +0200
[PATCH 3.12 26/82] rds: rds_ib_device.refcount overflow Jiri Slaby <jslaby@suse.cz> - 2015-08-24 11:40 +0200
[PATCH 3.12 23/82] xhci: report U3 when link is in resume state Jiri Slaby <jslaby@suse.cz> - 2015-08-24 11:40 +0200
[PATCH 3.12 03/82] HID: usbhid: add Chicony/Pixart usb optical mouse that needs QUIRK_ALWAYS_POLL Jiri Slaby <jslaby@suse.cz> - 2015-08-24 11:40 +0200
[PATCH 3.12 18/82] Input: usbtouchscreen - avoid unresponsive TSC-30 touch screen Jiri Slaby <jslaby@suse.cz> - 2015-08-24 11:40 +0200
[PATCH 3.12 25/82] xhci: do not report PLC when link is in internal resume state Jiri Slaby <jslaby@suse.cz> - 2015-08-24 11:40 +0200
[PATCH 3.12 22/82] xhci: Calculate old endpoints correctly on device reset Jiri Slaby <jslaby@suse.cz> - 2015-08-24 11:40 +0200
[PATCH 3.12 24/82] xhci: prevent bus_suspend if SS port resuming in phase 1 Jiri Slaby <jslaby@suse.cz> - 2015-08-24 11:40 +0200
[PATCH 3.12 06/82] s390/sclp: clear upper register halves in _sclp_print_early Jiri Slaby <jslaby@suse.cz> - 2015-08-24 11:40 +0200
[PATCH 3.12 17/82] tile: use free_bootmem_late() for initrd Jiri Slaby <jslaby@suse.cz> - 2015-08-24 11:40 +0200
[PATCH 3.12 04/82] mm: avoid setting up anonymous pages into file mapping Jiri Slaby <jslaby@suse.cz> - 2015-08-24 11:40 +0200
[PATCH 3.12 16/82] md/raid1: fix test for 'was read error from last working device'. Jiri Slaby <jslaby@suse.cz> - 2015-08-24 11:40 +0200
[PATCH 3.12 20/82] ata: pmp: add quirk for Marvell 4140 SATA PMP Jiri Slaby <jslaby@suse.cz> - 2015-08-24 11:40 +0200
[PATCH 3.12 11/82] ALSA: hda - Fix MacBook Pro 5,2 quirk Jiri Slaby <jslaby@suse.cz> - 2015-08-24 11:40 +0200
[PATCH 3.12 13/82] mac80211: clear subdir_stations when removing debugfs Jiri Slaby <jslaby@suse.cz> - 2015-08-24 11:40 +0200
[PATCH 3.12 21/82] usb-storage: ignore ZTE MF 823 card reader in mode 0x1225 Jiri Slaby <jslaby@suse.cz> - 2015-08-24 11:40 +0200
[PATCH 3.12 14/82] mmc: sdhci-esdhc: Make 8BIT bus work Jiri Slaby <jslaby@suse.cz> - 2015-08-24 11:40 +0200
[PATCH 3.12 12/82] st: null pointer dereference panic caused by use after kref_put by st_open Jiri Slaby <jslaby@suse.cz> - 2015-08-24 11:40 +0200
csiph-web