Re: [PATCH 1/2] mm: add PF_MEMALLOC_NOFS

From	Michal Hocko <mhocko@kernel.org>
Newsgroups	linux.kernel
Subject	Re: [PATCH 1/2] mm: add PF_MEMALLOC_NOFS
Date	2016-04-27 22:10 +0200
Message-ID	<rsDWA-7mg-57@gated-at.bofh.it> (permalink)
References	<rs9OO-76D-1@gated-at.bofh.it> <rs9OP-76D-21@gated-at.bofh.it> <rsvmh-8dS-1@gated-at.bofh.it> <rsvFE-dE-15@gated-at.bofh.it> <rsyWR-2P1-3@gated-at.bofh.it>
Organization	linux.* mail to news gateway

Show all headers | View raw

On Wed 27-04-16 23:44:35, Tetsuo Handa wrote:
> Michal Hocko wrote:
> > On Wed 27-04-16 19:53:21, Tetsuo Handa wrote:
> > [...]
> > > > Let's hope that filesystems will drop direct GFP_NOFS (resp. ~__GFP_FS)
> > > > usage as much and possible and only use a properly documented
> > > > memalloc_nofs_{save,restore} checkpoints where they are appropriate.
> > > 
> > > Is the story simple enough to monotonically replace GFP_NOFS/GFP_NOIO
> > > with GFP_KERNEL after memalloc_no{fs,io}_{save,restore} are inserted?
> > > We sometimes delegate some operations to somebody else. Don't we need to
> > > convey PF_MEMALLOC_NOFS/PF_MEMALLOC_NOIO flags to APIs which interact with
> > > other threads?
> > 
> > We can add an api to do that if that is really needed.
> > 
> 
> I'm not familiar with integrity subsystem.
> But if call traces shown below is possible and evm_verify_hmac() is called from
> genuine GFP_NOFS context, we are currently using GFP_KERNEL incorrectly.
> Therefore, inserting memalloc_nofs_{save,restore} would avoid possible memory
> reclaim deadlock by __GFP_FS.

I am not familiar with this code as well but you are definitely right
that scope GFP_NOFS would be better. I have a suspicious that NOFS is
used here improperly and it just copies the same gfp mask used for
all allocations in the same file without any good reason.

This would be a question for Mimi. Could you clarify please?

> ----------
> static enum integrity_status evm_verify_hmac(struct dentry *dentry, const char *xattr_name, char *xattr_value, size_t xattr_value_len, struct integrity_iint_cache *iint) {
>   rc = vfs_getxattr_alloc(dentry, XATTR_NAME_EVM, (char **)&xattr_data, 0, GFP_NOFS); /***** GFP_NOFS is used here. *****/
>   rc = integrity_digsig_verify(INTEGRITY_KEYRING_EVM, (const char *)xattr_data, xattr_len, calc.digest, sizeof(calc.digest)) {
>     keyring[id] = request_key(&key_type_keyring, keyring_name[id], NULL) {
>       key = request_key_and_link(type, description, callout_info, callout_len, NULL, NULL, KEY_ALLOC_IN_QUOTA) {
>         key = construct_key_and_link(&ctx, callout_info, callout_len, aux, dest_keyring, flags) {
>           ret = construct_alloc_key(ctx, dest_keyring, flags, user, &key) {
>             key = key_alloc(ctx->index_key.type, ctx->index_key.description, ctx->cred->fsuid, ctx->cred->fsgid, ctx->cred, perm, flags) {
>               key = kmem_cache_zalloc(key_jar, GFP_KERNEL); /***** Needs to use GFP_NOFS here if above GFP_NOFS usage is correct. *****/
>             }
>           }
>           ret = construct_key(key, callout_info, callout_len, aux, dest_keyring) {
>             cons = kmalloc(sizeof(*cons), GFP_KERNEL); /***** Ditto. *****/
>             actor = call_sbin_request_key;
>             ret = actor(cons, "create", aux) {
>               ret = call_usermodehelper_keys(argv[0], argv, envp, keyring, UMH_WAIT_PROC) {
>                 info = call_usermodehelper_setup(path, argv, envp, GFP_KERNEL, umh_keys_init, umh_keys_cleanup, session_keyring); /***** Ditto. *****/
>                 return call_usermodehelper_exec(info, wait) {
>                   queue_work(system_unbound_wq, &sub_info->work); /***** Queuing a GFP_NOFS work item here if above GFP_NOFS usage is correct. *****/
>                   wait_for_completion(&done); /***** But kworker uses GFP_KERNEL to create process for executing userspace program. *****/
>                 }
>               }
>             }
>           }
>         }
>       }
>     }
>   }
> }
> ----------
> 
> But there is a path where evm_verify_hmac() calls usermode helper.
> If evm_verify_hmac() calls usermode helper from genuine GFP_NOFS context,
> we will be still failing to tell kworker to use GFP_NOFS.

This would be a terrible thing to do. Because ...

> More problematic thing might be that we queue both GFP_KERNEL work item
> and GFP_NOFS work item into the same work queue. This means that the
> kworker will try __GFP_FS reclaim if current GFP_KERNEL work item
> and be blocked on a fs lock held by next GFP_NOFS work item. Then, simply
> conveying PF_MEMALLOC_NOFS/PF_MEMALLOC_NOIO flags to other threads is not
> sufficient, and we need to create separate workqueues (and respective
> consumers) for GFP_KERNEL work items and GFP_NOFS work items?

... of this very reason. If some GFP_NOFS code path relies on kworkers
and wait for the work synchronously then it really has to make sure that
the WQ has a rescuer and there are no __GFP_FS allocation requeuests
enqueued on the same WQ.

> (Or we have no such problem because khelper_wq was replaced with
> system_unbound_wq ?)

I do not think so. system_unbound_wq still depends to have some workers
and that might be not true under memory pressure.

Thanks!
-- 
Michal Hocko
SUSE Labs

Thread

[PATCH 0/2] scop GFP_NOFS api Michal Hocko <mhocko@kernel.org> - 2016-04-26 14:00 +0200
  [PATCH 2/2] mm, debug: report when GFP_NO{FS,IO} is used explicitly from memalloc_no{fs,io}_{save,restore} context Michal Hocko <mhocko@kernel.org> - 2016-04-26 14:00 +0200
    Re: [PATCH 2/2] mm, debug: report when GFP_NO{FS,IO} is used  explicitly from memalloc_no{fs,io}_{save,restore} context Dave Chinner <david@fromorbit.com> - 2016-04-27 01:00 +0200
      Re: [PATCH 2/2] mm, debug: report when GFP_NO{FS,IO} is used  explicitly from memalloc_no{fs,io}_{save,restore} context Michal Hocko <mhocko@kernel.org> - 2016-04-27 10:10 +0200
        Re: [PATCH 2/2] mm, debug: report when GFP_NO{FS,IO} is used  explicitly from memalloc_no{fs,io}_{save,restore} context Dave Chinner <david@fromorbit.com> - 2016-04-28 01:00 +0200
      Re: [PATCH 2/2] mm, debug: report when GFP_NO{FS,IO} is used  explicitly from memalloc_no{fs,io}_{save,restore} context Michal Hocko <mhocko@kernel.org> - 2016-04-28 10:20 +0200
        Re: [PATCH 2/2] mm, debug: report when GFP_NO{FS,IO} is used  explicitly from memalloc_no{fs,io}_{save,restore} context Dave Chinner <david@fromorbit.com> - 2016-04-29 00:00 +0200
          Re: [PATCH 2/2] mm, debug: report when GFP_NO{FS,IO} is used  explicitly from memalloc_no{fs,io}_{save,restore} context Michal Hocko <mhocko@kernel.org> - 2016-04-29 14:20 +0200
            Re: [PATCH 2/2] mm, debug: report when GFP_NO{FS,IO} is used  explicitly from memalloc_no{fs,io}_{save,restore} context Dave Chinner <david@fromorbit.com> - 2016-04-30 01:50 +0200
              Re: [PATCH 2/2] mm, debug: report when GFP_NO{FS,IO} is used  explicitly from memalloc_no{fs,io}_{save,restore} context Michal Hocko <mhocko@kernel.org> - 2016-05-03 17:40 +0200
                Re: [PATCH 2/2] mm, debug: report when GFP_NO{FS,IO} is used  explicitly from memalloc_no{fs,io}_{save,restore} context Dave Chinner <david@fromorbit.com> - 2016-05-04 02:10 +0200
  [PATCH 1/2] mm: add PF_MEMALLOC_NOFS Michal Hocko <mhocko@kernel.org> - 2016-04-26 14:00 +0200
    Re: [PATCH 1/2] mm: add PF_MEMALLOC_NOFS Dave Chinner <david@fromorbit.com> - 2016-04-27 01:10 +0200
      Re: [PATCH 1/2] mm: add PF_MEMALLOC_NOFS Michal Hocko <mhocko@kernel.org> - 2016-04-27 10:00 +0200
    Re: [PATCH 1/2] mm: add PF_MEMALLOC_NOFS Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp> - 2016-04-27 13:00 +0200
      Re: [PATCH 1/2] mm: add PF_MEMALLOC_NOFS Michal Hocko <mhocko@kernel.org> - 2016-04-27 13:20 +0200
        Re: [PATCH 1/2] mm: add PF_MEMALLOC_NOFS Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp> - 2016-04-27 16:50 +0200
          Re: [PATCH 1/2] mm: add PF_MEMALLOC_NOFS Michal Hocko <mhocko@kernel.org> - 2016-04-27 22:10 +0200
    [PATCH 1.1/2] xfs: abstract PF_FSTRANS to PF_MEMALLOC_NOFS Michal Hocko <mhocko@kernel.org> - 2016-04-27 14:00 +0200
      [PATCH 1.2/2] mm: introduce memalloc_nofs_{save,restore} API Michal Hocko <mhocko@kernel.org> - 2016-04-27 14:00 +0200
        Re: [PATCH 1.2/2] mm: introduce memalloc_nofs_{save,restore} API Michal Hocko <mhocko@kernel.org> - 2016-04-27 15:10 +0200
        Re: [PATCH 1.2/2] mm: introduce memalloc_nofs_{save,restore} API Michal Hocko <mhocko@kernel.org> - 2016-04-27 22:10 +0200
          Re: [PATCH 1.2/2] mm: introduce memalloc_nofs_{save,restore} API Michal Hocko <mhocko@kernel.org> - 2016-04-27 22:40 +0200
        Re: [PATCH 1.2/2] mm: introduce memalloc_nofs_{save,restore} API Michal Hocko <mhocko@kernel.org> - 2016-04-27 23:20 +0200
      Re: [PATCH 1.1/2] xfs: abstract PF_FSTRANS to PF_MEMALLOC_NOFS Andreas Dilger <adilger@dilger.ca> - 2016-04-27 19:50 +0200
        Re: [PATCH 1.1/2] xfs: abstract PF_FSTRANS to PF_MEMALLOC_NOFS Michal Hocko <mhocko@kernel.org> - 2016-04-27 21:50 +0200
  Re: [PATCH 0/2] scop GFP_NOFS api NeilBrown <mr@neil.brown.name> - 2016-04-29 08:20 +0200
    Re: [Cluster-devel] [PATCH 0/2] scop GFP_NOFS api Steven Whitehouse <swhiteho@redhat.com> - 2016-04-29 12:30 +0200
      Re: [Cluster-devel] [PATCH 0/2] scop GFP_NOFS api NeilBrown <mr@neil.brown.name> - 2016-04-30 23:20 +0200
    Re: [PATCH 0/2] scop GFP_NOFS api Michal Hocko <mhocko@kernel.org> - 2016-04-29 14:10 +0200
      Re: [PATCH 0/2] scop GFP_NOFS api Dave Chinner <david@fromorbit.com> - 2016-04-30 02:30 +0200
      Re: [PATCH 0/2] scop GFP_NOFS api NeilBrown <mr@neil.brown.name> - 2016-05-01 00:00 +0200
        Re: [PATCH 0/2] scop GFP_NOFS api Michal Hocko <mhocko@kernel.org> - 2016-05-03 17:20 +0200
          Re: [PATCH 0/2] scop GFP_NOFS api NeilBrown <mr@neil.brown.name> - 2016-05-04 01:30 +0200
    Re: [PATCH 0/2] scop GFP_NOFS api Dave Chinner <david@fromorbit.com> - 2016-04-30 02:20 +0200
      Re: [PATCH 0/2] scop GFP_NOFS api NeilBrown <mr@neil.brown.name> - 2016-05-01 00:30 +0200
        Re: [PATCH 0/2] scop GFP_NOFS api Dave Chinner <david@fromorbit.com> - 2016-05-04 03:10 +0200
          Re: [PATCH 0/2] scop GFP_NOFS api NeilBrown <mr@neil.brown.name> - 2016-05-06 05:30 +0200

csiph-web