Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]


Groups > linux.kernel > #1569103 > unrolled thread

[PATCH 00/60] staging: lustre: batches of fixes for lustre client

Started byJames Simmons <jsimmons@infradead.org>
First post2017-01-29 01:20 +0100
Last post2017-01-29 06:00 +0100
Articles 20 on this page of 50 — 2 participants

Back to article view | Back to linux.kernel


Contents

  [PATCH 00/60] staging: lustre: batches of fixes for lustre client James Simmons <jsimmons@infradead.org> - 2017-01-29 01:20 +0100
    [PATCH 21/60] staging: lustre: ptlrpc: correct use of list_add_tail() James Simmons <jsimmons@infradead.org> - 2017-01-29 01:20 +0100
    [PATCH 11/60] staging: lustre: obd: RCU stalls in lu_cache_shrink_count() James Simmons <jsimmons@infradead.org> - 2017-01-29 01:20 +0100
    [PATCH 12/60] staging: lustre: lmv: Error not handled for lmv_find_target James Simmons <jsimmons@infradead.org> - 2017-01-29 01:20 +0100
    [PATCH 28/60] staging: lustre: ldlm: ASSERTION(flock->blocking_export!=0) failed James Simmons <jsimmons@infradead.org> - 2017-01-29 01:20 +0100
    [PATCH 34/60] staging: lustre: libcfs: default CPT matches NUMA topology James Simmons <jsimmons@infradead.org> - 2017-01-29 01:20 +0100
    [PATCH 25/60] staging: lustre: llite: normal user can't set FS default stripe James Simmons <jsimmons@infradead.org> - 2017-01-29 01:20 +0100
    [PATCH 07/60] staging: lustre: ptlrpc: set proper mbits for EINPROGRESS resend James Simmons <jsimmons@infradead.org> - 2017-01-29 01:20 +0100
    [PATCH 19/60] staging: lustre: llite: handle inactive OSTs better in statfs James Simmons <jsimmons@infradead.org> - 2017-01-29 01:20 +0100
    [PATCH 05/60] staging: lustre: llite: check request != NULL in ll_migrate James Simmons <jsimmons@infradead.org> - 2017-01-29 01:20 +0100
      Re: [PATCH 05/60] staging: lustre: llite: check request != NULL in  ll_migrate Dan Carpenter <dan.carpenter@oracle.com> - 2017-01-30 12:40 +0100
    [PATCH 03/60] staging: lustre: clio: add cl_page LRU shrinker James Simmons <jsimmons@infradead.org> - 2017-01-29 01:20 +0100
    [PATCH 09/60] staging: lustre: lmv: Correctly generate target_obd James Simmons <jsimmons@infradead.org> - 2017-01-29 01:20 +0100
    [PATCH 04/60] staging: lustre: mdc: quiet console message for known -EINTR James Simmons <jsimmons@infradead.org> - 2017-01-29 01:20 +0100
    [PATCH 33/60] staging: lustre: libcfs: avoid stomping on module param cpu_pattern James Simmons <jsimmons@infradead.org> - 2017-01-29 01:20 +0100
    [PATCH 31/60] staging: lustre: clio: sync write should update mtime James Simmons <jsimmons@infradead.org> - 2017-01-29 01:20 +0100
    [PATCH 35/60] staging: lustre: lov: ld_target could be NULL James Simmons <jsimmons@infradead.org> - 2017-01-29 01:20 +0100
    [PATCH 18/60] staging: lustre: ptlrpc: skip lock if export failed James Simmons <jsimmons@infradead.org> - 2017-01-29 01:20 +0100
    [PATCH 27/60] staging: lustre: mgc: handle config_llog_data::cld_refcount properly James Simmons <jsimmons@infradead.org> - 2017-01-29 01:20 +0100
    [PATCH 16/60] staging: lustre: llite: don't ignore layout for group lock request James Simmons <jsimmons@infradead.org> - 2017-01-29 01:20 +0100
    [PATCH 36/60] staging: lustre: header: remove assert from interval_set() James Simmons <jsimmons@infradead.org> - 2017-01-29 01:20 +0100
    [PATCH 13/60] staging: lustre: obdclass: health_check to report unhealthy upon LBUG James Simmons <jsimmons@infradead.org> - 2017-01-29 01:20 +0100
      Re: [PATCH 13/60] staging: lustre: obdclass: health_check to report  unhealthy upon LBUG Dan Carpenter <dan.carpenter@oracle.com> - 2017-01-30 13:50 +0100
    [PATCH 15/60] staging: lustre: hsm: stack overrun in hai_dump_data_field James Simmons <jsimmons@infradead.org> - 2017-01-29 01:20 +0100
    [PATCH 55/60] staging: lustre: obd: move s3 in lmd_parse to inner loop James Simmons <jsimmons@infradead.org> - 2017-01-29 01:30 +0100
    [PATCH 42/60] staging: lustre: ptlrpc: allow blocking asts to be delayed James Simmons <jsimmons@infradead.org> - 2017-01-29 01:30 +0100
    [PATCH 53/60] staging: lustre: ptlrpc: update replay cursor when close during replay James Simmons <jsimmons@infradead.org> - 2017-01-29 01:30 +0100
    [PATCH 54/60] staging: lustre: fid: Change positional struct initializers to C99 James Simmons <jsimmons@infradead.org> - 2017-01-29 01:30 +0100
    [PATCH 43/60] staging: lustre: obd: remove OBD_NOTIFY_CREATE James Simmons <jsimmons@infradead.org> - 2017-01-29 01:30 +0100
    [PATCH 41/60] staging: lustre: osc: osc_match_base prototype differs from declaration James Simmons <jsimmons@infradead.org> - 2017-01-29 01:30 +0100
    [PATCH 51/60] staging: lustre: ptlrpc: update MODULE_PARAM_DESC in ptlrpcd.c James Simmons <jsimmons@infradead.org> - 2017-01-29 01:30 +0100
    [PATCH 50/60] staging: lustre: ptlrpc: remove unused pc->pc_env James Simmons <jsimmons@infradead.org> - 2017-01-29 01:30 +0100
    [PATCH 49/60] staging: lustre: socklnd: remove socklnd_init_msg James Simmons <jsimmons@infradead.org> - 2017-01-29 01:30 +0100
    [PATCH 52/60] staging: lustre: linkea: linkEA size limitation James Simmons <jsimmons@infradead.org> - 2017-01-29 01:30 +0100
    [PATCH 45/60] staging: lustre: libcfs: Change positional struct initializers to C99 James Simmons <jsimmons@infradead.org> - 2017-01-29 01:30 +0100
    [PATCH 39/60] staging: libcfs: remove integer types abstraction from libcfs James Simmons <jsimmons@infradead.org> - 2017-01-29 01:30 +0100
    [PATCH 48/60] staging: lustre: ksocklnd: ignore timedout TX on closing connection James Simmons <jsimmons@infradead.org> - 2017-01-29 01:30 +0100
    [PATCH 58/60] staging: lustre: osc: avoid 64 divide in osc_cache_too_much James Simmons <jsimmons@infradead.org> - 2017-01-29 01:40 +0100
    [PATCH 57/60] staging: lustre: lmv: remove nlink check in lmv_revalidate_slaves James Simmons <jsimmons@infradead.org> - 2017-01-29 01:40 +0100
    [PATCH 60/60] staging: lustre: libcfs: fix minimum size check for libcfs ioctl James Simmons <jsimmons@infradead.org> - 2017-01-29 01:40 +0100
      Re: [PATCH 60/60] staging: lustre: libcfs: fix minimum size check  for libcfs ioctl Dan Carpenter <dan.carpenter@oracle.com> - 2017-01-30 12:00 +0100
      Re: [PATCH 60/60] staging: lustre: libcfs: fix minimum size check  for libcfs ioctl Dan Carpenter <dan.carpenter@oracle.com> - 2017-01-30 12:00 +0100
    [PATCH 56/60] staging: lustre: llite: don't invoke direct_IO for the EOF case James Simmons <jsimmons@infradead.org> - 2017-01-29 01:40 +0100
    [PATCH 01/60] staging: lustre: llite: Remove access of stripe in ll_setattr_raw James Simmons <jsimmons@infradead.org> - 2017-01-29 01:40 +0100
    [PATCH 59/60] staging: lustre: ptlrpc : remove userland usage from ptlrpc James Simmons <jsimmons@infradead.org> - 2017-01-29 01:40 +0100
    [PATCH 44/60] staging: lustre: libcfs: fix error messages James Simmons <jsimmons@infradead.org> - 2017-01-29 01:50 +0100
    [PATCH 38/60] staging: lustre: llite: Adding timed wait in ll_umount_begin James Simmons <jsimmons@infradead.org> - 2017-01-29 02:20 +0100
    [PATCH 46/60] staging: lustre: mdc: Make IT_OPEN take lookup bits lock James Simmons <jsimmons@infradead.org> - 2017-01-29 02:50 +0100
    [PATCH 47/60] staging: lustre: mdc: avoid returning freed request James Simmons <jsimmons@infradead.org> - 2017-01-29 02:50 +0100
    [PATCH 40/60] staging: ptlrpc: leaked rs on difficult reply James Simmons <jsimmons@infradead.org> - 2017-01-29 06:00 +0100

Page 1 of 3  [1] 2 3  Next page →


#1569103 — [PATCH 00/60] staging: lustre: batches of fixes for lustre client

FromJames Simmons <jsimmons@infradead.org>
Date2017-01-29 01:20 +0100
Subject[PATCH 00/60] staging: lustre: batches of fixes for lustre client
Message-ID<t4Lea-8ik-3@gated-at.bofh.it>
Batch of missing fixes for lustre for the upstream client.

Alex Zhuravlev (1):
  staging: lustre: obdclass: do not call lu_site_purge() for single object exceed

Alexander Boyko (1):
  staging: lustre: ptlrpc: skip lock if export failed

Andreas Dilger (3):
  staging: lustre: mdc: quiet console message for known -EINTR
  staging: lustre: obdclass: add more info to sysfs version string
  staging: lustre: llite: handle inactive OSTs better in statfs

Andriy Skulysh (1):
  staging: lustre: ldlm: ASSERTION(flock->blocking_export!=0) failed

Ann Koehler (1):
  staging: lustre: obd: RCU stalls in lu_cache_shrink_count()

Ben Evans (1):
  staging: lustre: lustre: Remove old commented out code

Bobi Jam (3):
  staging: lustre: clio: add cl_page LRU shrinker
  staging: lustre: lov: ld_target could be NULL
  staging: lustre: llite: specify READA debug mask for ras_update

Bruno Faccini (1):
  staging: lustre: obdclass: health_check to report unhealthy upon LBUG

Dmitry Eremin (6):
  staging: lustre: llite: Setting xattr are properly checked with and without ACLs
  staging: lustre: libcfs: avoid stomping on module param cpu_pattern
  staging: lustre: libcfs: default CPT matches NUMA topology
  staging: lustre: libcfs: fix error messages
  staging: lustre: ptlrpc: remove unused pc->pc_env
  staging: lustre: ptlrpc: update MODULE_PARAM_DESC in ptlrpcd.c

Fan Yong (4):
  staging: lustre: fid: fix race in fid allocation
  staging: lustre: mgc: handle config_llog_data::cld_refcount properly
  staging: lustre: ptlrpc: comment for FLD_QUERY RPC reply swab
  staging: lustre: linkea: linkEA size limitation

Giuseppe Di Natale (1):
  staging: lustre: lmv: Correctly generate target_obd

James Simmons (7):
  staging: lustre: header: remove assert from interval_set()
  staging: libcfs: remove integer types abstraction from libcfs
  staging: lustre: socklnd: remove socklnd_init_msg
  staging: lustre: obd: move s3 in lmd_parse to inner loop
  staging: lustre: osc: avoid 64 divide in osc_cache_too_much
  staging: lustre: ptlrpc : remove userland usage from ptlrpc
  staging: lustre: libcfs: fix minimum size check for libcfs ioctl

Jeremy Filizetti (1):
  staging: lustre: ldlm: Restore connect flags on failure

Jinshan Xiong (4):
  staging: lustre: llite: Remove access of stripe in ll_setattr_raw
  staging: lustre: clio: revise readahead to support 16MB IO
  staging: lustre: llite: don't ignore layout for group lock request
  staging: lustre: osc: limits the number of chunks in write RPC

John L. Hammond (5):
  staging: lustre: llite: remove obsolete comment for ll_unlink()
  staging: lustre: ptlrpc: correct use of list_add_tail()
  staging: lustre: lmv: remove unused placement parameter
  staging: lustre: obd: remove OBD_NOTIFY_CREATE
  staging: lustre: mdc: avoid returning freed request

Lai Siyao (2):
  staging: lustre: statahead: drop support for remote entry
  staging: lustre: llite: normal user can't set FS default stripe

Liang Zhen (1):
  staging: lustre: ksocklnd: ignore timedout TX on closing connection

Nathaniel Clark (1):
  staging: lustre: lov: Ensure correct operation for large object sizes

Niu Yawei (4):
  staging: lustre: ptlrpc: set proper mbits for EINPROGRESS resend
  staging: lustre: clio: sync write should update mtime
  staging: ptlrpc: leaked rs on difficult reply
  staging: lustre: ptlrpc: update replay cursor when close during replay

Oleg Drokin (1):
  staging: lustre: llite: Trust creates in revalidate too.

Patrick Farrell (1):
  staging: lustre: mdc: Make IT_OPEN take lookup bits lock

Rahul Deshmukh (1):
  staging: lustre: llite: Adding timed wait in ll_umount_begin

Steve Guminski (3):
  staging: lustre: osc: osc_match_base prototype differs from declaration
  staging: lustre: libcfs: Change positional struct initializers to C99
  staging: lustre: fid: Change positional struct initializers to C99

Ulka Vaze (1):
  staging: lustre: lmv: Error not handled for lmv_find_target

Vladimir Saveliev (1):
  staging: lustre: ptlrpc: allow blocking asts to be delayed

Yang Sheng (1):
  staging: lustre: llite: don't invoke direct_IO for the EOF case

frank zago (1):
  staging: lustre: hsm: stack overrun in hai_dump_data_field

wang di (2):
  staging: lustre: llite: check request != NULL in ll_migrate
  staging: lustre: lmv: remove nlink check in lmv_revalidate_slaves

 .../lustre/include/linux/libcfs/libcfs_crypto.h    |  60 +++++--
 .../lustre/include/linux/libcfs/linux/libcfs.h     |   4 -
 .../staging/lustre/include/linux/lnet/socklnd.h    |   9 -
 .../staging/lustre/lnet/klnds/o2iblnd/o2iblnd_cb.c |   2 +-
 .../staging/lustre/lnet/klnds/socklnd/socklnd.c    |   2 +-
 .../staging/lustre/lnet/klnds/socklnd/socklnd_cb.c |  29 +--
 drivers/staging/lustre/lnet/libcfs/debug.c         |   2 +-
 .../staging/lustre/lnet/libcfs/linux/linux-cpu.c   |  17 +-
 .../lustre/lnet/libcfs/linux/linux-module.c        |   2 +-
 drivers/staging/lustre/lnet/libcfs/workitem.c      |   2 +-
 drivers/staging/lustre/lnet/lnet/acceptor.c        |   4 +-
 drivers/staging/lustre/lnet/selftest/module.c      |   3 +-
 drivers/staging/lustre/lustre/fid/fid_lib.c        |   7 +-
 drivers/staging/lustre/lustre/fid/fid_request.c    |  55 +++---
 drivers/staging/lustre/lustre/fid/lproc_fid.c      |  12 +-
 drivers/staging/lustre/lustre/include/cl_object.h  |  10 +-
 .../staging/lustre/lustre/include/interval_tree.h  |  12 +-
 drivers/staging/lustre/lustre/include/lu_object.h  |  14 +-
 .../lustre/lustre/include/lustre/lustre_idl.h      |   5 +-
 .../lustre/lustre/include/lustre/lustre_user.h     |  18 +-
 .../staging/lustre/lustre/include/lustre_linkea.h  |  15 +-
 drivers/staging/lustre/lustre/include/lustre_net.h |   4 -
 .../lustre/lustre/include/lustre_req_layout.h      |  10 +-
 drivers/staging/lustre/lustre/include/obd.h        |  18 +-
 drivers/staging/lustre/lustre/include/obd_class.h  |   5 +-
 drivers/staging/lustre/lustre/ldlm/ldlm_extent.c   |   6 +-
 drivers/staging/lustre/lustre/ldlm/ldlm_flock.c    |   1 -
 .../staging/lustre/lustre/ldlm/ldlm_inodebits.c    |   1 -
 drivers/staging/lustre/lustre/ldlm/ldlm_lib.c      |  13 +-
 drivers/staging/lustre/lustre/ldlm/ldlm_lock.c     |   7 +
 drivers/staging/lustre/lustre/llite/dcache.c       |  13 +-
 drivers/staging/lustre/lustre/llite/dir.c          |  14 +-
 drivers/staging/lustre/lustre/llite/file.c         |  83 +++++----
 drivers/staging/lustre/lustre/llite/lcommon_cl.c   |   9 +-
 drivers/staging/lustre/lustre/llite/lcommon_misc.c |   2 +-
 .../staging/lustre/lustre/llite/llite_internal.h   |  16 +-
 drivers/staging/lustre/lustre/llite/llite_lib.c    | 124 ++++++-------
 drivers/staging/lustre/lustre/llite/namei.c        |   5 -
 drivers/staging/lustre/lustre/llite/range_lock.c   |  10 +-
 drivers/staging/lustre/lustre/llite/range_lock.h   |   2 +-
 drivers/staging/lustre/lustre/llite/rw.c           | 199 ++++++++++-----------
 drivers/staging/lustre/lustre/llite/rw26.c         |   4 +
 drivers/staging/lustre/lustre/llite/statahead.c    |  94 ++++------
 drivers/staging/lustre/lustre/llite/vvp_io.c       |  17 +-
 drivers/staging/lustre/lustre/llite/xattr.c        |   9 +
 drivers/staging/lustre/lustre/lmv/lmv_intent.c     |  16 +-
 drivers/staging/lustre/lustre/lmv/lmv_obd.c        |  65 ++++---
 drivers/staging/lustre/lustre/lmv/lproc_lmv.c      |  85 ++-------
 drivers/staging/lustre/lustre/lov/lov_ea.c         |  22 +--
 drivers/staging/lustre/lustre/lov/lov_io.c         |   7 +-
 drivers/staging/lustre/lustre/lov/lov_lock.c       |   5 +
 drivers/staging/lustre/lustre/lov/lov_obd.c        |   2 -
 drivers/staging/lustre/lustre/lov/lov_object.c     |  33 +++-
 drivers/staging/lustre/lustre/lov/lov_request.c    |   6 +-
 drivers/staging/lustre/lustre/mdc/mdc_internal.h   |   3 +-
 drivers/staging/lustre/lustre/mdc/mdc_locks.c      |  18 +-
 drivers/staging/lustre/lustre/mdc/mdc_request.c    |   9 +-
 drivers/staging/lustre/lustre/mgc/mgc_request.c    | 183 ++++++++++---------
 drivers/staging/lustre/lustre/obdclass/linkea.c    |  70 ++++++--
 .../lustre/lustre/obdclass/linux/linux-module.c    |   8 +-
 drivers/staging/lustre/lustre/obdclass/lu_object.c | 106 +++++------
 drivers/staging/lustre/lustre/obdclass/obd_mount.c |   3 +-
 drivers/staging/lustre/lustre/osc/osc_cache.c      | 125 +++++++++----
 drivers/staging/lustre/lustre/osc/osc_internal.h   |  15 +-
 drivers/staging/lustre/lustre/osc/osc_io.c         |  15 +-
 drivers/staging/lustre/lustre/osc/osc_page.c       |  98 +++++++++-
 drivers/staging/lustre/lustre/osc/osc_request.c    |  21 +++
 drivers/staging/lustre/lustre/ptlrpc/client.c      |  24 ++-
 drivers/staging/lustre/lustre/ptlrpc/events.c      |   3 +-
 drivers/staging/lustre/lustre/ptlrpc/layout.c      |  26 ++-
 drivers/staging/lustre/lustre/ptlrpc/niobuf.c      |   5 +-
 drivers/staging/lustre/lustre/ptlrpc/ptlrpcd.c     |  18 +-
 drivers/staging/lustre/lustre/ptlrpc/recover.c     |  24 +--
 drivers/staging/lustre/lustre/ptlrpc/sec_gc.c      |   2 +-
 drivers/staging/lustre/lustre/ptlrpc/service.c     |  21 +--
 drivers/staging/lustre/lustre/ptlrpc/wiretest.c    |  16 +-
 76 files changed, 1136 insertions(+), 868 deletions(-)

-- 
1.8.3.1

[toc] | [next] | [standalone]


#1569104 — [PATCH 21/60] staging: lustre: ptlrpc: correct use of list_add_tail()

FromJames Simmons <jsimmons@infradead.org>
Date2017-01-29 01:20 +0100
Subject[PATCH 21/60] staging: lustre: ptlrpc: correct use of list_add_tail()
Message-ID<t4LnQ-8lZ-37@gated-at.bofh.it>
In reply to#1569103
From: "John L. Hammond" <john.hammond@intel.com>

In sptlrpc_gc_add_sec() swap the arguments to list_add_tail() so that
it does what we meant it to do.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-8270
Reviewed-on: http://review.whamcloud.com/20784
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Signed-off-by: James Simmons <jsimmons@infradead.org>
---
 drivers/staging/lustre/lustre/ptlrpc/sec_gc.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/staging/lustre/lustre/ptlrpc/sec_gc.c b/drivers/staging/lustre/lustre/ptlrpc/sec_gc.c
index 8ffd000..026bec7 100644
--- a/drivers/staging/lustre/lustre/ptlrpc/sec_gc.c
+++ b/drivers/staging/lustre/lustre/ptlrpc/sec_gc.c
@@ -66,7 +66,7 @@ void sptlrpc_gc_add_sec(struct ptlrpc_sec *sec)
 	sec->ps_gc_next = ktime_get_real_seconds() + sec->ps_gc_interval;
 
 	spin_lock(&sec_gc_list_lock);
-	list_add_tail(&sec_gc_list, &sec->ps_gc_list);
+	list_add_tail(&sec->ps_gc_list, &sec_gc_list);
 	spin_unlock(&sec_gc_list_lock);
 
 	CDEBUG(D_SEC, "added sec %p(%s)\n", sec, sec->ps_policy->sp_name);
-- 
1.8.3.1

[toc] | [prev] | [next] | [standalone]


#1569105 — [PATCH 11/60] staging: lustre: obd: RCU stalls in lu_cache_shrink_count()

FromJames Simmons <jsimmons@infradead.org>
Date2017-01-29 01:20 +0100
Subject[PATCH 11/60] staging: lustre: obd: RCU stalls in lu_cache_shrink_count()
Message-ID<t4LnQ-8lZ-27@gated-at.bofh.it>
In reply to#1569103
From: Ann Koehler <amk@cray.com>

The algorithm for counting freeable objects in the lu_cache shrinker
does not scale with the number of cpus. The LU_SS_LRU_LEN counter
for each cpu is read and summed at shrink time while holding the
lu_sites_guard mutex. With a large number of cpus and low memory
conditions, processes bottleneck on the mutex.

This mod reduces the time spent counting by using the kernel's percpu
counter functions to maintain the length of a site's lru. The summing
occurs when a percpu value is incremented or decremented and a
threshold is exceeded. lu_cache_shrink_count() simply returns the
last such computed sum.

This mod also replaces the lu_sites_guard mutex with a rw semaphore.
The lock protects the lu_site list, which is modified when a file
system is mounted/umounted or when the lu_site is purged.
lu_cache_shrink_count simply reads data so it does not need to wait
for other readers. lu_cache_shrink_scan, which actually frees the
unused objects, is still serialized.

Signed-off-by: Ann Koehler <amk@cray.com>
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-7997
Reviewed-on: http://review.whamcloud.com/19390
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Signed-off-by: James Simmons <jsimmons@infradead.org>
---
 drivers/staging/lustre/lustre/include/lu_object.h  |  6 +-
 drivers/staging/lustre/lustre/obdclass/lu_object.c | 80 ++++++++++------------
 2 files changed, 43 insertions(+), 43 deletions(-)

diff --git a/drivers/staging/lustre/lustre/include/lu_object.h b/drivers/staging/lustre/lustre/include/lu_object.h
index 69b2812..f442a96 100644
--- a/drivers/staging/lustre/lustre/include/lu_object.h
+++ b/drivers/staging/lustre/lustre/include/lu_object.h
@@ -34,6 +34,7 @@
 #define __LUSTRE_LU_OBJECT_H
 
 #include <stdarg.h>
+#include <linux/percpu_counter.h>
 #include "../../include/linux/libcfs/libcfs.h"
 #include "lustre/lustre_idl.h"
 #include "lu_ref.h"
@@ -580,7 +581,6 @@ enum {
 	LU_SS_CACHE_RACE,
 	LU_SS_CACHE_DEATH_RACE,
 	LU_SS_LRU_PURGED,
-	LU_SS_LRU_LEN,	/* # of objects in lsb_lru lists */
 	LU_SS_LAST_STAT
 };
 
@@ -635,6 +635,10 @@ struct lu_site {
 	 * XXX: a hack! fld has to find md_site via site, remove when possible
 	 */
 	struct seq_server_site	*ld_seq_site;
+	/**
+	 * Number of objects in lsb_lru_lists - used for shrinking
+	 */
+	struct percpu_counter	 ls_lru_len_counter;
 };
 
 static inline struct lu_site_bkt_data *
diff --git a/drivers/staging/lustre/lustre/obdclass/lu_object.c b/drivers/staging/lustre/lustre/obdclass/lu_object.c
index 7971562..1805861 100644
--- a/drivers/staging/lustre/lustre/obdclass/lu_object.c
+++ b/drivers/staging/lustre/lustre/obdclass/lu_object.c
@@ -151,7 +151,7 @@ void lu_object_put(const struct lu_env *env, struct lu_object *o)
 		LASSERT(list_empty(&top->loh_lru));
 		list_add_tail(&top->loh_lru, &bkt->lsb_lru);
 		bkt->lsb_lru_len++;
-		lprocfs_counter_incr(site->ls_stats, LU_SS_LRU_LEN);
+		percpu_counter_inc(&site->ls_lru_len_counter);
 		CDEBUG(D_INODE, "Add %p to site lru. hash: %p, bkt: %p, lru_len: %ld\n",
 		       o, site->ls_obj_hash, bkt, bkt->lsb_lru_len);
 		cfs_hash_bd_unlock(site->ls_obj_hash, &bd, 1);
@@ -202,7 +202,7 @@ void lu_object_unhash(const struct lu_env *env, struct lu_object *o)
 			list_del_init(&top->loh_lru);
 			bkt = cfs_hash_bd_extra_get(obj_hash, &bd);
 			bkt->lsb_lru_len--;
-			lprocfs_counter_decr(site->ls_stats, LU_SS_LRU_LEN);
+			percpu_counter_dec(&site->ls_lru_len_counter);
 		}
 		cfs_hash_bd_del_locked(obj_hash, &bd, &top->loh_hash);
 		cfs_hash_bd_unlock(obj_hash, &bd, 1);
@@ -379,7 +379,7 @@ int lu_site_purge(const struct lu_env *env, struct lu_site *s, int nr)
 					       &bd2, &h->loh_hash);
 			list_move(&h->loh_lru, &dispose);
 			bkt->lsb_lru_len--;
-			lprocfs_counter_decr(s->ls_stats, LU_SS_LRU_LEN);
+			percpu_counter_dec(&s->ls_lru_len_counter);
 			if (did_sth == 0)
 				did_sth = 1;
 
@@ -578,7 +578,7 @@ static struct lu_object *htable_lookup(struct lu_site *s,
 		if (!list_empty(&h->loh_lru)) {
 			list_del_init(&h->loh_lru);
 			bkt->lsb_lru_len--;
-			lprocfs_counter_decr(s->ls_stats, LU_SS_LRU_LEN);
+			percpu_counter_dec(&s->ls_lru_len_counter);
 		}
 		return lu_object_top(h);
 	}
@@ -820,7 +820,7 @@ void lu_device_type_fini(struct lu_device_type *ldt)
  * Global list of all sites on this node
  */
 static LIST_HEAD(lu_sites);
-static DEFINE_MUTEX(lu_sites_guard);
+static DECLARE_RWSEM(lu_sites_guard);
 
 /**
  * Global environment used by site shrinker.
@@ -994,9 +994,15 @@ int lu_site_init(struct lu_site *s, struct lu_device *top)
 	unsigned long bits;
 	unsigned long i;
 	char name[16];
+	int rc;
 
 	memset(s, 0, sizeof(*s));
 	mutex_init(&s->ls_purge_mutex);
+
+	rc = percpu_counter_init(&s->ls_lru_len_counter, 0, GFP_NOFS);
+	if (rc)
+		return -ENOMEM;
+
 	snprintf(name, sizeof(name), "lu_site_%s", top->ld_type->ldt_name);
 	for (bits = lu_htable_order(top); bits >= LU_SITE_BITS_MIN; bits--) {
 		s->ls_obj_hash = cfs_hash_create(name, bits, bits,
@@ -1042,12 +1048,6 @@ int lu_site_init(struct lu_site *s, struct lu_device *top)
 			     0, "cache_death_race", "cache_death_race");
 	lprocfs_counter_init(s->ls_stats, LU_SS_LRU_PURGED,
 			     0, "lru_purged", "lru_purged");
-	/*
-	 * Unlike other counters, lru_len can be decremented so
-	 * need lc_sum instead of just lc_count
-	 */
-	lprocfs_counter_init(s->ls_stats, LU_SS_LRU_LEN,
-			     LPROCFS_CNTR_AVGMINMAX, "lru_len", "lru_len");
 
 	INIT_LIST_HEAD(&s->ls_linkage);
 	s->ls_top_dev = top;
@@ -1069,9 +1069,11 @@ int lu_site_init(struct lu_site *s, struct lu_device *top)
  */
 void lu_site_fini(struct lu_site *s)
 {
-	mutex_lock(&lu_sites_guard);
+	down_write(&lu_sites_guard);
 	list_del_init(&s->ls_linkage);
-	mutex_unlock(&lu_sites_guard);
+	up_write(&lu_sites_guard);
+
+	percpu_counter_destroy(&s->ls_lru_len_counter);
 
 	if (s->ls_obj_hash) {
 		cfs_hash_putref(s->ls_obj_hash);
@@ -1097,11 +1099,11 @@ int lu_site_init_finish(struct lu_site *s)
 {
 	int result;
 
-	mutex_lock(&lu_sites_guard);
+	down_write(&lu_sites_guard);
 	result = lu_context_refill(&lu_shrink_env.le_ctx);
 	if (result == 0)
 		list_add(&s->ls_linkage, &lu_sites);
-	mutex_unlock(&lu_sites_guard);
+	up_write(&lu_sites_guard);
 	return result;
 }
 EXPORT_SYMBOL(lu_site_init_finish);
@@ -1820,12 +1822,15 @@ static void lu_site_stats_get(struct cfs_hash *hs,
 }
 
 /*
- * lu_cache_shrink_count returns the number of cached objects that are
- * candidates to be freed by shrink_slab(). A counter, which tracks
- * the number of items in the site's lru, is maintained in the per cpu
- * stats of each site. The counter is incremented when an object is added
- * to a site's lru and decremented when one is removed. The number of
- * free-able objects is the sum of all per cpu counters for all sites.
+ * lu_cache_shrink_count() returns an approximate number of cached objects
+ * that can be freed by shrink_slab(). A counter, which tracks the
+ * number of items in the site's lru, is maintained in a percpu_counter
+ * for each site. The percpu values are incremented and decremented as
+ * objects are added or removed from the lru. The percpu values are summed
+ * and saved whenever a percpu value exceeds a threshold. Thus the saved,
+ * summed value at any given time may not accurately reflect the current
+ * lru length. But this value is sufficiently accurate for the needs of
+ * a shrinker.
  *
  * Using a per cpu counter is a compromise solution to concurrent access:
  * lu_object_put() can update the counter without locking the site and
@@ -1842,11 +1847,10 @@ static unsigned long lu_cache_shrink_count(struct shrinker *sk,
 	if (!(sc->gfp_mask & __GFP_FS))
 		return 0;
 
-	mutex_lock(&lu_sites_guard);
-	list_for_each_entry_safe(s, tmp, &lu_sites, ls_linkage) {
-		cached += ls_stats_read(s->ls_stats, LU_SS_LRU_LEN);
-	}
-	mutex_unlock(&lu_sites_guard);
+	down_read(&lu_sites_guard);
+	list_for_each_entry_safe(s, tmp, &lu_sites, ls_linkage)
+		cached += percpu_counter_read_positive(&s->ls_lru_len_counter);
+	up_read(&lu_sites_guard);
 
 	cached = (cached / 100) * sysctl_vfs_cache_pressure;
 	CDEBUG(D_INODE, "%ld objects cached, cache pressure %d\n",
@@ -1877,7 +1881,7 @@ static unsigned long lu_cache_shrink_scan(struct shrinker *sk,
 		 */
 		return SHRINK_STOP;
 
-	mutex_lock(&lu_sites_guard);
+	down_write(&lu_sites_guard);
 	list_for_each_entry_safe(s, tmp, &lu_sites, ls_linkage) {
 		freed = lu_site_purge(&lu_shrink_env, s, remain);
 		remain -= freed;
@@ -1888,7 +1892,7 @@ static unsigned long lu_cache_shrink_scan(struct shrinker *sk,
 		list_move_tail(&s->ls_linkage, &splice);
 	}
 	list_splice(&splice, lu_sites.prev);
-	mutex_unlock(&lu_sites_guard);
+	up_write(&lu_sites_guard);
 
 	return sc->nr_to_scan - remain;
 }
@@ -1925,9 +1929,9 @@ int lu_global_init(void)
 	 * conservatively. This should not be too bad, because this
 	 * environment is global.
 	 */
-	mutex_lock(&lu_sites_guard);
+	down_write(&lu_sites_guard);
 	result = lu_env_init(&lu_shrink_env, LCT_SHRINKER);
-	mutex_unlock(&lu_sites_guard);
+	up_write(&lu_sites_guard);
 	if (result != 0)
 		return result;
 
@@ -1953,9 +1957,9 @@ void lu_global_fini(void)
 	 * Tear shrinker environment down _after_ de-registering
 	 * lu_global_key, because the latter has a value in the former.
 	 */
-	mutex_lock(&lu_sites_guard);
+	down_write(&lu_sites_guard);
 	lu_env_fini(&lu_shrink_env);
-	mutex_unlock(&lu_sites_guard);
+	up_write(&lu_sites_guard);
 
 	lu_ref_global_fini();
 }
@@ -1965,13 +1969,6 @@ static __u32 ls_stats_read(struct lprocfs_stats *stats, int idx)
 	struct lprocfs_counter ret;
 
 	lprocfs_stats_collect(stats, idx, &ret);
-	if (idx == LU_SS_LRU_LEN)
-		/*
-		 * protect against counter on cpu A being decremented
-		 * before counter is incremented on cpu B; unlikely
-		 */
-		return (__u32)((ret.lc_sum > 0) ? ret.lc_sum : 0);
-
 	return (__u32)ret.lc_count;
 }
 
@@ -1986,7 +1983,7 @@ int lu_site_stats_print(const struct lu_site *s, struct seq_file *m)
 	memset(&stats, 0, sizeof(stats));
 	lu_site_stats_get(s->ls_obj_hash, &stats, 1);
 
-	seq_printf(m, "%d/%d %d/%ld %d %d %d %d %d %d %d %d\n",
+	seq_printf(m, "%d/%d %d/%ld %d %d %d %d %d %d %d\n",
 		   stats.lss_busy,
 		   stats.lss_total,
 		   stats.lss_populated,
@@ -1997,8 +1994,7 @@ int lu_site_stats_print(const struct lu_site *s, struct seq_file *m)
 		   ls_stats_read(s->ls_stats, LU_SS_CACHE_MISS),
 		   ls_stats_read(s->ls_stats, LU_SS_CACHE_RACE),
 		   ls_stats_read(s->ls_stats, LU_SS_CACHE_DEATH_RACE),
-		   ls_stats_read(s->ls_stats, LU_SS_LRU_PURGED),
-		   ls_stats_read(s->ls_stats, LU_SS_LRU_LEN));
+		   ls_stats_read(s->ls_stats, LU_SS_LRU_PURGED));
 	return 0;
 }
 EXPORT_SYMBOL(lu_site_stats_print);
-- 
1.8.3.1

[toc] | [prev] | [next] | [standalone]


#1569106 — [PATCH 12/60] staging: lustre: lmv: Error not handled for lmv_find_target

FromJames Simmons <jsimmons@infradead.org>
Date2017-01-29 01:20 +0100
Subject[PATCH 12/60] staging: lustre: lmv: Error not handled for lmv_find_target
Message-ID<t4LnQ-8lZ-29@gated-at.bofh.it>
In reply to#1569103
From: Ulka Vaze <ulka.vaze@yahoo.in>

This issue is found by smatch; has been reported as-
Unchecked usage of potential ERR_PTR result in lmv_hsm_req_count
and lmv_hsm_req_build. Added ERR_PTR in both functions and also
return value check added.

Signed-off-by: Ulka Vaze <ulka.vaze@yahoo.in>
Signed-off-by: Aditya Pandit <panditadityashreesh@yahoo.com>
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-6523
Reviewed-on: http://review.whamcloud.com/14918
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Signed-off-by: James Simmons <jsimmons@infradead.org>
---
 drivers/staging/lustre/lustre/lmv/lmv_obd.c | 25 ++++++++++++++++++-------
 1 file changed, 18 insertions(+), 7 deletions(-)

diff --git a/drivers/staging/lustre/lustre/lmv/lmv_obd.c b/drivers/staging/lustre/lustre/lmv/lmv_obd.c
index 6a3b83f..915415c 100644
--- a/drivers/staging/lustre/lustre/lmv/lmv_obd.c
+++ b/drivers/staging/lustre/lustre/lmv/lmv_obd.c
@@ -736,16 +736,18 @@ static int lmv_hsm_req_count(struct lmv_obd *lmv,
 	/* count how many requests must be sent to the given target */
 	for (i = 0; i < hur->hur_request.hr_itemcount; i++) {
 		curr_tgt = lmv_find_target(lmv, &hur->hur_user_item[i].hui_fid);
+		if (IS_ERR(curr_tgt))
+			return PTR_ERR(curr_tgt);
 		if (obd_uuid_equals(&curr_tgt->ltd_uuid, &tgt_mds->ltd_uuid))
 			nr++;
 	}
 	return nr;
 }
 
-static void lmv_hsm_req_build(struct lmv_obd *lmv,
-			      struct hsm_user_request *hur_in,
-			      const struct lmv_tgt_desc *tgt_mds,
-			      struct hsm_user_request *hur_out)
+static int lmv_hsm_req_build(struct lmv_obd *lmv,
+			     struct hsm_user_request *hur_in,
+			     const struct lmv_tgt_desc *tgt_mds,
+			     struct hsm_user_request *hur_out)
 {
 	int			i, nr_out;
 	struct lmv_tgt_desc    *curr_tgt;
@@ -756,6 +758,8 @@ static void lmv_hsm_req_build(struct lmv_obd *lmv,
 	for (i = 0; i < hur_in->hur_request.hr_itemcount; i++) {
 		curr_tgt = lmv_find_target(lmv,
 					   &hur_in->hur_user_item[i].hui_fid);
+		if (IS_ERR(curr_tgt))
+			return PTR_ERR(curr_tgt);
 		if (obd_uuid_equals(&curr_tgt->ltd_uuid, &tgt_mds->ltd_uuid)) {
 			hur_out->hur_user_item[nr_out] =
 				hur_in->hur_user_item[i];
@@ -765,6 +769,8 @@ static void lmv_hsm_req_build(struct lmv_obd *lmv,
 	hur_out->hur_request.hr_itemcount = nr_out;
 	memcpy(hur_data(hur_out), hur_data(hur_in),
 	       hur_in->hur_request.hr_data_len);
+
+	return 0;
 }
 
 static int lmv_hsm_ct_unregister(struct lmv_obd *lmv, unsigned int cmd, int len,
@@ -1041,15 +1047,17 @@ static int lmv_iocontrol(unsigned int cmd, struct obd_export *exp,
 		} else {
 			/* split fid list to their respective MDS */
 			for (i = 0; i < count; i++) {
-				unsigned int		nr, reqlen;
-				int			rc1;
 				struct hsm_user_request *req;
+				size_t reqlen;
+				int nr, rc1;
 
 				tgt = lmv->tgts[i];
 				if (!tgt || !tgt->ltd_exp)
 					continue;
 
 				nr = lmv_hsm_req_count(lmv, hur, tgt);
+				if (nr < 0)
+					return nr;
 				if (nr == 0) /* nothing for this MDS */
 					continue;
 
@@ -1061,10 +1069,13 @@ static int lmv_iocontrol(unsigned int cmd, struct obd_export *exp,
 				if (!req)
 					return -ENOMEM;
 
-				lmv_hsm_req_build(lmv, hur, tgt, req);
+				rc1 = lmv_hsm_req_build(lmv, hur, tgt, req);
+				if (rc1 < 0)
+					goto hsm_req_err;
 
 				rc1 = obd_iocontrol(cmd, tgt->ltd_exp, reqlen,
 						    req, uarg);
+hsm_req_err:
 				if (rc1 != 0 && rc == 0)
 					rc = rc1;
 				kvfree(req);
-- 
1.8.3.1

[toc] | [prev] | [next] | [standalone]


#1569107 — [PATCH 28/60] staging: lustre: ldlm: ASSERTION(flock->blocking_export!=0) failed

FromJames Simmons <jsimmons@infradead.org>
Date2017-01-29 01:20 +0100
Subject[PATCH 28/60] staging: lustre: ldlm: ASSERTION(flock->blocking_export!=0) failed
Message-ID<t4LnQ-8lZ-39@gated-at.bofh.it>
In reply to#1569103
From: Andriy Skulysh <andriy.skulysh@seagate.com>

Whole policy structure was zeroed twice. Once during enqueue
and second time during resend or replay. Policy structure
should be initialized with default values only in ldlm_lock_new().

Signed-off-by: Andriy Skulysh <andriy.skulysh@seagate.com>
Signed-off-by: Ben Evans <bevans@cray.com>
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-8349
Seagate-bug-id: MRP-2536, MRP-2909
Reviewed-on: http://review.whamcloud.com/21061
Reviewed-by: Alexander Boyko <alexander.boyko@seagate.com>
Reviewed-by: Vitaly Fertman <vitaly.fertman@seagate.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Signed-off-by: James Simmons <jsimmons@infradead.org>
---
 drivers/staging/lustre/lustre/ldlm/ldlm_extent.c    | 1 -
 drivers/staging/lustre/lustre/ldlm/ldlm_flock.c     | 1 -
 drivers/staging/lustre/lustre/ldlm/ldlm_inodebits.c | 1 -
 3 files changed, 3 deletions(-)

diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_extent.c b/drivers/staging/lustre/lustre/ldlm/ldlm_extent.c
index 32b73ee..5616ea4 100644
--- a/drivers/staging/lustre/lustre/ldlm/ldlm_extent.c
+++ b/drivers/staging/lustre/lustre/ldlm/ldlm_extent.c
@@ -243,7 +243,6 @@ void ldlm_extent_unlink_lock(struct ldlm_lock *lock)
 void ldlm_extent_policy_wire_to_local(const union ldlm_wire_policy_data *wpolicy,
 				      union ldlm_policy_data *lpolicy)
 {
-	memset(lpolicy, 0, sizeof(*lpolicy));
 	lpolicy->l_extent.start = wpolicy->l_extent.start;
 	lpolicy->l_extent.end = wpolicy->l_extent.end;
 	lpolicy->l_extent.gid = wpolicy->l_extent.gid;
diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_flock.c b/drivers/staging/lustre/lustre/ldlm/ldlm_flock.c
index f815827..b7f28b3 100644
--- a/drivers/staging/lustre/lustre/ldlm/ldlm_flock.c
+++ b/drivers/staging/lustre/lustre/ldlm/ldlm_flock.c
@@ -615,7 +615,6 @@ struct ldlm_flock_wait_data {
 void ldlm_flock_policy_wire_to_local(const union ldlm_wire_policy_data *wpolicy,
 				     union ldlm_policy_data *lpolicy)
 {
-	memset(lpolicy, 0, sizeof(*lpolicy));
 	lpolicy->l_flock.start = wpolicy->l_flock.lfw_start;
 	lpolicy->l_flock.end = wpolicy->l_flock.lfw_end;
 	lpolicy->l_flock.pid = wpolicy->l_flock.lfw_pid;
diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_inodebits.c b/drivers/staging/lustre/lustre/ldlm/ldlm_inodebits.c
index 8e1709d..ae37c36 100644
--- a/drivers/staging/lustre/lustre/ldlm/ldlm_inodebits.c
+++ b/drivers/staging/lustre/lustre/ldlm/ldlm_inodebits.c
@@ -57,7 +57,6 @@
 void ldlm_ibits_policy_wire_to_local(const union ldlm_wire_policy_data *wpolicy,
 				     union ldlm_policy_data *lpolicy)
 {
-	memset(lpolicy, 0, sizeof(*lpolicy));
 	lpolicy->l_inodebits.bits = wpolicy->l_inodebits.bits;
 }
 
-- 
1.8.3.1

[toc] | [prev] | [next] | [standalone]


#1569108 — [PATCH 34/60] staging: lustre: libcfs: default CPT matches NUMA topology

FromJames Simmons <jsimmons@infradead.org>
Date2017-01-29 01:20 +0100
Subject[PATCH 34/60] staging: lustre: libcfs: default CPT matches NUMA topology
Message-ID<t4LnQ-8lZ-43@gated-at.bofh.it>
In reply to#1569103
From: Dmitry Eremin <dmitry.eremin@intel.com>

Change default value of CPT pattern and make it match NUMA topology

Signed-off-by: Liang Zhen <liang.zhen@intel.com>
Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-5050
Reviewed-on: http://review.whamcloud.com/22377
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Olaf Weber <olaf@sgi.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Signed-off-by: James Simmons <jsimmons@infradead.org>
---
 drivers/staging/lustre/lnet/libcfs/linux/linux-cpu.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/staging/lustre/lnet/libcfs/linux/linux-cpu.c b/drivers/staging/lustre/lnet/libcfs/linux/linux-cpu.c
index 71a5b19..62ab76e 100644
--- a/drivers/staging/lustre/lnet/libcfs/linux/linux-cpu.c
+++ b/drivers/staging/lustre/lnet/libcfs/linux/linux-cpu.c
@@ -59,7 +59,7 @@
  *
  * NB: If user specified cpu_pattern, cpu_npartitions will be ignored
  */
-static char	*cpu_pattern = "";
+static char	*cpu_pattern = "N";
 module_param(cpu_pattern, charp, 0444);
 MODULE_PARM_DESC(cpu_pattern, "CPU partitions pattern");
 
-- 
1.8.3.1

[toc] | [prev] | [next] | [standalone]


#1569109 — [PATCH 25/60] staging: lustre: llite: normal user can't set FS default stripe

FromJames Simmons <jsimmons@infradead.org>
Date2017-01-29 01:20 +0100
Subject[PATCH 25/60] staging: lustre: llite: normal user can't set FS default stripe
Message-ID<t4LnQ-8lZ-45@gated-at.bofh.it>
In reply to#1569103
From: Lai Siyao <lai.siyao@intel.com>

Current client doesn't check permission before updating filesystem
default stripe on MGS, which isn't secure and obvious.

Since we setattr on MDS first, and then set default stripe on MGS,
we can just return error upon setattr failure.

Now filesystem default stripe is stored in ROOT in MDT, so saving
it in system config is for compatibility with old servers, this
will be removed in the future.

Signed-off-by: Lai Siyao <lai.siyao@intel.com>
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-8454
Reviewed-on: http://review.whamcloud.com/21612
Reviewed-on: http://review.whamcloud.com/22580
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Signed-off-by: James Simmons <jsimmons@infradead.org>
---
 drivers/staging/lustre/lustre/llite/dir.c | 14 +++++++++-----
 1 file changed, 9 insertions(+), 5 deletions(-)

diff --git a/drivers/staging/lustre/lustre/llite/dir.c b/drivers/staging/lustre/lustre/llite/dir.c
index 526fea2..13b3592 100644
--- a/drivers/staging/lustre/lustre/llite/dir.c
+++ b/drivers/staging/lustre/lustre/llite/dir.c
@@ -521,12 +521,15 @@ int ll_dir_setstripe(struct inode *inode, struct lov_user_md *lump,
 	rc = md_setattr(sbi->ll_md_exp, op_data, lump, lum_size, &req);
 	ll_finish_md_op_data(op_data);
 	ptlrpc_req_finished(req);
-	if (rc) {
-		if (rc != -EPERM && rc != -EACCES)
-			CERROR("mdc_setattr fails: rc = %d\n", rc);
-	}
+	if (rc)
+		return rc;
 
-	/* In the following we use the fact that LOV_USER_MAGIC_V1 and
+#if OBD_OCD_VERSION(2, 13, 53, 0) > LUSTRE_VERSION_CODE
+	/*
+	 * 2.9 server has stored filesystem default stripe in ROOT xattr,
+	 * and it's stored into system config for backward compatibility.
+	 *
+	 * In the following we use the fact that LOV_USER_MAGIC_V1 and
 	 * LOV_USER_MAGIC_V3 have the same initial fields so we do not
 	 * need to make the distinction between the 2 versions
 	 */
@@ -567,6 +570,7 @@ int ll_dir_setstripe(struct inode *inode, struct lov_user_md *lump,
 end:
 		kfree(param);
 	}
+#endif
 	return rc;
 }
 
-- 
1.8.3.1

[toc] | [prev] | [next] | [standalone]


#1569110 — [PATCH 07/60] staging: lustre: ptlrpc: set proper mbits for EINPROGRESS resend

FromJames Simmons <jsimmons@infradead.org>
Date2017-01-29 01:20 +0100
Subject[PATCH 07/60] staging: lustre: ptlrpc: set proper mbits for EINPROGRESS resend
Message-ID<t4LnQ-8lZ-47@gated-at.bofh.it>
In reply to#1569103
From: Niu Yawei <yawei.niu@intel.com>

Set mbits for EINPROGRESS resend in ptl_send_rpc().

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-8193
Reviewed-on: http://review.whamcloud.com/20377
Reviewed-by: Liang Zhen <liang.zhen@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Signed-off-by: James Simmons <jsimmons@infradead.org>
---
 drivers/staging/lustre/lustre/ptlrpc/client.c | 7 +++++--
 drivers/staging/lustre/lustre/ptlrpc/niobuf.c | 5 +++--
 2 files changed, 8 insertions(+), 4 deletions(-)

diff --git a/drivers/staging/lustre/lustre/ptlrpc/client.c b/drivers/staging/lustre/lustre/ptlrpc/client.c
index 8047413..3c18ab6 100644
--- a/drivers/staging/lustre/lustre/ptlrpc/client.c
+++ b/drivers/staging/lustre/lustre/ptlrpc/client.c
@@ -3123,8 +3123,11 @@ void ptlrpc_set_bulk_mbits(struct ptlrpc_request *req)
 			req->rq_mbits = ptlrpc_next_xid();
 		} else {
 			/* old version transfers rq_xid to peer as matchbits */
-			req->rq_mbits = ptlrpc_next_xid();
-			req->rq_xid = req->rq_mbits;
+			spin_lock(&req->rq_import->imp_lock);
+			list_del_init(&req->rq_unreplied_list);
+			ptlrpc_assign_next_xid_nolock(req);
+			req->rq_mbits = req->rq_xid;
+			spin_unlock(&req->rq_import->imp_lock);
 		}
 
 		CDEBUG(D_HA, "resend bulk old x%llu new x%llu\n",
diff --git a/drivers/staging/lustre/lustre/ptlrpc/niobuf.c b/drivers/staging/lustre/lustre/ptlrpc/niobuf.c
index da1209e..b870184 100644
--- a/drivers/staging/lustre/lustre/ptlrpc/niobuf.c
+++ b/drivers/staging/lustre/lustre/ptlrpc/niobuf.c
@@ -522,13 +522,14 @@ int ptl_send_rpc(struct ptlrpc_request *request, int noreply)
 		 */
 		spin_lock(&imp->imp_lock);
 		ptlrpc_assign_next_xid_nolock(request);
-		request->rq_mbits = request->rq_xid;
 		min_xid = ptlrpc_known_replied_xid(imp);
 		spin_unlock(&imp->imp_lock);
 
 		lustre_msg_set_last_xid(request->rq_reqmsg, min_xid);
 		DEBUG_REQ(D_RPCTRACE, request, "Allocating new xid for resend on EINPROGRESS");
-	} else if (request->rq_bulk) {
+	}
+
+	if (request->rq_bulk) {
 		ptlrpc_set_bulk_mbits(request);
 		lustre_msg_set_mbits(request->rq_reqmsg, request->rq_mbits);
 	}
-- 
1.8.3.1

[toc] | [prev] | [next] | [standalone]


#1569111 — [PATCH 19/60] staging: lustre: llite: handle inactive OSTs better in statfs

FromJames Simmons <jsimmons@infradead.org>
Date2017-01-29 01:20 +0100
Subject[PATCH 19/60] staging: lustre: llite: handle inactive OSTs better in statfs
Message-ID<t4LnQ-8lZ-49@gated-at.bofh.it>
In reply to#1569103
From: Andreas Dilger <andreas.dilger@intel.com>

Change the order of checks for inactive OSCs in lov_prep_statfs_set()
so that administratively disabled OSTs do not generate any output in
"lfs df" at all, to avoid needlessly cluttering the output.

Enable the lazystatfs mount option by default, so that "df" does not
hang when an OST is temporarily offline.

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-7759
Reviewed-on: http://review.whamcloud.com/19195
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Signed-off-by: James Simmons <jsimmons@infradead.org>
---
 drivers/staging/lustre/lustre/llite/llite_lib.c | 1 +
 drivers/staging/lustre/lustre/lov/lov_request.c | 6 +++---
 2 files changed, 4 insertions(+), 3 deletions(-)

diff --git a/drivers/staging/lustre/lustre/llite/llite_lib.c b/drivers/staging/lustre/lustre/llite/llite_lib.c
index 769b307..0a87058 100644
--- a/drivers/staging/lustre/lustre/llite/llite_lib.c
+++ b/drivers/staging/lustre/lustre/llite/llite_lib.c
@@ -103,6 +103,7 @@ static struct ll_sb_info *ll_init_sbi(struct super_block *sb)
 	sbi->ll_flags |= LL_SBI_CHECKSUM;
 
 	sbi->ll_flags |= LL_SBI_LRU_RESIZE;
+	sbi->ll_flags |= LL_SBI_LAZYSTATFS;
 
 	for (i = 0; i <= LL_PROCESS_HIST_MAX; i++) {
 		spin_lock_init(&sbi->ll_rw_extents_info.pp_extents[i].
diff --git a/drivers/staging/lustre/lustre/lov/lov_request.c b/drivers/staging/lustre/lustre/lov/lov_request.c
index d43cc88..3a74791 100644
--- a/drivers/staging/lustre/lustre/lov/lov_request.c
+++ b/drivers/staging/lustre/lustre/lov/lov_request.c
@@ -344,9 +344,6 @@ int lov_prep_statfs_set(struct obd_device *obd, struct obd_info *oinfo,
 			continue;
 		}
 
-		if (!lov->lov_tgts[i]->ltd_active)
-			lov_check_and_wait_active(lov, i);
-
 		/* skip targets that have been explicitly disabled by the
 		 * administrator
 		 */
@@ -355,6 +352,9 @@ int lov_prep_statfs_set(struct obd_device *obd, struct obd_info *oinfo,
 			continue;
 		}
 
+		if (!lov->lov_tgts[i]->ltd_active)
+			lov_check_and_wait_active(lov, i);
+
 		req = kzalloc(sizeof(*req), GFP_NOFS);
 		if (!req) {
 			rc = -ENOMEM;
-- 
1.8.3.1

[toc] | [prev] | [next] | [standalone]


#1569112 — [PATCH 05/60] staging: lustre: llite: check request != NULL in ll_migrate

FromJames Simmons <jsimmons@infradead.org>
Date2017-01-29 01:20 +0100
Subject[PATCH 05/60] staging: lustre: llite: check request != NULL in ll_migrate
Message-ID<t4LnR-8lZ-55@gated-at.bofh.it>
In reply to#1569103
From: wang di <di.wang@intel.com>

Check if the request is NULL, before retrieve reply body
from the request.

Signed-off-by: wang di <di.wang@intel.com>
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-7396
Reviewed-on: http://review.whamcloud.com/17079
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Signed-off-by: James Simmons <jsimmons@infradead.org>
---
 drivers/staging/lustre/lustre/llite/file.c | 41 +++++++++++++++++-------------
 1 file changed, 23 insertions(+), 18 deletions(-)

diff --git a/drivers/staging/lustre/lustre/llite/file.c b/drivers/staging/lustre/lustre/llite/file.c
index a1e51a5..b681e15 100644
--- a/drivers/staging/lustre/lustre/llite/file.c
+++ b/drivers/staging/lustre/lustre/llite/file.c
@@ -2656,28 +2656,33 @@ int ll_migrate(struct inode *parent, struct file *file, int mdtidx,
 	if (!rc)
 		ll_update_times(request, parent);
 
-	body = req_capsule_server_get(&request->rq_pill, &RMF_MDT_BODY);
-	if (!body) {
-		rc = -EPROTO;
-		goto out_free;
-	}
+	if (request) {
+		body = req_capsule_server_get(&request->rq_pill, &RMF_MDT_BODY);
+		if (!body) {
+			rc = -EPROTO;
+			goto out_free;
+		}
 
-	/*
-	 * If the server does release layout lock, then we cleanup
-	 * the client och here, otherwise release it in out_free:
-	 */
-	if (och && body->mbo_valid & OBD_MD_CLOSE_INTENT_EXECED) {
-		obd_mod_put(och->och_mod);
-		md_clear_open_replay_data(ll_i2sbi(parent)->ll_md_exp, och);
-		och->och_fh.cookie = DEAD_HANDLE_MAGIC;
-		kfree(och);
-		och = NULL;
-	}
+		/*
+		 * If the server does release layout lock, then we cleanup
+		 * the client och here, otherwise release it in out_free:
+		 */
+		if (och && body->mbo_valid & OBD_MD_CLOSE_INTENT_EXECED) {
+			obd_mod_put(och->och_mod);
+			md_clear_open_replay_data(ll_i2sbi(parent)->ll_md_exp,
+						  och);
+			och->och_fh.cookie = DEAD_HANDLE_MAGIC;
+			kfree(och);
+			och = NULL;
+		}
 
-	ptlrpc_req_finished(request);
+		ptlrpc_req_finished(request);
+	}
 	/* Try again if the file layout has changed. */
-	if (rc == -EAGAIN && S_ISREG(child_inode->i_mode))
+	if (rc == -EAGAIN && S_ISREG(child_inode->i_mode)) {
+		request = NULL;
 		goto again;
+	}
 out_free:
 	if (child_inode) {
 		if (och) /* close the file */
-- 
1.8.3.1

[toc] | [prev] | [next] | [standalone]


#1569623 — Re: [PATCH 05/60] staging: lustre: llite: check request != NULL in ll_migrate

FromDan Carpenter <dan.carpenter@oracle.com>
Date2017-01-30 12:40 +0100
SubjectRe: [PATCH 05/60] staging: lustre: llite: check request != NULL in ll_migrate
Message-ID<t5its-3q1-13@gated-at.bofh.it>
In reply to#1569112
On Sat, Jan 28, 2017 at 07:04:33PM -0500, James Simmons wrote:
> From: wang di <di.wang@intel.com>
> 
> Check if the request is NULL, before retrieve reply body
> from the request.
> 
> Signed-off-by: wang di <di.wang@intel.com>
> Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-7396
> Reviewed-on: http://review.whamcloud.com/17079
> Reviewed-by: John L. Hammond <john.hammond@intel.com>
> Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
> Signed-off-by: James Simmons <jsimmons@infradead.org>
> ---
>  drivers/staging/lustre/lustre/llite/file.c | 41 +++++++++++++++++-------------
>  1 file changed, 23 insertions(+), 18 deletions(-)
> 
> diff --git a/drivers/staging/lustre/lustre/llite/file.c b/drivers/staging/lustre/lustre/llite/file.c
> index a1e51a5..b681e15 100644
> --- a/drivers/staging/lustre/lustre/llite/file.c
> +++ b/drivers/staging/lustre/lustre/llite/file.c
> @@ -2656,28 +2656,33 @@ int ll_migrate(struct inode *parent, struct file *file, int mdtidx,
>  	if (!rc)
>  		ll_update_times(request, parent);
>  

I don't like how we return a non-NULL request on many error paths.  It
would be simpler to understand if mdc_rename() freed request on error.
So mdc_reint() fails but we still continue?  I don't understand that but
there are no comments about it.


> -	body = req_capsule_server_get(&request->rq_pill, &RMF_MDT_BODY);
> -	if (!body) {
> -		rc = -EPROTO;
> -		goto out_free;
> -	}
> +	if (request) {
> +		body = req_capsule_server_get(&request->rq_pill, &RMF_MDT_BODY);
> +		if (!body) {
> +			rc = -EPROTO;
> +			goto out_free;

We should call ptlrpc_req_finished(request) before returning on this
path.

regards,
dan carpenter

[toc] | [prev] | [next] | [standalone]


#1569113 — [PATCH 03/60] staging: lustre: clio: add cl_page LRU shrinker

FromJames Simmons <jsimmons@infradead.org>
Date2017-01-29 01:20 +0100
Subject[PATCH 03/60] staging: lustre: clio: add cl_page LRU shrinker
Message-ID<t4LnR-8lZ-53@gated-at.bofh.it>
In reply to#1569103
From: Bobi Jam <bobijam.xu@intel.com>

Register cache shrinker to reclaim memory from cl_page LRU list.

Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-6842
Reviewed-on: http://review.whamcloud.com/15630
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Signed-off-by: James Simmons <jsimmons@infradead.org>
---
 drivers/staging/lustre/lustre/include/obd.h      |  2 +
 drivers/staging/lustre/lustre/ldlm/ldlm_lib.c    |  1 +
 drivers/staging/lustre/lustre/osc/osc_internal.h |  9 +++
 drivers/staging/lustre/lustre/osc/osc_page.c     | 87 ++++++++++++++++++++++++
 drivers/staging/lustre/lustre/osc/osc_request.c  | 21 ++++++
 5 files changed, 120 insertions(+)

diff --git a/drivers/staging/lustre/lustre/include/obd.h b/drivers/staging/lustre/lustre/include/obd.h
index 7f0fc44..6d3bd05 100644
--- a/drivers/staging/lustre/lustre/include/obd.h
+++ b/drivers/staging/lustre/lustre/include/obd.h
@@ -287,6 +287,8 @@ struct client_obd {
 	 * the transaction has NOT yet committed.
 	 */
 	atomic_long_t		 cl_unstable_count;
+	/** Link to osc_shrinker_list */
+	struct list_head	 cl_shrink_list;
 
 	/* number of in flight destroy rpcs is limited to max_rpcs_in_flight */
 	atomic_t	     cl_destroy_in_flight;
diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_lib.c b/drivers/staging/lustre/lustre/ldlm/ldlm_lib.c
index 9be0142..675e25b 100644
--- a/drivers/staging/lustre/lustre/ldlm/ldlm_lib.c
+++ b/drivers/staging/lustre/lustre/ldlm/ldlm_lib.c
@@ -336,6 +336,7 @@ int client_obd_setup(struct obd_device *obddev, struct lustre_cfg *lcfg)
 	INIT_LIST_HEAD(&cli->cl_lru_list);
 	spin_lock_init(&cli->cl_lru_list_lock);
 	atomic_long_set(&cli->cl_unstable_count, 0);
+	INIT_LIST_HEAD(&cli->cl_shrink_list);
 
 	init_waitqueue_head(&cli->cl_destroy_waitq);
 	atomic_set(&cli->cl_destroy_in_flight, 0);
diff --git a/drivers/staging/lustre/lustre/osc/osc_internal.h b/drivers/staging/lustre/lustre/osc/osc_internal.h
index ff7c9ec..43a43e4 100644
--- a/drivers/staging/lustre/lustre/osc/osc_internal.h
+++ b/drivers/staging/lustre/lustre/osc/osc_internal.h
@@ -222,4 +222,13 @@ struct ldlm_lock *osc_dlmlock_at_pgoff(const struct lu_env *env,
 
 int osc_object_invalidate(const struct lu_env *env, struct osc_object *osc);
 
+/** osc shrink list to link all osc client obd */
+extern struct list_head osc_shrink_list;
+/** spin lock to protect osc_shrink_list */
+extern spinlock_t osc_shrink_lock;
+unsigned long osc_cache_shrink_count(struct shrinker *sk,
+				     struct shrink_control *sc);
+unsigned long osc_cache_shrink_scan(struct shrinker *sk,
+				    struct shrink_control *sc);
+
 #endif /* OSC_INTERNAL_H */
diff --git a/drivers/staging/lustre/lustre/osc/osc_page.c b/drivers/staging/lustre/lustre/osc/osc_page.c
index e356e4a..0461408 100644
--- a/drivers/staging/lustre/lustre/osc/osc_page.c
+++ b/drivers/staging/lustre/lustre/osc/osc_page.c
@@ -943,4 +943,91 @@ bool osc_over_unstable_soft_limit(struct client_obd *cli)
 				    cli->cl_max_rpcs_in_flight;
 }
 
+/**
+ * Return how many LRU pages in the cache of all OSC devices
+ *
+ * Return:	return # of cached LRU pages times reclaimation tendency
+ *		SHRINK_STOP if it cannot do any scanning in this time
+ */
+unsigned long osc_cache_shrink_count(struct shrinker *sk,
+				     struct shrink_control *sc)
+{
+	struct client_obd *cli;
+	unsigned long cached = 0;
+
+	spin_lock(&osc_shrink_lock);
+	list_for_each_entry(cli, &osc_shrink_list, cl_shrink_list)
+		cached += atomic_long_read(&cli->cl_lru_in_list);
+	spin_unlock(&osc_shrink_lock);
+
+	return (cached  * sysctl_vfs_cache_pressure) / 100;
+}
+
+/**
+ * Scan and try to reclaim sc->nr_to_scan cached LRU pages
+ *
+ * Return:	number of cached LRU pages reclaimed
+ *		SHRINK_STOP if it cannot do any scanning in this time
+ *
+ * Linux kernel will loop calling this shrinker scan routine with
+ * sc->nr_to_scan = SHRINK_BATCH(128 for now) until kernel got enough memory.
+ *
+ * If sc->nr_to_scan is 0, the VM is querying the cache size, we don't need
+ * to scan and try to reclaim LRU pages, just return 0 and
+ * osc_cache_shrink_count() will report the LRU page number.
+ */
+unsigned long osc_cache_shrink_scan(struct shrinker *sk,
+				    struct shrink_control *sc)
+{
+	struct client_obd *stop_anchor = NULL;
+	struct client_obd *cli;
+	struct lu_env *env;
+	long shrank = 0;
+	int refcheck;
+	int rc;
+
+	if (!sc->nr_to_scan)
+		return 0;
+
+	if (!(sc->gfp_mask & __GFP_FS))
+		return SHRINK_STOP;
+
+	env = cl_env_get(&refcheck);
+	if (IS_ERR(env))
+		return SHRINK_STOP;
+
+	spin_lock(&osc_shrink_lock);
+	while (!list_empty(&osc_shrink_list)) {
+		cli = list_entry(osc_shrink_list.next, struct client_obd,
+				 cl_shrink_list);
+
+		if (!stop_anchor)
+			stop_anchor = cli;
+		else if (cli == stop_anchor)
+			break;
+
+		list_move_tail(&cli->cl_shrink_list, &osc_shrink_list);
+		spin_unlock(&osc_shrink_lock);
+
+		/* shrink no more than max_pages_per_rpc for an OSC */
+		rc = osc_lru_shrink(env, cli, (sc->nr_to_scan - shrank) >
+				    cli->cl_max_pages_per_rpc ?
+				    cli->cl_max_pages_per_rpc :
+				    sc->nr_to_scan - shrank, true);
+		if (rc > 0)
+			shrank += rc;
+
+		if (shrank >= sc->nr_to_scan)
+			goto out;
+
+		spin_lock(&osc_shrink_lock);
+	}
+	spin_unlock(&osc_shrink_lock);
+
+out:
+	cl_env_put(env, &refcheck);
+
+	return shrank;
+}
+
 /** @} osc */
diff --git a/drivers/staging/lustre/lustre/osc/osc_request.c b/drivers/staging/lustre/lustre/osc/osc_request.c
index 3efae75..c2c0385 100644
--- a/drivers/staging/lustre/lustre/osc/osc_request.c
+++ b/drivers/staging/lustre/lustre/osc/osc_request.c
@@ -2675,6 +2675,11 @@ int osc_setup(struct obd_device *obd, struct lustre_cfg *lcfg)
 
 	INIT_LIST_HEAD(&cli->cl_grant_shrink_list);
 	ns_register_cancel(obd->obd_namespace, osc_cancel_weight);
+
+	spin_lock(&osc_shrink_lock);
+	list_add_tail(&cli->cl_shrink_list, &osc_shrink_list);
+	spin_unlock(&osc_shrink_lock);
+
 	return rc;
 
 out_ptlrpcd_work:
@@ -2728,6 +2733,10 @@ static int osc_cleanup(struct obd_device *obd)
 	struct client_obd *cli = &obd->u.cli;
 	int rc;
 
+	spin_lock(&osc_shrink_lock);
+	list_del(&cli->cl_shrink_list);
+	spin_unlock(&osc_shrink_lock);
+
 	/* lru cleanup */
 	if (cli->cl_cache) {
 		LASSERT(atomic_read(&cli->cl_cache->ccc_users) > 0);
@@ -2795,6 +2804,15 @@ static int osc_process_config(struct obd_device *obd, u32 len, void *buf)
 	.quotactl       = osc_quotactl,
 };
 
+struct list_head osc_shrink_list = LIST_HEAD_INIT(osc_shrink_list);
+DEFINE_SPINLOCK(osc_shrink_lock);
+
+static struct shrinker osc_cache_shrinker = {
+	.count_objects	= osc_cache_shrink_count,
+	.scan_objects	= osc_cache_shrink_scan,
+	.seeks		= DEFAULT_SEEKS,
+};
+
 static int __init osc_init(void)
 {
 	struct lprocfs_static_vars lvars = { NULL };
@@ -2819,6 +2837,8 @@ static int __init osc_init(void)
 	if (rc)
 		goto out_kmem;
 
+	register_shrinker(&osc_cache_shrinker);
+
 	/* This is obviously too much memory, only prevent overflow here */
 	if (osc_reqpool_mem_max >= 1 << 12 || osc_reqpool_mem_max == 0) {
 		rc = -EINVAL;
@@ -2857,6 +2877,7 @@ static int __init osc_init(void)
 
 static void /*__exit*/ osc_exit(void)
 {
+	unregister_shrinker(&osc_cache_shrinker);
 	class_unregister_type(LUSTRE_OSC_NAME);
 	lu_kmem_fini(osc_caches);
 	ptlrpc_free_rq_pool(osc_rq_pool);
-- 
1.8.3.1

[toc] | [prev] | [next] | [standalone]


#1569114 — [PATCH 09/60] staging: lustre: lmv: Correctly generate target_obd

FromJames Simmons <jsimmons@infradead.org>
Date2017-01-29 01:20 +0100
Subject[PATCH 09/60] staging: lustre: lmv: Correctly generate target_obd
Message-ID<t4LnQ-8lZ-41@gated-at.bofh.it>
In reply to#1569103
From: Giuseppe Di Natale <dinatale2@llnl.gov>

The target_obd debugfs file was not being generated correctly
in cases where nonconsecutive MDT indices were used when
generating a filesystem.

Signed-off-by: Giuseppe Di Natale <dinatale2@llnl.gov>
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-8100
Reviewed-on: http://review.whamcloud.com/20336
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Signed-off-by: James Simmons <jsimmons@infradead.org>
---
 drivers/staging/lustre/lustre/lmv/lproc_lmv.c | 17 +++++++++++++++--
 1 file changed, 15 insertions(+), 2 deletions(-)

diff --git a/drivers/staging/lustre/lustre/lmv/lproc_lmv.c b/drivers/staging/lustre/lustre/lmv/lproc_lmv.c
index 20bbdfc..14fbc9c 100644
--- a/drivers/staging/lustre/lustre/lmv/lproc_lmv.c
+++ b/drivers/staging/lustre/lustre/lmv/lproc_lmv.c
@@ -147,7 +147,13 @@ static void *lmv_tgt_seq_start(struct seq_file *p, loff_t *pos)
 	struct obd_device       *dev = p->private;
 	struct lmv_obd	  *lmv = &dev->u.lmv;
 
-	return (*pos >= lmv->desc.ld_tgt_count) ? NULL : lmv->tgts[*pos];
+	while (*pos < lmv->tgts_size) {
+		if (lmv->tgts[*pos])
+			return lmv->tgts[*pos];
+		++*pos;
+	}
+
+	return  NULL;
 }
 
 static void lmv_tgt_seq_stop(struct seq_file *p, void *v)
@@ -159,8 +165,15 @@ static void *lmv_tgt_seq_next(struct seq_file *p, void *v, loff_t *pos)
 {
 	struct obd_device       *dev = p->private;
 	struct lmv_obd	  *lmv = &dev->u.lmv;
+
 	++*pos;
-	return (*pos >= lmv->desc.ld_tgt_count) ? NULL : lmv->tgts[*pos];
+	while (*pos < lmv->tgts_size) {
+		if (lmv->tgts[*pos])
+			return lmv->tgts[*pos];
+		++*pos;
+	}
+
+	return  NULL;
 }
 
 static int lmv_tgt_seq_show(struct seq_file *p, void *v)
-- 
1.8.3.1

[toc] | [prev] | [next] | [standalone]


#1569115 — [PATCH 04/60] staging: lustre: mdc: quiet console message for known -EINTR

FromJames Simmons <jsimmons@infradead.org>
Date2017-01-29 01:20 +0100
Subject[PATCH 04/60] staging: lustre: mdc: quiet console message for known -EINTR
Message-ID<t4LnR-8lZ-59@gated-at.bofh.it>
In reply to#1569103
From: Andreas Dilger <andreas.dilger@intel.com>

If a user process is waiting for MDS recovery during close, but the
process is interrupted, the file is still closed but it prints a
message on the console. Quiet the console message for -EINTR, since
this is expected behaviour.

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-6627
Reviewed-on: http://review.whamcloud.com/14911
Reviewed-by: Frank Zago <fzago@cray.com>
Reviewed-by: Emoly Liu <emoly.liu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Signed-off-by: James Simmons <jsimmons@infradead.org>
---
 drivers/staging/lustre/lustre/llite/file.c | 29 +++++++++++++----------------
 1 file changed, 13 insertions(+), 16 deletions(-)

diff --git a/drivers/staging/lustre/lustre/llite/file.c b/drivers/staging/lustre/lustre/llite/file.c
index 0ee02f1..a1e51a5 100644
--- a/drivers/staging/lustre/lustre/llite/file.c
+++ b/drivers/staging/lustre/lustre/llite/file.c
@@ -122,26 +122,25 @@ static int ll_close_inode_openhandle(struct obd_export *md_exp,
 				     enum mds_op_bias bias,
 				     void *data)
 {
-	struct obd_export *exp = ll_i2mdexp(inode);
+	const struct ll_inode_info *lli = ll_i2info(inode);
 	struct md_op_data *op_data;
 	struct ptlrpc_request *req = NULL;
-	struct obd_device *obd = class_exp2obd(exp);
 	int rc;
 
-	if (!obd) {
-		/*
-		 * XXX: in case of LMV, is this correct to access
-		 * ->exp_handle?
-		 */
-		CERROR("Invalid MDC connection handle %#llx\n",
-		       ll_i2mdexp(inode)->exp_handle.h_cookie);
+	if (!class_exp2obd(md_exp)) {
+		CERROR("%s: invalid MDC connection handle closing " DFID "\n",
+		       ll_get_fsname(inode->i_sb, NULL, 0),
+		       PFID(&lli->lli_fid));
 		rc = 0;
 		goto out;
 	}
 
 	op_data = kzalloc(sizeof(*op_data), GFP_NOFS);
+	/*
+	 * We leak openhandle and request here on error, but not much to be
+	 * done in OOM case since app won't retry close on error either.
+	 */
 	if (!op_data) {
-		/* XXX We leak openhandle and request here. */
 		rc = -ENOMEM;
 		goto out;
 	}
@@ -170,10 +169,9 @@ static int ll_close_inode_openhandle(struct obd_export *md_exp,
 	}
 
 	rc = md_close(md_exp, op_data, och->och_mod, &req);
-	if (rc) {
-		CERROR("%s: inode "DFID" mdc close failed: rc = %d\n",
-		       ll_i2mdexp(inode)->exp_obd->obd_name,
-		       PFID(ll_inode2fid(inode)), rc);
+	if (rc && rc != -EINTR) {
+		CERROR("%s: inode " DFID " mdc close failed: rc = %d\n",
+		       md_exp->exp_obd->obd_name, PFID(&lli->lli_fid), rc);
 	}
 
 	if (op_data->op_bias & (MDS_HSM_RELEASE | MDS_CLOSE_LAYOUT_SWAP) &&
@@ -192,8 +190,7 @@ static int ll_close_inode_openhandle(struct obd_export *md_exp,
 	och->och_fh.cookie = DEAD_HANDLE_MAGIC;
 	kfree(och);
 
-	if (req) /* This is close request */
-		ptlrpc_req_finished(req);
+	ptlrpc_req_finished(req);
 	return rc;
 }
 
-- 
1.8.3.1

[toc] | [prev] | [next] | [standalone]


#1569116 — [PATCH 33/60] staging: lustre: libcfs: avoid stomping on module param cpu_pattern

FromJames Simmons <jsimmons@infradead.org>
Date2017-01-29 01:20 +0100
Subject[PATCH 33/60] staging: lustre: libcfs: avoid stomping on module param cpu_pattern
Message-ID<t4LnR-8lZ-51@gated-at.bofh.it>
In reply to#1569103
From: Dmitry Eremin <dmitry.eremin@intel.com>

The function cfs_cpt_table_create_pattern() alters the string
passed to it. Currently we are passing in the module parameter
string cpu_pattern which is incorrect. Instead lets duplicate
the module parameter string and pass that to the function
cfs_cpt_table_create_pattern().

Signed-off-by: Liang Zhen <liang.zhen@intel.com>
Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-5050
Reviewed-on: http://review.whamcloud.com/22377
Reviewed-by: James Simmons <uja.ornl@yahoo.com>
Reviewed-by: Olaf Weber <olaf@sgi.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Signed-off-by: James Simmons <jsimmons@infradead.org>
---
 drivers/staging/lustre/lnet/libcfs/linux/linux-cpu.c | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/drivers/staging/lustre/lnet/libcfs/linux/linux-cpu.c b/drivers/staging/lustre/lnet/libcfs/linux/linux-cpu.c
index 427e219..71a5b19 100644
--- a/drivers/staging/lustre/lnet/libcfs/linux/linux-cpu.c
+++ b/drivers/staging/lustre/lnet/libcfs/linux/linux-cpu.c
@@ -1050,7 +1050,15 @@ static int cfs_cpu_dead(unsigned int cpu)
 	ret = -EINVAL;
 
 	if (*cpu_pattern) {
-		cfs_cpt_table = cfs_cpt_table_create_pattern(cpu_pattern);
+		char *cpu_pattern_dup = kstrdup(cpu_pattern, GFP_KERNEL);
+
+		if (!cpu_pattern_dup) {
+			CERROR("Failed to duplicate cpu_pattern\n");
+			goto failed;
+		}
+
+		cfs_cpt_table = cfs_cpt_table_create_pattern(cpu_pattern_dup);
+		kfree(cpu_pattern_dup);
 		if (!cfs_cpt_table) {
 			CERROR("Failed to create cptab from pattern %s\n",
 			       cpu_pattern);
-- 
1.8.3.1

[toc] | [prev] | [next] | [standalone]


#1569117 — [PATCH 31/60] staging: lustre: clio: sync write should update mtime

FromJames Simmons <jsimmons@infradead.org>
Date2017-01-29 01:20 +0100
Subject[PATCH 31/60] staging: lustre: clio: sync write should update mtime
Message-ID<t4LnR-8lZ-69@gated-at.bofh.it>
In reply to#1569103
From: Niu Yawei <yawei.niu@intel.com>

Sync write should update m/ctime promptly, otherwise, stale m/ctime
could be updated on the OST object by the sync write RPC.

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-7310
Reviewed-on: http://review.whamcloud.com/21063
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Bobi Jam <bobijam@hotmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Signed-off-by: James Simmons <jsimmons@infradead.org>
---
 drivers/staging/lustre/lustre/osc/osc_io.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/drivers/staging/lustre/lustre/osc/osc_io.c b/drivers/staging/lustre/lustre/osc/osc_io.c
index 7e5cd3a..3e61f5e 100644
--- a/drivers/staging/lustre/lustre/osc/osc_io.c
+++ b/drivers/staging/lustre/lustre/osc/osc_io.c
@@ -210,6 +210,18 @@ static int osc_io_submit(const struct lu_env *env,
 	if (queued > 0)
 		result = osc_queue_sync_pages(env, osc, &list, cmd, brw_flags);
 
+	/* Update c/mtime for sync write. LU-7310 */
+	if (qout->pl_nr > 0 && !result) {
+		struct cl_attr *attr = &osc_env_info(env)->oti_attr;
+		struct cl_object *obj = ios->cis_obj;
+
+		cl_object_attr_lock(obj);
+		attr->cat_mtime = LTIME_S(CURRENT_TIME);
+		attr->cat_ctime = attr->cat_mtime;
+		cl_object_attr_update(env, obj, attr, CAT_MTIME | CAT_CTIME);
+		cl_object_attr_unlock(obj);
+	}
+
 	CDEBUG(D_INFO, "%d/%d %d\n", qin->pl_nr, qout->pl_nr, result);
 	return qout->pl_nr > 0 ? 0 : result;
 }
-- 
1.8.3.1

[toc] | [prev] | [next] | [standalone]


#1569118 — [PATCH 35/60] staging: lustre: lov: ld_target could be NULL

FromJames Simmons <jsimmons@infradead.org>
Date2017-01-29 01:20 +0100
Subject[PATCH 35/60] staging: lustre: lov: ld_target could be NULL
Message-ID<t4LnR-8lZ-67@gated-at.bofh.it>
In reply to#1569103
From: Bobi Jam <bobijam.xu@intel.com>

lov_device::ld_target[ost_idx] could be NULL if the OST target is
not filled in lov_device::ld_lov::lov_tgt_desc[ost_idx] yet.

Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-8018
Reviewed-on: http://review.whamcloud.com/21411
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Signed-off-by: James Simmons <jsimmons@infradead.org>
---
 drivers/staging/lustre/lustre/lov/lov_object.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/drivers/staging/lustre/lustre/lov/lov_object.c b/drivers/staging/lustre/lustre/lov/lov_object.c
index 9c4b5ab..977579c 100644
--- a/drivers/staging/lustre/lustre/lov/lov_object.c
+++ b/drivers/staging/lustre/lustre/lov/lov_object.c
@@ -266,6 +266,13 @@ static int lov_init_raid0(const struct lu_env *env, struct lov_device *dev,
 			if (result != 0)
 				goto out;
 
+			if (!dev->ld_target[ost_idx]) {
+				CERROR("%s: OST %04x is not initialized\n",
+				lov2obd(dev->ld_lov)->obd_name, ost_idx);
+				result = -EIO;
+				goto out;
+			}
+
 			subdev = lovsub2cl_dev(dev->ld_target[ost_idx]);
 			subconf->u.coc_oinfo = oinfo;
 			LASSERTF(subdev, "not init ost %d\n", ost_idx);
-- 
1.8.3.1

[toc] | [prev] | [next] | [standalone]


#1569119 — [PATCH 18/60] staging: lustre: ptlrpc: skip lock if export failed

FromJames Simmons <jsimmons@infradead.org>
Date2017-01-29 01:20 +0100
Subject[PATCH 18/60] staging: lustre: ptlrpc: skip lock if export failed
Message-ID<t4LnR-8lZ-65@gated-at.bofh.it>
In reply to#1569103
From: Alexander Boyko <alexander.boyko@seagate.com>

This patch resolves IO vs eviction race.
After eviction failed export stayed at stale list,
a client had IO processing and reconnected during it.
A client sent brw rpc with last lock cookie and new connection.
The lock with failed export was found and assert was happened.
 (ost_handler.c:1812:ost_prolong_lock_one())
  ASSERTION( lock->l_export == opd->opd_exp ) failed:

 1. Skip the lock at ldlm_handle2lock if lock export failed.
 2. Validation of lock for IO was added at hpreq_check(). The lock
    searching is based on granted interval tree. If server doesn`t
    have a valid lock, it reply to client with ESTALE.

Signed-off-by: Alexander Boyko <alexander.boyko@seagate.com>
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-7702
Seagate-bug-id: MRP-2787
Reviewed-on: http://review.whamcloud.com/18120
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Vitaly Fertman <vitaly.fertman@seagate.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Signed-off-by: James Simmons <jsimmons@infradead.org>
---
 drivers/staging/lustre/lustre/ldlm/ldlm_lock.c |  7 +++++++
 drivers/staging/lustre/lustre/ptlrpc/service.c | 21 ++++++++-------------
 2 files changed, 15 insertions(+), 13 deletions(-)

diff --git a/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c b/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c
index afef5a2..5a94265 100644
--- a/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c
+++ b/drivers/staging/lustre/lustre/ldlm/ldlm_lock.c
@@ -533,6 +533,13 @@ struct ldlm_lock *__ldlm_handle2lock(const struct lustre_handle *handle,
 	if (!lock)
 		return NULL;
 
+	if (lock->l_export && lock->l_export->exp_failed) {
+		CDEBUG(D_INFO, "lock export failed: lock %p, exp %p\n",
+		       lock, lock->l_export);
+		LDLM_LOCK_PUT(lock);
+		return NULL;
+	}
+
 	/* It's unlikely but possible that someone marked the lock as
 	 * destroyed after we did handle2object on it
 	 */
diff --git a/drivers/staging/lustre/lustre/ptlrpc/service.c b/drivers/staging/lustre/lustre/ptlrpc/service.c
index 70c7055..b8091c1 100644
--- a/drivers/staging/lustre/lustre/ptlrpc/service.c
+++ b/drivers/staging/lustre/lustre/ptlrpc/service.c
@@ -1264,20 +1264,15 @@ static int ptlrpc_server_hpreq_init(struct ptlrpc_service_part *svcpt,
 		 */
 		if (req->rq_ops->hpreq_check) {
 			rc = req->rq_ops->hpreq_check(req);
-			/**
-			 * XXX: Out of all current
-			 * ptlrpc_hpreq_ops::hpreq_check(), only
-			 * ldlm_cancel_hpreq_check() can return an error code;
-			 * other functions assert in similar places, which seems
-			 * odd. What also does not seem right is that handlers
-			 * for those RPCs do not assert on the same checks, but
-			 * rather handle the error cases. e.g. see
-			 * ost_rw_hpreq_check(), and ost_brw_read(),
-			 * ost_brw_write().
+			if (rc == -ESTALE) {
+				req->rq_status = rc;
+				ptlrpc_error(req);
+			}
+			/** can only return error,
+			 * 0 for normal request,
+			 *  or 1 for high priority request
 			 */
-			if (rc < 0)
-				return rc;
-			LASSERT(rc == 0 || rc == 1);
+			LASSERT(rc <= 1);
 		}
 
 		spin_lock_bh(&req->rq_export->exp_rpc_lock);
-- 
1.8.3.1

[toc] | [prev] | [next] | [standalone]


#1569120 — [PATCH 27/60] staging: lustre: mgc: handle config_llog_data::cld_refcount properly

FromJames Simmons <jsimmons@infradead.org>
Date2017-01-29 01:20 +0100
Subject[PATCH 27/60] staging: lustre: mgc: handle config_llog_data::cld_refcount properly
Message-ID<t4LnS-8lZ-75@gated-at.bofh.it>
In reply to#1569103
From: Fan Yong <fan.yong@intel.com>

Originally, the logic of handling config_llog_data::cld_refcount
is some confusing, it may cause the cld_refcount to be leaked or
trigger "LASSERT(atomic_read(&cld->cld_refcount) > 0);" when put
the reference. This patch clean related logic as following:

1) When the 'cld' is created, its reference is set as 1.

2) No need additional reference when add the 'cld' into the list
   'config_llog_list'.

3) Inrease 'cld_refcount' when set lock data after mgc_enqueue()
   done successfully by mgc_process_log().

4) When mgc_requeue_thread() traversals the 'config_llog_list',
   it needs to take additional reference on each 'cld' to avoid
   being freed during subsequent processing. The reference also
   prevents the 'cld' to be dropped from the 'config_llog_list',
   then the mgc_requeue_thread() can safely locate next 'cld',
   and then decrease the 'cld_refcount' for previous one.

5) mgc_blocking_ast() will drop the reference of 'cld_refcount'
   that is taken in mgc_process_log().

6) The others need to call config_log_find() to find the 'cld'
   if want to access related config log data. That will increase
   the 'cld_refcount' to avoid being freed during accessing. The
   sponsor needs to call config_log_put() after using the 'cld'.

7) Other confused or redundant logic are dropped.

   On the other hand, the patch also enhances the protection for
   'config_llog_data' flags, such as 'cld_stopping'/'cld_lostlock'
   as following.

a) Use 'config_list_lock' (spinlock) to handle the possible
   parallel accessing of these flags among mgc_requeue_thread()
   and others config llog data visitors, such as mount/umount,
   blocking_ast, and so on.

b) Use 'config_llog_data::cld_lock' (mutex) to pretect other
   parallel accessing of these flags among kinds of blockable
   operations, such as mount, umount, and blocking ast.

The 'config_llog_data::cld_lock' is also used for protecting
the sub-cld members, such as 'cld_sptlrpc'/'cld_params', and
so on.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-8408
Reviewed-on: http://review.whamcloud.com/21616
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Hongchao Zhang <hongchao.zhang@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Signed-off-by: James Simmons <jsimmons@infradead.org>
---
 drivers/staging/lustre/lustre/mgc/mgc_request.c | 183 ++++++++++++------------
 1 file changed, 94 insertions(+), 89 deletions(-)

diff --git a/drivers/staging/lustre/lustre/mgc/mgc_request.c b/drivers/staging/lustre/lustre/mgc/mgc_request.c
index b9c522a..6a76605 100644
--- a/drivers/staging/lustre/lustre/mgc/mgc_request.c
+++ b/drivers/staging/lustre/lustre/mgc/mgc_request.c
@@ -142,10 +142,10 @@ static void config_log_put(struct config_llog_data *cld)
 
 		if (cld->cld_recover)
 			config_log_put(cld->cld_recover);
-		if (cld->cld_sptlrpc)
-			config_log_put(cld->cld_sptlrpc);
 		if (cld->cld_params)
 			config_log_put(cld->cld_params);
+		if (cld->cld_sptlrpc)
+			config_log_put(cld->cld_sptlrpc);
 		if (cld_is_sptlrpc(cld))
 			sptlrpc_conf_log_stop(cld->cld_logname);
 
@@ -175,13 +175,10 @@ struct config_llog_data *config_log_find(char *logname,
 		/* instance may be NULL, should check name */
 		if (strcmp(logname, cld->cld_logname) == 0) {
 			found = cld;
+			config_log_get(found);
 			break;
 		}
 	}
-	if (found) {
-		atomic_inc(&found->cld_refcount);
-		LASSERT(found->cld_stopping == 0 || cld_is_sptlrpc(found) == 0);
-	}
 	spin_unlock(&config_list_lock);
 	return found;
 }
@@ -203,6 +200,12 @@ struct config_llog_data *do_config_log_add(struct obd_device *obd,
 	if (!cld)
 		return ERR_PTR(-ENOMEM);
 
+	rc = mgc_logname2resid(logname, &cld->cld_resid, type);
+	if (rc) {
+		kfree(cld);
+		return ERR_PTR(rc);
+	}
+
 	strcpy(cld->cld_logname, logname);
 	if (cfg)
 		cld->cld_cfg = *cfg;
@@ -223,17 +226,10 @@ struct config_llog_data *do_config_log_add(struct obd_device *obd,
 		cld->cld_cfg.cfg_obdname = obd->obd_name;
 	}
 
-	rc = mgc_logname2resid(logname, &cld->cld_resid, type);
-
 	spin_lock(&config_list_lock);
 	list_add(&cld->cld_list_chain, &config_llog_list);
 	spin_unlock(&config_list_lock);
 
-	if (rc) {
-		config_log_put(cld);
-		return ERR_PTR(rc);
-	}
-
 	if (cld_is_sptlrpc(cld)) {
 		rc = mgc_process_log(obd, cld);
 		if (rc && rc != -ENOENT)
@@ -284,14 +280,15 @@ struct config_llog_data *do_config_log_add(struct obd_device *obd,
  * We have one active log per "mount" - client instance or servername.
  * Each instance may be at a different point in the log.
  */
-static int config_log_add(struct obd_device *obd, char *logname,
-			  struct config_llog_instance *cfg,
-			  struct super_block *sb)
+static struct config_llog_data *
+config_log_add(struct obd_device *obd, char *logname,
+	       struct config_llog_instance *cfg, struct super_block *sb)
 {
 	struct lustre_sb_info *lsi = s2lsi(sb);
 	struct config_llog_data *cld;
 	struct config_llog_data *sptlrpc_cld;
 	struct config_llog_data *params_cld;
+	bool locked = false;
 	char			seclogname[32];
 	char			*ptr;
 	int			rc;
@@ -305,7 +302,7 @@ static int config_log_add(struct obd_device *obd, char *logname,
 	ptr = strrchr(logname, '-');
 	if (!ptr || ptr - logname > 8) {
 		CERROR("logname %s is too long\n", logname);
-		return -EINVAL;
+		return ERR_PTR(-EINVAL);
 	}
 
 	memcpy(seclogname, logname, ptr - logname);
@@ -326,14 +323,14 @@ static int config_log_add(struct obd_device *obd, char *logname,
 		rc = PTR_ERR(params_cld);
 		CERROR("%s: can't create params log: rc = %d\n",
 		       obd->obd_name, rc);
-		goto out_err1;
+		goto out_sptlrpc;
 	}
 
 	cld = do_config_log_add(obd, logname, CONFIG_T_CONFIG, cfg, sb);
 	if (IS_ERR(cld)) {
 		CERROR("can't create log: %s\n", logname);
 		rc = PTR_ERR(cld);
-		goto out_err2;
+		goto out_params;
 	}
 
 	cld->cld_sptlrpc = sptlrpc_cld;
@@ -350,33 +347,52 @@ static int config_log_add(struct obd_device *obd, char *logname,
 			CERROR("%s: sptlrpc log name not correct, %s: rc = %d\n",
 			       obd->obd_name, seclogname, -EINVAL);
 			config_log_put(cld);
-			return -EINVAL;
+			rc = -EINVAL;
+			goto out_cld;
 		}
 		recover_cld = config_recover_log_add(obd, seclogname, cfg, sb);
 		if (IS_ERR(recover_cld)) {
 			rc = PTR_ERR(recover_cld);
-			goto out_err3;
+			goto out_cld;
 		}
+
+		mutex_lock(&cld->cld_lock);
+		locked = true;
 		cld->cld_recover = recover_cld;
 	}
 
-	return 0;
+	if (!locked)
+		mutex_lock(&cld->cld_lock);
+	cld->cld_params = params_cld;
+	cld->cld_sptlrpc = sptlrpc_cld;
+	mutex_unlock(&cld->cld_lock);
+
+	return cld;
 
-out_err3:
+out_cld:
 	config_log_put(cld);
 
-out_err2:
+out_params:
 	config_log_put(params_cld);
 
-out_err1:
+out_sptlrpc:
 	config_log_put(sptlrpc_cld);
 
 out_err:
-	return rc;
+	return ERR_PTR(rc);
 }
 
 static DEFINE_MUTEX(llog_process_lock);
 
+static inline void config_mark_cld_stop(struct config_llog_data *cld)
+{
+	mutex_lock(&cld->cld_lock);
+	spin_lock(&config_list_lock);
+	cld->cld_stopping = 1;
+	spin_unlock(&config_list_lock);
+	mutex_unlock(&cld->cld_lock);
+}
+
 /** Stop watching for updates on this log.
  */
 static int config_log_end(char *logname, struct config_llog_instance *cfg)
@@ -406,36 +422,32 @@ static int config_log_end(char *logname, struct config_llog_instance *cfg)
 		return rc;
 	}
 
+	spin_lock(&config_list_lock);
 	cld->cld_stopping = 1;
+	spin_unlock(&config_list_lock);
 
 	cld_recover = cld->cld_recover;
 	cld->cld_recover = NULL;
+
+	cld_params = cld->cld_params;
+	cld->cld_params = NULL;
+	cld_sptlrpc = cld->cld_sptlrpc;
+	cld->cld_sptlrpc = NULL;
 	mutex_unlock(&cld->cld_lock);
 
 	if (cld_recover) {
-		mutex_lock(&cld_recover->cld_lock);
-		cld_recover->cld_stopping = 1;
-		mutex_unlock(&cld_recover->cld_lock);
+		config_mark_cld_stop(cld_recover);
 		config_log_put(cld_recover);
 	}
 
-	spin_lock(&config_list_lock);
-	cld_sptlrpc = cld->cld_sptlrpc;
-	cld->cld_sptlrpc = NULL;
-	cld_params = cld->cld_params;
-	cld->cld_params = NULL;
-	spin_unlock(&config_list_lock);
-
-	if (cld_sptlrpc)
-		config_log_put(cld_sptlrpc);
-
 	if (cld_params) {
-		mutex_lock(&cld_params->cld_lock);
-		cld_params->cld_stopping = 1;
-		mutex_unlock(&cld_params->cld_lock);
+		config_mark_cld_stop(cld_params);
 		config_log_put(cld_params);
 	}
 
+	if (cld_sptlrpc)
+		config_log_put(cld_sptlrpc);
+
 	/* drop the ref from the find */
 	config_log_put(cld);
 	/* drop the start ref */
@@ -531,11 +543,10 @@ static int mgc_requeue_thread(void *data)
 	/* Keep trying failed locks periodically */
 	spin_lock(&config_list_lock);
 	rq_state |= RQ_RUNNING;
-	while (1) {
+	while (!(rq_state & RQ_STOP)) {
 		struct l_wait_info lwi;
 		struct config_llog_data *cld, *cld_prev;
 		int rand = cfs_rand() & MGC_TIMEOUT_RAND_CENTISEC;
-		int stopped = !!(rq_state & RQ_STOP);
 		int to;
 
 		/* Any new or requeued lostlocks will change the state */
@@ -571,44 +582,40 @@ static int mgc_requeue_thread(void *data)
 		spin_lock(&config_list_lock);
 		rq_state &= ~RQ_PRECLEANUP;
 		list_for_each_entry(cld, &config_llog_list, cld_list_chain) {
-			if (!cld->cld_lostlock)
+			if (!cld->cld_lostlock || cld->cld_stopping)
 				continue;
 
+			/*
+			 * hold reference to avoid being freed during
+			 * subsequent processing.
+			 */
+			config_log_get(cld);
+			cld->cld_lostlock = 0;
 			spin_unlock(&config_list_lock);
 
-			LASSERT(atomic_read(&cld->cld_refcount) > 0);
-
-			/* Whether we enqueued again or not in mgc_process_log,
-			 * we're done with the ref from the old enqueue
-			 */
 			if (cld_prev)
 				config_log_put(cld_prev);
 			cld_prev = cld;
 
-			cld->cld_lostlock = 0;
-			if (likely(!stopped))
+			if (likely(!(rq_state & RQ_STOP))) {
 				do_requeue(cld);
-
-			spin_lock(&config_list_lock);
+				spin_lock(&config_list_lock);
+			} else {
+				spin_lock(&config_list_lock);
+				break;
+			}
 		}
 		spin_unlock(&config_list_lock);
 		if (cld_prev)
 			config_log_put(cld_prev);
 
-		/* break after scanning the list so that we can drop
-		 * refcount to losing lock clds
-		 */
-		if (unlikely(stopped)) {
-			spin_lock(&config_list_lock);
-			break;
-		}
-
 		/* Wait a bit to see if anyone else needs a requeue */
 		lwi = (struct l_wait_info) { 0 };
 		l_wait_event(rq_waitq, rq_state & (RQ_NOW | RQ_STOP),
 			     &lwi);
 		spin_lock(&config_list_lock);
 	}
+
 	/* spinlock and while guarantee RQ_NOW and RQ_LATER are not set */
 	rq_state &= ~RQ_RUNNING;
 	spin_unlock(&config_list_lock);
@@ -624,32 +631,24 @@ static int mgc_requeue_thread(void *data)
  */
 static void mgc_requeue_add(struct config_llog_data *cld)
 {
+	bool wakeup = false;
+
 	CDEBUG(D_INFO, "log %s: requeue (r=%d sp=%d st=%x)\n",
 	       cld->cld_logname, atomic_read(&cld->cld_refcount),
 	       cld->cld_stopping, rq_state);
 	LASSERT(atomic_read(&cld->cld_refcount) > 0);
 
 	mutex_lock(&cld->cld_lock);
-	if (cld->cld_stopping || cld->cld_lostlock) {
-		mutex_unlock(&cld->cld_lock);
-		return;
-	}
-	/* this refcount will be released in mgc_requeue_thread. */
-	config_log_get(cld);
-	cld->cld_lostlock = 1;
-	mutex_unlock(&cld->cld_lock);
-
-	/* Hold lock for rq_state */
 	spin_lock(&config_list_lock);
-	if (rq_state & RQ_STOP) {
-		spin_unlock(&config_list_lock);
-		cld->cld_lostlock = 0;
-		config_log_put(cld);
-	} else {
+	if (!(rq_state & RQ_STOP) && !cld->cld_stopping && !cld->cld_lostlock) {
+		cld->cld_lostlock = 1;
 		rq_state |= RQ_NOW;
-		spin_unlock(&config_list_lock);
-		wake_up(&rq_waitq);
+		wakeup = true;
 	}
+	spin_unlock(&config_list_lock);
+	mutex_unlock(&cld->cld_lock);
+	if (wakeup)
+		wake_up(&rq_waitq);
 }
 
 static int mgc_llog_init(const struct lu_env *env, struct obd_device *obd)
@@ -812,6 +811,8 @@ static int mgc_blocking_ast(struct ldlm_lock *lock, struct ldlm_lock_desc *desc,
 
 		/* held at mgc_process_log(). */
 		LASSERT(atomic_read(&cld->cld_refcount) > 0);
+
+		lock->l_ast_data = NULL;
 		/* Are we done with this log? */
 		if (cld->cld_stopping) {
 			CDEBUG(D_MGC, "log %s: stopping, won't requeue\n",
@@ -1661,16 +1662,18 @@ int mgc_process_log(struct obd_device *mgc, struct config_llog_data *cld)
 				goto restart;
 			} else {
 				mutex_lock(&cld->cld_lock);
+				spin_lock(&config_list_lock);
 				cld->cld_lostlock = 1;
+				spin_unlock(&config_list_lock);
 			}
 		} else {
 			/* mark cld_lostlock so that it will requeue
 			 * after MGC becomes available.
 			 */
+			spin_lock(&config_list_lock);
 			cld->cld_lostlock = 1;
+			spin_unlock(&config_list_lock);
 		}
-		/* Get extra reference, it will be put in requeue thread */
-		config_log_get(cld);
 	}
 
 	if (cld_is_recover(cld)) {
@@ -1681,7 +1684,9 @@ int mgc_process_log(struct obd_device *mgc, struct config_llog_data *cld)
 				CERROR("%s: recover log %s failed: rc = %d not fatal.\n",
 				       mgc->obd_name, cld->cld_logname, rc);
 				rc = 0;
+				spin_lock(&config_list_lock);
 				cld->cld_lostlock = 1;
+				spin_unlock(&config_list_lock);
 			}
 		}
 	} else {
@@ -1749,12 +1754,9 @@ static int mgc_process_config(struct obd_device *obd, u32 len, void *buf)
 		       cfg->cfg_last_idx);
 
 		/* We're only called through here on the initial mount */
-		rc = config_log_add(obd, logname, cfg, sb);
-		if (rc)
-			break;
-		cld = config_log_find(logname, cfg);
-		if (!cld) {
-			rc = -ENOENT;
+		cld = config_log_add(obd, logname, cfg, sb);
+		if (IS_ERR(cld)) {
+			rc = PTR_ERR(cld);
 			break;
 		}
 
@@ -1770,11 +1772,15 @@ static int mgc_process_config(struct obd_device *obd, u32 len, void *buf)
 					 imp_connect_data, IMP_RECOV)) {
 				rc = mgc_process_log(obd, cld->cld_recover);
 			} else {
-				struct config_llog_data *cir = cld->cld_recover;
+				struct config_llog_data *cir;
 
+				mutex_lock(&cld->cld_lock);
+				cir = cld->cld_recover;
 				cld->cld_recover = NULL;
+				mutex_unlock(&cld->cld_lock);
 				config_log_put(cir);
 			}
+
 			if (rc)
 				CERROR("Cannot process recover llog %d\n", rc);
 		}
@@ -1792,7 +1798,6 @@ static int mgc_process_config(struct obd_device *obd, u32 len, void *buf)
 				       "%s: can't process params llog: rc = %d\n",
 				       obd->obd_name, rc);
 		}
-		config_log_put(cld);
 
 		break;
 	}
-- 
1.8.3.1

[toc] | [prev] | [next] | [standalone]


#1569121 — [PATCH 16/60] staging: lustre: llite: don't ignore layout for group lock request

FromJames Simmons <jsimmons@infradead.org>
Date2017-01-29 01:20 +0100
Subject[PATCH 16/60] staging: lustre: llite: don't ignore layout for group lock request
Message-ID<t4LnR-8lZ-63@gated-at.bofh.it>
In reply to#1569103
From: Jinshan Xiong <jinshan.xiong@intel.com>

ignore_layout can be set for operations that layout won't be changed,
typically page operations. Ignoring layout change in group lock
request will confuse layout change code at LOV layer and hit
assertion.

Signed-off-by: Henri Doreau <henri.doreau@cea.fr>
Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-2766
Reviewed-on: http://review.whamcloud.com/6828
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Signed-off-by: James Simmons <jsimmons@infradead.org>
---
 drivers/staging/lustre/lustre/llite/lcommon_misc.c |  2 +-
 drivers/staging/lustre/lustre/llite/vvp_io.c       |  7 -------
 drivers/staging/lustre/lustre/lov/lov_lock.c       |  5 +++++
 drivers/staging/lustre/lustre/lov/lov_object.c     | 23 +++++++++++++++++++++-
 drivers/staging/lustre/lustre/osc/osc_cache.c      |  1 +
 5 files changed, 29 insertions(+), 9 deletions(-)

diff --git a/drivers/staging/lustre/lustre/llite/lcommon_misc.c b/drivers/staging/lustre/lustre/llite/lcommon_misc.c
index f48660e..f0c132e 100644
--- a/drivers/staging/lustre/lustre/llite/lcommon_misc.c
+++ b/drivers/staging/lustre/lustre/llite/lcommon_misc.c
@@ -33,6 +33,7 @@
  * future).
  *
  */
+#define DEBUG_SUBSYSTEM S_LLITE
 #include "../include/obd_class.h"
 #include "../include/obd_support.h"
 #include "../include/obd.h"
@@ -132,7 +133,6 @@ int cl_get_grouplock(struct cl_object *obj, unsigned long gid, int nonblock,
 
 	io = vvp_env_thread_io(env);
 	io->ci_obj = obj;
-	io->ci_ignore_layout = 1;
 
 	rc = cl_io_init(env, io, CIT_MISC, io->ci_obj);
 	if (rc != 0) {
diff --git a/drivers/staging/lustre/lustre/llite/vvp_io.c b/drivers/staging/lustre/lustre/llite/vvp_io.c
index 19f85fc..3e9cf71 100644
--- a/drivers/staging/lustre/lustre/llite/vvp_io.c
+++ b/drivers/staging/lustre/lustre/llite/vvp_io.c
@@ -1348,13 +1348,6 @@ int vvp_io_init(const struct lu_env *env, struct cl_object *obj,
 			io->ci_lockreq = CILR_MANDATORY;
 	}
 
-	/* ignore layout change for generic CIT_MISC but not for glimpse.
-	 * io context for glimpse must set ci_verify_layout to true,
-	 * see cl_glimpse_size0() for details.
-	 */
-	if (io->ci_type == CIT_MISC && !io->ci_verify_layout)
-		io->ci_ignore_layout = 1;
-
 	/* Enqueue layout lock and get layout version. We need to do this
 	 * even for operations requiring to open file, such as read and write,
 	 * because it might not grant layout lock in IT_OPEN.
diff --git a/drivers/staging/lustre/lustre/lov/lov_lock.c b/drivers/staging/lustre/lustre/lov/lov_lock.c
index f3a0583..8502128 100644
--- a/drivers/staging/lustre/lustre/lov/lov_lock.c
+++ b/drivers/staging/lustre/lustre/lov/lov_lock.c
@@ -134,6 +134,11 @@ static struct lov_lock *lov_lock_sub_init(const struct lu_env *env,
 	struct lov_layout_raid0 *r0     = lov_r0(loo);
 	struct lov_lock		*lovlck;
 
+	CDEBUG(D_INODE, "%p: lock/io FID " DFID "/" DFID ", lock/io clobj %p/%p\n",
+	       loo, PFID(lu_object_fid(lov2lu(loo))),
+	       PFID(lu_object_fid(&obj->co_lu)),
+	       lov2cl(loo), obj);
+
 	file_start = cl_offset(lov2cl(loo), lock->cll_descr.cld_start);
 	file_end   = cl_offset(lov2cl(loo), lock->cll_descr.cld_end + 1) - 1;
 
diff --git a/drivers/staging/lustre/lustre/lov/lov_object.c b/drivers/staging/lustre/lustre/lov/lov_object.c
index 46ec46e..9c4b5ab 100644
--- a/drivers/staging/lustre/lustre/lov/lov_object.c
+++ b/drivers/staging/lustre/lustre/lov/lov_object.c
@@ -650,12 +650,16 @@ static enum lov_layout_type lov_type(struct lov_stripe_md *lsm)
 
 static inline void lov_conf_freeze(struct lov_object *lov)
 {
+	CDEBUG(D_INODE, "To take share lov(%p) owner %p/%p\n",
+	       lov, lov->lo_owner, current);
 	if (lov->lo_owner != current)
 		down_read(&lov->lo_type_guard);
 }
 
 static inline void lov_conf_thaw(struct lov_object *lov)
 {
+	CDEBUG(D_INODE, "To release share lov(%p) owner %p/%p\n",
+	       lov, lov->lo_owner, current);
 	if (lov->lo_owner != current)
 		up_read(&lov->lo_type_guard);
 }
@@ -698,10 +702,14 @@ static void lov_conf_lock(struct lov_object *lov)
 	down_write(&lov->lo_type_guard);
 	LASSERT(!lov->lo_owner);
 	lov->lo_owner = current;
+	CDEBUG(D_INODE, "Took exclusive lov(%p) owner %p\n",
+	       lov, lov->lo_owner);
 }
 
 static void lov_conf_unlock(struct lov_object *lov)
 {
+	CDEBUG(D_INODE, "To release exclusive lov(%p) owner %p\n",
+	       lov, lov->lo_owner);
 	lov->lo_owner = NULL;
 	up_write(&lov->lo_type_guard);
 }
@@ -725,6 +733,7 @@ static int lov_layout_change(const struct lu_env *unused,
 			     struct lov_object *lov, struct lov_stripe_md *lsm,
 			     const struct cl_object_conf *conf)
 {
+	struct lov_device *lov_dev = lov_object_dev(lov);
 	enum lov_layout_type llt = lov_type(lsm);
 	union lov_layout_state *state = &lov->u;
 	const struct lov_layout_operations *old_ops;
@@ -760,14 +769,21 @@ static int lov_layout_change(const struct lu_env *unused,
 
 	LASSERT(!atomic_read(&lov->lo_active_ios));
 
+	CDEBUG(D_INODE, DFID "Apply new layout lov %p, type %d\n",
+	       PFID(lu_object_fid(lov2lu(lov))), lov, llt);
+
 	lov->lo_type = LLT_EMPTY;
 
 	/* page bufsize fixup */
 	cl_object_header(&lov->lo_cl)->coh_page_bufsize -=
 			lov_page_slice_fixup(lov, NULL);
 
-	rc = new_ops->llo_init(env, lov_object_dev(lov), lov, lsm, conf, state);
+	rc = new_ops->llo_init(env, lov_dev, lov, lsm, conf, state);
 	if (rc) {
+		struct obd_device *obd = lov2obd(lov_dev->ld_lov);
+
+		CERROR("%s: cannot apply new layout on " DFID " : rc = %d\n",
+		       obd->obd_name, PFID(lu_object_fid(lov2lu(lov))), rc);
 		new_ops->llo_delete(env, lov, state);
 		new_ops->llo_fini(env, lov, state);
 		/* this file becomes an EMPTY file. */
@@ -923,6 +939,11 @@ int lov_io_init(const struct lu_env *env, struct cl_object *obj,
 		struct cl_io *io)
 {
 	CL_IO_SLICE_CLEAN(lov_env_io(env), lis_cl);
+
+	CDEBUG(D_INODE, DFID "io %p type %d ignore/verify layout %d/%d\n",
+	       PFID(lu_object_fid(&obj->co_lu)), io, io->ci_type,
+	       io->ci_ignore_layout, io->ci_verify_layout);
+
 	return LOV_2DISPATCH_MAYLOCK(cl2lov(obj), llo_io_init,
 				     !io->ci_ignore_layout, env, obj, io);
 }
diff --git a/drivers/staging/lustre/lustre/osc/osc_cache.c b/drivers/staging/lustre/lustre/osc/osc_cache.c
index 5ac0e14..72dd554 100644
--- a/drivers/staging/lustre/lustre/osc/osc_cache.c
+++ b/drivers/staging/lustre/lustre/osc/osc_cache.c
@@ -1001,6 +1001,7 @@ static int osc_extent_truncate(struct osc_extent *ext, pgoff_t trunc_index,
 	env = cl_env_get(&refcheck);
 	io  = &osc_env_info(env)->oti_io;
 	io->ci_obj = cl_object_top(osc2cl(obj));
+	io->ci_ignore_layout = 1;
 	rc = cl_io_init(env, io, CIT_MISC, io->ci_obj);
 	if (rc < 0)
 		goto out;
-- 
1.8.3.1

[toc] | [prev] | [next] | [standalone]


Page 1 of 3  [1] 2 3  Next page →

Back to top | Article view | linux.kernel


csiph-web