Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1576435 > unrolled thread

[PATCH] proc/sysctl: drop unregistered stale dentries as soon as possible

Back to article view | Back to linux.kernel

Contents

  [PATCH] proc/sysctl: drop unregistered stale dentries as soon as  possible Konstantin Khlebnikov <khlebnikov@yandex-team.ru> - 2017-02-08 12:00 +0100
    Re: [PATCH] proc/sysctl: drop unregistered stale dentries as soon  as possible Andrew Morton <akpm@linux-foundation.org> - 2017-02-08 22:50 +0100
      Re: [PATCH] proc/sysctl: drop unregistered stale dentries as soon as  possible Al Viro <viro@ZenIV.linux.org.uk> - 2017-02-09 05:00 +0100
        Re: [PATCH] proc/sysctl: drop unregistered stale dentries as soon as possible Konstantin Khlebnikov <koct9i@gmail.com> - 2017-02-09 08:40 +0100
          Re: [PATCH] proc/sysctl: drop unregistered stale dentries as soon as  possible Al Viro <viro@ZenIV.linux.org.uk> - 2017-02-09 10:10 +0100
            [PATCH] proc/sysctl: prune stale dentries during unregistering Konstantin Khlebnikov <khlebnikov@yandex-team.ru> - 2017-02-10 08:40 +0100
              Re: [PATCH] proc/sysctl: prune stale dentries during unregistering Al Viro <viro@ZenIV.linux.org.uk> - 2017-02-10 08:50 +0100
                Re: [PATCH] proc/sysctl: prune stale dentries during unregistering Konstantin Khlebnikov <khlebnikov@yandex-team.ru> - 2017-02-10 09:00 +0100
                  Re: [PATCH] proc/sysctl: prune stale dentries during unregistering ebiederm@xmission.com (Eric W. Biederman) - 2017-02-13 11:00 +0100

#1576435 — [PATCH] proc/sysctl: drop unregistered stale dentries as soon as possible

Currently unregistering sysctl does not prune its dentries.
Stale sysctl dentries could slowdown sysctl operations significantly.

For example, command:

# for i in {1..100000} ; do unshare -n -- sysctl -a &> /dev/null ; done

creates a millions of stale denties around sysctls of loopback interface:

# sysctl fs.dentry-state
fs.dentry-state = 25812579	24724135	45	0	0	0

All of them have matching names thus lookup have to scan though whole
hash chain and call d_compare (proc_sys_compare) which checks them
under system-wide spinlock (sysctl_lock).

# time sysctl -a > /dev/null
real    1m12.806s
user    0m0.016s
sys     1m12.400s

Currently only memory reclaimer could remove this garbage.
But without significant memory pressure this never happens.

This patch detects stale dentry in proc_sys_compare and pretends that
it has matching name - revalidation will kill it and lookup restarts.
As a result each stale dentry will be seen only once and will not
contaminate hash endlessly.

Signed-off-by: Konstantin Khlebnikov <khlebnikov@yandex-team.ru>
---
 fs/proc/proc_sysctl.c |   10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/fs/proc/proc_sysctl.c b/fs/proc/proc_sysctl.c
index d4e37acd4821..1af7230c2c9e 100644
--- a/fs/proc/proc_sysctl.c
+++ b/fs/proc/proc_sysctl.c
@@ -852,11 +852,19 @@ static int proc_sys_compare(const struct dentry *dentry,
 	inode = d_inode_rcu(dentry);
 	if (!inode)
 		return 1;
+
+	/*
+	 * Stale dentry: we cannot invalidate it right here, instead we
+	 * pretend that it matches and revalidation will kill it later.
+	 */
+	head = rcu_dereference(PROC_I(inode)->sysctl);
+	if (head && head->unregistering)
+		return 0;
+
 	if (name->len != len)
 		return 1;
 	if (memcmp(name->name, str, len))
 		return 1;
-	head = rcu_dereference(PROC_I(inode)->sysctl);
 	return !head || !sysctl_is_seen(head);
 }
 

[toc] | [next] | [standalone]

#1577117 — Re: [PATCH] proc/sysctl: drop unregistered stale dentries as soon as possible

On Wed, 08 Feb 2017 13:48:24 +0300 Konstantin Khlebnikov <khlebnikov@yandex-team.ru> wrote:

> Currently unregistering sysctl does not prune its dentries.
> Stale sysctl dentries could slowdown sysctl operations significantly.
> 
> For example, command:
> 
> # for i in {1..100000} ; do unshare -n -- sysctl -a &> /dev/null ; done
> 
> creates a millions of stale denties around sysctls of loopback interface:
> 
> # sysctl fs.dentry-state
> fs.dentry-state = 25812579	24724135	45	0	0	0
> 
> All of them have matching names thus lookup have to scan though whole
> hash chain and call d_compare (proc_sys_compare) which checks them
> under system-wide spinlock (sysctl_lock).
> 
> # time sysctl -a > /dev/null
> real    1m12.806s
> user    0m0.016s
> sys     1m12.400s
> 
> Currently only memory reclaimer could remove this garbage.
> But without significant memory pressure this never happens.
> 
> This patch detects stale dentry in proc_sys_compare and pretends that
> it has matching name - revalidation will kill it and lookup restarts.
> As a result each stale dentry will be seen only once and will not
> contaminate hash endlessly.
> 

What are "stale" dentries?  Unused dentries?  If so, why doesn't the
creation of a new dentry immediately invalidate the old dentry with a
matching path?  What do other filesystems do to prevent this issue?

IOW I'm wondering if this should be fixed in some other place.  Al?

> --- a/fs/proc/proc_sysctl.c
> +++ b/fs/proc/proc_sysctl.c
> @@ -852,11 +852,19 @@ static int proc_sys_compare(const struct dentry *dentry,
>  	inode = d_inode_rcu(dentry);
>  	if (!inode)
>  		return 1;
> +
> +	/*
> +	 * Stale dentry: we cannot invalidate it right here, instead we
> +	 * pretend that it matches and revalidation will kill it later.
> +	 */
> +	head = rcu_dereference(PROC_I(inode)->sysctl);
> +	if (head && head->unregistering)
> +		return 0;
> +
>  	if (name->len != len)
>  		return 1;
>  	if (memcmp(name->name, str, len))
>  		return 1;
> -	head = rcu_dereference(PROC_I(inode)->sysctl);
>  	return !head || !sysctl_is_seen(head);
>  }
>  

[toc] | [prev] | [next] | [standalone]

#1577290

On Wed, Feb 08, 2017 at 01:48:04PM -0800, Andrew Morton wrote:

> > This patch detects stale dentry in proc_sys_compare and pretends that
> > it has matching name - revalidation will kill it and lookup restarts.
> > As a result each stale dentry will be seen only once and will not
> > contaminate hash endlessly.
> > 
> 
> What are "stale" dentries?  Unused dentries?  If so, why doesn't the
> creation of a new dentry immediately invalidate the old dentry with a
> matching path?  What do other filesystems do to prevent this issue?

The whole point is that it's *NOT* a matching path.  Currently ->d_compare()
for /proc/sys tries to make sure that sysctl getting unregistered means
that no extra references will be added to dentries of the stuff we are
trying to kick out.  If it's getting unregistered, ->d_compare() won't be
seeing them and from that point on dentry refcount can only go down -
no new lookups will increase it.

This kludge tries to have _any_ lookup in the same hash chain pick the
first dentry of such stuff, no matter what name/parent it has.  Then
it relies upon ->d_revalidate() refusing to accept that sucker, so that
it gets unhashed and we (hopefully) repeat the lookup.

This is complete garbage.  Lookups won't be repeated indefinitely -
if there are several such dentries in the hash chain we search, syscall
will end up failing with ESTALE on thus buggered ->d_compare(), even though
none of those dentries are anywhere near the path we are trying to resolve.
No other filesystem attempts that kind of insanity, and for a good reason.

The problem it tries to address is that sysctl unregistration doesn't
unhash the now-stale dentries.  Before the unregistration we kept them
even with refcount 0, until memory pressure evicts the suckers.  After
unregistration we make sure that refcount reaching 0 will cause the
instant eviction.  The problem is with the case when they had refcount 0
to start with.  Then the eviction rule does not get triggered - it would have
happened when dropping the last reference, but we don't have any.

The kludge proposed in that patch is nowhere near being a sane way to deal
with that.  Having ->d_compare() notice such dentries and quietly kick
them out would be borderline saner, but
	* it's a potentially blocking operation and ->d_compare() is called
in non-blocking contexts, including deep under rcu_read_lock().
	* it's done when walking a hash chain; having that chain modified
by ->d_compare() itself would require some modifications of callers and
those are very hot codepaths.

I agree that the problem is real, but this is no way to deal with it.
What we want is something along the lines of d_prune_aliases() done for all
inodes corresponding to given sysctl.  Done just before erase_header()
in start_unregistering().  That would require maintaining the list of
inodes in question (e.g. anchored in ctl_table_header) and a bit of care
in traversing it (use of igrab(), etc.)

In the current form - NAK.  Sorry.

[toc] | [prev] | [next] | [standalone]

#1577357 — Re: [PATCH] proc/sysctl: drop unregistered stale dentries as soon as possible

On Thu, Feb 9, 2017 at 6:53 AM, Al Viro <viro@zeniv.linux.org.uk> wrote:
> On Wed, Feb 08, 2017 at 01:48:04PM -0800, Andrew Morton wrote:
>
>> > This patch detects stale dentry in proc_sys_compare and pretends that
>> > it has matching name - revalidation will kill it and lookup restarts.
>> > As a result each stale dentry will be seen only once and will not
>> > contaminate hash endlessly.
>> >
>>
>> What are "stale" dentries?  Unused dentries?  If so, why doesn't the
>> creation of a new dentry immediately invalidate the old dentry with a
>> matching path?  What do other filesystems do to prevent this issue?
>
> The whole point is that it's *NOT* a matching path.  Currently ->d_compare()
> for /proc/sys tries to make sure that sysctl getting unregistered means
> that no extra references will be added to dentries of the stuff we are
> trying to kick out.  If it's getting unregistered, ->d_compare() won't be
> seeing them and from that point on dentry refcount can only go down -
> no new lookups will increase it.
>
> This kludge tries to have _any_ lookup in the same hash chain pick the
> first dentry of such stuff, no matter what name/parent it has.  Then
> it relies upon ->d_revalidate() refusing to accept that sucker, so that
> it gets unhashed and we (hopefully) repeat the lookup.
>
> This is complete garbage.  Lookups won't be repeated indefinitely -
> if there are several such dentries in the hash chain we search, syscall
> will end up failing with ESTALE on thus buggered ->d_compare(), even though
> none of those dentries are anywhere near the path we are trying to resolve.
> No other filesystem attempts that kind of insanity, and for a good reason.
>
> The problem it tries to address is that sysctl unregistration doesn't
> unhash the now-stale dentries.  Before the unregistration we kept them
> even with refcount 0, until memory pressure evicts the suckers.  After
> unregistration we make sure that refcount reaching 0 will cause the
> instant eviction.  The problem is with the case when they had refcount 0
> to start with.  Then the eviction rule does not get triggered - it would have
> happened when dropping the last reference, but we don't have any.
>
> The kludge proposed in that patch is nowhere near being a sane way to deal
> with that.  Having ->d_compare() notice such dentries and quietly kick
> them out would be borderline saner, but
>         * it's a potentially blocking operation and ->d_compare() is called
> in non-blocking contexts, including deep under rcu_read_lock().
>         * it's done when walking a hash chain; having that chain modified
> by ->d_compare() itself would require some modifications of callers and
> those are very hot codepaths.
>
> I agree that the problem is real, but this is no way to deal with it.
> What we want is something along the lines of d_prune_aliases() done for all
> inodes corresponding to given sysctl.  Done just before erase_header()
> in start_unregistering().  That would require maintaining the list of
> inodes in question (e.g. anchored in ctl_table_header) and a bit of care
> in traversing it (use of igrab(), etc.)
>
> In the current form - NAK.  Sorry.

Ok, Thank you. I've expected that this fix isn't sane,

Maybe we could minimize changes for now. For example: keep these
stale dentries in memory but silently unhash them in ->d_compare().
Memory processure and reclaimer will kill them later.

[toc] | [prev] | [next] | [standalone]

#1577395

On Thu, Feb 09, 2017 at 10:36:15AM +0300, Konstantin Khlebnikov wrote:

> Ok, Thank you. I've expected that this fix isn't sane,
> 
> Maybe we could minimize changes for now. For example: keep these
> stale dentries in memory but silently unhash them in ->d_compare().
> Memory processure and reclaimer will kill them later.

->d_compare() is called by the code walking the hash chains.  What's worse,
in the most common case all we have is rcu_read_lock().  Modifying the chain
in rcu reader is no-go.  Turning __d_lookup_rcu() into a writer on the
off-chance that we'll walk onto a visibly stale sysctl dentry - even more so.

If you want to deal with that, do it right, please.  Have sysctl inodes
on a list of some kind anchored in struct ctl_table_header; insert them
there in proc_sys_make_inode(), remove - in proc_evict_inode() (or
have it pass the inode to sysctl_head_put() and do the removal there).
Use sysctl_lock for serialization.

In start_unregistering(), just before the erase_header() call, check
if the list is non-empty and if it is -
	grab sysctl_lock
	last = NULL
	walk the list
		igrab(inode we are looking at)
		if succeeded
			drop sysctl_lock
			iput(last)
			last = that inode
			d_prune_aliases(last)
			retake sysctl_lock
			// inode is still not evicted, so it's still on the list
	drop sysctl_lock
	iput(last)

list would pass through struct proc_inode, and I would probably use
hlist rather than the normal one; might be more convenient to initialize
that way.  Getting from containing struct proc_inode to inode - &ei->vfs_inode.

It's not that much work; if you have time - go for it, or remind me after
-rc1...

[toc] | [prev] | [next] | [standalone]

#1578226 — [PATCH] proc/sysctl: prune stale dentries during unregistering

Currently unregistering sysctl table does not prune its dentries.
Stale dentries could slowdown sysctl operations significantly.

For example, command:

# for i in {1..100000} ; do unshare -n -- sysctl -a &> /dev/null ; done

creates a millions of stale denties around sysctls of loopback interface:

# sysctl fs.dentry-state
fs.dentry-state = 25812579  24724135        45      0       0       0

All of them have matching names thus lookup have to scan though whole
hash chain and call d_compare (proc_sys_compare) which checks them
under system-wide spinlock (sysctl_lock).

# time sysctl -a > /dev/null
real    1m12.806s
user    0m0.016s
sys     1m12.400s

Currently only memory reclaimer could remove this garbage.
But without significant memory pressure this never happens.

This patch collects sysctl inodes into list on sysctl table header and
prunes all their dentries once that table unregisters.

Signed-off-by: Konstantin Khlebnikov <khlebnikov@yandex-team.ru>
Suggested-by: Al Viro <viro@zeniv.linux.org.uk>
---
 fs/proc/inode.c        |    3 ++
 fs/proc/internal.h     |    7 ++++--
 fs/proc/proc_sysctl.c  |   59 +++++++++++++++++++++++++++++++++++-------------
 include/linux/sysctl.h |    1 +
 4 files changed, 51 insertions(+), 19 deletions(-)

diff --git a/fs/proc/inode.c b/fs/proc/inode.c
index 842a5ff5b85c..7ad9ed7958af 100644
--- a/fs/proc/inode.c
+++ b/fs/proc/inode.c
@@ -43,10 +43,11 @@ static void proc_evict_inode(struct inode *inode)
 	de = PDE(inode);
 	if (de)
 		pde_put(de);
+
 	head = PROC_I(inode)->sysctl;
 	if (head) {
 		RCU_INIT_POINTER(PROC_I(inode)->sysctl, NULL);
-		sysctl_head_put(head);
+		proc_sys_evict_inode(inode, head);
 	}
 }
 
diff --git a/fs/proc/internal.h b/fs/proc/internal.h
index 2de5194ba378..ed1d762160e6 100644
--- a/fs/proc/internal.h
+++ b/fs/proc/internal.h
@@ -65,6 +65,7 @@ struct proc_inode {
 	struct proc_dir_entry *pde;
 	struct ctl_table_header *sysctl;
 	struct ctl_table *sysctl_entry;
+	struct list_head sysctl_inodes;
 	const struct proc_ns_operations *ns_ops;
 	struct inode vfs_inode;
 };
@@ -249,10 +250,12 @@ extern void proc_thread_self_init(void);
  */
 #ifdef CONFIG_PROC_SYSCTL
 extern int proc_sys_init(void);
-extern void sysctl_head_put(struct ctl_table_header *);
+extern void proc_sys_evict_inode(struct inode *inode,
+				 struct ctl_table_header *head);
 #else
 static inline void proc_sys_init(void) { }
-static inline void sysctl_head_put(struct ctl_table_header *head) { }
+static inline void proc_sys_evict_inode(struct  inode *inode,
+					struct ctl_table_header *head) { }
 #endif
 
 /*
diff --git a/fs/proc/proc_sysctl.c b/fs/proc/proc_sysctl.c
index d4e37acd4821..8efb1e10b025 100644
--- a/fs/proc/proc_sysctl.c
+++ b/fs/proc/proc_sysctl.c
@@ -190,6 +190,7 @@ static void init_header(struct ctl_table_header *head,
 	head->set = set;
 	head->parent = NULL;
 	head->node = node;
+	INIT_LIST_HEAD(&head->inodes);
 	if (node) {
 		struct ctl_table *entry;
 		for (entry = table; entry->procname; entry++, node++)
@@ -259,6 +260,29 @@ static void unuse_table(struct ctl_table_header *p)
 			complete(p->unregistering);
 }
 
+/* called under sysctl_lock */
+static void proc_sys_prune_dcache(struct ctl_table_header *head)
+{
+	struct inode *inode, *prev = NULL;
+	struct proc_inode *ei;
+
+	list_for_each_entry(ei, &head->inodes, sysctl_inodes) {
+		inode = igrab(&ei->vfs_inode);
+		if (inode) {
+			spin_unlock(&sysctl_lock);
+			iput(prev);
+			prev = inode;
+			d_prune_aliases(inode);
+			spin_lock(&sysctl_lock);
+		}
+	}
+	if (prev) {
+		spin_unlock(&sysctl_lock);
+		iput(prev);
+		spin_lock(&sysctl_lock);
+	}
+}
+
 /* called under sysctl_lock, will reacquire if has to wait */
 static void start_unregistering(struct ctl_table_header *p)
 {
@@ -278,27 +302,17 @@ static void start_unregistering(struct ctl_table_header *p)
 		p->unregistering = ERR_PTR(-EINVAL);
 	}
 	/*
+	 * Prune dentries for unregistered sysctls: namespaced sysctls
+	 * can have duplicate names and contaminate dcache very badly.
+	 */
+	proc_sys_prune_dcache(p);
+	/*
 	 * do not remove from the list until nobody holds it; walking the
 	 * list in do_sysctl() relies on that.
 	 */
 	erase_header(p);
 }
 
-static void sysctl_head_get(struct ctl_table_header *head)
-{
-	spin_lock(&sysctl_lock);
-	head->count++;
-	spin_unlock(&sysctl_lock);
-}
-
-void sysctl_head_put(struct ctl_table_header *head)
-{
-	spin_lock(&sysctl_lock);
-	if (!--head->count)
-		kfree_rcu(head, rcu);
-	spin_unlock(&sysctl_lock);
-}
-
 static struct ctl_table_header *sysctl_head_grab(struct ctl_table_header *head)
 {
 	BUG_ON(!head);
@@ -440,11 +454,15 @@ static struct inode *proc_sys_make_inode(struct super_block *sb,
 
 	inode->i_ino = get_next_ino();
 
-	sysctl_head_get(head);
 	ei = PROC_I(inode);
 	ei->sysctl = head;
 	ei->sysctl_entry = table;
 
+	spin_lock(&sysctl_lock);
+	list_add(&ei->sysctl_inodes, &head->inodes);
+	head->count++;
+	spin_unlock(&sysctl_lock);
+
 	inode->i_mtime = inode->i_atime = inode->i_ctime = current_time(inode);
 	inode->i_mode = table->mode;
 	if (!S_ISDIR(table->mode)) {
@@ -466,6 +484,15 @@ static struct inode *proc_sys_make_inode(struct super_block *sb,
 	return inode;
 }
 
+void proc_sys_evict_inode(struct inode *inode, struct ctl_table_header *head)
+{
+	spin_lock(&sysctl_lock);
+	list_del(&PROC_I(inode)->sysctl_inodes);
+	if (!--head->count)
+		kfree_rcu(head, rcu);
+	spin_unlock(&sysctl_lock);
+}
+
 static struct ctl_table_header *grab_header(struct inode *inode)
 {
 	struct ctl_table_header *head = PROC_I(inode)->sysctl;
diff --git a/include/linux/sysctl.h b/include/linux/sysctl.h
index adf4e51cf597..b7e82049fec7 100644
--- a/include/linux/sysctl.h
+++ b/include/linux/sysctl.h
@@ -143,6 +143,7 @@ struct ctl_table_header
 	struct ctl_table_set *set;
 	struct ctl_dir *parent;
 	struct ctl_node *node;
+	struct list_head inodes; /* head for proc_inode->sysctl_inodes */
 };
 
 struct ctl_dir {

[toc] | [prev] | [next] | [standalone]

#1578231 — Re: [PATCH] proc/sysctl: prune stale dentries during unregistering

On Fri, Feb 10, 2017 at 10:35:02AM +0300, Konstantin Khlebnikov wrote:

> # time sysctl -a > /dev/null
> real    1m12.806s
> user    0m0.016s
> sys     1m12.400s
> 
> Currently only memory reclaimer could remove this garbage.
> But without significant memory pressure this never happens.
> 
> This patch collects sysctl inodes into list on sysctl table header and
> prunes all their dentries once that table unregisters.

I'd probably go for hlist, but that's mostly cosmetic difference; how about
the matching stats *after* that patch?

[toc] | [prev] | [next] | [standalone]

#1578258 — Re: [PATCH] proc/sysctl: prune stale dentries during unregistering

On 10.02.2017 10:47, Al Viro wrote:
> On Fri, Feb 10, 2017 at 10:35:02AM +0300, Konstantin Khlebnikov wrote:
>
>> # time sysctl -a > /dev/null
>> real    1m12.806s
>> user    0m0.016s
>> sys     1m12.400s
>>
>> Currently only memory reclaimer could remove this garbage.
>> But without significant memory pressure this never happens.
>>
>> This patch collects sysctl inodes into list on sysctl table header and
>> prunes all their dentries once that table unregisters.
>
> I'd probably go for hlist, but that's mostly cosmetic difference; how about
> the matching stats *after* that patch?
>

dcache size doesn't grow endlessly, so stats are fine

# sysctl fs.dentry-state
fs.dentry-state = 92712	58376	45	0	0	0

# time sysctl -a &>/dev/null

real	0m0.013s
user	0m0.004s
sys	0m0.008s

[toc] | [prev] | [next] | [standalone]

#1579565 — Re: [PATCH] proc/sysctl: prune stale dentries during unregistering

Konstantin Khlebnikov <khlebnikov@yandex-team.ru> writes:

> On 10.02.2017 10:47, Al Viro wrote:
>> On Fri, Feb 10, 2017 at 10:35:02AM +0300, Konstantin Khlebnikov wrote:
>>
>>> # time sysctl -a > /dev/null
>>> real    1m12.806s
>>> user    0m0.016s
>>> sys     1m12.400s
>>>
>>> Currently only memory reclaimer could remove this garbage.
>>> But without significant memory pressure this never happens.
>>>
>>> This patch collects sysctl inodes into list on sysctl table header and
>>> prunes all their dentries once that table unregisters.
>>
>> I'd probably go for hlist, but that's mostly cosmetic difference; how about
>> the matching stats *after* that patch?
>>
>
> dcache size doesn't grow endlessly, so stats are fine
>
> # sysctl fs.dentry-state
> fs.dentry-state = 92712	58376	45	0	0	0
>
> # time sysctl -a &>/dev/null
>
> real	0m0.013s
> user	0m0.004s
> sys	0m0.008s

Applied thanks,

Eric

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web