Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1169419 > unrolled thread

Re: [RFC][PATCH] fs: optimize inotify/fsnotify code for unwatched files

Back to article view | Back to linux.kernel

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: [RFC][PATCH] fs: optimize inotify/fsnotify code for unwatched  files Dave Hansen <dave@sr71.net> - 2015-06-20 20:10 +0200
    Re: [RFC][PATCH] fs: optimize inotify/fsnotify code for unwatched  files "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2015-06-21 03:40 +0200
      Re: [RFC][PATCH] fs: optimize inotify/fsnotify code for unwatched  files "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2015-06-23 02:40 +0200
      Re: [RFC][PATCH] fs: optimize inotify/fsnotify code for unwatched  files "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> - 2015-06-23 02:40 +0200

#1169419 — Re: [RFC][PATCH] fs: optimize inotify/fsnotify code for unwatched files

On 06/19/2015 07:21 PM, Paul E. McKenney wrote:
>>> > > What is so expensive in it? Just the memory barrier in it?
>> > 
>> > The profiling doesn't hit on the mfence directly, but I assume that the
>> > overhead is coming from there.  The "mov    0x8(%rdi),%rcx" is identical
>> > before and after the barrier, but it appears much more expensive
>> > _after_.  That makes no sense unless the barrier is the thing causing it.
> OK, one thing to try is to simply delete the memory barrier.  The
> resulting code will be unsafe, but will probably run well enough to
> get benchmark results.  If it is the memory barrier, you should of
> course get increased throughput.

So I took the smp_mb() out of __srcu_read_lock().  The benchmark didn't
improve at all.  Looking at the profile, all of the overhead had just
shifted to __srcu_read_unlock() and its memory barrier!  Removing the
barrier in __srcu_read_unlock() got essentially the same gains out of
the benchmark as the original patch in this thread that just avoids RCU.

I think that's fairly conclusive that the source of the overhead is,
indeed, the memory barriers.

Although I said this test was single threaded, I also had another
thought.  The benchmark is single-threaded, but 'perf' is sitting doing
profiling and who knows what else on the other core, and the profiling
NMIs are certainly writing plenty of data to memory.  So, there might be
plenty of work for that smp_mb()/mfence to do _despite_ the benchmark
itself being single threaded.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [next] | [standalone]

#1169508

On Sat, Jun 20, 2015 at 11:02:08AM -0700, Dave Hansen wrote:
> On 06/19/2015 07:21 PM, Paul E. McKenney wrote:
> >>> > > What is so expensive in it? Just the memory barrier in it?
> >> > 
> >> > The profiling doesn't hit on the mfence directly, but I assume that the
> >> > overhead is coming from there.  The "mov    0x8(%rdi),%rcx" is identical
> >> > before and after the barrier, but it appears much more expensive
> >> > _after_.  That makes no sense unless the barrier is the thing causing it.
> > OK, one thing to try is to simply delete the memory barrier.  The
> > resulting code will be unsafe, but will probably run well enough to
> > get benchmark results.  If it is the memory barrier, you should of
> > course get increased throughput.
> 
> So I took the smp_mb() out of __srcu_read_lock().  The benchmark didn't
> improve at all.  Looking at the profile, all of the overhead had just
> shifted to __srcu_read_unlock() and its memory barrier!  Removing the
> barrier in __srcu_read_unlock() got essentially the same gains out of
> the benchmark as the original patch in this thread that just avoids RCU.
> 
> I think that's fairly conclusive that the source of the overhead is,
> indeed, the memory barriers.
> 
> Although I said this test was single threaded, I also had another
> thought.  The benchmark is single-threaded, but 'perf' is sitting doing
> profiling and who knows what else on the other core, and the profiling
> NMIs are certainly writing plenty of data to memory.  So, there might be
> plenty of work for that smp_mb()/mfence to do _despite_ the benchmark
> itself being single threaded.

Well, it is not hard to have an SRCU-like thing that doesn't have
read-side memory barriers, given that older versions of SRCU didn't
have them.  However, the price is increased latency for the analog to
synchronize_srcu().  I am guessing that this would not be a problem
for notification-group destruction, which is presumably rare.

That said, if empty *_fsnotify_mask is the common case or if the
overhead of processing notification overwhelms srcu_read_lock(),
your original patch seems a bit simpler.

							Thanx, Paul

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1170309

On Mon, Jun 22, 2015 at 08:52:29PM +0200, Peter Zijlstra wrote:
> On Mon, Jun 22, 2015 at 08:11:21AM -0700, Paul E. McKenney wrote:
> > That depends on how slow the resulting slow global state would be.
> > We have some use cases (definitely KVM, perhaps also some of the VFS
> > code) that need the current speed, as opposed to the profound slowness
> > that three trips through synchronize_sched() would provide.
> 
> So what we have with that percpu-rwsem code that I send out earlier
> today is a conditional smp_mb(), and I think we can do the same for
> SRCU.
> 
> I'm just not sure !GP is common enough for all SRCU cases to be worth
> doing.

Especially given that we don't want the readers to have to acquire a
lock in order to get a consistent view of whether or not a grace period
is in progress.

> Those that rely on sync_srcu() and who do it rarely would definitely
> benefit. The same with those that rarely do call_srcu().
> 
> But those that heavily use call_srcu() would be better off with the
> prolonged GP with 3 sync_sched() calls in.

Those are indeed two likely possibilities.  Other possibilities include
cases where synchronize_srcu() is invoked rarely, but where its latency
is visible to userspace, and those where there really is a need to
wait synchronously for a grace period, so that call_srcu() doesn't buy
you anything.

							Thanx, Paul

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1170312

On Mon, Jun 22, 2015 at 09:03:08PM +0200, Peter Zijlstra wrote:
> On Mon, Jun 22, 2015 at 09:29:49AM -0700, Paul E. McKenney wrote:
> 
> > I believe that there still are some cases.  But why would offline
> > CPUs seem so iffy?  CPUs coming up execute code before they are fully
> > operational, and during that time, much of the kernel views them as
> > being offline.  Yet they do have to execute significant code in order
> > to get themselves set up.
> 
> I'm thinking we do far too much during bringup and tear-down as it is.
> But yes maybe.

Boot, suspend, and hibernation indeed would be faster if we did less,
but we still will have to do something.

							Thanx, Paul

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web