Re: [RFC PATCH] mm, compaction: allow compaction for GFP_NOFS requests

From	Michal Hocko <mhocko@kernel.org>
Newsgroups	linux.kernel
Subject	Re: [RFC PATCH] mm, compaction: allow compaction for GFP_NOFS requests
Date	2016-10-13 13:10 +0200
Message-ID	<srM3E-2I3-43@gated-at.bofh.it> (permalink)
References	(2 earlier) <soSIi-mA-9@gated-at.bofh.it> <sp6rT-2cz-5@gated-at.bofh.it> <spDea-V4-21@gated-at.bofh.it> <srC4i-3JZ-13@gated-at.bofh.it> <srL7z-24J-3@gated-at.bofh.it>
Organization	linux.* mail to news gateway

Show all headers | View raw

On Thu 13-10-16 09:39:47, Michal Hocko wrote:
> On Thu 13-10-16 11:29:24, Dave Chinner wrote:
> > On Fri, Oct 07, 2016 at 03:18:14PM +0200, Michal Hocko wrote:
> [...]
> > > Unpatched kernel:
> > > #       Version 3.3, 16 thread(s) starting at Fri Oct  7 09:55:05 2016
> > > #       Sync method: NO SYNC: Test does not issue sync() or fsync() calls.
> > > #       Directories:  Time based hash between directories across 10000 subdirectories with 180 seconds per subdirectory.
> > > #       File names: 40 bytes long, (16 initial bytes of time stamp with 24 random bytes at end of name)
> > > #       Files info: size 0 bytes, written with an IO size of 16384 bytes per write
> > > #       App overhead is time in microseconds spent in the test not doing file writing related system calls.
> > > #
> > > FSUse%        Count         Size    Files/sec     App Overhead
> > >      1      1600000            0       4300.1         20745838
> > >      3      3200000            0       4239.9         23849857
> > >      5      4800000            0       4243.4         25939543
> > >      6      6400000            0       4248.4         19514050
> > >      8      8000000            0       4262.1         20796169
> > >      9      9600000            0       4257.6         21288675
> > >     11     11200000            0       4259.7         19375120
> > >     13     12800000            0       4220.7         22734141
> > >     14     14400000            0       4238.5         31936458
> > >     16     16000000            0       4231.5         23409901
> > >     18     17600000            0       4045.3         23577700
> > >     19     19200000            0       2783.4         58299526
> > >     21     20800000            0       2678.2         40616302
> > >     23     22400000            0       2693.5         83973996
> > > Ctrl+C because it just took too long.
> > 
> > Try running it on a larger filesystem, or configure the fs with more
> > AGs and a larger log (i.e. mkfs.xfs -f -dagcount=24 -l size=512m
> > <dev>). That will speed up modifications and increase concurrency.
> > This test should be able to run 5-10x faster than this (it
> > sustains 55,000 files/s @ 300MB/s write on my test fs on a cheap
> > SSD).
> 
> Will add more memory to the machine. Will report back on that.

increasing the memory to 1G didn't help. So I've tried to add
-dagcount=24 -l size=512m and that didn't help much either. I am at 5k
files/s so nowhere close to your 55k. I thought this is more about CPUs
count than about the amount of memory. So I've tried a larger machine
with 24 CPUs (no dagcount etc...), this one doesn't have a fast storage
so I've backed the fs image by ramdisk but even then I am getting very
similar results. No idea what is wrong with my kvm setup.
-- 
Michal Hocko
SUSE Labs

Thread

[RFC PATCH] mm, compaction: allow compaction for GFP_NOFS requests Michal Hocko <mhocko@kernel.org> - 2016-10-04 10:20 +0200
  Re: [RFC PATCH] mm, compaction: allow compaction for GFP_NOFS  requests Dave Chinner <david@fromorbit.com> - 2016-10-04 22:40 +0200
    Re: [RFC PATCH] mm, compaction: allow compaction for GFP_NOFS  requests Michal Hocko <mhocko@kernel.org> - 2016-10-05 13:40 +0200
      Re: [RFC PATCH] mm, compaction: allow compaction for GFP_NOFS  requests Dave Chinner <david@fromorbit.com> - 2016-10-06 04:20 +0200
        Re: [RFC PATCH] mm, compaction: allow compaction for GFP_NOFS  requests Michal Hocko <mhocko@kernel.org> - 2016-10-07 15:20 +0200
          Re: [RFC PATCH] mm, compaction: allow compaction for GFP_NOFS  requests Dave Chinner <david@fromorbit.com> - 2016-10-13 02:30 +0200
            Re: [RFC PATCH] mm, compaction: allow compaction for GFP_NOFS  requests Michal Hocko <mhocko@kernel.org> - 2016-10-13 12:10 +0200
              Re: [RFC PATCH] mm, compaction: allow compaction for GFP_NOFS  requests Michal Hocko <mhocko@kernel.org> - 2016-10-13 13:10 +0200
                Re: [RFC PATCH] mm, compaction: allow compaction for GFP_NOFS  requests Dave Chinner <david@fromorbit.com> - 2016-10-17 00:10 +0200
                Re: [RFC PATCH] mm, compaction: allow compaction for GFP_NOFS  requests Michal Hocko <mhocko@kernel.org> - 2016-10-17 10:30 +0200
                Re: [RFC PATCH] mm, compaction: allow compaction for GFP_NOFS  requests Dave Chinner <david@fromorbit.com> - 2016-10-18 08:30 +0200
                Re: [RFC PATCH] mm, compaction: allow compaction for GFP_NOFS  requests Michal Hocko <mhocko@kernel.org> - 2016-10-18 14:40 +0200
  Re: [RFC PATCH] mm, compaction: allow compaction for GFP_NOFS  requests Vlastimil Babka <vbabka@suse.cz> - 2016-10-07 07:30 +0200
    Re: [RFC PATCH] mm, compaction: allow compaction for GFP_NOFS  requests Michal Hocko <mhocko@kernel.org> - 2016-10-07 09:00 +0200
      Re: [RFC PATCH] mm, compaction: allow compaction for GFP_NOFS  requests Vlastimil Babka <vbabka@suse.cz> - 2016-10-07 10:20 +0200
        Re: [RFC PATCH] mm, compaction: allow compaction for GFP_NOFS  requests Michal Hocko <mhocko@kernel.org> - 2016-10-07 11:30 +0200
          Re: [RFC PATCH] mm, compaction: allow compaction for GFP_NOFS  requests Vlastimil Babka <vbabka@suse.cz> - 2016-10-10 08:50 +0200

csiph-web