Re: [PATCHSET 00/16] perf top: Add multi-thread support (v1)

From	Namhyung Kim <namhyung@kernel.org>
Newsgroups	linux.kernel
Subject	Re: [PATCHSET 00/16] perf top: Add multi-thread support (v1)
Date	2015-12-14 17:40 +0100
Message-ID	<qFEki-9n-21@gated-at.bofh.it> (permalink)
References	(2 earlier) <qE55f-3L0-3@gated-at.bofh.it> <qEr5L-1BY-11@gated-at.bofh.it> <qExuy-69x-11@gated-at.bofh.it> <qFxCa-4ar-15@gated-at.bofh.it> <qFxLP-4dP-17@gated-at.bofh.it>
Organization	linux.* mail to news gateway

Show all headers | View raw

Hi Ingo,

On Mon, Dec 14, 2015 at 10:38:41AM +0100, Ingo Molnar wrote:
> 
> * Peter Zijlstra <peterz@infradead.org> wrote:
> 
> > On Fri, Dec 11, 2015 at 08:01:31AM -0700, David Ahern wrote:
> > > On 12/11/15 1:11 AM, Ingo Molnar wrote:
> > > >
> > > >* Namhyung Kim <namhyung@gmail.com> wrote:
> > > >
> > > >>IIRC David said that thread per cpu seems too much especially on a large system
> > > >>(like ~1024 cpu). [...]
> > > >
> > > >Too much in what fashion? For recording I think it's the fastest, most natural
> > > >model - anything else will create cache line bounces.
> > > 
> > > The intrusiveness of perf on the system under observation. I understand
> > > there are a lot of factors that go into it.
> > 
> > So I can see some of that, if every cpu has its own thread then every
> > cpu will occasionally schedule that thread. Whereas if there were less,
> > you'd not have that.
> > 
> > Still, I think it makes sense to implement it, we need the multi-file
> > option anyway. Once we have that, we can also implement a per-node
> > option, which should be a fairly simple hybrid of the two approaches.
> > 
> > The thing is, perf-record is really struggling on big machines.
> > 
> > And in an unrelated note, I absolutely detest --buildid being the
> > default, it makes perf-record blow chunks.
> 
> So I'd absolutely _love_ to split up the singular perf.data into a hierarchy of 
> files in a .perf directory, with a structure like this (4-core system):
> 
> 	.perf/cmdline
> 	.perf/features
> 	.perf/evlist
> 	.perf/ring_buffers/cpu0/raw.trace
> 	.perf/ring_buffers/cpu1/raw.trace
> 	.perf/ring_buffers/cpu2/raw.trace
> 	.perf/ring_buffers/cpu3/raw.trace
> 	...
> 
> I.e. the current single file format of perf.data would be split up into individual 
> files. Each CPU would get its own trace file output - any sorting and ordering 
> would be done afterwards. 'perf record' itself would never by default have to do 
> any of that, it's a pure recording session.
> 
> 'perf archive' would still create a single file to make transport between machines 
> easy.
> 
> perf.data.old would be replaced by a .perf.old directory or so.
> 
> Debugging would be easier too I think, as there's no complex perf data format 
> anymore, it's all in individual (typically text, or binary dump) files in the 
> .perf directory.
> 
> This would solve all the scalability problems - and would make the format more 
> extensible and generally more accessible as well.
> 
> What do you think?

It requires many changes, but basically I also like the split-up since
it's easier to deal with.  IIRC there was an opinion (Andi?) regarding
single-file vs multi-file.  The file access will be better for single
file so I changed my earlier implementation to use indexed single data
file instead of multiple files.

But anyway, I'll work on perf-top first :)

Thanks,
Namhyung
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Thread

Re: [PATCHSET 00/16] perf top: Add multi-thread support (v1) Peter Zijlstra <peterz@infradead.org> - 2015-12-14 10:30 +0100
  Re: [PATCHSET 00/16] perf top: Add multi-thread support (v1) Ingo Molnar <mingo@kernel.org> - 2015-12-14 10:40 +0100
    Re: [PATCHSET 00/16] perf top: Add multi-thread support (v1) David Ahern <dsahern@gmail.com> - 2015-12-14 16:00 +0100
      Re: [PATCHSET 00/16] perf top: Add multi-thread support (v1) Arnaldo Carvalho de Melo <acme@kernel.org> - 2015-12-14 17:30 +0100
        Re: [PATCHSET 00/16] perf top: Add multi-thread support (v1) Peter Zijlstra <peterz@infradead.org> - 2015-12-14 17:50 +0100
          Re: [PATCHSET 00/16] perf top: Add multi-thread support (v1) Arnaldo Carvalho de Melo <acme@kernel.org> - 2015-12-14 19:00 +0100
    Re: [PATCHSET 00/16] perf top: Add multi-thread support (v1) Namhyung Kim <namhyung@kernel.org> - 2015-12-14 17:40 +0100
      Re: [PATCHSET 00/16] perf top: Add multi-thread support (v1) Peter Zijlstra <peterz@infradead.org> - 2015-12-14 18:00 +0100
        Re: [PATCHSET 00/16] perf top: Add multi-thread support (v1) Namhyung Kim <namhyung@kernel.org> - 2015-12-14 18:20 +0100
  Re: [PATCHSET 00/16] perf top: Add multi-thread support (v1) David Ahern <dsahern@gmail.com> - 2015-12-14 15:50 +0100
    Re: [PATCHSET 00/16] perf top: Add multi-thread support (v1) Namhyung Kim <namhyung@kernel.org> - 2015-12-14 18:10 +0100
      Re: [PATCHSET 00/16] perf top: Add multi-thread support (v1) Arnaldo Carvalho de Melo <acme@kernel.org> - 2015-12-14 19:00 +0100
  Re: [PATCHSET 00/16] perf top: Add multi-thread support (v1) Namhyung Kim <namhyung@kernel.org> - 2015-12-14 17:30 +0100
    Re: [PATCHSET 00/16] perf top: Add multi-thread support (v1) Peter Zijlstra <peterz@infradead.org> - 2015-12-14 17:50 +0100

csiph-web