Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1291001 > unrolled thread

Re: [PATCHSET 00/16] perf top: Add multi-thread support (v1)

Back to article view | Back to linux.kernel

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Re: [PATCHSET 00/16] perf top: Add multi-thread support (v1) Peter Zijlstra <peterz@infradead.org> - 2015-12-14 10:30 +0100
    Re: [PATCHSET 00/16] perf top: Add multi-thread support (v1) Ingo Molnar <mingo@kernel.org> - 2015-12-14 10:40 +0100
      Re: [PATCHSET 00/16] perf top: Add multi-thread support (v1) David Ahern <dsahern@gmail.com> - 2015-12-14 16:00 +0100
        Re: [PATCHSET 00/16] perf top: Add multi-thread support (v1) Arnaldo Carvalho de Melo <acme@kernel.org> - 2015-12-14 17:30 +0100
          Re: [PATCHSET 00/16] perf top: Add multi-thread support (v1) Peter Zijlstra <peterz@infradead.org> - 2015-12-14 17:50 +0100
            Re: [PATCHSET 00/16] perf top: Add multi-thread support (v1) Arnaldo Carvalho de Melo <acme@kernel.org> - 2015-12-14 19:00 +0100
      Re: [PATCHSET 00/16] perf top: Add multi-thread support (v1) Namhyung Kim <namhyung@kernel.org> - 2015-12-14 17:40 +0100
        Re: [PATCHSET 00/16] perf top: Add multi-thread support (v1) Peter Zijlstra <peterz@infradead.org> - 2015-12-14 18:00 +0100
          Re: [PATCHSET 00/16] perf top: Add multi-thread support (v1) Namhyung Kim <namhyung@kernel.org> - 2015-12-14 18:20 +0100
    Re: [PATCHSET 00/16] perf top: Add multi-thread support (v1) David Ahern <dsahern@gmail.com> - 2015-12-14 15:50 +0100
      Re: [PATCHSET 00/16] perf top: Add multi-thread support (v1) Namhyung Kim <namhyung@kernel.org> - 2015-12-14 18:10 +0100
        Re: [PATCHSET 00/16] perf top: Add multi-thread support (v1) Arnaldo Carvalho de Melo <acme@kernel.org> - 2015-12-14 19:00 +0100
    Re: [PATCHSET 00/16] perf top: Add multi-thread support (v1) Namhyung Kim <namhyung@kernel.org> - 2015-12-14 17:30 +0100
      Re: [PATCHSET 00/16] perf top: Add multi-thread support (v1) Peter Zijlstra <peterz@infradead.org> - 2015-12-14 17:50 +0100

#1291001 — Re: [PATCHSET 00/16] perf top: Add multi-thread support (v1)

On Fri, Dec 11, 2015 at 08:01:31AM -0700, David Ahern wrote:
> On 12/11/15 1:11 AM, Ingo Molnar wrote:
> >
> >* Namhyung Kim <namhyung@gmail.com> wrote:
> >
> >>IIRC David said that thread per cpu seems too much especially on a large system
> >>(like ~1024 cpu). [...]
> >
> >Too much in what fashion? For recording I think it's the fastest, most natural
> >model - anything else will create cache line bounces.
> 
> The intrusiveness of perf on the system under observation. I understand
> there are a lot of factors that go into it.

So I can see some of that, if every cpu has its own thread then every
cpu will occasionally schedule that thread. Whereas if there were less,
you'd not have that.

Still, I think it makes sense to implement it, we need the multi-file
option anyway. Once we have that, we can also implement a per-node
option, which should be a fairly simple hybrid of the two approaches.

The thing is, perf-record is really struggling on big machines.

And in an unrelated note, I absolutely detest --buildid being the
default, it makes perf-record blow chunks.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [next] | [standalone]

#1291006

* Peter Zijlstra <peterz@infradead.org> wrote:

> On Fri, Dec 11, 2015 at 08:01:31AM -0700, David Ahern wrote:
> > On 12/11/15 1:11 AM, Ingo Molnar wrote:
> > >
> > >* Namhyung Kim <namhyung@gmail.com> wrote:
> > >
> > >>IIRC David said that thread per cpu seems too much especially on a large system
> > >>(like ~1024 cpu). [...]
> > >
> > >Too much in what fashion? For recording I think it's the fastest, most natural
> > >model - anything else will create cache line bounces.
> > 
> > The intrusiveness of perf on the system under observation. I understand
> > there are a lot of factors that go into it.
> 
> So I can see some of that, if every cpu has its own thread then every
> cpu will occasionally schedule that thread. Whereas if there were less,
> you'd not have that.
> 
> Still, I think it makes sense to implement it, we need the multi-file
> option anyway. Once we have that, we can also implement a per-node
> option, which should be a fairly simple hybrid of the two approaches.
> 
> The thing is, perf-record is really struggling on big machines.
> 
> And in an unrelated note, I absolutely detest --buildid being the
> default, it makes perf-record blow chunks.

So I'd absolutely _love_ to split up the singular perf.data into a hierarchy of 
files in a .perf directory, with a structure like this (4-core system):

	.perf/cmdline
	.perf/features
	.perf/evlist
	.perf/ring_buffers/cpu0/raw.trace
	.perf/ring_buffers/cpu1/raw.trace
	.perf/ring_buffers/cpu2/raw.trace
	.perf/ring_buffers/cpu3/raw.trace
	...

I.e. the current single file format of perf.data would be split up into individual 
files. Each CPU would get its own trace file output - any sorting and ordering 
would be done afterwards. 'perf record' itself would never by default have to do 
any of that, it's a pure recording session.

'perf archive' would still create a single file to make transport between machines 
easy.

perf.data.old would be replaced by a .perf.old directory or so.

Debugging would be easier too I think, as there's no complex perf data format 
anymore, it's all in individual (typically text, or binary dump) files in the 
.perf directory.

This would solve all the scalability problems - and would make the format more 
extensible and generally more accessible as well.

What do you think?

Thanks,

	Ingo
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1291246

On 12/14/15 2:38 AM, Ingo Molnar wrote:
>
>> And in an unrelated note, I absolutely detest --buildid being the
>> default, it makes perf-record blow chunks.

Yes, a .debug directory that gets bloated fast and being a dot-directory 
is off the primary radar. I forget about it and too often forget to add 
the option to disable it.

>
> So I'd absolutely _love_ to split up the singular perf.data into a hierarchy of
> files in a .perf directory, with a structure like this (4-core system):
>
> 	.perf/cmdline
> 	.perf/features
> 	.perf/evlist
> 	.perf/ring_buffers/cpu0/raw.trace
> 	.perf/ring_buffers/cpu1/raw.trace
> 	.perf/ring_buffers/cpu2/raw.trace
> 	.perf/ring_buffers/cpu3/raw.trace
> 	...

On a related note why a .perf directory?

>
> I.e. the current single file format of perf.data would be split up into individual
> files. Each CPU would get its own trace file output - any sorting and ordering
> would be done afterwards. 'perf record' itself would never by default have to do
> any of that, it's a pure recording session.
>
> 'perf archive' would still create a single file to make transport between machines
> easy.
>
> perf.data.old would be replaced by a .perf.old directory or so.
>
> Debugging would be easier too I think, as there's no complex perf data format
> anymore, it's all in individual (typically text, or binary dump) files in the
> .perf directory.
>
> This would solve all the scalability problems - and would make the format more
> extensible and generally more accessible as well.
>
> What do you think?

Big change to user experience.

I realize perf-archive has been around since I started using perf in 
mid-2010, but I for one never use it. I suspect it is not widely used 
(definitely not in the circles I have been involved and helped with 
perf), so suddenly requiring it is a change in user experience.

The only 2 files on the system I pull off the box are kallsyms and 
perf.data. Most of the systems where I use perf have limited symbols and 
there is nothing in .debug I need to pull of the box.

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1291329

Em Mon, Dec 14, 2015 at 07:55:32AM -0700, David Ahern escreveu:
> On 12/14/15 2:38 AM, Ingo Molnar wrote:
> >
> >>And in an unrelated note, I absolutely detest --buildid being the
> >>default, it makes perf-record blow chunks.
> 
> Yes, a .debug directory that gets bloated fast and being a dot-directory is
> off the primary radar. I forget about it and too often forget to add the
> option to disable it.

Multiple things here, .debug/ should be size limited, and buildid
processing doesn't need necessarily to store things in that cache, I bet
what PeterZ is complaining about is the reprocessing of events at the
end of the session, to find out about the PERF_RECORD_MMAP* events to
then read the build-ids and insert then into the perf.data file header.

All this can be disabled by default, the downside is that when samples
get resolved to something random because the binary used in the session
was replaced by some other we will not be able to notice.

This would be solved by inserting the buildid (20-some bytes) into the
PERF_RECORD_MMAP record, but that remains to be done...
 
> >
> >So I'd absolutely _love_ to split up the singular perf.data into a hierarchy of
> >files in a .perf directory, with a structure like this (4-core system):
> >
> >	.perf/cmdline
> >	.perf/features
> >	.perf/evlist
> >	.perf/ring_buffers/cpu0/raw.trace
> >	.perf/ring_buffers/cpu1/raw.trace
> >	.perf/ring_buffers/cpu2/raw.trace
> >	.perf/ring_buffers/cpu3/raw.trace
> >	...
> 
> On a related note why a .perf directory?
> 
> >
> >I.e. the current single file format of perf.data would be split up into individual
> >files. Each CPU would get its own trace file output - any sorting and ordering
> >would be done afterwards. 'perf record' itself would never by default have to do
> >any of that, it's a pure recording session.
> >
> >'perf archive' would still create a single file to make transport between machines
> >easy.
> >
> >perf.data.old would be replaced by a .perf.old directory or so.
> >
> >Debugging would be easier too I think, as there's no complex perf data format
> >anymore, it's all in individual (typically text, or binary dump) files in the
> >.perf directory.
> >
> >This would solve all the scalability problems - and would make the format more
> >extensible and generally more accessible as well.
> >
> >What do you think?
> 
> Big change to user experience.
> 
> I realize perf-archive has been around since I started using perf in
> mid-2010, but I for one never use it. I suspect it is not widely used
> (definitely not in the circles I have been involved and helped with perf),
> so suddenly requiring it is a change in user experience.
> 
> The only 2 files on the system I pull off the box are kallsyms and
> perf.data. Most of the systems where I use perf have limited symbols and
> there is nothing in .debug I need to pull of the box.

Well, we don't have to use perf archive for that, doing what Ingo
suggests and on top of it putting it into a perf.data file that in fact
is a cpio or tarball would keep the one-file while introducing the split
up for later processing bits.

- Arnaldo
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1291349

On Mon, Dec 14, 2015 at 01:26:55PM -0300, Arnaldo Carvalho de Melo wrote:
> Em Mon, Dec 14, 2015 at 07:55:32AM -0700, David Ahern escreveu:
> > On 12/14/15 2:38 AM, Ingo Molnar wrote:
> > >
> > >>And in an unrelated note, I absolutely detest --buildid being the
> > >>default, it makes perf-record blow chunks.

> Multiple things here, .debug/ should be size limited, and buildid
> processing doesn't need necessarily to store things in that cache, I bet
> what PeterZ is complaining about is the reprocessing of events at the
> end of the session, to find out about the PERF_RECORD_MMAP* events to
> then read the build-ids and insert then into the perf.data file header.
> 

Yeah, its the reprocessing that is taking forever.. On my moderately
sized system with 40 CPUs, the reprocessing is taking about as long as
the actual workload, which is tedious.

Once I figured out what was happening Jiri was quick to point out I
should be using -B.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1291398

Em Mon, Dec 14, 2015 at 05:41:30PM +0100, Peter Zijlstra escreveu:
> On Mon, Dec 14, 2015 at 01:26:55PM -0300, Arnaldo Carvalho de Melo wrote:
> > Em Mon, Dec 14, 2015 at 07:55:32AM -0700, David Ahern escreveu:
> > > On 12/14/15 2:38 AM, Ingo Molnar wrote:
> > > >
> > > >>And in an unrelated note, I absolutely detest --buildid being the
> > > >>default, it makes perf-record blow chunks.
> 
> > Multiple things here, .debug/ should be size limited, and buildid
> > processing doesn't need necessarily to store things in that cache, I bet
> > what PeterZ is complaining about is the reprocessing of events at the
> > end of the session, to find out about the PERF_RECORD_MMAP* events to
> > then read the build-ids and insert then into the perf.data file header.
> > 
> 
> Yeah, its the reprocessing that is taking forever.. On my moderately
> sized system with 40 CPUs, the reprocessing is taking about as long as
> the actual workload, which is tedious.

Right, I thought about using some dummy event for tracking mmaps, which
I think is a technique used by the Intel PT code (well, there it uses it
for sched_switches) will check...
 
> Once I figured out what was happening Jiri was quick to point out I
> should be using -B.

Right, we have to have sane behaviour by default, damn long/high freq
workloads, duh ;-)

- Arnaldo
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1291339

Hi Ingo,

On Mon, Dec 14, 2015 at 10:38:41AM +0100, Ingo Molnar wrote:
> 
> * Peter Zijlstra <peterz@infradead.org> wrote:
> 
> > On Fri, Dec 11, 2015 at 08:01:31AM -0700, David Ahern wrote:
> > > On 12/11/15 1:11 AM, Ingo Molnar wrote:
> > > >
> > > >* Namhyung Kim <namhyung@gmail.com> wrote:
> > > >
> > > >>IIRC David said that thread per cpu seems too much especially on a large system
> > > >>(like ~1024 cpu). [...]
> > > >
> > > >Too much in what fashion? For recording I think it's the fastest, most natural
> > > >model - anything else will create cache line bounces.
> > > 
> > > The intrusiveness of perf on the system under observation. I understand
> > > there are a lot of factors that go into it.
> > 
> > So I can see some of that, if every cpu has its own thread then every
> > cpu will occasionally schedule that thread. Whereas if there were less,
> > you'd not have that.
> > 
> > Still, I think it makes sense to implement it, we need the multi-file
> > option anyway. Once we have that, we can also implement a per-node
> > option, which should be a fairly simple hybrid of the two approaches.
> > 
> > The thing is, perf-record is really struggling on big machines.
> > 
> > And in an unrelated note, I absolutely detest --buildid being the
> > default, it makes perf-record blow chunks.
> 
> So I'd absolutely _love_ to split up the singular perf.data into a hierarchy of 
> files in a .perf directory, with a structure like this (4-core system):
> 
> 	.perf/cmdline
> 	.perf/features
> 	.perf/evlist
> 	.perf/ring_buffers/cpu0/raw.trace
> 	.perf/ring_buffers/cpu1/raw.trace
> 	.perf/ring_buffers/cpu2/raw.trace
> 	.perf/ring_buffers/cpu3/raw.trace
> 	...
> 
> I.e. the current single file format of perf.data would be split up into individual 
> files. Each CPU would get its own trace file output - any sorting and ordering 
> would be done afterwards. 'perf record' itself would never by default have to do 
> any of that, it's a pure recording session.
> 
> 'perf archive' would still create a single file to make transport between machines 
> easy.
> 
> perf.data.old would be replaced by a .perf.old directory or so.
> 
> Debugging would be easier too I think, as there's no complex perf data format 
> anymore, it's all in individual (typically text, or binary dump) files in the 
> .perf directory.
> 
> This would solve all the scalability problems - and would make the format more 
> extensible and generally more accessible as well.
> 
> What do you think?

It requires many changes, but basically I also like the split-up since
it's easier to deal with.  IIRC there was an opinion (Andi?) regarding
single-file vs multi-file.  The file access will be better for single
file so I changed my earlier implementation to use indexed single data
file instead of multiple files.

But anyway, I'll work on perf-top first :)

Thanks,
Namhyung
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1291350

On Tue, Dec 15, 2015 at 01:38:30AM +0900, Namhyung Kim wrote:
> It requires many changes, but basically I also like the split-up since
> it's easier to deal with.  IIRC there was an opinion (Andi?) regarding
> single-file vs multi-file.  The file access will be better for single
> file so I changed my earlier implementation to use indexed single data
> file instead of multiple files.

The page-cache has a lock per inode, so by having all CPUs populate the
one file you get contention on that.

Also, I suppose you'll have to arbitrate ranges in that file for each
cpu to make it work, that too could get you some contention.

Having a file per cpu avoids all that.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1291360

On Mon, Dec 14, 2015 at 05:56:14PM +0100, Peter Zijlstra wrote:
> On Tue, Dec 15, 2015 at 01:38:30AM +0900, Namhyung Kim wrote:
> > It requires many changes, but basically I also like the split-up since
> > it's easier to deal with.  IIRC there was an opinion (Andi?) regarding
> > single-file vs multi-file.  The file access will be better for single
> > file so I changed my earlier implementation to use indexed single data
> > file instead of multiple files.
> 
> The page-cache has a lock per inode, so by having all CPUs populate the
> one file you get contention on that.
> 
> Also, I suppose you'll have to arbitrate ranges in that file for each
> cpu to make it work, that too could get you some contention.
> 
> Having a file per cpu avoids all that.

Right.  Now I recall that it was about *report* (not record) time
accessing single file vs. multi files.

At record time we should use file per cpu.  I combined them into
one with index at post-processing time in my earlier work.

Thanks for clarification!
Namhyung
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1291236

On 12/14/15 2:26 AM, Peter Zijlstra wrote:
> On Fri, Dec 11, 2015 at 08:01:31AM -0700, David Ahern wrote:
>> On 12/11/15 1:11 AM, Ingo Molnar wrote:
>>>
>>> * Namhyung Kim <namhyung@gmail.com> wrote:
>>>
>>>> IIRC David said that thread per cpu seems too much especially on a large system
>>>> (like ~1024 cpu). [...]
>>>
>>> Too much in what fashion? For recording I think it's the fastest, most natural
>>> model - anything else will create cache line bounces.
>>
>> The intrusiveness of perf on the system under observation. I understand
>> there are a lot of factors that go into it.
>
> So I can see some of that, if every cpu has its own thread then every
> cpu will occasionally schedule that thread. Whereas if there were less,
> you'd not have that.
>
> Still, I think it makes sense to implement it, we need the multi-file
> option anyway. Once we have that, we can also implement a per-node
> option, which should be a fairly simple hybrid of the two approaches.
>
> The thing is, perf-record is really struggling on big machines.

I've gone from the 1024-cpu sparc systems earlier this year down to 
small PPC and Rangeley-based switches. For both ends of the scale (and 
in between) I constantly struggle with the options to manage memory, cpu 
and disk consumption.

There definitely needs to be options (e.g., multi-threaded on/off). For 
the threading options I get the appeal for 1-thread per cpu but other 
options make sense as well -- 1 thread per core, 1 per NUMA node. perf 
has the CPU topology so should not be too difficult.

If you have 1-thread per cpu that means you are pinning the threads to 
the cpu? That brings in additional permissions problems.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1291353

Hi David,

On Mon, Dec 14, 2015 at 07:46:28AM -0700, David Ahern wrote:
> On 12/14/15 2:26 AM, Peter Zijlstra wrote:
> >On Fri, Dec 11, 2015 at 08:01:31AM -0700, David Ahern wrote:
> >>On 12/11/15 1:11 AM, Ingo Molnar wrote:
> >>>
> >>>* Namhyung Kim <namhyung@gmail.com> wrote:
> >>>
> >>>>IIRC David said that thread per cpu seems too much especially on a large system
> >>>>(like ~1024 cpu). [...]
> >>>
> >>>Too much in what fashion? For recording I think it's the fastest, most natural
> >>>model - anything else will create cache line bounces.
> >>
> >>The intrusiveness of perf on the system under observation. I understand
> >>there are a lot of factors that go into it.
> >
> >So I can see some of that, if every cpu has its own thread then every
> >cpu will occasionally schedule that thread. Whereas if there were less,
> >you'd not have that.
> >
> >Still, I think it makes sense to implement it, we need the multi-file
> >option anyway. Once we have that, we can also implement a per-node
> >option, which should be a fairly simple hybrid of the two approaches.
> >
> >The thing is, perf-record is really struggling on big machines.
> 
> I've gone from the 1024-cpu sparc systems earlier this year down to small
> PPC and Rangeley-based switches. For both ends of the scale (and in between)
> I constantly struggle with the options to manage memory, cpu and disk
> consumption.
> 
> There definitely needs to be options (e.g., multi-threaded on/off). For the
> threading options I get the appeal for 1-thread per cpu but other options
> make sense as well -- 1 thread per core, 1 per NUMA node. perf has the CPU
> topology so should not be too difficult.

I think we can use --num-thread option to control multi-threading: 1
for disabling and others for enabling.  In the current implementation,
using 1 thread still use same logic so 1 reader + 1 collector will be
created as well as 1 display thread.  Not sure it'd be better special
casing 1 thread to use different code path.

Anyway, I think it'd be nice to have per-core, per-socket and per-node
options and per-core is a good default then.

> 
> If you have 1-thread per cpu that means you are pinning the threads to the
> cpu? That brings in additional permissions problems.

Did you mean setting sched affinity?  It seems not a privileged
operation doing it for its own threads..

Thanks,
Namhyung
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1291394

Em Tue, Dec 15, 2015 at 02:06:54AM +0900, Namhyung Kim escreveu:
> On Mon, Dec 14, 2015 at 07:46:28AM -0700, David Ahern wrote:
> > If you have 1-thread per cpu that means you are pinning the threads to the
> > cpu? That brings in additional permissions problems.
> 
> Did you mean setting sched affinity?  It seems not a privileged
> operation doing it for its own threads..

Right, and we should do it (create per cpu rb consuming threads) just
for the cpus in the workload affinity mask.

- Arnaldo
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1291331

Hi Peter,

On Mon, Dec 14, 2015 at 10:26:13AM +0100, Peter Zijlstra wrote:
> On Fri, Dec 11, 2015 at 08:01:31AM -0700, David Ahern wrote:
> > On 12/11/15 1:11 AM, Ingo Molnar wrote:
> > >
> > >* Namhyung Kim <namhyung@gmail.com> wrote:
> > >
> > >>IIRC David said that thread per cpu seems too much especially on a large system
> > >>(like ~1024 cpu). [...]
> > >
> > >Too much in what fashion? For recording I think it's the fastest, most natural
> > >model - anything else will create cache line bounces.
> > 
> > The intrusiveness of perf on the system under observation. I understand
> > there are a lot of factors that go into it.
> 
> So I can see some of that, if every cpu has its own thread then every
> cpu will occasionally schedule that thread. Whereas if there were less,
> you'd not have that.
> 
> Still, I think it makes sense to implement it, we need the multi-file
> option anyway. Once we have that, we can also implement a per-node
> option, which should be a fairly simple hybrid of the two approaches.
> 
> The thing is, perf-record is really struggling on big machines.

Yes, but perf-record and perf-top is different.  The perf-record
merely saves the data into file while perf-top read events and process
them at the same time without file.  So we should choose different
default IMHO.

I want to focus on perf-top for now, once it's in a good shape, I'll
work on perf record/report too.


> 
> And in an unrelated note, I absolutely detest --buildid being the
> default, it makes perf-record blow chunks.

Maybe we can add a config option?

Thanks,
Namhyung
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1291342

On Tue, Dec 15, 2015 at 01:25:35AM +0900, Namhyung Kim wrote:
> I want to focus on perf-top for now, once it's in a good shape, I'll
> work on perf record/report too.

Fair enough and thanks!

> 
> > 
> > And in an unrelated note, I absolutely detest --buildid being the
> > default, it makes perf-record blow chunks.
> 
> Maybe we can add a config option?

That would be good, until I get a new machine and have forgotten all
about this again :-)

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web