Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1457022 > unrolled thread

[BUG] perf: memory leak in perf top

Started by"Wangnan (F)" <wangnan0@huawei.com>
First post2016-08-05 13:00 +0200
Last post2016-08-08 15:30 +0200
Articles 4 — 2 participants

Back to article view | Back to linux.kernel

Contents

  [BUG] perf: memory leak in perf top "Wangnan (F)" <wangnan0@huawei.com> - 2016-08-05 13:00 +0200
    Re: [BUG] perf: memory leak in perf top Arnaldo Carvalho de Melo <acme@kernel.org> - 2016-08-05 16:20 +0200
      Re: [BUG] perf: memory leak in perf top "Wangnan (F)" <wangnan0@huawei.com> - 2016-08-08 13:40 +0200
        Re: [BUG] perf: memory leak in perf top Arnaldo Carvalho de Melo <acme@kernel.org> - 2016-08-08 15:30 +0200

#1457022 — [BUG] perf: memory leak in perf top

From"Wangnan (F)" <wangnan0@huawei.com>
Date2016-08-05 13:00 +0200
Subject[BUG] perf: memory leak in perf top
Message-ID<s2L17-6pn-5@gated-at.bofh.it>
perf top comsumes too much memory than it need.

Using following commands:

  # yes '' > /dev/null
  # perf top -e raw_syscalls:sys_enter

System will quickly become unresponsive because of out of memory.

Using GDB I find two call stacks of malloc. See below.

[call stack 1]

#0 __GI___libc_malloc (bytes=140723951319552) at malloc.c:2841
#1 0x00000000004b7246 in memdup (src=0x7f515e04b054, len=68) at 
../lib/string.c:29
#2 0x00000000004f9299 in hist_entry__init (sample_self=true, 
template=0x7ffcd9213a80, he=0xa37da50) at util/hist.c:401
#3 hist_entry__new (template=template@entry=0x7ffcd9213a80 
<mailto:%28template=template@entry=0x7ffcd9213a80>, 
sample_self=sample_self@entry=true) 
<mailto:sample_self=sample_self@entry=true%29> at util/hist.c:458
#4 0x00000000004fa13f in hists__findnew_entry (hists=0x3549d30, 
hists=0x3549d30, al=0x7ffcd9213ca0, sample_self=true, entry=0x
7ffcd9213a80) at util/hist.c:544
#5 __hists__add_entry (hists=0x3549d30, al=0x7ffcd9213ca0, 
sym_parent=<optimized out>, bi=bi@entry=0x0 <mailto:bi=bi@entry=0x0>, 
mi=mi@entry=0x0 <mailto:mi=mi@entry=0x0>, sampl
e=<optimized out>, sample_self=sample_self@entry=true 
<mailto:sample_self=sample_self@entry=true>, ops=ops@entry=0x0) 
<mailto:ops=ops@entry=0x0%29> at util/hist.c:600
#6 0x00000000004fa3d6 in hists__add_entry (sample_self=true, 
sample=<optimized out>, mi=0x0, bi=0x0, sym_parent=<optimized out
 >, al=<optimized out>, hists=<optimized out>) at util/hist.c:611
#7 iter_add_single_normal_entry (iter=0x7ffcd9213ce0, al=<optimized 
out>) at util/hist.c:823
#8 0x00000000004fc3c5 in hist_entry_iter__add (iter=0x7ffcd9213ce0, 
al=0x7ffcd9213ca0, max_stack_depth=<optimized out>, arg=0x
7ffcd9214120) at util/hist.c:1030
#9 0x00000000004536d7 in perf_event__process_sample (machine=<optimized 
out>, sample=0x7ffcd9213d30, evsel=0x3549bb0, event=<o
ptimized out>, tool=0x7ffcd9214120) at builtin-top.c:800
#10 perf_top__mmap_read_idx (top=top@entry=0x7ffcd9214120 
<mailto:%28top=top@entry=0x7ffcd9214120>, idx=idx@entry=0) 
<mailto:idx=idx@entry=0%29> at builtin-top.c:866
#11 0x00000000004559fd in perf_top__mmap_read (top=0x7ffcd9214120) at 
builtin-top.c:883
#12 __cmd_top (top=0x7ffcd9214120) at builtin-top.c:1018
#13 cmd_top (argc=<optimized out>, argv=<optimized out>, 
prefix=<optimized out>) at builtin-top.c:1344
#14 0x0000000000493561 in run_builtin (p=p@entry=0x948000 
<mailto:%28p=p@entry=0x948000> <commands+288>, argc=argc@entry=4 
<mailto:argc=argc@entry=4>, argv=argv@entry=0x7ffcd92199b0) 
<mailto:argv=argv@entry=0x7ffcd92199b0%29> at
perf.c:357
#15 0x0000000000437157 in handle_internal_command (argv=0x7ffcd92199b0, 
argc=4) at perf.c:419
#16 run_argv (argv=0x7ffcd9219720, argcp=0x7ffcd921972c) at perf.c:465
#17 main (argc=4, argv=0x7ffcd92199b0) at perf.c:609

[call stack 2]

#0 __GI___libc_malloc (bytes=140723951319456) at malloc.c:2841
#1 0x00000000005766de in trace_seq_init (s=0x7ffcd92139b0) at trace-seq.c:60
#2 0x00000000004f7047 in get_trace_output (he=0x7ffcd9213a80) at 
util/sort.c:584
#3 sort__trace_cmp (left=0x40e6d30, right=0x7ffcd9213a80) at util/sort.c:606
#4 0x00000000004fa007 in hist_entry__cmp (right=0x7ffcd9213a80, 
left=0x40e6d30) at util/hist.c:1072
#5 hists__findnew_entry (hists=0x3549d30, hists=0x3549d30, 
al=0x7ffcd9213ca0, sample_self=true, entry=0x7ffcd9213a80) at util/
hist.c:509
#6 __hists__add_entry (hists=0x3549d30, al=0x7ffcd9213ca0, 
sym_parent=<optimized out>, bi=bi@entry=0x0 <mailto:bi=bi@entry=0x0>, 
mi=mi@entry=0x0 <mailto:mi=mi@entry=0x0>, sampl
e=<optimized out>, sample_self=sample_self@entry=true 
<mailto:sample_self=sample_self@entry=true>, ops=ops@entry=0x0) 
<mailto:ops=ops@entry=0x0%29> at util/hist.c:600
#7 0x00000000004fa3d6 in hists__add_entry (sample_self=true, 
sample=<optimized out>, mi=0x0, bi=0x0, sym_parent=<optimized out
 >, al=<optimized out>, hists=<optimized out>) at util/hist.c:611
#8 iter_add_single_normal_entry (iter=0x7ffcd9213ce0, al=<optimized 
out>) at util/hist.c:823
#9 0x00000000004fc3c5 in hist_entry_iter__add (iter=0x7ffcd9213ce0, 
al=0x7ffcd9213ca0, max_stack_depth=<optimized out>, arg=0x
7ffcd9214120) at util/hist.c:1030
#10 0x00000000004536d7 in perf_event__process_sample (machine=<optimized 
out>, sample=0x7ffcd9213d30, evsel=0x3549bb0, event=<o
ptimized out>, tool=0x7ffcd9214120) at builtin-top.c:800
#11 perf_top__mmap_read_idx (top=top@entry=0x7ffcd9214120 
<mailto:%28top=top@entry=0x7ffcd9214120>, idx=idx@entry=0) 
<mailto:idx=idx@entry=0%29> at builtin-top.c:866
#12 0x00000000004559fd in perf_top__mmap_read (top=0x7ffcd9214120) at 
builtin-top.c:883
#13 __cmd_top (top=0x7ffcd9214120) at builtin-top.c:1018
#14 cmd_top (argc=<optimized out>, argv=<optimized out>, 
prefix=<optimized out>) at builtin-top.c:1344
#15 0x0000000000493561 in run_builtin (p=p@entry=0x948000 
<mailto:%28p=p@entry=0x948000> <commands+288>, argc=argc@entry=4 
<mailto:argc=argc@entry=4>, argv=argv@entry=0x7ffcd92199b0) 
<mailto:argv=argv@entry=0x7ffcd92199b0%29> at
perf.c:357
#16 0x0000000000437157 in handle_internal_command (argv=0x7ffcd92199b0, 
argc=4) at perf.c:419
#17 run_argv (argv=0x7ffcd9219720, argcp=0x7ffcd921972c) at perf.c:465
#18 main (argc=4, argv=0x7ffcd92199b0) at perf.c:609

[toc] | [next] | [standalone]

#1457120

FromArnaldo Carvalho de Melo <acme@kernel.org>
Date2016-08-05 16:20 +0200
Message-ID<s2O8G-bi-7@gated-at.bofh.it>
In reply to#1457022
Em Fri, Aug 05, 2016 at 06:58:12PM +0800, Wangnan (F) escreveu:
> perf top comsumes too much memory than it need.
> 
> Using following commands:
> 
>  # yes '' > /dev/null
>  # perf top -e raw_syscalls:sys_enter
> 
> System will quickly become unresponsive because of out of memory.
> 
> Using GDB I find two call stacks of malloc. See below.
> 
> [call stack 1]
> 
> #0 __GI___libc_malloc (bytes=140723951319552) at malloc.c:2841
> #1 0x00000000004b7246 in memdup (src=0x7f515e04b054, len=68) at
> ../lib/string.c:29
> #2 0x00000000004f9299 in hist_entry__init (sample_self=true,
> template=0x7ffcd9213a80, he=0xa37da50) at util/hist.c:401
> #3 hist_entry__new (template=template@entry=0x7ffcd9213a80
> <mailto:%28template=template@entry=0x7ffcd9213a80>,
> sample_self=sample_self@entry=true)
> <mailto:sample_self=sample_self@entry=true%29> at util/hist.c:458
> #4 0x00000000004fa13f in hists__findnew_entry (hists=0x3549d30,
> hists=0x3549d30, al=0x7ffcd9213ca0, sample_self=true, entry=0x
> 7ffcd9213a80) at util/hist.c:544
> #5 __hists__add_entry (hists=0x3549d30, al=0x7ffcd9213ca0,
> sym_parent=<optimized out>, bi=bi@entry=0x0 <mailto:bi=bi@entry=0x0>,
> mi=mi@entry=0x0 <mailto:mi=mi@entry=0x0>, sampl
> e=<optimized out>, sample_self=sample_self@entry=true
> <mailto:sample_self=sample_self@entry=true>, ops=ops@entry=0x0)
> <mailto:ops=ops@entry=0x0%29> at util/hist.c:600
> #6 0x00000000004fa3d6 in hists__add_entry (sample_self=true,
> sample=<optimized out>, mi=0x0, bi=0x0, sym_parent=<optimized out
> >, al=<optimized out>, hists=<optimized out>) at util/hist.c:611
> #7 iter_add_single_normal_entry (iter=0x7ffcd9213ce0, al=<optimized out>) at
> util/hist.c:823
> #8 0x00000000004fc3c5 in hist_entry_iter__add (iter=0x7ffcd9213ce0,
> al=0x7ffcd9213ca0, max_stack_depth=<optimized out>, arg=0x
> 7ffcd9214120) at util/hist.c:1030
> #9 0x00000000004536d7 in perf_event__process_sample (machine=<optimized
> out>, sample=0x7ffcd9213d30, evsel=0x3549bb0, event=<o
> ptimized out>, tool=0x7ffcd9214120) at builtin-top.c:800
> #10 perf_top__mmap_read_idx (top=top@entry=0x7ffcd9214120
> <mailto:%28top=top@entry=0x7ffcd9214120>, idx=idx@entry=0)
> <mailto:idx=idx@entry=0%29> at builtin-top.c:866
> #11 0x00000000004559fd in perf_top__mmap_read (top=0x7ffcd9214120) at
> builtin-top.c:883
> #12 __cmd_top (top=0x7ffcd9214120) at builtin-top.c:1018
> #13 cmd_top (argc=<optimized out>, argv=<optimized out>, prefix=<optimized
> out>) at builtin-top.c:1344
> #14 0x0000000000493561 in run_builtin (p=p@entry=0x948000
> <mailto:%28p=p@entry=0x948000> <commands+288>, argc=argc@entry=4
> <mailto:argc=argc@entry=4>, argv=argv@entry=0x7ffcd92199b0)
> <mailto:argv=argv@entry=0x7ffcd92199b0%29> at
> perf.c:357
> #15 0x0000000000437157 in handle_internal_command (argv=0x7ffcd92199b0,
> argc=4) at perf.c:419
> #16 run_argv (argv=0x7ffcd9219720, argcp=0x7ffcd921972c) at perf.c:465
> #17 main (argc=4, argv=0x7ffcd92199b0) at perf.c:609
> 
> [call stack 2]
 
> #0 __GI___libc_malloc (bytes=140723951319456) at malloc.c:2841
> #1 0x00000000005766de in trace_seq_init (s=0x7ffcd92139b0) at trace-seq.c:60
> #2 0x00000000004f7047 in get_trace_output (he=0x7ffcd9213a80) at
> util/sort.c:584

Yeah, this allocates 4K, way too much, can you try with this patch,
which amounts to a trace_seq_trim()?

diff --git a/tools/perf/util/sort.c b/tools/perf/util/sort.c
index 947d21f38398..37ddcf57500f 100644
--- a/tools/perf/util/sort.c
+++ b/tools/perf/util/sort.c
@@ -588,7 +588,7 @@ static char *get_trace_output(struct hist_entry *he)
 	} else {
 		pevent_event_info(&seq, evsel->tp_format, &rec);
 	}
-	return seq.buffer;
+	return realloc(seq.buffer, seq.len);
 }
 
 static int64_t
> #3 sort__trace_cmp (left=0x40e6d30, right=0x7ffcd9213a80) at util/sort.c:606
> #4 0x00000000004fa007 in hist_entry__cmp (right=0x7ffcd9213a80,
> left=0x40e6d30) at util/hist.c:1072
> #5 hists__findnew_entry (hists=0x3549d30, hists=0x3549d30,
> al=0x7ffcd9213ca0, sample_self=true, entry=0x7ffcd9213a80) at util/
> hist.c:509
> #6 __hists__add_entry (hists=0x3549d30, al=0x7ffcd9213ca0,
> sym_parent=<optimized out>, bi=bi@entry=0x0 <mailto:bi=bi@entry=0x0>,
> mi=mi@entry=0x0 <mailto:mi=mi@entry=0x0>, sampl
> e=<optimized out>, sample_self=sample_self@entry=true
> <mailto:sample_self=sample_self@entry=true>, ops=ops@entry=0x0)
> <mailto:ops=ops@entry=0x0%29> at util/hist.c:600
> #7 0x00000000004fa3d6 in hists__add_entry (sample_self=true,
> sample=<optimized out>, mi=0x0, bi=0x0, sym_parent=<optimized out
> >, al=<optimized out>, hists=<optimized out>) at util/hist.c:611
> #8 iter_add_single_normal_entry (iter=0x7ffcd9213ce0, al=<optimized out>) at
> util/hist.c:823
> #9 0x00000000004fc3c5 in hist_entry_iter__add (iter=0x7ffcd9213ce0,
> al=0x7ffcd9213ca0, max_stack_depth=<optimized out>, arg=0x
> 7ffcd9214120) at util/hist.c:1030
> #10 0x00000000004536d7 in perf_event__process_sample (machine=<optimized
> out>, sample=0x7ffcd9213d30, evsel=0x3549bb0, event=<o
> ptimized out>, tool=0x7ffcd9214120) at builtin-top.c:800
> #11 perf_top__mmap_read_idx (top=top@entry=0x7ffcd9214120
> <mailto:%28top=top@entry=0x7ffcd9214120>, idx=idx@entry=0)
> <mailto:idx=idx@entry=0%29> at builtin-top.c:866
> #12 0x00000000004559fd in perf_top__mmap_read (top=0x7ffcd9214120) at
> builtin-top.c:883
> #13 __cmd_top (top=0x7ffcd9214120) at builtin-top.c:1018
> #14 cmd_top (argc=<optimized out>, argv=<optimized out>, prefix=<optimized
> out>) at builtin-top.c:1344
> #15 0x0000000000493561 in run_builtin (p=p@entry=0x948000
> <mailto:%28p=p@entry=0x948000> <commands+288>, argc=argc@entry=4
> <mailto:argc=argc@entry=4>, argv=argv@entry=0x7ffcd92199b0)
> <mailto:argv=argv@entry=0x7ffcd92199b0%29> at
> perf.c:357
> #16 0x0000000000437157 in handle_internal_command (argv=0x7ffcd92199b0,
> argc=4) at perf.c:419
> #17 run_argv (argv=0x7ffcd9219720, argcp=0x7ffcd921972c) at perf.c:465
> #18 main (argc=4, argv=0x7ffcd92199b0) at perf.c:609

[toc] | [prev] | [next] | [standalone]

#1457738

From"Wangnan (F)" <wangnan0@huawei.com>
Date2016-08-08 13:40 +0200
Message-ID<s3R4u-PQ-17@gated-at.bofh.it>
In reply to#1457120

On 2016/8/5 22:12, Arnaldo Carvalho de Melo wrote:
> Em Fri, Aug 05, 2016 at 06:58:12PM +0800, Wangnan (F) escreveu:
>> perf top comsumes too much memory than it need.
>>
>> Using following commands:
>>
>>   # yes '' > /dev/null
>>   # perf top -e raw_syscalls:sys_enter
>>
>> System will quickly become unresponsive because of out of memory.
>>
>> Using GDB I find two call stacks of malloc. See below.
>>
>> [call stack 1]
>>
>> #0 __GI___libc_malloc (bytes=140723951319552) at malloc.c:2841
>> #1 0x00000000004b7246 in memdup (src=0x7f515e04b054, len=68) at
>> ../lib/string.c:29
>> #2 0x00000000004f9299 in hist_entry__init (sample_self=true,
>> template=0x7ffcd9213a80, he=0xa37da50) at util/hist.c:401
>> #3 hist_entry__new (template=template@entry=0x7ffcd9213a80
>> <mailto:%28template=template@entry=0x7ffcd9213a80>,
>> sample_self=sample_self@entry=true)
>> <mailto:sample_self=sample_self@entry=true%29> at util/hist.c:458
>> #4 0x00000000004fa13f in hists__findnew_entry (hists=0x3549d30,
>> hists=0x3549d30, al=0x7ffcd9213ca0, sample_self=true, entry=0x
>> 7ffcd9213a80) at util/hist.c:544
>> #5 __hists__add_entry (hists=0x3549d30, al=0x7ffcd9213ca0,
>> sym_parent=<optimized out>, bi=bi@entry=0x0 <mailto:bi=bi@entry=0x0>,
>> mi=mi@entry=0x0 <mailto:mi=mi@entry=0x0>, sampl
>> e=<optimized out>, sample_self=sample_self@entry=true
>> <mailto:sample_self=sample_self@entry=true>, ops=ops@entry=0x0)
>> <mailto:ops=ops@entry=0x0%29> at util/hist.c:600
>> #6 0x00000000004fa3d6 in hists__add_entry (sample_self=true,
>> sample=<optimized out>, mi=0x0, bi=0x0, sym_parent=<optimized out
>>> , al=<optimized out>, hists=<optimized out>) at util/hist.c:611
>> #7 iter_add_single_normal_entry (iter=0x7ffcd9213ce0, al=<optimized out>) at
>> util/hist.c:823
>> #8 0x00000000004fc3c5 in hist_entry_iter__add (iter=0x7ffcd9213ce0,
>> al=0x7ffcd9213ca0, max_stack_depth=<optimized out>, arg=0x
>> 7ffcd9214120) at util/hist.c:1030
>> #9 0x00000000004536d7 in perf_event__process_sample (machine=<optimized
>> out>, sample=0x7ffcd9213d30, evsel=0x3549bb0, event=<o
>> ptimized out>, tool=0x7ffcd9214120) at builtin-top.c:800
>> #10 perf_top__mmap_read_idx (top=top@entry=0x7ffcd9214120
>> <mailto:%28top=top@entry=0x7ffcd9214120>, idx=idx@entry=0)
>> <mailto:idx=idx@entry=0%29> at builtin-top.c:866
>> #11 0x00000000004559fd in perf_top__mmap_read (top=0x7ffcd9214120) at
>> builtin-top.c:883
>> #12 __cmd_top (top=0x7ffcd9214120) at builtin-top.c:1018
>> #13 cmd_top (argc=<optimized out>, argv=<optimized out>, prefix=<optimized
>> out>) at builtin-top.c:1344
>> #14 0x0000000000493561 in run_builtin (p=p@entry=0x948000
>> <mailto:%28p=p@entry=0x948000> <commands+288>, argc=argc@entry=4
>> <mailto:argc=argc@entry=4>, argv=argv@entry=0x7ffcd92199b0)
>> <mailto:argv=argv@entry=0x7ffcd92199b0%29> at
>> perf.c:357
>> #15 0x0000000000437157 in handle_internal_command (argv=0x7ffcd92199b0,
>> argc=4) at perf.c:419
>> #16 run_argv (argv=0x7ffcd9219720, argcp=0x7ffcd921972c) at perf.c:465
>> #17 main (argc=4, argv=0x7ffcd92199b0) at perf.c:609
>>
>> [call stack 2]
>   
>> #0 __GI___libc_malloc (bytes=140723951319456) at malloc.c:2841
>> #1 0x00000000005766de in trace_seq_init (s=0x7ffcd92139b0) at trace-seq.c:60
>> #2 0x00000000004f7047 in get_trace_output (he=0x7ffcd9213a80) at
>> util/sort.c:584
> Yeah, this allocates 4K, way too much, can you try with this patch,
> which amounts to a trace_seq_trim()?
>
> diff --git a/tools/perf/util/sort.c b/tools/perf/util/sort.c
> index 947d21f38398..37ddcf57500f 100644
> --- a/tools/perf/util/sort.c
> +++ b/tools/perf/util/sort.c
> @@ -588,7 +588,7 @@ static char *get_trace_output(struct hist_entry *he)
>   	} else {
>   		pevent_event_info(&seq, evsel->tp_format, &rec);
>   	}
> -	return seq.buffer;
> +	return realloc(seq.buffer, seq.len);
>   }
>   
>   static int64_t

It works: from 1.8g/10s to 40m/10s, but unfortunately your
patch is wrong: please use realloc(seq.buffer, seq.len + 1)
because of the ending '\0' :)

In my experiment RSS continuously increases because I use
raw_syscalls, which can result in too much entries to record
in perf top because of the nature of argument list (garbage
arguments are also considered by perf). It becomes much better
if I use --fields to reduce the number of possible entries.
However, theoretically perf will eventually consume all
system memory, unless we carefully choose tracepoints and
fields to make sure the possible entries is limited.

Thank you.

[toc] | [prev] | [next] | [standalone]

#1457785

FromArnaldo Carvalho de Melo <acme@kernel.org>
Date2016-08-08 15:30 +0200
Message-ID<s3SMW-1UI-15@gated-at.bofh.it>
In reply to#1457738
Em Mon, Aug 08, 2016 at 07:34:54PM +0800, Wangnan (F) escreveu:
> 
> 
> On 2016/8/5 22:12, Arnaldo Carvalho de Melo wrote:
> > Em Fri, Aug 05, 2016 at 06:58:12PM +0800, Wangnan (F) escreveu:
> > > perf top comsumes too much memory than it need.
> > > 
> > > Using following commands:
> > > 
> > >   # yes '' > /dev/null
> > >   # perf top -e raw_syscalls:sys_enter
> > > 
> > > System will quickly become unresponsive because of out of memory.
> > > 
> > > Using GDB I find two call stacks of malloc. See below.
> > > 
> > > [call stack 1]
> > > 
> > > #0 __GI___libc_malloc (bytes=140723951319552) at malloc.c:2841
> > > #1 0x00000000004b7246 in memdup (src=0x7f515e04b054, len=68) at
> > > ../lib/string.c:29
> > > #2 0x00000000004f9299 in hist_entry__init (sample_self=true,
> > > template=0x7ffcd9213a80, he=0xa37da50) at util/hist.c:401
> > > #3 hist_entry__new (template=template@entry=0x7ffcd9213a80
> > > <mailto:%28template=template@entry=0x7ffcd9213a80>,
> > > sample_self=sample_self@entry=true)
> > > <mailto:sample_self=sample_self@entry=true%29> at util/hist.c:458
> > > #4 0x00000000004fa13f in hists__findnew_entry (hists=0x3549d30,
> > > hists=0x3549d30, al=0x7ffcd9213ca0, sample_self=true, entry=0x
> > > 7ffcd9213a80) at util/hist.c:544
> > > #5 __hists__add_entry (hists=0x3549d30, al=0x7ffcd9213ca0,
> > > sym_parent=<optimized out>, bi=bi@entry=0x0 <mailto:bi=bi@entry=0x0>,
> > > mi=mi@entry=0x0 <mailto:mi=mi@entry=0x0>, sampl
> > > e=<optimized out>, sample_self=sample_self@entry=true
> > > <mailto:sample_self=sample_self@entry=true>, ops=ops@entry=0x0)
> > > <mailto:ops=ops@entry=0x0%29> at util/hist.c:600
> > > #6 0x00000000004fa3d6 in hists__add_entry (sample_self=true,
> > > sample=<optimized out>, mi=0x0, bi=0x0, sym_parent=<optimized out
> > > > , al=<optimized out>, hists=<optimized out>) at util/hist.c:611
> > > #7 iter_add_single_normal_entry (iter=0x7ffcd9213ce0, al=<optimized out>) at
> > > util/hist.c:823
> > > #8 0x00000000004fc3c5 in hist_entry_iter__add (iter=0x7ffcd9213ce0,
> > > al=0x7ffcd9213ca0, max_stack_depth=<optimized out>, arg=0x
> > > 7ffcd9214120) at util/hist.c:1030
> > > #9 0x00000000004536d7 in perf_event__process_sample (machine=<optimized
> > > out>, sample=0x7ffcd9213d30, evsel=0x3549bb0, event=<o
> > > ptimized out>, tool=0x7ffcd9214120) at builtin-top.c:800
> > > #10 perf_top__mmap_read_idx (top=top@entry=0x7ffcd9214120
> > > <mailto:%28top=top@entry=0x7ffcd9214120>, idx=idx@entry=0)
> > > <mailto:idx=idx@entry=0%29> at builtin-top.c:866
> > > #11 0x00000000004559fd in perf_top__mmap_read (top=0x7ffcd9214120) at
> > > builtin-top.c:883
> > > #12 __cmd_top (top=0x7ffcd9214120) at builtin-top.c:1018
> > > #13 cmd_top (argc=<optimized out>, argv=<optimized out>, prefix=<optimized
> > > out>) at builtin-top.c:1344
> > > #14 0x0000000000493561 in run_builtin (p=p@entry=0x948000
> > > <mailto:%28p=p@entry=0x948000> <commands+288>, argc=argc@entry=4
> > > <mailto:argc=argc@entry=4>, argv=argv@entry=0x7ffcd92199b0)
> > > <mailto:argv=argv@entry=0x7ffcd92199b0%29> at
> > > perf.c:357
> > > #15 0x0000000000437157 in handle_internal_command (argv=0x7ffcd92199b0,
> > > argc=4) at perf.c:419
> > > #16 run_argv (argv=0x7ffcd9219720, argcp=0x7ffcd921972c) at perf.c:465
> > > #17 main (argc=4, argv=0x7ffcd92199b0) at perf.c:609
> > > 
> > > [call stack 2]
> > > #0 __GI___libc_malloc (bytes=140723951319456) at malloc.c:2841
> > > #1 0x00000000005766de in trace_seq_init (s=0x7ffcd92139b0) at trace-seq.c:60
> > > #2 0x00000000004f7047 in get_trace_output (he=0x7ffcd9213a80) at
> > > util/sort.c:584
> > Yeah, this allocates 4K, way too much, can you try with this patch,
> > which amounts to a trace_seq_trim()?
> > 
> > diff --git a/tools/perf/util/sort.c b/tools/perf/util/sort.c
> > index 947d21f38398..37ddcf57500f 100644
> > --- a/tools/perf/util/sort.c
> > +++ b/tools/perf/util/sort.c
> > @@ -588,7 +588,7 @@ static char *get_trace_output(struct hist_entry *he)
> >   	} else {
> >   		pevent_event_info(&seq, evsel->tp_format, &rec);
> >   	}
> > -	return seq.buffer;
> > +	return realloc(seq.buffer, seq.len);
> >   }
> >   static int64_t
> 
> It works: from 1.8g/10s to 40m/10s, but unfortunately your
> patch is wrong: please use realloc(seq.buffer, seq.len + 1)
> because of the ending '\0' :)

Ouch, I thought those were accounted in seq.len :-\ Will check and fix,
thanks for testing it.
 
> In my experiment RSS continuously increases because I use
> raw_syscalls, which can result in too much entries to record
> in perf top because of the nature of argument list (garbage
> arguments are also considered by perf). It becomes much better
> if I use --fields to reduce the number of possible entries.
> However, theoretically perf will eventually consume all
> system memory, unless we carefully choose tracepoints and
> fields to make sure the possible entries is limited.

I need to continue work on reference counting more classes, so that we
can trow away more stuff as we decay histogram entries...

- Arnaldo

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web