Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1173313 > unrolled thread

[RFC] perf tools: Add hugetlbfs memory recognition

Back to article view | Back to linux.kernel

Contents

  [RFC] perf tools: Add hugetlbfs memory recognition Hou Pengyang <houpengyang@huawei.com> - 2015-06-27 11:00 +0200
    Re: [RFC] perf tools: Add hugetlbfs memory recognition Hou Pengyang <houpengyang@huawei.com> - 2015-06-27 11:10 +0200
      Re: [RFC] perf tools: Add hugetlbfs memory recognition Hou Pengyang <houpengyang@huawei.com> - 2015-06-30 11:40 +0200
        Re: [RFC] perf tools: Add hugetlbfs memory recognition Hou Pengyang <houpengyang@huawei.com> - 2015-07-03 12:30 +0200
          Re: [RFC] perf tools: Add hugetlbfs memory recognition Arnaldo Carvalho de Melo <acme@kernel.org> - 2015-07-03 17:20 +0200

#1173313 — [RFC] perf tools: Add hugetlbfs memory recognition

Maps for JIT is helpful for symbols-parsing for anon-executable-memory.
What we need to do is to add (START, SIZE, symbolname) to /tmp/perf-%d.map
(%d = pid of process), and perf would parse symbol located in this area
according to /tmp/perf-%d.map. It works well for normal mmap.

However, when we alloc such memory from hugetlbfs by the following code:

......

    fd = open("/mnt/huge/hugepagefile", O_CREAT | O_RDWR, 0755);
    if (fd < 0) {
        perror("Open failed");
        exit(1);
    }   
    addr = mmap(ADDR, LENGTH, PROT_READ | PROT_WRITE |     \   
        PROT_EXEC, MAP_SHARED, fd, 0); 

......
where hugetlbfs is mounted in /mnt/huge. Symbols could not be parsed correctly:

#perf report
   86.96%     0.00%  hugepage-mmap  hugepagefile      [.] 0xffffffe00000005c      
   86.96%     0.00%  hugepage-mmap  hugepagefile      [.] 0xffffffe0000000ac

This is because such mmap area's file backed is "/mnt/huge/hugepagefile", a node 
in pseudo filesystem, it is useless for symbol-parsing. so wo had better recognize 
such hugetlbfs area, and rename it's filename to /tmp/perf-%d.map, just as anon-memory
and no_dso_memory do. 

This patch imports a new function named is_hugetlb_memory to check if this memory
is from hugetlbfs. If true, change its name.  

After this patch:
#perf report 
   86.96%     0.00%  hugepage-mmap  perf-182.map        [.] 0x000000200000005c 
   86.96%     0.00%  hugepage-mmap  perf-182.map        [.] 0x00000020000000ac

We can add maps info to perf-182.map for further symbols-parsing.

Signed-off-by: Hou Pengyang <houpengyang@huawei.com>
---
 tools/perf/util/map.c | 11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)

diff --git a/tools/perf/util/map.c b/tools/perf/util/map.c
index b5a5e9c..796db08 100644
--- a/tools/perf/util/map.c
+++ b/tools/perf/util/map.c
@@ -15,6 +15,7 @@
 #include "debug.h"
 #include "machine.h"
 #include <linux/string.h>
+#include <sys/mman.h>
 
 static void __maps__insert(struct maps *maps, struct map *map);
 
@@ -43,6 +44,11 @@ static inline int is_android_lib(const char *filename)
 	       !strncmp(filename, "/system/lib", 11);
 }
 
+static inline int is_hugetlb_memory(u32 flags)
+{
+	return flags & MAP_HUGETLB;
+}
+
 static inline bool replace_android_lib(const char *filename, char *newfilename)
 {
 	const char *libname;
@@ -151,12 +157,13 @@ struct map *map__new(struct machine *machine, u64 start, u64 len,
 	if (map != NULL) {
 		char newfilename[PATH_MAX];
 		struct dso *dso;
-		int anon, no_dso, vdso, android;
+		int anon, no_dso, vdso, android, hugetlb;
 
 		android = is_android_lib(filename);
 		anon = is_anon_memory(filename);
 		vdso = is_vdso_map(filename);
 		no_dso = is_no_dso_memory(filename);
+		hugetlb = is_hugetlb_memory(flags);
 
 		map->maj = d_maj;
 		map->min = d_min;
@@ -165,7 +172,7 @@ struct map *map__new(struct machine *machine, u64 start, u64 len,
 		map->prot = prot;
 		map->flags = flags;
 
-		if ((anon || no_dso) && type == MAP__FUNCTION) {
+		if ((anon || no_dso || hugetlb) && type == MAP__FUNCTION) {
 			snprintf(newfilename, sizeof(newfilename), "/tmp/perf-%d.map", pid);
 			filename = newfilename;
 		}
-- 
1.8.3.4

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [next] | [standalone]

#1173319

On 2015/6/27 16:49, Hou Pengyang wrote:
> Maps for JIT is helpful for symbols-parsing for anon-executable-memory.
> What we need to do is to add (START, SIZE, symbolname) to /tmp/perf-%d.map
> (%d = pid of process), and perf would parse symbol located in this area
> according to /tmp/perf-%d.map. It works well for normal mmap.
>
> However, when we alloc such memory from hugetlbfs by the following code:
>
> ......
>
>      fd = open("/mnt/huge/hugepagefile", O_CREAT | O_RDWR, 0755);
>      if (fd < 0) {
>          perror("Open failed");
>          exit(1);
>      }
>      addr = mmap(ADDR, LENGTH, PROT_READ | PROT_WRITE |     \
>          PROT_EXEC, MAP_SHARED, fd, 0);
>
> ......
> where hugetlbfs is mounted in /mnt/huge. Symbols could not be parsed correctly:
>
> #perf report
>     86.96%     0.00%  hugepage-mmap  hugepagefile      [.] 0xffffffe00000005c
>     86.96%     0.00%  hugepage-mmap  hugepagefile      [.] 0xffffffe0000000ac
>
> This is because such mmap area's file backed is "/mnt/huge/hugepagefile", a node
> in pseudo filesystem, it is useless for symbol-parsing. so wo had better recognize
> such hugetlbfs area, and rename it's filename to /tmp/perf-%d.map, just as anon-memory
> and no_dso_memory do.
>
> This patch imports a new function named is_hugetlb_memory to check if this memory
> is from hugetlbfs. If true, change its name.
>
> After this patch:
> #perf report
>     86.96%     0.00%  hugepage-mmap  perf-182.map        [.] 0x000000200000005c
>     86.96%     0.00%  hugepage-mmap  perf-182.map        [.] 0x00000020000000ac
>
> We can add maps info to perf-182.map for further symbols-parsing.
>
> Signed-off-by: Hou Pengyang <houpengyang@huawei.com>
> ---
>   tools/perf/util/map.c | 11 +++++++++--
>   1 file changed, 9 insertions(+), 2 deletions(-)
>
> diff --git a/tools/perf/util/map.c b/tools/perf/util/map.c
> index b5a5e9c..796db08 100644
> --- a/tools/perf/util/map.c
> +++ b/tools/perf/util/map.c
> @@ -15,6 +15,7 @@
>   #include "debug.h"
>   #include "machine.h"
>   #include <linux/string.h>
> +#include <sys/mman.h>
>
>   static void __maps__insert(struct maps *maps, struct map *map);
>
> @@ -43,6 +44,11 @@ static inline int is_android_lib(const char *filename)
>   	       !strncmp(filename, "/system/lib", 11);
>   }
>
> +static inline int is_hugetlb_memory(u32 flags)
> +{
> +	return flags & MAP_HUGETLB;
> +}

There is something about MAP_HUGETLB.

In this patch, we check if a mmap area is hugetlbfs area by MAP_HUGETLB,
a bit in MMAP2 event.

However, if mmap area is hugetlb related, MAP_HUGETLB does not always 
appear. Because, there are two ways generating MMAP2 event.

1) when a new vm_area_struct is created, its info would be exported
    as a MMAP2 event.
2) perf reads /proc/pid/maps for generating MMAP2 event.

MAP_HUGETLB appears if MMAP2 event is generated on situation 1),
while not on situation 2).

This is because on situation 2), perf reads /proc/pid/maps, which
contains only PROT_READ/WRITE/EXEC, MAP_SHARED/MAP_PRIVATE, while more 
details appear in /proc/pid/smaps, such as MAP_HUGETLB.

So I wonder if there is a need to read /proc/pid/smaps instead of 
/proc/pid/maps to generate MMAP2 event. Or we should solve the problem 
by another way?

Thanks!

> +
>   static inline bool replace_android_lib(const char *filename, char *newfilename)
>   {
>   	const char *libname;
> @@ -151,12 +157,13 @@ struct map *map__new(struct machine *machine, u64 start, u64 len,
>   	if (map != NULL) {
>   		char newfilename[PATH_MAX];
>   		struct dso *dso;
> -		int anon, no_dso, vdso, android;
> +		int anon, no_dso, vdso, android, hugetlb;
>
>   		android = is_android_lib(filename);
>   		anon = is_anon_memory(filename);
>   		vdso = is_vdso_map(filename);
>   		no_dso = is_no_dso_memory(filename);
> +		hugetlb = is_hugetlb_memory(flags);
>
>   		map->maj = d_maj;
>   		map->min = d_min;
> @@ -165,7 +172,7 @@ struct map *map__new(struct machine *machine, u64 start, u64 len,
>   		map->prot = prot;
>   		map->flags = flags;
>
> -		if ((anon || no_dso) && type == MAP__FUNCTION) {
> +		if ((anon || no_dso || hugetlb) && type == MAP__FUNCTION) {
>   			snprintf(newfilename, sizeof(newfilename), "/tmp/perf-%d.map", pid);
>   			filename = newfilename;
>   		}
>


--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1174412

On 2015/6/29 21:42, Arnaldo Carvalho de Melo wrote:
> Em Mon, Jun 29, 2015 at 10:23:29AM -0300, Arnaldo Carvalho de Melo escreveu:
>> Em Sat, Jun 27, 2015 at 05:08:20PM +0800, Hou Pengyang escreveu:
>>> There is something about MAP_HUGETLB.
>>
>>> In this patch, we check if a mmap area is hugetlbfs area by MAP_HUGETLB,
>>> a bit in MMAP2 event.
>>
>>> However, if mmap area is hugetlb related, MAP_HUGETLB does not always
>>> appear. Because, there are two ways generating MMAP2 event.
>>
>>> 1) when a new vm_area_struct is created, its info would be exported
>>>     as a MMAP2 event.
>>> 2) perf reads /proc/pid/maps for generating MMAP2 event.
>>
>>> MAP_HUGETLB appears if MMAP2 event is generated on situation 1),
>>> while not on situation 2).
>>
>>> This is because on situation 2), perf reads /proc/pid/maps, which
>>> contains only PROT_READ/WRITE/EXEC, MAP_SHARED/MAP_PRIVATE, while more
>>> details appear in /proc/pid/smaps, such as MAP_HUGETLB.
>>
>> Humm:
>>
>> [root@zoo ~]# wc -l /proc/`pidof firefox`/maps
>> 934 /proc/4551/maps
>> [root@zoo ~]# wc -l /proc/`pidof firefox`/smaps
>> 14944 /proc/4551/smaps
>> [root@zoo ~]#
>>
>>> So I wonder if there is a need to read /proc/pid/smaps instead of
>>> /proc/pid/maps to generate MMAP2 event. Or we should solve the problem by
>>> another way?
>>
>> Doing some research now...
>
> Bummer, seems that only smaps -> ... -> show_smap_vma_flags() will
> expose that to userspace...
>
> Perhaps we can look at some global stat for HugeTLB fs to figure out if
> we really, really need to parse smaps instead of just maps? I.e. in my
> system, a desktop one, F21, I have:
>

It seems no other info tell us if one process is using hugetlbfs.
So how about an option to tell perf explicitly which file is from
hugetlbfs, like:

./perf report --hugetlb-file=/mnt/huge/hugepagefile

So that, we can check if a mmap area is from hugetlbfs by its name
instead of MAP_HUGETLB.

> [root@zoo ~]# find /proc -name smaps | xargs grep -w ht
> grep: /proc/31971/task/31971/smaps: No such file or directory
> grep: /proc/31971/smaps: No such file or directory
> [root@zoo ~]#
>
> No "ht" HugeTLB areas, so no need to parse 16 times more bytes than by
> just using /proc/PID/maps.



>
> - Arnaldo
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>
> .
>


--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1176468

On 2015/6/30 22:50, Arnaldo Carvalho de Melo wrote:
> Em Tue, Jun 30, 2015 at 05:33:05PM +0800, Hou Pengyang escreveu:
>> On 2015/6/29 21:42, Arnaldo Carvalho de Melo wrote:
>>> Em Mon, Jun 29, 2015 at 10:23:29AM -0300, Arnaldo Carvalho de Melo escreveu:
>>>> Em Sat, Jun 27, 2015 at 05:08:20PM +0800, Hou Pengyang escreveu:
>>>>> So I wonder if there is a need to read /proc/pid/smaps instead of
>>>>> /proc/pid/maps to generate MMAP2 event. Or we should solve the problem by
>>>>> another way?
>>>>
>>>> Doing some research now...
>>>
>>> Bummer, seems that only smaps -> ... -> show_smap_vma_flags() will
>>> expose that to userspace...
>>>
>>> Perhaps we can look at some global stat for HugeTLB fs to figure out if
>>> we really, really need to parse smaps instead of just maps? I.e. in my
>>> system, a desktop one, F21, I have:
>>>
>>
>> It seems no other info tell us if one process is using hugetlbfs.
>> So how about an option to tell perf explicitly which file is from
>> hugetlbfs, like:
>>
>> ./perf report --hugetlb-file=/mnt/huge/hugepagefile
>>
>> So that, we can check if a mmap area is from hugetlbfs by its name
>> instead of MAP_HUGETLB.
>
> Well, we _can_ detect this automatically, its just a matter of parsing
> /proc/PID/smaps, right?
>
> What I was discussing was a way only parse smaps when we know we need
> to, i.e. when we, for instance, parsing /proc/PID/maps, find a map that
> we think may be a hugetlb one, maybe some other way to tell us that
> hugetlb maps are in place, looking at the hugetlbfs stats somehow?
>

Another solution, we can parse /proc/self/mounts. Here is the
/proc/self/mounts in my system:

.....
/dev/root / ext2 rw,relatime,errors=remount-ro 0 0
tracefs /sys/kernel/debug/tracing tracefs rw,relatime 0 0
nodev /mnt/huge hugetlbfs rw,relatime 0 0
....

from /proc/self/mounts, we can get mount point of hugetlbfs.
in my system, it is "/mnt/huge". So that, when mmap2 event
comes from userspace, we can compare prefix of filename with
hugetlbfs mount point (e.g "/mnt/huge") to check if it is a
file in hugetlbfs. if it is, set MAP_HUGELTB, otherwise, not set.

There is no need to parse /proc/PID/smaps now, what's more,
it is not difficult to parse "/proc/self/mounts".

> - Arnaldo
>
> .
>


--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [next] | [standalone]

#1176650

Em Fri, Jul 03, 2015 at 06:21:48PM +0800, Hou Pengyang escreveu:
> On 2015/6/30 22:50, Arnaldo Carvalho de Melo wrote:
> >Well, we _can_ detect this automatically, its just a matter of parsing
> >/proc/PID/smaps, right?

<SNIP>

> >What I was discussing was a way only parse smaps when we know we need
> >to, i.e. when we, for instance, parsing /proc/PID/maps, find a map that
> >we think may be a hugetlb one, maybe some other way to tell us that
> >hugetlb maps are in place, looking at the hugetlbfs stats somehow?

<SNIP>

> from /proc/self/mounts, we can get mount point of hugetlbfs.
> in my system, it is "/mnt/huge". So that, when mmap2 event
> comes from userspace, we can compare prefix of filename with
> hugetlbfs mount point (e.g "/mnt/huge") to check if it is a
> file in hugetlbfs. if it is, set MAP_HUGELTB, otherwise, not set.
 
> There is no need to parse /proc/PID/smaps now, what's more,
> it is not difficult to parse "/proc/self/mounts".

A-ha! please take a look at using the find_mountpoint() function
in tools/lib/api/fs/, I guess that should be enough for you?

- Arnaldo
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web