Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1304606 > unrolled thread

[PATCH 0/6] perf tools: Improve 'make build-test'

Started byWang Nan <wangnan0@huawei.com>
First post2016-01-08 15:30 +0100
Last post2016-01-11 09:50 +0100
Articles 12 — 5 participants

Back to article view | Back to linux.kernel

Contents

  [PATCH 0/6] perf tools: Improve 'make build-test' Wang Nan <wangnan0@huawei.com> - 2016-01-08 15:30 +0100
    [PATCH 5/6] perf tools: Pass O option to Makefile.perf in build-test Wang Nan <wangnan0@huawei.com> - 2016-01-08 15:30 +0100
    [PATCH 2/6] perf tools: Add missing headers in perf's MANIFEST Wang Nan <wangnan0@huawei.com> - 2016-01-08 15:30 +0100
      [tip:perf/core] perf tools: Add missing headers in perf'  s MANIFEST tip-bot for Wang Nan <tipbot@zytor.com> - 2016-01-09 17:40 +0100
      Re: [PATCH] perf tools: Add missing sources in perf's MANIFEST "Wangnan (F)" <wangnan0@huawei.com> - 2016-01-11 10:50 +0100
        Re: [PATCH] perf tools: Add missing sources in perf's MANIFEST Arnaldo Carvalho de Melo <acme@kernel.org> - 2016-01-11 15:20 +0100
      [PATCH] perf tools: Add missing sources in perf's MANIFEST Jiri Olsa <jolsa@redhat.com> - 2016-01-11 10:50 +0100
    [PATCH 4/6] perf tools: Set parallel making options build-test Wang Nan <wangnan0@huawei.com> - 2016-01-08 15:30 +0100
      Re: [PATCH 4/6] perf tools: Set parallel making options build-test Jiri Olsa <jolsa@redhat.com> - 2016-01-11 10:40 +0100
    Re: [PATCH 0/6] perf tools: Improve 'make build-test' Arnaldo Carvalho de Melo <acme@kernel.org> - 2016-01-08 15:40 +0100
      Re: [PATCH 0/6] perf tools: Improve 'make build-test' "Wangnan (F)" <wangnan0@huawei.com> - 2016-01-11 04:00 +0100
        Re: [PATCH 0/6] perf tools: Improve 'make build-test' Jiri Olsa <jolsa@redhat.com> - 2016-01-11 09:50 +0100

#1304606 — [PATCH 0/6] perf tools: Improve 'make build-test'

FromWang Nan <wangnan0@huawei.com>
Date2016-01-08 15:30 +0100
Subject[PATCH 0/6] perf tools: Improve 'make build-test'
Message-ID<qOGdc-7Wi-13@gated-at.bofh.it>
There are some bugs and inconveniences in 'make build-test', causes
people with strong motivation to avoid writing to source directory
and people don't have plenty of time skip build-test before sending
patches.

Patch 1 - 3 fixes some existing bug. Patch 4 speedup the test.
Patch 5 and 6 makes build-test obey 'O' option passes from cmdline
of the first 'make'.

With these 6 patches I make build-test success in my environment
fully success the first time.

Wang Nan (6):
  perf tools: Add -lutil in python lib list for broken python-config
  perf tools: Add missing headers in perf's MANIFEST
  perf tools: Fix phony build target for build-test
  perf tools: Set parallel making options build-test
  perf tools: Pass O option to Makefile.perf in build-test
  perf tools: Test correct path of perf in build-test

 tools/perf/MANIFEST        |  2 ++
 tools/perf/config/Makefile |  2 +-
 tools/perf/tests/make      | 56 +++++++++++++++++++++++++++++++++-------------
 3 files changed, 43 insertions(+), 17 deletions(-)

-- 
1.8.3.4

[toc] | [next] | [standalone]

#1304609 — [PATCH 5/6] perf tools: Pass O option to Makefile.perf in build-test

FromWang Nan <wangnan0@huawei.com>
Date2016-01-08 15:30 +0100
Subject[PATCH 5/6] perf tools: Pass O option to Makefile.perf in build-test
Message-ID<qOGdc-7Wi-23@gated-at.bofh.it>
In reply to#1304606
Doesn't like tools/perf/Makefile, tools/perf/Makefile.perf obey 'O'
option when it is passed through cmdline only, because of code in
tools/scripts/Makefile.include:

 ifneq ($(O),)
 ifeq ($(origin O), command line)
 	...
 	ABSOLUTE_O := $(shell cd $(O) ; pwd)
 	OUTPUT := $(ABSOLUTE_O)/$(if $(subdir),$(subdir)/)
 endif
 endif

This patch passes 'O' to Makefile.perf through cmdline explicitly
to make it follow O variable during build-test.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
---
 tools/perf/tests/make | 12 ++++++++++--
 1 file changed, 10 insertions(+), 2 deletions(-)

diff --git a/tools/perf/tests/make b/tools/perf/tests/make
index bd9c61a..a32615a3 100644
--- a/tools/perf/tests/make
+++ b/tools/perf/tests/make
@@ -3,7 +3,7 @@ ifeq ($(MAKECMDGOALS),)
 # no target specified, trigger the whole suite
 all:
 	@echo "Testing Makefile";      $(MAKE) -sf tests/make MK=Makefile
-	@echo "Testing Makefile.perf"; $(MAKE) -sf tests/make MK=Makefile.perf SET_PARALLEL=1
+	@echo "Testing Makefile.perf"; $(MAKE) -sf tests/make MK=Makefile.perf SET_PARALLEL=1 SET_O=1
 else
 # run only specific test over 'Makefile'
 %:
@@ -11,6 +11,14 @@ else
 endif
 else
 PERF := .
+O_OPT :=
+
+ifneq ($(O),)
+  FULL_O := $(shell readlink -f $(O) || echo $(O))
+  ifeq ($(SET_O),1)
+    O_OPT := 'O=$(FULL_O)'
+  endif
+endif
 
 PARALLEL_OPT=
 ifeq ($(SET_PARALLEL),1)
@@ -247,7 +255,7 @@ clean := @(cd $(PERF); make -s -f $(MK) clean >/dev/null)
 $(run):
 	$(call clean)
 	@TMP_DEST=$$(mktemp -d); \
-	cmd="cd $(PERF) && make -f $(MK) $(PARALLEL_OPT) DESTDIR=$$TMP_DEST $($@)"; \
+	cmd="cd $(PERF) && make -f $(MK) $(PARALLEL_OPT) $(O_OPT) DESTDIR=$$TMP_DEST $($@)"; \
 	echo "- $@: $$cmd" && echo $$cmd > $@ && \
 	( eval $$cmd ) >> $@ 2>&1; \
 	echo "  test: $(call test,$@)" >> $@ 2>&1; \
-- 
1.8.3.4

[toc] | [prev] | [next] | [standalone]

#1304610 — [PATCH 2/6] perf tools: Add missing headers in perf's MANIFEST

FromWang Nan <wangnan0@huawei.com>
Date2016-01-08 15:30 +0100
Subject[PATCH 2/6] perf tools: Add missing headers in perf's MANIFEST
Message-ID<qOGdd-7Wi-25@gated-at.bofh.it>
In reply to#1304606
These losted headers are found in arm64 cross building:
failed to build perf using tarball generated using

 $ make perf-targz-src-pkg

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
---
 tools/perf/MANIFEST | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/tools/perf/MANIFEST b/tools/perf/MANIFEST
index ce3932e..c0053af 100644
--- a/tools/perf/MANIFEST
+++ b/tools/perf/MANIFEST
@@ -1,6 +1,7 @@
 tools/perf
 tools/arch/alpha/include/asm/barrier.h
 tools/arch/arm/include/asm/barrier.h
+tools/arch/arm64/include/asm/barrier.h
 tools/arch/ia64/include/asm/barrier.h
 tools/arch/mips/include/asm/barrier.h
 tools/arch/powerpc/include/asm/barrier.h
@@ -30,6 +31,7 @@ tools/lib/util/find_next_bit.c
 tools/include/asm/atomic.h
 tools/include/asm/barrier.h
 tools/include/asm/bug.h
+tools/include/asm-generic/atomic-gcc.h
 tools/include/asm-generic/barrier.h
 tools/include/asm-generic/bitops/arch_hweight.h
 tools/include/asm-generic/bitops/atomic.h
-- 
1.8.3.4

[toc] | [prev] | [next] | [standalone]

#1305312 — [tip:perf/core] perf tools: Add missing headers in perf' s MANIFEST

Fromtip-bot for Wang Nan <tipbot@zytor.com>
Date2016-01-09 17:40 +0100
Subject[tip:perf/core] perf tools: Add missing headers in perf' s MANIFEST
Message-ID<qP4IA-87R-69@gated-at.bofh.it>
In reply to#1304610
Commit-ID:  2d7c03e6b0c604decae33b0ce03e69b79b2a39a1
Gitweb:     http://git.kernel.org/tip/2d7c03e6b0c604decae33b0ce03e69b79b2a39a1
Author:     Wang Nan <wangnan0@huawei.com>
AuthorDate: Fri, 8 Jan 2016 14:23:57 +0000
Committer:  Arnaldo Carvalho de Melo <acme@redhat.com>
CommitDate: Fri, 8 Jan 2016 12:48:40 -0300

perf tools: Add missing headers in perf's MANIFEST

These lost headers are found in arm64 cross buildings, failing to build
perf using tarballs generated using:

 $ make perf-targz-src-pkg

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Zefan Li <lizefan@huawei.com>
Cc: pi3orama@163.com
Link: http://lkml.kernel.org/r/1452263041-225488-3-git-send-email-wangnan0@huawei.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
---
 tools/perf/MANIFEST | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/tools/perf/MANIFEST b/tools/perf/MANIFEST
index b3db8df..ddf922f 100644
--- a/tools/perf/MANIFEST
+++ b/tools/perf/MANIFEST
@@ -1,6 +1,7 @@
 tools/perf
 tools/arch/alpha/include/asm/barrier.h
 tools/arch/arm/include/asm/barrier.h
+tools/arch/arm64/include/asm/barrier.h
 tools/arch/ia64/include/asm/barrier.h
 tools/arch/mips/include/asm/barrier.h
 tools/arch/powerpc/include/asm/barrier.h
@@ -30,6 +31,7 @@ tools/lib/find_bit.c
 tools/include/asm/atomic.h
 tools/include/asm/barrier.h
 tools/include/asm/bug.h
+tools/include/asm-generic/atomic-gcc.h
 tools/include/asm-generic/barrier.h
 tools/include/asm-generic/bitops/arch_hweight.h
 tools/include/asm-generic/bitops/atomic.h

[toc] | [prev] | [next] | [standalone]

#1305980 — Re: [PATCH] perf tools: Add missing sources in perf's MANIFEST

From"Wangnan (F)" <wangnan0@huawei.com>
Date2016-01-11 10:50 +0100
SubjectRe: [PATCH] perf tools: Add missing sources in perf's MANIFEST
Message-ID<qPHgS-v6-37@gated-at.bofh.it>
In reply to#1304610

On 2016/1/11 17:43, Jiri Olsa wrote:
> On Fri, Jan 08, 2016 at 02:23:57PM +0000, Wang Nan wrote:
>> These losted headers are found in arm64 cross building:
>> failed to build perf using tarball generated using
>>
>>   $ make perf-targz-src-pkg
>>
>> Signed-off-by: Wang Nan <wangnan0@huawei.com>
>> Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
>> Cc: Jiri Olsa <jolsa@kernel.org>
>> Cc: Namhyung Kim <namhyung@kernel.org>
>> ---
>>   tools/perf/MANIFEST | 2 ++
>>   1 file changed, 2 insertions(+)
>>
>> diff --git a/tools/perf/MANIFEST b/tools/perf/MANIFEST
>> index ce3932e..c0053af 100644
>> --- a/tools/perf/MANIFEST
>> +++ b/tools/perf/MANIFEST
>> @@ -1,6 +1,7 @@
>>   tools/perf
>>   tools/arch/alpha/include/asm/barrier.h
>>   tools/arch/arm/include/asm/barrier.h
>> +tools/arch/arm64/include/asm/barrier.h
>>   tools/arch/ia64/include/asm/barrier.h
>>   tools/arch/mips/include/asm/barrier.h
>>   tools/arch/powerpc/include/asm/barrier.h
>> @@ -30,6 +31,7 @@ tools/lib/util/find_next_bit.c
>>   tools/include/asm/atomic.h
>>   tools/include/asm/barrier.h
>>   tools/include/asm/bug.h
>> +tools/include/asm-generic/atomic-gcc.h
>>   tools/include/asm-generic/barrier.h
>>   tools/include/asm-generic/bitops/arch_hweight.h
>>   tools/include/asm-generic/bitops/atomic.h
> tarpkg test just failed for me due to missing bitmap sources
> I needed to add.. seems related to:
>    915b0882c310 tools lib: Move bitmap.[ch] from tools/perf/ to tools/{lib,include}/
>

I also saw this:

http://lkml.kernel.org/g/56931E12.8030506@huawei.com

tarpkg works for me when I posted this patchset at Friday, but
fail after I return back today...

Thank you.

[toc] | [prev] | [next] | [standalone]

#1306297 — Re: [PATCH] perf tools: Add missing sources in perf's MANIFEST

FromArnaldo Carvalho de Melo <acme@kernel.org>
Date2016-01-11 15:20 +0100
SubjectRe: [PATCH] perf tools: Add missing sources in perf's MANIFEST
Message-ID<qPLuc-3uR-45@gated-at.bofh.it>
In reply to#1305980
Em Mon, Jan 11, 2016 at 05:46:10PM +0800, Wangnan (F) escreveu:
> On 2016/1/11 17:43, Jiri Olsa wrote:
> >On Fri, Jan 08, 2016 at 02:23:57PM +0000, Wang Nan wrote:
> >>These losted headers are found in arm64 cross building:
> >>failed to build perf using tarball generated using
> >>
> >>  $ make perf-targz-src-pkg
> >>
> >>Signed-off-by: Wang Nan <wangnan0@huawei.com>
> >>Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
> >>Cc: Jiri Olsa <jolsa@kernel.org>
> >>Cc: Namhyung Kim <namhyung@kernel.org>
> >>---
> >>  tools/perf/MANIFEST | 2 ++
> >>  1 file changed, 2 insertions(+)
> >>
> >>diff --git a/tools/perf/MANIFEST b/tools/perf/MANIFEST
> >>index ce3932e..c0053af 100644
> >>--- a/tools/perf/MANIFEST
> >>+++ b/tools/perf/MANIFEST
> >>@@ -1,6 +1,7 @@
> >>  tools/perf
> >>  tools/arch/alpha/include/asm/barrier.h
> >>  tools/arch/arm/include/asm/barrier.h
> >>+tools/arch/arm64/include/asm/barrier.h
> >>  tools/arch/ia64/include/asm/barrier.h
> >>  tools/arch/mips/include/asm/barrier.h
> >>  tools/arch/powerpc/include/asm/barrier.h
> >>@@ -30,6 +31,7 @@ tools/lib/util/find_next_bit.c
> >>  tools/include/asm/atomic.h
> >>  tools/include/asm/barrier.h
> >>  tools/include/asm/bug.h
> >>+tools/include/asm-generic/atomic-gcc.h
> >>  tools/include/asm-generic/barrier.h
> >>  tools/include/asm-generic/bitops/arch_hweight.h
> >>  tools/include/asm-generic/bitops/atomic.h
> >tarpkg test just failed for me due to missing bitmap sources
> >I needed to add.. seems related to:
> >   915b0882c310 tools lib: Move bitmap.[ch] from tools/perf/ to tools/{lib,include}/
 
> I also saw this:
 
> http://lkml.kernel.org/g/56931E12.8030506@huawei.com
 
> tarpkg works for me when I posted this patchset at Friday, but
> fail after I return back today...

Right, my turn, we all make mistakes ;-\

Anyway, I'm looking at git commit hooks to try and get this checked
automatically for future renames or when new files gets added that are
not listed in the MANIFEST file...

Thank you guys for spotting this! :-)

- Arnaldo

[toc] | [prev] | [next] | [standalone]

#1305982 — [PATCH] perf tools: Add missing sources in perf's MANIFEST

FromJiri Olsa <jolsa@redhat.com>
Date2016-01-11 10:50 +0100
Subject[PATCH] perf tools: Add missing sources in perf's MANIFEST
Message-ID<qPHgS-v6-35@gated-at.bofh.it>
In reply to#1304610
On Fri, Jan 08, 2016 at 02:23:57PM +0000, Wang Nan wrote:
> These losted headers are found in arm64 cross building:
> failed to build perf using tarball generated using
> 
>  $ make perf-targz-src-pkg
> 
> Signed-off-by: Wang Nan <wangnan0@huawei.com>
> Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
> Cc: Jiri Olsa <jolsa@kernel.org>
> Cc: Namhyung Kim <namhyung@kernel.org>
> ---
>  tools/perf/MANIFEST | 2 ++
>  1 file changed, 2 insertions(+)
> 
> diff --git a/tools/perf/MANIFEST b/tools/perf/MANIFEST
> index ce3932e..c0053af 100644
> --- a/tools/perf/MANIFEST
> +++ b/tools/perf/MANIFEST
> @@ -1,6 +1,7 @@
>  tools/perf
>  tools/arch/alpha/include/asm/barrier.h
>  tools/arch/arm/include/asm/barrier.h
> +tools/arch/arm64/include/asm/barrier.h
>  tools/arch/ia64/include/asm/barrier.h
>  tools/arch/mips/include/asm/barrier.h
>  tools/arch/powerpc/include/asm/barrier.h
> @@ -30,6 +31,7 @@ tools/lib/util/find_next_bit.c
>  tools/include/asm/atomic.h
>  tools/include/asm/barrier.h
>  tools/include/asm/bug.h
> +tools/include/asm-generic/atomic-gcc.h
>  tools/include/asm-generic/barrier.h
>  tools/include/asm-generic/bitops/arch_hweight.h
>  tools/include/asm-generic/bitops/atomic.h

tarpkg test just failed for me due to missing bitmap sources
I needed to add.. seems related to:
  915b0882c310 tools lib: Move bitmap.[ch] from tools/perf/ to tools/{lib,include}/


thanks,
jirka


---
Adding missing bitmap.[ch] sources to the MINIFEST file.

Signed-off-by: Jiri Olsa <jolsa@kernel.org>
---
diff --git a/tools/perf/MANIFEST b/tools/perf/MANIFEST
index ddf922f93aa1..2e1fa2357528 100644
--- a/tools/perf/MANIFEST
+++ b/tools/perf/MANIFEST
@@ -28,6 +28,7 @@ tools/lib/string.c
 tools/lib/symbol/kallsyms.c
 tools/lib/symbol/kallsyms.h
 tools/lib/find_bit.c
+tools/lib/bitmap.c
 tools/include/asm/atomic.h
 tools/include/asm/barrier.h
 tools/include/asm/bug.h
@@ -57,6 +58,7 @@ tools/include/linux/rbtree_augmented.h
 tools/include/linux/string.h
 tools/include/linux/types.h
 tools/include/linux/err.h
+tools/include/linux/bitmap.h
 include/asm-generic/bitops/arch_hweight.h
 include/asm-generic/bitops/const_hweight.h
 include/asm-generic/bitops/fls64.h

[toc] | [prev] | [next] | [standalone]

#1304612 — [PATCH 4/6] perf tools: Set parallel making options build-test

FromWang Nan <wangnan0@huawei.com>
Date2016-01-08 15:30 +0100
Subject[PATCH 4/6] perf tools: Set parallel making options build-test
Message-ID<qOGdd-7Wi-35@gated-at.bofh.it>
In reply to#1304606
'make build-test' is painful because of time consuming. In a full test,
all test cases are built twice with tools/perf/Makefile and
tools/perf/Makefile.perf. 'Makefile' automatically computes parallel
options for make, but 'Makefile.perf' not, so all test cases is built
with one job. It is very slow.

This patch adds '-j' options to Makefile.perf testing. It computes
parallel building options like what tools/perf/Makefile does, and pass
'-j' option to Makefile.perf test.

Signed-off-by: Wang Nan <wangnan0@huawei.com>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
---
 tools/perf/tests/make | 23 ++++++++++++++++-------
 1 file changed, 16 insertions(+), 7 deletions(-)

diff --git a/tools/perf/tests/make b/tools/perf/tests/make
index 130be7c..bd9c61a 100644
--- a/tools/perf/tests/make
+++ b/tools/perf/tests/make
@@ -3,7 +3,7 @@ ifeq ($(MAKECMDGOALS),)
 # no target specified, trigger the whole suite
 all:
 	@echo "Testing Makefile";      $(MAKE) -sf tests/make MK=Makefile
-	@echo "Testing Makefile.perf"; $(MAKE) -sf tests/make MK=Makefile.perf
+	@echo "Testing Makefile.perf"; $(MAKE) -sf tests/make MK=Makefile.perf SET_PARALLEL=1
 else
 # run only specific test over 'Makefile'
 %:
@@ -12,6 +12,15 @@ endif
 else
 PERF := .
 
+PARALLEL_OPT=
+ifeq ($(SET_PARALLEL),1)
+  cores := $(shell (getconf _NPROCESSORS_ONLN || egrep -c '^processor|^CPU[0-9]' /proc/cpuinfo) 2>/dev/null)
+  ifeq ($(cores),0)
+    cores := 1
+  endif
+  PARALLEL_OPT="-j$(cores)"
+endif
+
 include config/Makefile.arch
 
 # FIXME looks like x86 is the only arch running tests ;-)
@@ -238,7 +247,7 @@ clean := @(cd $(PERF); make -s -f $(MK) clean >/dev/null)
 $(run):
 	$(call clean)
 	@TMP_DEST=$$(mktemp -d); \
-	cmd="cd $(PERF) && make -f $(MK) DESTDIR=$$TMP_DEST $($@)"; \
+	cmd="cd $(PERF) && make -f $(MK) $(PARALLEL_OPT) DESTDIR=$$TMP_DEST $($@)"; \
 	echo "- $@: $$cmd" && echo $$cmd > $@ && \
 	( eval $$cmd ) >> $@ 2>&1; \
 	echo "  test: $(call test,$@)" >> $@ 2>&1; \
@@ -249,7 +258,7 @@ $(run_O):
 	$(call clean)
 	@TMP_O=$$(mktemp -d); \
 	TMP_DEST=$$(mktemp -d); \
-	cmd="cd $(PERF) && make -f $(MK) O=$$TMP_O DESTDIR=$$TMP_DEST $($(patsubst %_O,%,$@))"; \
+	cmd="cd $(PERF) && make -f $(MK) $(PARALLEL_OPT) O=$$TMP_O DESTDIR=$$TMP_DEST $($(patsubst %_O,%,$@))"; \
 	echo "- $@: $$cmd" && echo $$cmd > $@ && \
 	( eval $$cmd ) >> $@ 2>&1 && \
 	echo "  test: $(call test_O,$@)" >> $@ 2>&1; \
@@ -263,15 +272,15 @@ tarpkg:
 	rm -f $@
 
 make_kernelsrc:
-	@echo "- make -C <kernelsrc> tools/perf"
+	@echo "- make -C <kernelsrc> $(PARALLEL_OPT) tools/perf"
 	$(call clean); \
-	(make -C ../.. tools/perf) > $@ 2>&1 && \
+	(make -C ../.. $(PARALLEL_OPT) tools/perf) > $@ 2>&1 && \
 	test -x perf && rm -f $@ || (cat $@ ; false)
 
 make_kernelsrc_tools:
-	@echo "- make -C <kernelsrc>/tools perf"
+	@echo "- make -C <kernelsrc>/tools $(PARALLEL_OPT) perf"
 	$(call clean); \
-	(make -C ../../tools perf) > $@ 2>&1 && \
+	(make -C ../../tools $(PARALLEL_OPT) perf) > $@ 2>&1 && \
 	test -x perf && rm -f $@ || (cat $@ ; false)
 
 all: $(run) $(run_O) tarpkg make_kernelsrc make_kernelsrc_tools
-- 
1.8.3.4

[toc] | [prev] | [next] | [standalone]

#1305976 — Re: [PATCH 4/6] perf tools: Set parallel making options build-test

FromJiri Olsa <jolsa@redhat.com>
Date2016-01-11 10:40 +0100
SubjectRe: [PATCH 4/6] perf tools: Set parallel making options build-test
Message-ID<qPH7b-r2-7@gated-at.bofh.it>
In reply to#1304612
On Fri, Jan 08, 2016 at 02:23:59PM +0000, Wang Nan wrote:
> 'make build-test' is painful because of time consuming. In a full test,
> all test cases are built twice with tools/perf/Makefile and
> tools/perf/Makefile.perf. 'Makefile' automatically computes parallel
> options for make, but 'Makefile.perf' not, so all test cases is built
> with one job. It is very slow.
> 
> This patch adds '-j' options to Makefile.perf testing. It computes
> parallel building options like what tools/perf/Makefile does, and pass
> '-j' option to Makefile.perf test.
> 
> Signed-off-by: Wang Nan <wangnan0@huawei.com>
> Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
> Cc: Jiri Olsa <jolsa@kernel.org>
> Cc: Namhyung Kim <namhyung@kernel.org>
> ---
>  tools/perf/tests/make | 23 ++++++++++++++++-------
>  1 file changed, 16 insertions(+), 7 deletions(-)
> 
> diff --git a/tools/perf/tests/make b/tools/perf/tests/make
> index 130be7c..bd9c61a 100644
> --- a/tools/perf/tests/make
> +++ b/tools/perf/tests/make
> @@ -3,7 +3,7 @@ ifeq ($(MAKECMDGOALS),)
>  # no target specified, trigger the whole suite
>  all:
>  	@echo "Testing Makefile";      $(MAKE) -sf tests/make MK=Makefile
> -	@echo "Testing Makefile.perf"; $(MAKE) -sf tests/make MK=Makefile.perf
> +	@echo "Testing Makefile.perf"; $(MAKE) -sf tests/make MK=Makefile.perf SET_PARALLEL=1
>  else
>  # run only specific test over 'Makefile'
>  %:
> @@ -12,6 +12,15 @@ endif
>  else
>  PERF := .
>  
> +PARALLEL_OPT=
> +ifeq ($(SET_PARALLEL),1)
> +  cores := $(shell (getconf _NPROCESSORS_ONLN || egrep -c '^processor|^CPU[0-9]' /proc/cpuinfo) 2>/dev/null)
> +  ifeq ($(cores),0)
> +    cores := 1
> +  endif
> +  PARALLEL_OPT="-j$(cores)"
> +endif

we could share this with Makefile, but there's probably no point to librarize single line ATM

Acked-by: Jiri Olsa <jolsa@kernel.org>

thanks,
jirka

[toc] | [prev] | [next] | [standalone]

#1304617

FromArnaldo Carvalho de Melo <acme@kernel.org>
Date2016-01-08 15:40 +0100
Message-ID<qOGmR-812-7@gated-at.bofh.it>
In reply to#1304606
Em Fri, Jan 08, 2016 at 02:23:55PM +0000, Wang Nan escreveu:
> There are some bugs and inconveniences in 'make build-test', causes
> people with strong motivation to avoid writing to source directory
> and people don't have plenty of time skip build-test before sending
> patches.
> 
> Patch 1 - 3 fixes some existing bug. Patch 4 speedup the test.
> Patch 5 and 6 makes build-test obey 'O' option passes from cmdline
> of the first 'make'.
> 
> With these 6 patches I make build-test success in my environment
> fully success the first time.

Hey, I owe you some beer or other stuff you like first time we meet, now
waiting for Jiri to chime in...

Thanks!

- Arnaldo
 
> Wang Nan (6):
>   perf tools: Add -lutil in python lib list for broken python-config
>   perf tools: Add missing headers in perf's MANIFEST
>   perf tools: Fix phony build target for build-test
>   perf tools: Set parallel making options build-test
>   perf tools: Pass O option to Makefile.perf in build-test
>   perf tools: Test correct path of perf in build-test
> 
>  tools/perf/MANIFEST        |  2 ++
>  tools/perf/config/Makefile |  2 +-
>  tools/perf/tests/make      | 56 +++++++++++++++++++++++++++++++++-------------
>  3 files changed, 43 insertions(+), 17 deletions(-)
> 
> -- 
> 1.8.3.4

[toc] | [prev] | [next] | [standalone]

#1305791

From"Wangnan (F)" <wangnan0@huawei.com>
Date2016-01-11 04:00 +0100
Message-ID<qPAS6-4ya-19@gated-at.bofh.it>
In reply to#1304617
Hi Jiri,

I think if we can fold the feature checking in build-test then
it can be even faster. We need only 2 feature check result (for
normal building and static building), but in a full build-test
we run 138 rounds feature checking. Would you please give some
help on it?

Now I have a 160 cores server for building perf, currently busy
time and idle time are 5:3 (result from top. I use 1s interval.
Each test case takes about 8s,  in about 5 seconds I can see
full of 'cc1' in perf top result). A full test-build still require
more than 0.3 hours (8s * 34 cases * 2 (with and without O) *
2 (Makefile and Makefile.perf) = 1088 seconds).

Thank you.



On 2016/1/8 22:32, Arnaldo Carvalho de Melo wrote:
> Em Fri, Jan 08, 2016 at 02:23:55PM +0000, Wang Nan escreveu:
>> There are some bugs and inconveniences in 'make build-test', causes
>> people with strong motivation to avoid writing to source directory
>> and people don't have plenty of time skip build-test before sending
>> patches.
>>
>> Patch 1 - 3 fixes some existing bug. Patch 4 speedup the test.
>> Patch 5 and 6 makes build-test obey 'O' option passes from cmdline
>> of the first 'make'.
>>
>> With these 6 patches I make build-test success in my environment
>> fully success the first time.
> Hey, I owe you some beer or other stuff you like first time we meet, now
> waiting for Jiri to chime in...
>
> Thanks!
>
> - Arnaldo
>   
>> Wang Nan (6):
>>    perf tools: Add -lutil in python lib list for broken python-config
>>    perf tools: Add missing headers in perf's MANIFEST
>>    perf tools: Fix phony build target for build-test
>>    perf tools: Set parallel making options build-test
>>    perf tools: Pass O option to Makefile.perf in build-test
>>    perf tools: Test correct path of perf in build-test
>>
>>   tools/perf/MANIFEST        |  2 ++
>>   tools/perf/config/Makefile |  2 +-
>>   tools/perf/tests/make      | 56 +++++++++++++++++++++++++++++++++-------------
>>   3 files changed, 43 insertions(+), 17 deletions(-)
>>
>> -- 
>> 1.8.3.4

[toc] | [prev] | [next] | [standalone]

#1305956

FromJiri Olsa <jolsa@redhat.com>
Date2016-01-11 09:50 +0100
Message-ID<qPGkN-8j2-15@gated-at.bofh.it>
In reply to#1305791
On Mon, Jan 11, 2016 at 10:55:50AM +0800, Wangnan (F) wrote:
> Hi Jiri,
> 
> I think if we can fold the feature checking in build-test then
> it can be even faster. We need only 2 feature check result (for
> normal building and static building), but in a full build-test
> we run 138 rounds feature checking. Would you please give some
> help on it?

I think we can add separate target that will skip the feature
detection, given that there's extra call before generating
FEATURE-DUMP file.. I'll check on this

> 
> Now I have a 160 cores server for building perf, currently busy
> time and idle time are 5:3 (result from top. I use 1s interval.
> Each test case takes about 8s,  in about 5 seconds I can see
> full of 'cc1' in perf top result). A full test-build still require
> more than 0.3 hours (8s * 34 cases * 2 (with and without O) *
> 2 (Makefile and Makefile.perf) = 1088 seconds).

nice

thanks,
jirka

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web