Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.kernel > #1345087 > unrolled thread

[PATCH] perf tools: Fix python extension build

Started byJiri Olsa <jolsa@redhat.com>
First post2016-02-27 21:30 +0100
Last post2016-02-29 21:00 +0100
Articles 4 — 3 participants

Back to article view | Back to linux.kernel

Contents

  [PATCH] perf tools: Fix python extension build Jiri Olsa <jolsa@redhat.com> - 2016-02-27 21:30 +0100
    Re: [PATCH] perf tools: Fix python extension build Josh Boyer <jwboyer@fedoraproject.org> - 2016-02-29 14:30 +0100
      Re: [PATCH] perf tools: Fix python extension build Arnaldo Carvalho de Melo <acme@kernel.org> - 2016-02-29 15:10 +0100
        Re: [PATCH] perf tools: Fix python extension build Josh Boyer <jwboyer@fedoraproject.org> - 2016-02-29 21:00 +0100

#1345087 — [PATCH] perf tools: Fix python extension build

FromJiri Olsa <jolsa@redhat.com>
Date2016-02-27 21:30 +0100
Subject[PATCH] perf tools: Fix python extension build
Message-ID<r6TEZ-8jB-7@gated-at.bofh.it>
On Fri, Feb 26, 2016 at 11:59:03PM +0100, Jiri Olsa wrote:
> On Fri, Feb 26, 2016 at 05:49:02PM -0500, Josh Boyer wrote:
> 
> SNIP
> 
> > > ./python_ext_build/tmp/home/jolsa/kernel/linux-perf/tools/perf/util/evsel.o
> > > ./python_ext_build/tmp/home/jolsa/kernel/linux-perf/tools/perf/util/trace-event.o
> > > ./python_ext_build/tmp/home/jolsa/kernel/linux-perf/tools/perf/util/xyarray.o
> > > ./python_ext_build/tmp/home/jolsa/kernel/linux-perf/tools/perf/util/python.o
> > > ./python_ext_build/tmp/home/jolsa/kernel/linux-perf/tools/perf/util/ctype.o
> > > ./python_ext_build/tmp/home/jolsa/kernel/linux-perf/tools/perf/util/cgroup.o
> > > ./python_ext_build/tmp/home/jolsa/kernel/linux-perf/tools/perf/util/rblist.o
> > > ./python_ext_build/tmp/home/jolsa/kernel/linux-perf/tools/perf/util/string.o
> > > ./python_ext_build/tmp/home/jolsa/kernel/linux-perf/tools/lib
> > > ./python_ext_build/tmp/home/jolsa/kernel/linux-perf/tools/lib/hweight.o
> > > ./python_ext_build/tmp/home/jolsa/kernel/linux-perf/tools/lib/rbtree.o
> > > ./python_ext_build/tmp/home/jolsa/kernel/linux-perf/tools/lib/bitmap.o
> > > ./python_ext_build/tmp/home/jolsa/kernel/linux-perf/tools/lib/find_bit.o
> > >
> > > not sure we want to come up with some 'nicer' solution
> > 
> > I don't think anything under python_ext_build/tmp/ actually matters in
> > any significant way.  As long as this doesn't negatively impact
> > something via side-effect, it's probably good enough.
> 
> right, and it's probably generic sollution too
> 
> > 
> > Will you write up a full patch?
> 
> yep, will post it tomorrow
> 
> jirka

sending full patch

jirka


---
The util/python-ext-sources file contains sources files
for python extension relative to $(srctree)/tools/perf,

Such a file path $(FILE).c is handed over to the python
extension build system, which builts final object in
$(PYTHON_EXTBUILD)/tmp/$(FILE).o path.

After build is done all files from $(PYTHON_EXTBUILD)lib/
are carried as the result binaries.

Above system fails when we add source file relative to
../lib, which we do for:

  ../lib/bitmap.c
  ../lib/find_bit.c
  ../lib/hweight.c
  ../lib/rbtree.c

All above objects will be built like:

  $(PYTHON_EXTBUILD)/tmp/../lib/bitmap.c
  $(PYTHON_EXTBUILD)/tmp/../lib/find_bit.c
  $(PYTHON_EXTBUILD)/tmp/../lib/hweight.c
  $(PYTHON_EXTBUILD)/tmp/../lib/rbtree.c

which accidentally happens to be final library path:
  $(PYTHON_EXTBUILD)/lib/

Changing setup.py to pass full paths of source files
to Extension build class and thus keep all built objects
under $(PYTHON_EXTBUILD)tmp directory.

Cc: David Ahern <dsahern@gmail.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Reported-by: Josh Boyer <jwboyer@fedoraproject.org>
Link: http://lkml.kernel.org/n/tip-z14ivktbyblnz9ivuo0zau8l@git.kernel.org
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
---
 tools/perf/util/setup.py | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/tools/perf/util/setup.py b/tools/perf/util/setup.py
index 1833103768cb..c8680984d2d6 100644
--- a/tools/perf/util/setup.py
+++ b/tools/perf/util/setup.py
@@ -22,6 +22,7 @@ cflags = getenv('CFLAGS', '').split()
 # switch off several checks (need to be at the end of cflags list)
 cflags += ['-fno-strict-aliasing', '-Wno-write-strings', '-Wno-unused-parameter' ]
 
+src_perf  = getenv('srctree') + '/tools/perf'
 build_lib = getenv('PYTHON_EXTBUILD_LIB')
 build_tmp = getenv('PYTHON_EXTBUILD_TMP')
 libtraceevent = getenv('LIBTRACEEVENT')
@@ -30,6 +31,9 @@ libapikfs = getenv('LIBAPI')
 ext_sources = [f.strip() for f in file('util/python-ext-sources')
 				if len(f.strip()) > 0 and f[0] != '#']
 
+# use full paths with source files
+ext_sources = map(lambda x: '%s/%s' % (src_perf, x) , ext_sources)
+
 perf = Extension('perf',
 		  sources = ext_sources,
 		  include_dirs = ['util/include'],
-- 
2.4.3

[toc] | [next] | [standalone]

#1345811

FromJosh Boyer <jwboyer@fedoraproject.org>
Date2016-02-29 14:30 +0100
Message-ID<r7w3E-3yu-17@gated-at.bofh.it>
In reply to#1345087
On Sat, Feb 27, 2016 at 3:21 PM, Jiri Olsa <jolsa@redhat.com> wrote:
> On Fri, Feb 26, 2016 at 11:59:03PM +0100, Jiri Olsa wrote:
>> On Fri, Feb 26, 2016 at 05:49:02PM -0500, Josh Boyer wrote:
>>
>> SNIP
>>
>> > > ./python_ext_build/tmp/home/jolsa/kernel/linux-perf/tools/perf/util/evsel.o
>> > > ./python_ext_build/tmp/home/jolsa/kernel/linux-perf/tools/perf/util/trace-event.o
>> > > ./python_ext_build/tmp/home/jolsa/kernel/linux-perf/tools/perf/util/xyarray.o
>> > > ./python_ext_build/tmp/home/jolsa/kernel/linux-perf/tools/perf/util/python.o
>> > > ./python_ext_build/tmp/home/jolsa/kernel/linux-perf/tools/perf/util/ctype.o
>> > > ./python_ext_build/tmp/home/jolsa/kernel/linux-perf/tools/perf/util/cgroup.o
>> > > ./python_ext_build/tmp/home/jolsa/kernel/linux-perf/tools/perf/util/rblist.o
>> > > ./python_ext_build/tmp/home/jolsa/kernel/linux-perf/tools/perf/util/string.o
>> > > ./python_ext_build/tmp/home/jolsa/kernel/linux-perf/tools/lib
>> > > ./python_ext_build/tmp/home/jolsa/kernel/linux-perf/tools/lib/hweight.o
>> > > ./python_ext_build/tmp/home/jolsa/kernel/linux-perf/tools/lib/rbtree.o
>> > > ./python_ext_build/tmp/home/jolsa/kernel/linux-perf/tools/lib/bitmap.o
>> > > ./python_ext_build/tmp/home/jolsa/kernel/linux-perf/tools/lib/find_bit.o
>> > >
>> > > not sure we want to come up with some 'nicer' solution
>> >
>> > I don't think anything under python_ext_build/tmp/ actually matters in
>> > any significant way.  As long as this doesn't negatively impact
>> > something via side-effect, it's probably good enough.
>>
>> right, and it's probably generic sollution too
>>
>> >
>> > Will you write up a full patch?
>>
>> yep, will post it tomorrow
>>
>> jirka
>
> sending full patch
>
> jirka
>
>
> ---
> The util/python-ext-sources file contains sources files
> for python extension relative to $(srctree)/tools/perf,
>
> Such a file path $(FILE).c is handed over to the python
> extension build system, which builts final object in
> $(PYTHON_EXTBUILD)/tmp/$(FILE).o path.
>
> After build is done all files from $(PYTHON_EXTBUILD)lib/
> are carried as the result binaries.
>
> Above system fails when we add source file relative to
> ../lib, which we do for:
>
>   ../lib/bitmap.c
>   ../lib/find_bit.c
>   ../lib/hweight.c
>   ../lib/rbtree.c
>
> All above objects will be built like:
>
>   $(PYTHON_EXTBUILD)/tmp/../lib/bitmap.c
>   $(PYTHON_EXTBUILD)/tmp/../lib/find_bit.c
>   $(PYTHON_EXTBUILD)/tmp/../lib/hweight.c
>   $(PYTHON_EXTBUILD)/tmp/../lib/rbtree.c
>
> which accidentally happens to be final library path:
>   $(PYTHON_EXTBUILD)/lib/
>
> Changing setup.py to pass full paths of source files
> to Extension build class and thus keep all built objects
> under $(PYTHON_EXTBUILD)tmp directory.
>
> Cc: David Ahern <dsahern@gmail.com>
> Cc: Namhyung Kim <namhyung@kernel.org>
> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
> Reported-by: Josh Boyer <jwboyer@fedoraproject.org>
> Link: http://lkml.kernel.org/n/tip-z14ivktbyblnz9ivuo0zau8l@git.kernel.org
> Signed-off-by: Jiri Olsa <jolsa@kernel.org>

The Reported-by: tag should probably be changed to Jeff Bastian
<jbastian@redhat.com> if there's still time.  He reported the original
bug.  Maybe also include this in stable for 4.2+?

I'll be putting this into Fedora Rawhide today.  Thanks!

josh

> ---
>  tools/perf/util/setup.py | 4 ++++
>  1 file changed, 4 insertions(+)
>
> diff --git a/tools/perf/util/setup.py b/tools/perf/util/setup.py
> index 1833103768cb..c8680984d2d6 100644
> --- a/tools/perf/util/setup.py
> +++ b/tools/perf/util/setup.py
> @@ -22,6 +22,7 @@ cflags = getenv('CFLAGS', '').split()
>  # switch off several checks (need to be at the end of cflags list)
>  cflags += ['-fno-strict-aliasing', '-Wno-write-strings', '-Wno-unused-parameter' ]
>
> +src_perf  = getenv('srctree') + '/tools/perf'
>  build_lib = getenv('PYTHON_EXTBUILD_LIB')
>  build_tmp = getenv('PYTHON_EXTBUILD_TMP')
>  libtraceevent = getenv('LIBTRACEEVENT')
> @@ -30,6 +31,9 @@ libapikfs = getenv('LIBAPI')
>  ext_sources = [f.strip() for f in file('util/python-ext-sources')
>                                 if len(f.strip()) > 0 and f[0] != '#']
>
> +# use full paths with source files
> +ext_sources = map(lambda x: '%s/%s' % (src_perf, x) , ext_sources)
> +
>  perf = Extension('perf',
>                   sources = ext_sources,
>                   include_dirs = ['util/include'],
> --
> 2.4.3

[toc] | [prev] | [next] | [standalone]

#1345853

FromArnaldo Carvalho de Melo <acme@kernel.org>
Date2016-02-29 15:10 +0100
Message-ID<r7wGn-44T-13@gated-at.bofh.it>
In reply to#1345811
Em Mon, Feb 29, 2016 at 08:20:33AM -0500, Josh Boyer escreveu:
> On Sat, Feb 27, 2016 at 3:21 PM, Jiri Olsa <jolsa@redhat.com> wrote:
> > On Fri, Feb 26, 2016 at 11:59:03PM +0100, Jiri Olsa wrote:
> >> On Fri, Feb 26, 2016 at 05:49:02PM -0500, Josh Boyer wrote:
> >>
> >> SNIP
> >>
> >> > > ./python_ext_build/tmp/home/jolsa/kernel/linux-perf/tools/perf/util/evsel.o
> >> > > ./python_ext_build/tmp/home/jolsa/kernel/linux-perf/tools/perf/util/trace-event.o
> >> > > ./python_ext_build/tmp/home/jolsa/kernel/linux-perf/tools/perf/util/xyarray.o
> >> > > ./python_ext_build/tmp/home/jolsa/kernel/linux-perf/tools/perf/util/python.o
> >> > > ./python_ext_build/tmp/home/jolsa/kernel/linux-perf/tools/perf/util/ctype.o
> >> > > ./python_ext_build/tmp/home/jolsa/kernel/linux-perf/tools/perf/util/cgroup.o
> >> > > ./python_ext_build/tmp/home/jolsa/kernel/linux-perf/tools/perf/util/rblist.o
> >> > > ./python_ext_build/tmp/home/jolsa/kernel/linux-perf/tools/perf/util/string.o
> >> > > ./python_ext_build/tmp/home/jolsa/kernel/linux-perf/tools/lib
> >> > > ./python_ext_build/tmp/home/jolsa/kernel/linux-perf/tools/lib/hweight.o
> >> > > ./python_ext_build/tmp/home/jolsa/kernel/linux-perf/tools/lib/rbtree.o
> >> > > ./python_ext_build/tmp/home/jolsa/kernel/linux-perf/tools/lib/bitmap.o
> >> > > ./python_ext_build/tmp/home/jolsa/kernel/linux-perf/tools/lib/find_bit.o
> >> > >
> >> > > not sure we want to come up with some 'nicer' solution
> >> >
> >> > I don't think anything under python_ext_build/tmp/ actually matters in
> >> > any significant way.  As long as this doesn't negatively impact
> >> > something via side-effect, it's probably good enough.
> >>
> >> right, and it's probably generic sollution too
> >>
> >> >
> >> > Will you write up a full patch?
> >>
> >> yep, will post it tomorrow
> >>
> >> jirka
> >
> > sending full patch
> >
> > jirka
> >
> >
> > ---
> > The util/python-ext-sources file contains sources files
> > for python extension relative to $(srctree)/tools/perf,
> >
> > Such a file path $(FILE).c is handed over to the python
> > extension build system, which builts final object in
> > $(PYTHON_EXTBUILD)/tmp/$(FILE).o path.
> >
> > After build is done all files from $(PYTHON_EXTBUILD)lib/
> > are carried as the result binaries.
> >
> > Above system fails when we add source file relative to
> > ../lib, which we do for:
> >
> >   ../lib/bitmap.c
> >   ../lib/find_bit.c
> >   ../lib/hweight.c
> >   ../lib/rbtree.c
> >
> > All above objects will be built like:
> >
> >   $(PYTHON_EXTBUILD)/tmp/../lib/bitmap.c
> >   $(PYTHON_EXTBUILD)/tmp/../lib/find_bit.c
> >   $(PYTHON_EXTBUILD)/tmp/../lib/hweight.c
> >   $(PYTHON_EXTBUILD)/tmp/../lib/rbtree.c
> >
> > which accidentally happens to be final library path:
> >   $(PYTHON_EXTBUILD)/lib/
> >
> > Changing setup.py to pass full paths of source files
> > to Extension build class and thus keep all built objects
> > under $(PYTHON_EXTBUILD)tmp directory.
> >
> > Cc: David Ahern <dsahern@gmail.com>
> > Cc: Namhyung Kim <namhyung@kernel.org>
> > Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
> > Reported-by: Josh Boyer <jwboyer@fedoraproject.org>
> > Link: http://lkml.kernel.org/n/tip-z14ivktbyblnz9ivuo0zau8l@git.kernel.org
> > Signed-off-by: Jiri Olsa <jolsa@kernel.org>
> 
> The Reported-by: tag should probably be changed to Jeff Bastian
> <jbastian@redhat.com> if there's still time.  He reported the original
> bug.  Maybe also include this in stable for 4.2+?

Sure, I'll use Jeff in the Reported-by: tag and have it CCed to stable,

I also will add a Tested-by: Josh Boyer, ok?

Thanks
 
> I'll be putting this into Fedora Rawhide today.  Thanks!
> 
> josh
> 
> > ---
> >  tools/perf/util/setup.py | 4 ++++
> >  1 file changed, 4 insertions(+)
> >
> > diff --git a/tools/perf/util/setup.py b/tools/perf/util/setup.py
> > index 1833103768cb..c8680984d2d6 100644
> > --- a/tools/perf/util/setup.py
> > +++ b/tools/perf/util/setup.py
> > @@ -22,6 +22,7 @@ cflags = getenv('CFLAGS', '').split()
> >  # switch off several checks (need to be at the end of cflags list)
> >  cflags += ['-fno-strict-aliasing', '-Wno-write-strings', '-Wno-unused-parameter' ]
> >
> > +src_perf  = getenv('srctree') + '/tools/perf'
> >  build_lib = getenv('PYTHON_EXTBUILD_LIB')
> >  build_tmp = getenv('PYTHON_EXTBUILD_TMP')
> >  libtraceevent = getenv('LIBTRACEEVENT')
> > @@ -30,6 +31,9 @@ libapikfs = getenv('LIBAPI')
> >  ext_sources = [f.strip() for f in file('util/python-ext-sources')
> >                                 if len(f.strip()) > 0 and f[0] != '#']
> >
> > +# use full paths with source files
> > +ext_sources = map(lambda x: '%s/%s' % (src_perf, x) , ext_sources)
> > +
> >  perf = Extension('perf',
> >                   sources = ext_sources,
> >                   include_dirs = ['util/include'],
> > --
> > 2.4.3

[toc] | [prev] | [next] | [standalone]

#1346131

FromJosh Boyer <jwboyer@fedoraproject.org>
Date2016-02-29 21:00 +0100
Message-ID<r7C93-7oN-11@gated-at.bofh.it>
In reply to#1345853
On Mon, Feb 29, 2016 at 9:08 AM, Arnaldo Carvalho de Melo
<acme@kernel.org> wrote:
> Em Mon, Feb 29, 2016 at 08:20:33AM -0500, Josh Boyer escreveu:
>> On Sat, Feb 27, 2016 at 3:21 PM, Jiri Olsa <jolsa@redhat.com> wrote:
>> > On Fri, Feb 26, 2016 at 11:59:03PM +0100, Jiri Olsa wrote:
>> >> On Fri, Feb 26, 2016 at 05:49:02PM -0500, Josh Boyer wrote:
>> >>
>> >> SNIP
>> >>
>> >> > > ./python_ext_build/tmp/home/jolsa/kernel/linux-perf/tools/perf/util/evsel.o
>> >> > > ./python_ext_build/tmp/home/jolsa/kernel/linux-perf/tools/perf/util/trace-event.o
>> >> > > ./python_ext_build/tmp/home/jolsa/kernel/linux-perf/tools/perf/util/xyarray.o
>> >> > > ./python_ext_build/tmp/home/jolsa/kernel/linux-perf/tools/perf/util/python.o
>> >> > > ./python_ext_build/tmp/home/jolsa/kernel/linux-perf/tools/perf/util/ctype.o
>> >> > > ./python_ext_build/tmp/home/jolsa/kernel/linux-perf/tools/perf/util/cgroup.o
>> >> > > ./python_ext_build/tmp/home/jolsa/kernel/linux-perf/tools/perf/util/rblist.o
>> >> > > ./python_ext_build/tmp/home/jolsa/kernel/linux-perf/tools/perf/util/string.o
>> >> > > ./python_ext_build/tmp/home/jolsa/kernel/linux-perf/tools/lib
>> >> > > ./python_ext_build/tmp/home/jolsa/kernel/linux-perf/tools/lib/hweight.o
>> >> > > ./python_ext_build/tmp/home/jolsa/kernel/linux-perf/tools/lib/rbtree.o
>> >> > > ./python_ext_build/tmp/home/jolsa/kernel/linux-perf/tools/lib/bitmap.o
>> >> > > ./python_ext_build/tmp/home/jolsa/kernel/linux-perf/tools/lib/find_bit.o
>> >> > >
>> >> > > not sure we want to come up with some 'nicer' solution
>> >> >
>> >> > I don't think anything under python_ext_build/tmp/ actually matters in
>> >> > any significant way.  As long as this doesn't negatively impact
>> >> > something via side-effect, it's probably good enough.
>> >>
>> >> right, and it's probably generic sollution too
>> >>
>> >> >
>> >> > Will you write up a full patch?
>> >>
>> >> yep, will post it tomorrow
>> >>
>> >> jirka
>> >
>> > sending full patch
>> >
>> > jirka
>> >
>> >
>> > ---
>> > The util/python-ext-sources file contains sources files
>> > for python extension relative to $(srctree)/tools/perf,
>> >
>> > Such a file path $(FILE).c is handed over to the python
>> > extension build system, which builts final object in
>> > $(PYTHON_EXTBUILD)/tmp/$(FILE).o path.
>> >
>> > After build is done all files from $(PYTHON_EXTBUILD)lib/
>> > are carried as the result binaries.
>> >
>> > Above system fails when we add source file relative to
>> > ../lib, which we do for:
>> >
>> >   ../lib/bitmap.c
>> >   ../lib/find_bit.c
>> >   ../lib/hweight.c
>> >   ../lib/rbtree.c
>> >
>> > All above objects will be built like:
>> >
>> >   $(PYTHON_EXTBUILD)/tmp/../lib/bitmap.c
>> >   $(PYTHON_EXTBUILD)/tmp/../lib/find_bit.c
>> >   $(PYTHON_EXTBUILD)/tmp/../lib/hweight.c
>> >   $(PYTHON_EXTBUILD)/tmp/../lib/rbtree.c
>> >
>> > which accidentally happens to be final library path:
>> >   $(PYTHON_EXTBUILD)/lib/
>> >
>> > Changing setup.py to pass full paths of source files
>> > to Extension build class and thus keep all built objects
>> > under $(PYTHON_EXTBUILD)tmp directory.
>> >
>> > Cc: David Ahern <dsahern@gmail.com>
>> > Cc: Namhyung Kim <namhyung@kernel.org>
>> > Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
>> > Reported-by: Josh Boyer <jwboyer@fedoraproject.org>
>> > Link: http://lkml.kernel.org/n/tip-z14ivktbyblnz9ivuo0zau8l@git.kernel.org
>> > Signed-off-by: Jiri Olsa <jolsa@kernel.org>
>>
>> The Reported-by: tag should probably be changed to Jeff Bastian
>> <jbastian@redhat.com> if there's still time.  He reported the original
>> bug.  Maybe also include this in stable for 4.2+?
>
> Sure, I'll use Jeff in the Reported-by: tag and have it CCed to stable,
>
> I also will add a Tested-by: Josh Boyer, ok?

Yes, that is fine.  I tested the build and the resulting python-perf
package only has the proper contents.  I see you've added this to the
formal patch, so thank you.  Apologies for the delayed reply.

josh

[toc] | [prev] | [standalone]

Back to top | Article view | linux.kernel

csiph-web