Groups > linux.kernel > #1455443 > unrolled thread

powerpc allyesconfig / allmodconfig linux-next next-20160729 - next-20160729 build failures

Started by	"Luis R. Rodriguez" <mcgrof@kernel.org>
First post	2016-08-02 22:10 +0200
Last post	2016-08-03 05:30 +0200
Articles	20 on this page of 36 — 7 participants

Back to article view | Back to linux.kernel

  powerpc allyesconfig / allmodconfig linux-next next-20160729 -  next-20160729 build failures "Luis R. Rodriguez" <mcgrof@kernel.org> - 2016-08-02 22:10 +0200
    Re: powerpc allyesconfig / allmodconfig linux-next next-20160729 -  next-20160729 build failures Guenter Roeck <linux@roeck-us.net> - 2016-08-03 00:00 +0200
      Re: powerpc allyesconfig / allmodconfig linux-next next-20160729 -  next-20160729 build failures "Luis R. Rodriguez" <mcgrof@kernel.org> - 2016-08-03 00:30 +0200
        Re: powerpc allyesconfig / allmodconfig linux-next next-20160729 - next-20160729 build failures Arnd Bergmann <arnd@arndb.de> - 2016-08-03 00:50 +0200
        Re: powerpc allyesconfig / allmodconfig linux-next next-20160729 -  next-20160729 build failures Stephen Rothwell <sfr@canb.auug.org.au> - 2016-08-03 02:30 +0200
          Re: powerpc allyesconfig / allmodconfig linux-next next-20160729 - next-20160729 build failures Arnd Bergmann <arnd@arndb.de> - 2016-08-03 10:00 +0200
            Re: powerpc allyesconfig / allmodconfig linux-next next-20160729 -  next-20160729 build failures Stephen Rothwell <sfr@canb.auug.org.au> - 2016-08-03 14:30 +0200
              Re: powerpc allyesconfig / allmodconfig linux-next next-20160729 - next-20160729 build failures Arnd Bergmann <arnd@arndb.de> - 2016-08-03 14:40 +0200
                Re: powerpc allyesconfig / allmodconfig linux-next next-20160729 -  next-20160729 build failures Nicholas Piggin <npiggin@gmail.com> - 2016-08-03 18:00 +0200
                  Re: powerpc allyesconfig / allmodconfig linux-next next-20160729 - next-20160729 build failures Arnd Bergmann <arnd@arndb.de> - 2016-08-03 21:10 +0200
                    Re: powerpc allyesconfig / allmodconfig linux-next next-20160729 - next-20160729 build failures Arnd Bergmann <arnd@arndb.de> - 2016-08-03 22:20 +0200
                      Re: powerpc allyesconfig / allmodconfig linux-next next-20160729 -  next-20160729 build failures Nicholas Piggin <npiggin@gmail.com> - 2016-08-11 14:50 +0200
                        Re: powerpc allyesconfig / allmodconfig linux-next next-20160729 - next-20160729 build failures Arnd Bergmann <arnd@arndb.de> - 2016-08-11 15:10 +0200
                          Re: powerpc allyesconfig / allmodconfig linux-next next-20160729 -  next-20160729 build failures Nicholas Piggin <npiggin@gmail.com> - 2016-08-11 15:20 +0200
                            [TESTING] kbuild: link drivers subdirectories separately Arnd Bergmann <arnd@arndb.de> - 2016-08-11 16:00 +0200
                              Re: [TESTING] kbuild: link drivers subdirectories separately Arnd Bergmann <arnd@arndb.de> - 2016-08-11 17:50 +0200
                    Re: powerpc allyesconfig / allmodconfig linux-next next-20160729 - next-20160729 build failures Segher Boessenkool <segher@kernel.crashing.org> - 2016-08-03 22:50 +0200
                    Re: powerpc allyesconfig / allmodconfig linux-next next-20160729 -  next-20160729 build failures Stephen Rothwell <sfr@canb.auug.org.au> - 2016-08-04 02:20 +0200
                      Re: powerpc allyesconfig / allmodconfig linux-next next-20160729 - next-20160729 build failures Arnd Bergmann <arnd@arndb.de> - 2016-08-04 11:10 +0200
                        Re: powerpc allyesconfig / allmodconfig linux-next next-20160729 - next-20160729 build failures Arnd Bergmann <arnd@arndb.de> - 2016-08-04 12:50 +0200
                          Re: powerpc allyesconfig / allmodconfig linux-next next-20160729 -  next-20160729 build failures Nicholas Piggin <npiggin@gmail.com> - 2016-08-04 13:50 +0200
                            Re: powerpc allyesconfig / allmodconfig linux-next next-20160729 - next-20160729 build failures Arnd Bergmann <arnd@arndb.de> - 2016-08-04 14:20 +0200
                              Re: powerpc allyesconfig / allmodconfig linux-next next-20160729 -  next-20160729 build failures Nicholas Piggin <npiggin@gmail.com> - 2016-08-04 14:40 +0200
                                Re: powerpc allyesconfig / allmodconfig linux-next next-20160729 - next-20160729 build failures Arnd Bergmann <arnd@arndb.de> - 2016-08-04 17:50 +0200
                                Re: powerpc allyesconfig / allmodconfig linux-next next-20160729 -  next-20160729 build failures Nicholas Piggin <npiggin@gmail.com> - 2016-08-04 18:10 +0200
                            Re: powerpc allyesconfig / allmodconfig linux-next next-20160729 - next-20160729 build failures Arnd Bergmann <arnd@arndb.de> - 2016-08-04 18:20 +0200
                              Re: powerpc allyesconfig / allmodconfig linux-next next-20160729 - next-20160729 build failures Segher Boessenkool <segher@kernel.crashing.org> - 2016-08-04 20:30 +0200
                                Re: powerpc allyesconfig / allmodconfig linux-next next-20160729 -  next-20160729 build failures Nicholas Piggin <npiggin@gmail.com> - 2016-08-05 10:50 +0200
                                  Re: powerpc allyesconfig / allmodconfig linux-next next-20160729 - next-20160729 build failures Arnd Bergmann <arnd@arndb.de> - 2016-08-05 12:20 +0200
                                    Re: powerpc allyesconfig / allmodconfig linux-next next-20160729 -  next-20160729 build failures Nicholas Piggin <npiggin@gmail.com> - 2016-08-05 14:30 +0200
                                      Re: powerpc allyesconfig / allmodconfig linux-next next-20160729 - next-20160729 build failures Arnd Bergmann <arnd@arndb.de> - 2016-08-05 18:10 +0200
                                        Re: powerpc allyesconfig / allmodconfig linux-next next-20160729 -  next-20160729 build failures Nicholas Piggin <npiggin@gmail.com> - 2016-08-05 18:20 +0200
                                          Re: powerpc allyesconfig / allmodconfig linux-next next-20160729 - next-20160729 build failures Arnd Bergmann <arnd@arndb.de> - 2016-08-05 21:20 +0200
                                            Re: powerpc allyesconfig / allmodconfig linux-next next-20160729 -  next-20160729 build failures Nicholas Piggin <npiggin@gmail.com> - 2016-08-06 22:50 +0200
                                              Re: powerpc allyesconfig / allmodconfig linux-next next-20160729 - next-20160729 build failures Arnd Bergmann <arnd@arndb.de> - 2016-08-06 23:20 +0200
    Re: powerpc allyesconfig / allmodconfig linux-next next-20160729 - next-20160729 build failures Michael Ellerman <mpe@ellerman.id.au> - 2016-08-03 05:30 +0200

Page 1 of 2 [1] 2 Next page →

#1455443 — powerpc allyesconfig / allmodconfig linux-next next-20160729 - next-20160729 build failures

From	"Luis R. Rodriguez" <mcgrof@kernel.org>
Date	2016-08-02 22:10 +0200
Subject	powerpc allyesconfig / allmodconfig linux-next next-20160729 - next-20160729 build failures
Message-ID	<s1OaK-WW-15@gated-at.bofh.it>

Are linux-next builds being tested for powerpc with allyesconfig and
allmodconfig ? I have some changes I'm making and while debugging my
build issues I decided to give a clean build a shot and see linux-next
next-20160729 up to next-20160729 all have build failures without my
changes. I get:

/opt/gcc-4.9.0-nolibc/powerpc64-linux/bin/powerpc64-linux-ld:
drivers/built-in.o: .opd is not a regular array of opd entries
  MODPOST vmlinux.o
  GEN     .version
  CHK     include/generated/compile.h
  UPD     include/generated/compile.h
  CC      init/version.o
  LD      init/built-in.o
/opt/gcc-4.9.0-nolibc/powerpc64-linux/bin/powerpc64-linux-ld:
drivers/built-in.o: .opd is not a regular array of opd entries
drivers/built-in.o: In function `.ipw2100_up':
ipw2100.c:(.text+0x1ff9c90): relocation truncated to fit:
R_PPC64_REL24 (stub) against symbol `.round_jiffies_relative' defined
in .text section in kernel/built-in.o
drivers/built-in.o: In function `.ipw2100_reset_adapter':
ipw2100.c:(.text+0x1ffa500): relocation truncated to fit:
R_PPC64_REL24 (stub) against symbol `._raw_spin_lock_irqsave' defined
in .spinlock.text section in kernel/built-in.o
drivers/built-in.o: In function `.ipw2100_irq_tasklet':
ipw2100.c:(.text+0x1ffa7cc): relocation truncated to fit:
R_PPC64_REL24 (stub) against symbol `._raw_spin_lock_irqsave' defined
in .spinlock.text section in kernel/built-in.o
ipw2100.c:(.text+0x1ffb6c8): relocation truncated to fit:
R_PPC64_REL24 (stub) against symbol `.printk' defined in
.text.unlikely section in kernel/built-in.o
ipw2100.c:(.text+0x1ffb6d8): relocation truncated to fit:
R_PPC64_REL24 (stub) against symbol `.printk' defined in
.text.unlikely section in kernel/built-in.o
ipw2100.c:(.text+0x1ffb740): relocation truncated to fit:
R_PPC64_REL24 (stub) against symbol `.printk' defined in
.text.unlikely section in kernel/built-in.o
ipw2100.c:(.text+0x1ffb750): relocation truncated to fit:
R_PPC64_REL24 (stub) against symbol `.printk' defined in
.text.unlikely section in kernel/built-in.o
ipw2100.c:(.text+0x1ffb7ec): relocation truncated to fit:
R_PPC64_REL24 (stub) against symbol `.debug_dma_unmap_page' defined in
.text section in lib/built-in.o
ipw2100.c:(.text+0x1ffb88c): relocation truncated to fit:
R_PPC64_REL24 (stub) against symbol `.__dev_kfree_skb_any' defined in
.text section in net/built-in.o
ipw2100.c:(.text+0x1ffb8b8): relocation truncated to fit:
R_PPC64_REL24 (stub) against symbol `.printk' defined in
.text.unlikely section in kernel/built-in.o
ipw2100.c:(.text+0x1ffb8f4): additional relocation overflows omitted
from the output
scripts/link-vmlinux.sh: line 52: 14580 Segmentation fault      (core
dumped) ${LD} ${LDFLAGS} ${LDFLAGS_vmlinux} -o ${2} -T ${lds}
${KBUILD_VMLINUX_INIT} --start-group ${KBUILD_VMLINUX_MAIN}
--end-group ${1}
make: *** [Makefile:952: vmlinux] Error 139

  Luis

[toc] | [next] | [standalone]

#1455496

From	Guenter Roeck <linux@roeck-us.net>
Date	2016-08-03 00:00 +0200
Message-ID	<s1PTc-1PI-9@gated-at.bofh.it>
In reply to	#1455443

On Tue, Aug 02, 2016 at 01:07:09PM -0700, Luis R. Rodriguez wrote:
> Are linux-next builds being tested for powerpc with allyesconfig and
> allmodconfig ? I have some changes I'm making and while debugging my
> build issues I decided to give a clean build a shot and see linux-next
> next-20160729 up to next-20160729 all have build failures without my
> changes. I get:
> 
> /opt/gcc-4.9.0-nolibc/powerpc64-linux/bin/powerpc64-linux-ld:
> drivers/built-in.o: .opd is not a regular array of opd entries
>   MODPOST vmlinux.o
>   GEN     .version
>   CHK     include/generated/compile.h
>   UPD     include/generated/compile.h
>   CC      init/version.o
>   LD      init/built-in.o
> /opt/gcc-4.9.0-nolibc/powerpc64-linux/bin/powerpc64-linux-ld:
> drivers/built-in.o: .opd is not a regular array of opd entries
> drivers/built-in.o: In function `.ipw2100_up':
> ipw2100.c:(.text+0x1ff9c90): relocation truncated to fit:

"relocation truncated to fit"  errors are typical for ppc:allyesconfig.
allmodconfig should work, though.

Guenter

[toc] | [prev] | [next] | [standalone]

#1455506

From	"Luis R. Rodriguez" <mcgrof@kernel.org>
Date	2016-08-03 00:30 +0200
Message-ID	<s1Qmd-2er-3@gated-at.bofh.it>
In reply to	#1455496

On Tue, Aug 02, 2016 at 02:58:39PM -0700, Guenter Roeck wrote:
> On Tue, Aug 02, 2016 at 01:07:09PM -0700, Luis R. Rodriguez wrote:
> > Are linux-next builds being tested for powerpc with allyesconfig and
> > allmodconfig ? I have some changes I'm making and while debugging my
> > build issues I decided to give a clean build a shot and see linux-next
> > next-20160729 up to next-20160729 all have build failures without my
> > changes. I get:
> > 
> > /opt/gcc-4.9.0-nolibc/powerpc64-linux/bin/powerpc64-linux-ld:
> > drivers/built-in.o: .opd is not a regular array of opd entries
> >   MODPOST vmlinux.o
> >   GEN     .version
> >   CHK     include/generated/compile.h
> >   UPD     include/generated/compile.h
> >   CC      init/version.o
> >   LD      init/built-in.o
> > /opt/gcc-4.9.0-nolibc/powerpc64-linux/bin/powerpc64-linux-ld:
> > drivers/built-in.o: .opd is not a regular array of opd entries
> > drivers/built-in.o: In function `.ipw2100_up':
> > ipw2100.c:(.text+0x1ff9c90): relocation truncated to fit:
> 
> "relocation truncated to fit"  errors are typical for ppc:allyesconfig.

Thanks for the confirmation. For how long is it known this is broken?
Does anyone care and fix these ? Or is this best effort?

> allmodconfig should work, though.

OK thanks.

  Luis

[toc] | [prev] | [next] | [standalone]

#1455514 — Re: powerpc allyesconfig / allmodconfig linux-next next-20160729 - next-20160729 build failures

From	Arnd Bergmann <arnd@arndb.de>
Date	2016-08-03 00:50 +0200
Subject	Re: powerpc allyesconfig / allmodconfig linux-next next-20160729 - next-20160729 build failures
Message-ID	<s1QFz-2lk-9@gated-at.bofh.it>
In reply to	#1455506

On Wednesday, August 3, 2016 12:02:43 AM CEST Luis R. Rodriguez wrote:
> On Tue, Aug 02, 2016 at 02:58:39PM -0700, Guenter Roeck wrote:
> > On Tue, Aug 02, 2016 at 01:07:09PM -0700, Luis R. Rodriguez wrote:
> > > Are linux-next builds being tested for powerpc with allyesconfig and
> > > allmodconfig ? I have some changes I'm making and while debugging my
> > > build issues I decided to give a clean build a shot and see linux-next
> > > next-20160729 up to next-20160729 all have build failures without my
> > > changes. I get:
> > > 
> > > /opt/gcc-4.9.0-nolibc/powerpc64-linux/bin/powerpc64-linux-ld:
> > > drivers/built-in.o: .opd is not a regular array of opd entries
> > >   MODPOST vmlinux.o
> > >   GEN     .version
> > >   CHK     include/generated/compile.h
> > >   UPD     include/generated/compile.h
> > >   CC      init/version.o
> > >   LD      init/built-in.o
> > > /opt/gcc-4.9.0-nolibc/powerpc64-linux/bin/powerpc64-linux-ld:
> > > drivers/built-in.o: .opd is not a regular array of opd entries
> > > drivers/built-in.o: In function `.ipw2100_up':
> > > ipw2100.c:(.text+0x1ff9c90): relocation truncated to fit:
> > 
> > "relocation truncated to fit"  errors are typical for ppc:allyesconfig.
> 
> Thanks for the confirmation. For how long is it known this is broken?
> Does anyone care and fix these ? Or is this best effort?

We used to have the same thing on ARM, but it's (mostly) fixed now.
In case of ARM, the solution was to ensure that all sections that
have long jumps or targets of long jumps are marked as executable
in the ELF headers, so the linker can insert trampolines.

The one remaining problem at the moment is related to recursive
linking of the drivers/ directory, which has .text section that
is larger than 32MB by itself. There is a patch to solve this by
linking each drivers/*/built-in.o object directly into vmlinux,
but that is a rather drastic change.

	Arnd

[toc] | [prev] | [next] | [standalone]

#1455542

From	Stephen Rothwell <sfr@canb.auug.org.au>
Date	2016-08-03 02:30 +0200
Message-ID	<s1Sem-3p4-13@gated-at.bofh.it>
In reply to	#1455506

Hi Luis,

On Wed, 3 Aug 2016 00:02:43 +0200 "Luis R. Rodriguez" <mcgrof@kernel.org> wrote:
>
> Thanks for the confirmation. For how long is it known this is broken?
> Does anyone care and fix these ? Or is this best effort?

This has been broken for many years :-(

I have a couple of times almost fixed it, but it requires that we
change from using "ld -r" to build the built-in.o objects and some
changes to the powerpc head.S code ... I will give it another shot now
that the merge window is almost over (and linux-next goes into its
quieter time).

-- 
Cheers,
Stephen Rothwell

[toc] | [prev] | [next] | [standalone]

#1455671 — Re: powerpc allyesconfig / allmodconfig linux-next next-20160729 - next-20160729 build failures

From	Arnd Bergmann <arnd@arndb.de>
Date	2016-08-03 10:00 +0200
Subject	Re: powerpc allyesconfig / allmodconfig linux-next next-20160729 - next-20160729 build failures
Message-ID	<s1ZfQ-89Q-15@gated-at.bofh.it>
In reply to	#1455542

On Wednesday, August 3, 2016 10:23:24 AM CEST Stephen Rothwell wrote:
> Hi Luis,
> 
> On Wed, 3 Aug 2016 00:02:43 +0200 "Luis R. Rodriguez" <mcgrof@kernel.org> wrote:
> >
> > Thanks for the confirmation. For how long is it known this is broken?
> > Does anyone care and fix these ? Or is this best effort?
> 
> This has been broken for many years 
> 
> I have a couple of times almost fixed it, but it requires that we
> change from using "ld -r" to build the built-in.o objects and some
> changes to the powerpc head.S code ... I will give it another shot now
> that the merge window is almost over (and linux-next goes into its
> quieter time).

Using a different way to link the kernel would also help us with
the remaining allyesconfig problem on ARM, as the problem is only in
'ld -r' not producing trampolines for symbols that later cannot get
them any more. It would probably also help building with ld.gold,
which is currently not working.

What is your suggested alternative?

	Arnd

[toc] | [prev] | [next] | [standalone]

#1455778

From	Stephen Rothwell <sfr@canb.auug.org.au>
Date	2016-08-03 14:30 +0200
Message-ID	<s23t7-2tz-1@gated-at.bofh.it>
In reply to	#1455671

Hi Arnd,

On Wed, 03 Aug 2016 09:52:23 +0200 Arnd Bergmann <arnd@arndb.de> wrote:
>
> Using a different way to link the kernel would also help us with
> the remaining allyesconfig problem on ARM, as the problem is only in
> 'ld -r' not producing trampolines for symbols that later cannot get
> them any more. It would probably also help building with ld.gold,
> which is currently not working.
> 
> What is your suggested alternative?

I have a patch that make the built-in.o files into thin archives (same
as archives, but the actual objects are replaced with the name of the
original object file).  That way the final link has all the original
objects.  I haven't checked to see what the overheads of doing it this
way is.

Nick Piggin has just today taken my old patch (it was last rebased to
v4.4-rc1) and tried it on a recent kernel and it still seems to mostly
work.  It probably needs some tidying up, but you are welcome to test
it if you want to.

-- 
Cheers,
Stephen Rothwell

[toc] | [prev] | [next] | [standalone]

#1455784 — Re: powerpc allyesconfig / allmodconfig linux-next next-20160729 - next-20160729 build failures

From	Arnd Bergmann <arnd@arndb.de>
Date	2016-08-03 14:40 +0200
Subject	Re: powerpc allyesconfig / allmodconfig linux-next next-20160729 - next-20160729 build failures
Message-ID	<s23CN-2wB-11@gated-at.bofh.it>
In reply to	#1455778

On Wednesday, August 3, 2016 10:19:11 PM CEST Stephen Rothwell wrote:
> Hi Arnd,
> 
> On Wed, 03 Aug 2016 09:52:23 +0200 Arnd Bergmann <arnd@arndb.de> wrote:
> >
> > Using a different way to link the kernel would also help us with
> > the remaining allyesconfig problem on ARM, as the problem is only in
> > 'ld -r' not producing trampolines for symbols that later cannot get
> > them any more. It would probably also help building with ld.gold,
> > which is currently not working.
> > 
> > What is your suggested alternative?
> 
> I have a patch that make the built-in.o files into thin archives (same
> as archives, but the actual objects are replaced with the name of the
> original object file).  That way the final link has all the original
> objects.  I haven't checked to see what the overheads of doing it this
> way is.
> 
> Nick Piggin has just today taken my old patch (it was last rebased to
> v4.4-rc1) and tried it on a recent kernel and it still seems to mostly
> work.  It probably needs some tidying up, but you are welcome to test
> it if you want to.

Sure, I'll certainly give it a try on ARM when you send me a copy.

	Arnd

[toc] | [prev] | [next] | [standalone]

#1455873

From	Nicholas Piggin <npiggin@gmail.com>
Date	2016-08-03 18:00 +0200
Message-ID	<s26Km-4pi-5@gated-at.bofh.it>
In reply to	#1455784

On Wed, 03 Aug 2016 14:29:13 +0200
Arnd Bergmann <arnd@arndb.de> wrote:

> On Wednesday, August 3, 2016 10:19:11 PM CEST Stephen Rothwell wrote:
> > Hi Arnd,
> > 
> > On Wed, 03 Aug 2016 09:52:23 +0200 Arnd Bergmann <arnd@arndb.de> wrote:  
> > >
> > > Using a different way to link the kernel would also help us with
> > > the remaining allyesconfig problem on ARM, as the problem is only in
> > > 'ld -r' not producing trampolines for symbols that later cannot get
> > > them any more. It would probably also help building with ld.gold,
> > > which is currently not working.
> > > 
> > > What is your suggested alternative?  
> > 
> > I have a patch that make the built-in.o files into thin archives (same
> > as archives, but the actual objects are replaced with the name of the
> > original object file).  That way the final link has all the original
> > objects.  I haven't checked to see what the overheads of doing it this
> > way is.
> > 
> > Nick Piggin has just today taken my old patch (it was last rebased to
> > v4.4-rc1) and tried it on a recent kernel and it still seems to mostly
> > work.  It probably needs some tidying up, but you are welcome to test
> > it if you want to.  
> 
> Sure, I'll certainly give it a try on ARM when you send me a copy.

I've attached what I'm using, which builds and runs for me without
any work. Your arch obviously has to select the option to use it.

    text      data     bss      dec       hex     filename
    11196784  1185024  1923820  14305628  da495c  vmlinuxppc64.before
    11187536  1181848  1923176  14292560  da1650  vmlinuxppc64.after
    
~9K text saving, ~3K data saving. I assume this comes from fewer
branch trampolines and toc entries, but haven't verified exactly.



commit 8bc3ca4798c215e9a9107b6d44408f0af259f84f
Author: Stephen Rothwell <sfr@canb.auug.org.au>
Date:   Tue Oct 30 12:14:18 2012 +1100

    kbuild: allow architectures to use thin archives instead of ld -r
    
    Alan Modra has been trying to convince the kernel developers that ld -r
    is "evil" for many years.  This is an alternative and means that the
    linker has much more information available to it when it links the
    kernel.
    
    Signed-off-by: Stephen Rothwell <sfr@canb.auug.org.au>

diff --git a/arch/Kconfig b/arch/Kconfig
index d794384..1330bf4 100644
--- a/arch/Kconfig
+++ b/arch/Kconfig
@@ -424,6 +424,12 @@ config CC_STACKPROTECTOR_STRONG
 
 endchoice
 
+config THIN_ARCHIVES
+	bool
+	help
+	  Select this if the architecture wants to use thin archives
+	  instead of ld -r to create the built-in.o files.
+
 config HAVE_CONTEXT_TRACKING
 	bool
 	help
diff --git a/scripts/Makefile.build b/scripts/Makefile.build
index 0d1ca5b..bbf60b3 100644
--- a/scripts/Makefile.build
+++ b/scripts/Makefile.build
@@ -358,10 +358,15 @@ $(sort $(subdir-obj-y)): $(subdir-ym) ;
 # Rule to compile a set of .o files into one .o file
 #
 ifdef builtin-target
+ifdef CONFIG_THIN_ARCHIVES
+  cmd_make_builtin = rm -f $@; $(AR) rcsT$(KBUILD_ARFLAGS)
+else
+  cmd_make_builtin = $(LD) $(ld_flags) -r -o
+endif
 quiet_cmd_link_o_target = LD      $@
 # If the list of objects to link is empty, just create an empty built-in.o
 cmd_link_o_target = $(if $(strip $(obj-y)),\
-		      $(LD) $(ld_flags) -r -o $@ $(filter $(obj-y), $^) \
+		      $(cmd_make_builtin) $@ $(filter $(obj-y), $^) \
 		      $(cmd_secanalysis),\
 		      rm -f $@; $(AR) rcs$(KBUILD_ARFLAGS) $@)
 
diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh
index f0f6d9d..ef4658f 100755
--- a/scripts/link-vmlinux.sh
+++ b/scripts/link-vmlinux.sh
@@ -41,8 +41,14 @@ info()
 # ${1} output file
 modpost_link()
 {
-	${LD} ${LDFLAGS} -r -o ${1} ${KBUILD_VMLINUX_INIT}                   \
-		--start-group ${KBUILD_VMLINUX_MAIN} --end-group
+	local objects
+
+	if [ -n "${CONFIG_THIN_ARCHIVES}" ]; then
+		objects="--whole-archive ${KBUILD_VMLINUX_INIT} ${KBUILD_VMLINUX_MAIN} --no-whole-archive"
+	else
+		objects="${KBUILD_VMLINUX_INIT} --start-group ${KBUILD_VMLINUX_MAIN} --end-group"
+	fi
+	${LD} ${LDFLAGS} -r -o ${1} ${objects}
 }
 
 # Link of vmlinux
@@ -51,11 +57,16 @@ modpost_link()
 vmlinux_link()
 {
 	local lds="${objtree}/${KBUILD_LDS}"
+	local objects
 
 	if [ "${SRCARCH}" != "um" ]; then
+		if [ -n "${CONFIG_THIN_ARCHIVES}" ]; then
+			objects="--whole-archive ${KBUILD_VMLINUX_INIT} ${KBUILD_VMLINUX_MAIN} --no-whole-archive"
+		else
+			objects="${KBUILD_VMLINUX_INIT} --start-group ${KBUILD_VMLINUX_MAIN} --end-group"
+		fi
 		${LD} ${LDFLAGS} ${LDFLAGS_vmlinux} -o ${2}                  \
-			-T ${lds} ${KBUILD_VMLINUX_INIT}                     \
-			--start-group ${KBUILD_VMLINUX_MAIN} --end-group ${1}
+			-T ${lds} ${objects} ${1}
 	else
 		${CC} ${CFLAGS_vmlinux} -o ${2}                              \
 			-Wl,-T,${lds} ${KBUILD_VMLINUX_INIT}                 \

[toc] | [prev] | [next] | [standalone]

#1455977 — Re: powerpc allyesconfig / allmodconfig linux-next next-20160729 - next-20160729 build failures

From	Arnd Bergmann <arnd@arndb.de>
Date	2016-08-03 21:10 +0200
Subject	Re: powerpc allyesconfig / allmodconfig linux-next next-20160729 - next-20160729 build failures
Message-ID	<s29Ie-6D8-23@gated-at.bofh.it>
In reply to	#1455873

On Thursday, August 4, 2016 1:37:29 AM CEST Nicholas Piggin wrote:
> 
> I've attached what I'm using, which builds and runs for me without
> any work. Your arch obviously has to select the option to use it.
> 
>     text      data     bss      dec       hex     filename
>     11196784  1185024  1923820  14305628  da495c  vmlinuxppc64.before
>     11187536  1181848  1923176  14292560  da1650  vmlinuxppc64.after
>     
> ~9K text saving, ~3K data saving. I assume this comes from fewer
> branch trampolines and toc entries, but haven't verified exactly.

The patch seems to work great, but for me it's getting bigger
(compared to my older patch, mainline allyesconfig doesn't build):

   text	   data	    bss	    dec	    hex	filename
51299868	42599559	23362148	117261575	6fd4507	vmlinuxarm.before
51302545	42595015	23361884	117259444	6fd3cb4	vmlinuxarm.after

Most of the difference appears to be in branch trampolines (634 added,
559 removed, 14837 unchanged) as you suspect, but I also see a couple
of symbols show up in vmlinux that were not there before:

-A __crc_dma_noop_ops
-D dma_noop_ops
-R __clz_tab
-r fdt_errtable
-r __kcrctab_dma_noop_ops
-r __kstrtab_dma_noop_ops
-R __ksymtab_dma_noop_ops
-t dma_noop_alloc
-t dma_noop_free
-t dma_noop_map_page
-t dma_noop_mapping_error
-t dma_noop_map_sg
-t dma_noop_supported
-T fdt_add_reservemap_entry
-T fdt_begin_node
-T fdt_create
-T fdt_create_empty_tree
-T fdt_end_node
-T fdt_finish
-T fdt_finish_reservemap
-T fdt_property
-T fdt_resize
-T fdt_strerror
-T find_cpio_data

From my first look, it seems that all of lib/*.o is now getting linked
into vmlinux, while we traditionally leave out everything from lib/
that is not referenced.

I also see a noticeable overhead in link time, the numbers are for
a cache-hot rebuild after a successful allyesconfig build, using a
24-way Opteron@2.5Ghz, just relinking vmlinux:

$ time make skj30 vmlinux # before
real	2m8.092s
user	3m41.008s
sys	0m48.172s

$ time make skj30 vmlinux # after
real	4m10.189s
user	5m43.804s
sys	0m52.988s

That is clearly a very sharp difference. Fortunately for the defconfig
build, the times are much lower, and I see no real difference other
than the noise between subsequent runs:

$ time make skj30 vmlinux # before
real	0m5.415s
user	0m19.716s
sys	0m9.356s
$ time make skj30 vmlinux # before
real	0m9.536s
user	0m21.320s
sys	0m9.224s


$ time make skj30 vmlinux # after
real	0m5.539s
user	0m20.360s
sys	0m9.224s

$ time make skj30 vmlinux # after
real	0m9.138s
user	0m21.932s
sys	0m8.988s

$ time make skj30 vmlinux # after
real	0m5.659s
user	0m20.332s
sys	0m9.620s

	Arnd

[toc] | [prev] | [next] | [standalone]

#1456011 — Re: powerpc allyesconfig / allmodconfig linux-next next-20160729 - next-20160729 build failures

From	Arnd Bergmann <arnd@arndb.de>
Date	2016-08-03 22:20 +0200
Subject	Re: powerpc allyesconfig / allmodconfig linux-next next-20160729 - next-20160729 build failures
Message-ID	<s2aNY-7my-17@gated-at.bofh.it>
In reply to	#1455977

On Wednesday, August 3, 2016 2:44:29 PM CEST Segher Boessenkool wrote:
> Hi Arnd,
> 
> On Wed, Aug 03, 2016 at 08:52:48PM +0200, Arnd Bergmann wrote:
> > From my first look, it seems that all of lib/*.o is now getting linked
> > into vmlinux, while we traditionally leave out everything from lib/
> > that is not referenced.
> > 
> > I also see a noticeable overhead in link time, the numbers are for
> > a cache-hot rebuild after a successful allyesconfig build, using a
> > 24-way Opteron@2.5Ghz, just relinking vmlinux:
> > 
> > $ time make skj30 vmlinux # before
> > real	2m8.092s
> > user	3m41.008s
> > sys	0m48.172s
> > 
> > $ time make skj30 vmlinux # after
> > real	4m10.189s
> > user	5m43.804s
> > sys	0m52.988s
> 
> Is it better when using rcT instead of rcsT?

It seems to be noticeably better for the clean rebuild case, though
not as good as the original:

real	3m34.015s
user	5m7.104s
sys	0m49.172s

I've also tried now with my own patch applied as well (linking
each drivers/*/built-in.o into vmlinux rather than having them
linked into drivers/built-in.o first), but that makes no
difference.

	Arnd

[toc] | [prev] | [next] | [standalone]

#1460476

From	Nicholas Piggin <npiggin@gmail.com>
Date	2016-08-11 14:50 +0200
Message-ID	<s4XAR-3T5-3@gated-at.bofh.it>
In reply to	#1456011

On Wed, 03 Aug 2016 22:13:28 +0200
Arnd Bergmann <arnd@arndb.de> wrote:

> On Wednesday, August 3, 2016 2:44:29 PM CEST Segher Boessenkool wrote:
> > Hi Arnd,
> > 
> > On Wed, Aug 03, 2016 at 08:52:48PM +0200, Arnd Bergmann wrote:  
> > > From my first look, it seems that all of lib/*.o is now getting linked
> > > into vmlinux, while we traditionally leave out everything from lib/
> > > that is not referenced.
> > > 
> > > I also see a noticeable overhead in link time, the numbers are for
> > > a cache-hot rebuild after a successful allyesconfig build, using a
> > > 24-way Opteron@2.5Ghz, just relinking vmlinux:
> > > 
> > > $ time make skj30 vmlinux # before
> > > real	2m8.092s
> > > user	3m41.008s
> > > sys	0m48.172s
> > > 
> > > $ time make skj30 vmlinux # after
> > > real	4m10.189s
> > > user	5m43.804s
> > > sys	0m52.988s  
> > 
> > Is it better when using rcT instead of rcsT?  
> 
> It seems to be noticeably better for the clean rebuild case, though
> not as good as the original:
> 
> real	3m34.015s
> user	5m7.104s
> sys	0m49.172s
> 
> I've also tried now with my own patch applied as well (linking
> each drivers/*/built-in.o into vmlinux rather than having them
> linked into drivers/built-in.o first), but that makes no
> difference.

I just want to come back to this, because I've subbmitted the thin
archives kbuild patch, I wanted to make sure we're doing okay on
ARM/ARM64. I cross compiled with my laptop.

For ARM64 allyesconfig:

After building then removing all built-in.o then rebuilding vmlinux:
inclink
time make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- -j8 vmlinux
real    1m18.977s
user    2m14.512s
sys     0m29.704s

thinarc
time make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- -j8 vmlinux
real    1m18.433s
user    2m6.128s
sys     0m28.372s


Final ld time
inclink
real    0m4.005s
user    0m3.464s
sys     0m0.536s

thinarc
real    0m5.841s
user    0m4.916s
sys     0m0.916s


Build directory size is of course much better (3953MB vs 5519MB).


For ARM, defconfig

After building then removing all built-in.o then rebuilding vmlinux:
inclink
real	0m19.593s
user	0m22.372s
sys	0m6.428s

thinarc
real	0m18.919s
user	0m21.924s
sys	0m6.400s


Final ld time
inclink
real	0m0.378s
user	0m0.304s
sys	0m0.076s

thinarc
real    0m0.894s
user    0m0.684s
sys     0m0.200s

For both cases final link gets slower with thin archives. I guess there is some
per-file overhead but I thought with --whole-archive it should not be that much
slower. Still, overall time for main ar/ld phases comes out about the same in
the end so I don't think it's too much problem. Unless ARM blows up significantly
worse with a bigger config.

Linking with thin archives takes significantly more time in bfd hash lookup code.
I haven't dug much further yet.

Thanks,
Nick

[toc] | [prev] | [next] | [standalone]

#1460498 — Re: powerpc allyesconfig / allmodconfig linux-next next-20160729 - next-20160729 build failures

From	Arnd Bergmann <arnd@arndb.de>
Date	2016-08-11 15:10 +0200
Subject	Re: powerpc allyesconfig / allmodconfig linux-next next-20160729 - next-20160729 build failures
Message-ID	<s4XUd-4eY-5@gated-at.bofh.it>
In reply to	#1460476

On Thursday, August 11, 2016 10:43:20 PM CEST Nicholas Piggin wrote:
> On Wed, 03 Aug 2016 22:13:28 +0200
> Arnd Bergmann <arnd@arndb.de> wrote:
> 
> > On Wednesday, August 3, 2016 2:44:29 PM CEST Segher Boessenkool wrote:
> > > Hi Arnd,
> > > 
> > > On Wed, Aug 03, 2016 at 08:52:48PM +0200, Arnd Bergmann wrote:  
> > > > From my first look, it seems that all of lib/*.o is now getting linked
> > > > into vmlinux, while we traditionally leave out everything from lib/
> > > > that is not referenced.
> > > > 
> > > > I also see a noticeable overhead in link time, the numbers are for
> > > > a cache-hot rebuild after a successful allyesconfig build, using a
> > > > 24-way Opteron@2.5Ghz, just relinking vmlinux:
> > > > 
> > > > $ time make skj30 vmlinux # before
> > > > real	2m8.092s
> > > > user	3m41.008s
> > > > sys	0m48.172s
> > > > 
> > > > $ time make skj30 vmlinux # after
> > > > real	4m10.189s
> > > > user	5m43.804s
> > > > sys	0m52.988s  
> > > 
> > > Is it better when using rcT instead of rcsT?  
> > 
> > It seems to be noticeably better for the clean rebuild case, though
> > not as good as the original:
> > 
> > real	3m34.015s
> > user	5m7.104s
> > sys	0m49.172s
> > 
> > I've also tried now with my own patch applied as well (linking
> > each drivers/*/built-in.o into vmlinux rather than having them
> > linked into drivers/built-in.o first), but that makes no
> > difference.
> 
> I just want to come back to this, because I've subbmitted the thin
> archives kbuild patch, I wanted to make sure we're doing okay on
> ARM/ARM64. I cross compiled with my laptop.
> 
> For ARM64 allyesconfig:
> 
> After building then removing all built-in.o then rebuilding vmlinux:
> inclink
> time make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- -j8 vmlinux
> real    1m18.977s
> user    2m14.512s
> sys     0m29.704s
> 
> thinarc
> time make ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- -j8 vmlinux
> real    1m18.433s
> user    2m6.128s
> sys     0m28.372s
> 
> 
> Final ld time
> inclink
> real    0m4.005s
> user    0m3.464s
> sys     0m0.536s
> 
> thinarc
> real    0m5.841s
> user    0m4.916s
> sys     0m0.916s
> 
> 
> Build directory size is of course much better (3953MB vs 5519MB).

Ok, looks great. Some downsides and some upsides here, but overall
I think this is a win.

> 
> For ARM, defconfig
> 
> After building then removing all built-in.o then rebuilding vmlinux:
> inclink
> real	0m19.593s
> user	0m22.372s
> sys	0m6.428s
> 
> thinarc
> real	0m18.919s
> user	0m21.924s
> sys	0m6.400s
> 
> 
> Final ld time
> inclink
> real	0m0.378s
> user	0m0.304s
> sys	0m0.076s
> 
> thinarc
> real    0m0.894s
> user    0m0.684s
> sys     0m0.200s

This also still seems fine.

> For both cases final link gets slower with thin archives. I guess there is some
> per-file overhead but I thought with --whole-archive it should not be that much
> slower. Still, overall time for main ar/ld phases comes out about the same in
> the end so I don't think it's too much problem. Unless ARM blows up significantly
> worse with a bigger config.

Unfortunately I think it does. I haven't tried your latest series yet,
but I think the total time for removing built-in.o and relinking went
up from around 4 minutes (already way too much) to 18 minutes for me.

> Linking with thin archives takes significantly more time in bfd hash lookup code.
> I haven't dug much further yet.

Can you try the ARM allyesconfig with thin archives? I'll follow up with two
patches: one to get ARM to link without thin archives, and one that I used
to get --gc-sections to work.

	Arnd

[toc] | [prev] | [next] | [standalone]

#1460512

From	Nicholas Piggin <npiggin@gmail.com>
Date	2016-08-11 15:20 +0200
Message-ID	<s4Y3U-4iG-19@gated-at.bofh.it>
In reply to	#1460498

On Thu, 11 Aug 2016 15:04:00 +0200
Arnd Bergmann <arnd@arndb.de> wrote:

> On Thursday, August 11, 2016 10:43:20 PM CEST Nicholas Piggin wrote:
> > On Wed, 03 Aug 2016 22:13:28 +0200

> > Final ld time
> > inclink
> > real	0m0.378s
> > user	0m0.304s
> > sys	0m0.076s
> > 
> > thinarc
> > real    0m0.894s
> > user    0m0.684s
> > sys     0m0.200s  
> 
> This also still seems fine.
> 
> > For both cases final link gets slower with thin archives. I guess there is some
> > per-file overhead but I thought with --whole-archive it should not be that much
> > slower. Still, overall time for main ar/ld phases comes out about the same in
> > the end so I don't think it's too much problem. Unless ARM blows up significantly
> > worse with a bigger config.  
> 
> Unfortunately I think it does. I haven't tried your latest series yet,
> but I think the total time for removing built-in.o and relinking went
> up from around 4 minutes (already way too much) to 18 minutes for me.
> 
> > Linking with thin archives takes significantly more time in bfd hash lookup code.
> > I haven't dug much further yet.  
> 
> Can you try the ARM allyesconfig with thin archives? I'll follow up with two
> patches: one to get ARM to link without thin archives, and one that I used
> to get --gc-sections to work.

Okay send them over, I'll try digging into it. There is not much kbuild
code to maintain so we don't have to switch every arch. It would be nice
to though.

Thanks,
Nick

[toc] | [prev] | [next] | [standalone]

#1460539 — [TESTING] kbuild: link drivers subdirectories separately

From	Arnd Bergmann <arnd@arndb.de>
Date	2016-08-11 16:00 +0200
Subject	[TESTING] kbuild: link drivers subdirectories separately
Message-ID	<s4YGB-4ye-9@gated-at.bofh.it>
In reply to	#1460512

On ARM, relative branches between functions can not span more than 32MB,
which limits the size of an ELF section. In the final link, the linker
will introduce trampolines that perform long calls to avoid the limit,
and during a recursive link, trampolines are added within the section.

However, this does not work for cross-section branches when the source
section is already larger than 32MB because there is no longer space
to put the trampoline.

We are unable to build an allyesconfig kernel on ARM because the
.text section in drivers/built-in.o has that problem.

This patch avoids it by linking drivers/*/built-in.o directly into
vmlinux.o, rather than first linking them into drivers/built-in.o.

Signed-off-by: Arnd Bergmann <arnd@arndb.de>
---
This patch gets allyesconfig to work for me on ARM. We have previously
decided that this is too ugly, but you can use it for comparing the
link times.

diff --git a/Makefile b/Makefile
index 2eae4bab0d9b..091ca3a3015b 100644
--- a/Makefile
+++ b/Makefile
@@ -557,13 +557,6 @@ scripts: scripts_basic include/config/auto.conf include/config/tristate.conf \
 	 asm-generic gcc-plugins
 	$(Q)$(MAKE) $(build)=$(@)
 
-# Objects we will link into vmlinux / subdirs we need to visit
-init-y		:= init/
-drivers-y	:= drivers/ sound/ firmware/
-net-y		:= net/
-libs-y		:= lib/
-core-y		:= usr/
-virt-y		:= virt/
 endif # KBUILD_EXTMOD
 
 ifeq ($(dot-config),1)
@@ -584,6 +577,20 @@ $(KCONFIG_CONFIG) include/config/auto.conf.cmd: ;
 # we execute the config step to be sure to catch updated Kconfig files
 include/config/%.conf: $(KCONFIG_CONFIG) include/config/auto.conf.cmd
 	$(Q)$(MAKE) -f $(srctree)/Makefile silentoldconfig
+
+# Objects we will link into vmlinux / subdirs we need to visit
+init-y		:= init/
+net-y		:= net/
+libs-y		:= lib/
+core-y		:= usr/
+virt-y		:= virt/
+
+# split out objects from drivers to avoid recursively linking large .o files
+include drivers/Makefile
+drivers-y	:= $(addprefix drivers/,$(obj-y) $(obj-m))
+drivers-y	+= sound/ firmware/
+obj-y		:=
+
 else
 # external modules needs include/generated/autoconf.h and include/config/auto.conf
 # but do not care if they are up-to-date. Use auto.conf to trigger the test
diff --git a/drivers/Makefile b/drivers/Makefile
index 9cfa547d67ce..38848742db1f 100644
--- a/drivers/Makefile
+++ b/drivers/Makefile
@@ -95,10 +95,7 @@ obj-$(CONFIG_ATA_OVER_ETH)	+= block/aoe/
 obj-$(CONFIG_PARIDE) 		+= block/paride/
 obj-$(CONFIG_TC)		+= tc/
 obj-$(CONFIG_UWB)		+= uwb/
-obj-$(CONFIG_USB_PHY)		+= usb/
-obj-$(CONFIG_USB)		+= usb/
-obj-$(CONFIG_PCI)		+= usb/
-obj-$(CONFIG_USB_GADGET)	+= usb/
+obj-y				+= usb/
 obj-$(CONFIG_SERIO)		+= input/serio/
 obj-$(CONFIG_GAMEPORT)		+= input/gameport/
 obj-$(CONFIG_INPUT)		+= input/
@@ -137,7 +134,8 @@ obj-$(CONFIG_PPC_PS3)		+= ps3/
 obj-$(CONFIG_OF)		+= of/
 obj-$(CONFIG_SSB)		+= ssb/
 obj-$(CONFIG_BCMA)		+= bcma/
-obj-y				+= vhost/
+obj-$(CONFIG_VHOST_RING)	+= vhost/
+obj-$(CONFIG_VHOST)		+= vhost/
 obj-$(CONFIG_VLYNQ)		+= vlynq/
 obj-$(CONFIG_STAGING)		+= staging/
 obj-y				+= platform/

[toc] | [prev] | [next] | [standalone]

#1460630 — Re: [TESTING] kbuild: link drivers subdirectories separately

From	Arnd Bergmann <arnd@arndb.de>
Date	2016-08-11 17:50 +0200
Subject	Re: [TESTING] kbuild: link drivers subdirectories separately
Message-ID	<s50p4-5GR-47@gated-at.bofh.it>
In reply to	#1460539

On Thursday, August 11, 2016 3:49:03 PM CEST Arnd Bergmann wrote:
> @@ -137,7 +134,8 @@ obj-$(CONFIG_PPC_PS3)               += ps3/
>  obj-$(CONFIG_OF)               += of/
>  obj-$(CONFIG_SSB)              += ssb/
>  obj-$(CONFIG_BCMA)             += bcma/
> -obj-y                          += vhost/
> +obj-$(CONFIG_VHOST_RING)       += vhost/
> +obj-$(CONFIG_VHOST)            += vhost/
>  obj-$(CONFIG_VLYNQ)            += vlynq/
>  obj-$(CONFIG_STAGING)          += staging/
>  obj-y                          += platform/
> 

This hunk should have been the other way round to apply and work correctly,
I mixed up the number of reverts I had on my tree before it.

	Arnd

[toc] | [prev] | [next] | [standalone]

#1456021 — Re: powerpc allyesconfig / allmodconfig linux-next next-20160729 - next-20160729 build failures

From	Segher Boessenkool <segher@kernel.crashing.org>
Date	2016-08-03 22:50 +0200
Subject	Re: powerpc allyesconfig / allmodconfig linux-next next-20160729 - next-20160729 build failures
Message-ID	<s2aNY-7my-19@gated-at.bofh.it>
In reply to	#1455977

Hi Arnd,

On Wed, Aug 03, 2016 at 08:52:48PM +0200, Arnd Bergmann wrote:
> From my first look, it seems that all of lib/*.o is now getting linked
> into vmlinux, while we traditionally leave out everything from lib/
> that is not referenced.
> 
> I also see a noticeable overhead in link time, the numbers are for
> a cache-hot rebuild after a successful allyesconfig build, using a
> 24-way Opteron@2.5Ghz, just relinking vmlinux:
> 
> $ time make skj30 vmlinux # before
> real	2m8.092s
> user	3m41.008s
> sys	0m48.172s
> 
> $ time make skj30 vmlinux # after
> real	4m10.189s
> user	5m43.804s
> sys	0m52.988s

Is it better when using rcT instead of rcsT?


Segher

[toc] | [prev] | [next] | [standalone]

#1456079

From	Stephen Rothwell <sfr@canb.auug.org.au>
Date	2016-08-04 02:20 +0200
Message-ID	<s2eye-1j2-5@gated-at.bofh.it>
In reply to	#1455977

Hi Arnd,

On Wed, 03 Aug 2016 20:52:48 +0200 Arnd Bergmann <arnd@arndb.de> wrote:
>
> Most of the difference appears to be in branch trampolines (634 added,
> 559 removed, 14837 unchanged) as you suspect, but I also see a couple
> of symbols show up in vmlinux that were not there before:
> 
> -A __crc_dma_noop_ops
> -D dma_noop_ops
> -R __clz_tab
> -r fdt_errtable
> -r __kcrctab_dma_noop_ops
> -r __kstrtab_dma_noop_ops
> -R __ksymtab_dma_noop_ops
> -t dma_noop_alloc
> -t dma_noop_free
> -t dma_noop_map_page
> -t dma_noop_mapping_error
> -t dma_noop_map_sg
> -t dma_noop_supported
> -T fdt_add_reservemap_entry
> -T fdt_begin_node
> -T fdt_create
> -T fdt_create_empty_tree
> -T fdt_end_node
> -T fdt_finish
> -T fdt_finish_reservemap
> -T fdt_property
> -T fdt_resize
> -T fdt_strerror
> -T find_cpio_data
> 
> From my first look, it seems that all of lib/*.o is now getting linked
> into vmlinux, while we traditionally leave out everything from lib/
> that is not referenced.

You could try removing the --{,no-}whole-archive arguments to ld in
scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh.  Last time I did
that, though, a whole lot of stuff failed to be linked in. (Especially
stuff only referenced by EXPORT_SYMBOL()s, bu that may have been fixed).

> I also see a noticeable overhead in link time, the numbers are for
> a cache-hot rebuild after a successful allyesconfig build, using a
> 24-way Opteron@2.5Ghz, just relinking vmlinux:

I was afraid of that, but it is offset by the time saved by not doing
the "ld -r"s along the way?  It may also be that (for powerpc anyway)
the linker is doing a better job.

-- 
Cheers,
Stephen Rothwell

[toc] | [prev] | [next] | [standalone]

#1456228 — Re: powerpc allyesconfig / allmodconfig linux-next next-20160729 - next-20160729 build failures

From	Arnd Bergmann <arnd@arndb.de>
Date	2016-08-04 11:10 +0200
Subject	Re: powerpc allyesconfig / allmodconfig linux-next next-20160729 - next-20160729 build failures
Message-ID	<s2mP8-6St-15@gated-at.bofh.it>
In reply to	#1456079

On Thursday, August 4, 2016 10:10:51 AM CEST Stephen Rothwell wrote:
> Hi Arnd,
> 
> On Wed, 03 Aug 2016 20:52:48 +0200 Arnd Bergmann <arnd@arndb.de> wrote:
> >
> > Most of the difference appears to be in branch trampolines (634 added,
> > 559 removed, 14837 unchanged) as you suspect, but I also see a couple
> > of symbols show up in vmlinux that were not there before:
> > 
> > -A __crc_dma_noop_ops
> > -D dma_noop_ops
> > -R __clz_tab
> > -r fdt_errtable
> > -r __kcrctab_dma_noop_ops
> > -r __kstrtab_dma_noop_ops
> > -R __ksymtab_dma_noop_ops
> > -t dma_noop_alloc
> > -t dma_noop_free
> > -t dma_noop_map_page
> > -t dma_noop_mapping_error
> > -t dma_noop_map_sg
> > -t dma_noop_supported
> > -T fdt_add_reservemap_entry
> > -T fdt_begin_node
> > -T fdt_create
> > -T fdt_create_empty_tree
> > -T fdt_end_node
> > -T fdt_finish
> > -T fdt_finish_reservemap
> > -T fdt_property
> > -T fdt_resize
> > -T fdt_strerror
> > -T find_cpio_data
> > 
> > From my first look, it seems that all of lib/*.o is now getting linked
> > into vmlinux, while we traditionally leave out everything from lib/
> > that is not referenced.
> 
> You could try removing the --{,no-}whole-archive arguments to ld in
> scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh.  Last time I did
> that, though, a whole lot of stuff failed to be linked in. (Especially
> stuff only referenced by EXPORT_SYMBOL()s, bu that may have been fixed).

I tried this

diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh
index b5e40ed86e60..89bca1a25916 100755
--- a/scripts/link-vmlinux.sh
+++ b/scripts/link-vmlinux.sh
@@ -44,7 +44,7 @@ modpost_link()
 	local objects
 
 	if [ -n "${CONFIG_THIN_ARCHIVES}" ]; then
-		objects="--whole-archive ${KBUILD_VMLINUX_INIT} ${KBUILD_VMLINUX_MAIN} --no-whole-archive"
+		objects="${KBUILD_VMLINUX_INIT} ${KBUILD_VMLINUX_MAIN}"
 	else
 		objects="${KBUILD_VMLINUX_INIT} --start-group ${KBUILD_VMLINUX_MAIN} --end-group"
 	fi

but that did not seem to change anything, the extra symbols are
still there. I have not tried to understand what that actually
does, so maybe I misunderstood your suggestion.

> > I also see a noticeable overhead in link time, the numbers are for
> > a cache-hot rebuild after a successful allyesconfig build, using a
> > 24-way Opteron@2.5Ghz, just relinking vmlinux:
> 
> I was afraid of that, but it is offset by the time saved by not doing
> the "ld -r"s along the way?  It may also be that (for powerpc anyway)
> the linker is doing a better job.

At least on a big SMP system, it doesn't seem to make much difference,
as the "ld -r" steps are easily parallized

$ find build/ -name built-in.o | xargs rm ; time make -skj30 vmlinux
real	2m12.092s
user	3m52.932s
sys	0m51.248s

$ time make -skj30 vmlinux
real	2m12.162s
user	3m44.788s
sys	0m47.788s

I tried this twice with identical results: "user" time increases
by eight seconds today when we have to rebuild all "built-in.o"
files rather than just relinking vmlinux, but elapsed time
is unchanged.

After your patch that difference becomes smaller (three seconds
in one run, could be within the noise), but we still have the
extra two minutes for the total build time:

$ find build/ -name built-in.o | xargs rm ; time make -skj30 vmlinux
real	4m20.717s
user	5m47.556s
sys	0m54.128s

$ time make -skj30 vmlinux
real	4m18.835s
user	5m44.552s
sys	0m53.152s

FWIW, here is a sample build output I get on an allyesconfig build,
with timestamps added:

$ time make W= -kj30 vmlinux 
make[1]: Entering directory '/git/arm-soc'
make[2]: Entering directory '/git/arm-soc/build/tmp'
10:46:12   CHK     include/config/kernel.release
10:46:13   GEN     ./Makefile
10:46:13   CHK     include/generated/uapi/linux/version.h
  Using /git/arm-soc as source for kernel
10:46:13   CHK     include/generated/utsrelease.h
10:46:13   CHK     include/generated/timeconst.h
10:46:13   CHK     include/generated/bounds.h
10:46:13   CHK     include/generated/asm-offsets.h
10:46:13   CALL    /git/arm-soc/scripts/checksyscalls.sh
10:46:14   CHK     include/generated/compile.h
10:46:18   CHK     kernel/config_data.h
10:46:20   CC      drivers/misc/lkdtm_rodata.o
10:46:20   OBJCOPY drivers/misc/lkdtm_rodata_objcopy.o
10:46:20   LD      drivers/misc/lkdtm.o
10:46:20   LD      drivers/misc/built-in.o
10:46:20   DTC     drivers/gpu/drm/tilcdc/tilcdc_slave_compat.dtb
10:46:20   DTB     drivers/gpu/drm/tilcdc/tilcdc_slave_compat.dtb.S
10:46:20   AS      drivers/gpu/drm/tilcdc/tilcdc_slave_compat.dtb.o
10:46:20   LD      drivers/gpu/drm/tilcdc/built-in.o
rm drivers/gpu/drm/tilcdc/tilcdc_slave_compat.dtb.S drivers/gpu/drm/tilcdc/tilcdc_slave_compat.dtb
10:46:33   LD      drivers/gpu/drm/built-in.o
10:46:33   LD      drivers/gpu/built-in.o
10:46:36   CHK     include/generated/uapi/linux/version.h
10:46:36   LINK    vmlinux
10:46:37   LD      vmlinux.o
10:47:14   MODPOST vmlinux.o
10:47:16   GEN     .version
10:47:17   CHK     include/generated/compile.h
10:47:17   UPD     include/generated/compile.h
10:47:17   CC      init/version.o
10:47:17   LD      init/built-in.o
10:48:09   KSYM    .tmp_kallsyms1.o
10:49:19   KSYM    .tmp_kallsyms2.o
10:49:33   LD      vmlinux
10:50:27   SORTEX  vmlinux
10:50:27   SYSMAP  System.map
make[2]: Leaving directory '/git/arm-soc/build/tmp'
make[1]: Leaving directory '/git/arm-soc'

real	4m18.033s
user	5m44.728s
sys	0m52.724s

(yes, I also just realized we should fix the tilcdc and lkdtm drivers
to not force a rebuild).

	Arnd

[toc] | [prev] | [next] | [standalone]

#1456324 — Re: powerpc allyesconfig / allmodconfig linux-next next-20160729 - next-20160729 build failures

From	Arnd Bergmann <arnd@arndb.de>
Date	2016-08-04 12:50 +0200
Subject	Re: powerpc allyesconfig / allmodconfig linux-next next-20160729 - next-20160729 build failures
Message-ID	<s2onT-7Kf-9@gated-at.bofh.it>
In reply to	#1456228

On Thursday, August 4, 2016 11:00:49 AM CEST Arnd Bergmann wrote:
> I tried this
> 
> diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh
> index b5e40ed86e60..89bca1a25916 100755
> --- a/scripts/link-vmlinux.sh
> +++ b/scripts/link-vmlinux.sh
> @@ -44,7 +44,7 @@ modpost_link()
>         local objects
>  
>         if [ -n "${CONFIG_THIN_ARCHIVES}" ]; then
> -               objects="--whole-archive ${KBUILD_VMLINUX_INIT} ${KBUILD_VMLINUX_MAIN} --no-whole-archive"
> +               objects="${KBUILD_VMLINUX_INIT} ${KBUILD_VMLINUX_MAIN}"
>         else
>                 objects="${KBUILD_VMLINUX_INIT} --start-group ${KBUILD_VMLINUX_MAIN} --end-group"
>         fi
> 
> but that did not seem to change anything, the extra symbols are
> still there. I have not tried to understand what that actually
> does, so maybe I misunderstood your suggestion.
> 

On a second attempt, I did the same change for vmlinux instead of the
module (d'oh), and got a link failure instead:

arch/arm/mm/proc-xscale.o: In function `cpu_xscale_do_resume':
(.text+0x3d4): undefined reference to `cpu_resume_mmu'
arch/arm/kernel/setup.o: In function `setup_arch':
setup.c:(.init.text+0x910): undefined reference to `init_uts_ns'
kernel/nsproxy.o:(.data+0x4): undefined reference to `init_uts_ns'
kernel/sched/core.o: In function `update_rq_clock':
core.c:(.text+0x6d8): undefined reference to `paravirt_steal_rq_enabled'
core.c:(.text+0x6dc): undefined reference to `pv_time_ops'
kernel/sched/cputime.o: In function `account_process_tick':
cputime.c:(.text+0x794): undefined reference to `paravirt_steal_enabled'
cputime.c:(.text+0x7a0): undefined reference to `pv_time_ops'
kernel/locking/lockdep.o: In function `save_trace':
lockdep.c:(.text+0xfe8): undefined reference to `save_stack_trace'
kernel/module.o: In function `load_module':
module.c:(.text+0x1b54): undefined reference to `elf_check_arch'
module.c:(.text+0x2024): undefined reference to `apply_relocate'
kernel/debug/debug_core.o: In function `kgdb_unregister_io_module':
debug_core.c:(.text+0x2e4): undefined reference to `kgdb_arch_exit'
kernel/debug/debug_core.o: In function `kgdb_arch_set_breakpoint':
debug_core.c:(.text+0x3bc): undefined reference to `arch_kgdb_ops'
kernel/debug/debug_core.o: In function `dbg_remove_all_break':
debug_core.c:(.text+0x6d0): undefined reference to `arch_kgdb_ops'
...

However, I also see a link failure in some rare configurations
with just your patch:

arch/arm/lib/lib.a(io-acorn.o): In function `outsl':
(.text+0x38): undefined reference to `printk'

The problem being a file in a library object that is not referenced,
but that references another symbol that is not defined
(CONFIG_PRINTK=n).

	Arnd

[toc] | [prev] | [next] | [standalone]

Page 1 of 2 [1] 2 Next page →

csiph-web

powerpc allyesconfig / allmodconfig linux-next next-20160729 - next-20160729 build failures

Contents

#1455443 — powerpc allyesconfig / allmodconfig linux-next next-20160729 - next-20160729 build failures

#1455496

#1455506

#1455514 — Re: powerpc allyesconfig / allmodconfig linux-next next-20160729 - next-20160729 build failures

#1455542

#1455671 — Re: powerpc allyesconfig / allmodconfig linux-next next-20160729 - next-20160729 build failures

#1455778

#1455784 — Re: powerpc allyesconfig / allmodconfig linux-next next-20160729 - next-20160729 build failures

#1455873

#1455977 — Re: powerpc allyesconfig / allmodconfig linux-next next-20160729 - next-20160729 build failures

#1456011 — Re: powerpc allyesconfig / allmodconfig linux-next next-20160729 - next-20160729 build failures

#1460476

#1460498 — Re: powerpc allyesconfig / allmodconfig linux-next next-20160729 - next-20160729 build failures

#1460512

#1460539 — [TESTING] kbuild: link drivers subdirectories separately

#1460630 — Re: [TESTING] kbuild: link drivers subdirectories separately

#1456021 — Re: powerpc allyesconfig / allmodconfig linux-next next-20160729 - next-20160729 build failures

#1456079

#1456228 — Re: powerpc allyesconfig / allmodconfig linux-next next-20160729 - next-20160729 build failures

#1456324 — Re: powerpc allyesconfig / allmodconfig linux-next next-20160729 - next-20160729 build failures