Path: csiph.com!fu-berlin.de!bofh.it!news.nic.it!robomod From: Soren Stoutner Newsgroups: linux.debian.maint.python Subject: Re: use of waf in pyinstaller (was: blhc) Date: Sat, 07 Dec 2024 18:40:01 +0100 Message-ID: References: X-Original-To: debian-python@lists.debian.org X-Mailbox-Line: From debian-python-request@lists.debian.org Sat Dec 7 17:30:00 2024 Old-Return-Path: X-Amavis-Spam-Status: No, score=-114.749 tagged_above=-10000 required=5.3 tests=[BAYES_00=-2, DKIMWL_WL_HIGH=-0.34, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FOURLA=0.1, LDO_WHITELIST=-5, PGPSIGNATURE=-5, RCVD_IN_DNSWL_MED=-2.3, UNPARSEABLE_RELAY=0.001, USER_IN_DKIM_WELCOMELIST=-0.01, USER_IN_DKIM_WHITELIST=-100] autolearn=ham autolearn_force=no Organization: Debian MIME-Version: 1.0 Content-Type: multipart/signed; boundary="nextPart2645065.dOs2k3PN8f"; micalg="pgp-sha512"; protocol="application/pgp-signature" X-Debian-User: soren X-Mailing-List: archive/latest/22640 List-ID: List-URL: List-Archive: https://lists.debian.org/msgid-search/6992686.2H1T9oXYi1@soren-desktop Approved: robomod@news.nic.it Lines: 156 Sender: robomod@news.nic.it X-Original-Cc: Simon McVittie X-Original-Date: Sat, 07 Dec 2024 10:29:39 -0700 X-Original-Message-ID: <6992686.2H1T9oXYi1@soren-desktop> X-Original-References: <7225104.MRShDI54Kn@soren-desktop> Xref: csiph.com linux.debian.maint.python:16552 --nextPart2645065.dOs2k3PN8f Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="UTF-8"; protected-headers="v1" From: Soren Stoutner To: debian-python@lists.debian.org Cc: Simon McVittie Subject: Re: use of waf in pyinstaller (was: blhc) Date: Sat, 07 Dec 2024 10:29:39 -0700 Message-ID: <6992686.2H1T9oXYi1@soren-desktop> Organization: Debian In-Reply-To: MIME-Version: 1.0 Simon, Thank you for taking the time to look at this. On Friday, December 6, 2024 3:27:55 AM MST Simon McVittie wrote: > On Thu, 05 Dec 2024 at 19:00:50 -0700, Soren Stoutner wrote: > > I am working on PyInstaller, which is mostly written in Python, but=20 compiles > > a bootloader written in c. blhc failes because the [logs] do not conta= in > > verbose compile flags. >=20 > You'll need to look at the implementation of the build for the C part, and > then do whatever is most appropriate for that build system. >=20 > >From a quick glance at setup.py, it seems to be (a vendored copy of) waf: > additional_args =3D os.getenv('PYINSTALLER_BOOTLOADER_WAF_ARGS', > '').strip().split() cmd =3D [sys.executable, './waf', 'configure', 'all'] > cmd +=3D additional_args >=20 > so hopefully there is something you can add to > PYINSTALLER_BOOTLOADER_WAF_ARGS that would make waf verbose, analogous > to `ninja -v` or Autotools `V=3D1`? This was a very helpful suggestion. I was able to produce a verbose build = by=20 adding the following to debian/rules: # Enable the verbose waf build argument so that blhc can analyze the build= =20 flags. waf is the system that builds the bootloader from C code. export PYINSTALLER_BOOTLOADER_WAF_ARGS =3D --verbose > After that, you'll also need to make sure that the intended build > options are actually used (I don't know whether waf uses CFLAGS, etc. by > default or has to be given them via waf-specific command-line options). > Looking at other packages that use a waf build system and implement build > flags correctly, if any such packages exist, will probably be useful. The above verbose flag then produces this output in the build logs: [ 1/21] Compiling src/pyi_utils.c 00:07:52 runner ['/usr/lib/ccache/gcc', '-g', '-O2', '-Werror=3Dimplicit- function-declaration', '-ffile-prefix-map=3D/builds/python-team/packages/ pyinstaller/debian/output/source_dir=3D.', '-fstack-protector-strong', '-fs= tack- clash-protection', '-Wformat', '-Werror=3Dformat-security', '-fcf-protectio= n',=20 '-Wdate-time', '-D_FORTIFY_SOURCE=3D2', '-m64', '-O2', '-Wall', '-Werror', = '- Wno-error=3Dunused-variable', '-Wno-error=3Dunused-function', '-Wno-error= =3Dunused- but-set-variable', '-U_FORTIFY_SOURCE', '-Isrc', '-I../../src', '-Iwindows'= ,=20 '-I../../windows', '-Izlib', '-I../../zlib', '-D_REENTRANT', '-D_BSD_SOURCE= ',=20 '-D_DEFAULT_SOURCE', '-D_FORTIFY_SOURCE=3D2', '-DHAVE_STDBOOL_H=3D1', '- DHAVE_UNSETENV=3D1', '-DHAVE_MKDTEMP=3D1', '-DHAVE_DIRNAME=3D1', '- DHAVE_BASENAME=3D1', '-DLAUNCH_DEBUG', '-DNDEBUG', '../../src/pyi_utils.c',= '- c', '-o/builds/python-team/packages/pyinstaller/debian/output/source_dir/ bootloader/build/debug/src/pyi_utils.c.1.o', '-Wdate-time', =E2=80=98- D_FORTIFY_SOURCE=3D2=E2=80=99] Blhc still reports the above as a NONVERBOSE build because there is a line= =20 break, so the first line is flagged separate from the second line. It turn= s out=20 there is an existing blhc bug report for this, which I have added some=20 addition information to. https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=3D976175 > We have had problems with waf in the past, both technical and social > (licensing-related and others), so please be careful to ensure that > this package meets Debian's quality standards and doesn't contain any > particularly attractive places to hide malware. >=20 > In particular, the recommended way to distribute waf-built code used to > be to vendor a generated script containing a bzip2-compressed tarball, > which is not straightforward to review or patch, and the ftp team does > not consider this to be acceptable in Debian [1]. Is this still the case, > or is PyInstaller redistributing waf as reviewable/patchable files in > something more closely resembling their preferred form for modification? >=20 > Has the maintainer of this package (possibly you, I don't know this > package's history) verified that the included copy of waf is something > that we can trust? From the fact that you didn't already know this > package is using waf, I would guess perhaps not? I have not had any experience with waf before, and so am not aware of DFSG = or=20 malware difficulties that other projects have faced. In the case of=20 PyInstaller, most of the waf code is contained in: https://salsa.debian.org/python-team/packages/pyinstaller/-/tree/debian/ master/bootloader/waflib?ref_type=3Dheads It is written in Python and licensed under the BSD-3-clause. It is used to= =20 compile the C code in: https://salsa.debian.org/python-team/packages/pyinstaller/-/tree/debian/ master/bootloader/src?ref_type=3Dheads Which is licensed under the GPL-2+~with-bootloader-exception, which is the= =20 main license of the project. The resulting bootloader (two files) is shipp= ed=20 in the binary package in /usr/lib/python3/dist-packages/PyInstaller/ bootloader/Linux-64bit-intel/*. None of this looks problematic to me. However, if there are any concerns I= =20 have missed I would be very interested to hear of them before I submit=20 PyInstaller to the NEW queue. =2D-=20 Soren Stoutner soren@debian.org --nextPart2645065.dOs2k3PN8f Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part. Content-Transfer-Encoding: 7Bit -----BEGIN PGP SIGNATURE----- iQIzBAABCgAdFiEEJKVN2yNUZnlcqOI+wufLJ66wtgMFAmdUhgMACgkQwufLJ66w tgNkpg/+MFkl2+ojCOZoNfaFxa5xf76vQ6SsPMq8YI+ua/mP1VVTP0YPcykQQPpp NcxnL8XoL+74dCHqCL4PAi0RAdxibXOXQ11RcO7A8Ml4iqYfjPjy4Fsq7WBSv2Nf 84o3108IKRBCqNSl+X+vy+ojHMVuYKzx4SFQ5xBngL4m1Vq83hbMAlPAqSclCU1c V8+NfZKlWX8XWq9yhoGnWxiplf7B8ybAGDmXGnPYVYw9CSBpbJrGoVDnbUPitCrW SvpkygrsF2uKWlJdDWyUuhFvOUBOcgRhKwtxQIQFr7o9DltsoyZ7MEj/D5EtygLS qlp/E5m65DeuRdo/lhci9ycl/jCbium18eWjmRU0jPOxmRMYM16z7YnzUr+nJw7r SaoyzrKsZKrLIYv9CTKpN5rHG/6s0HhrwdUDxmoXOMLeZJNE9+g5hH22dWH4wC1G GhUy4JeqiTLzQ8MhRSiZeWfLSky0ryNyMSvy945CxNSxrHZX7sCnyTF35Lam819Y 71l+3/by+7zzxmVNntF+j0Ke6dNCmcZbrcO5aj1GWaUo16ZS2tVuJ4kNcXoX0Czq 9EWD2vwYY7YTRbcAyO8pOm7ZqE/mZX1LzLB03/cVSP/lRTw2fRbUqMRXJUNvmWcP heoeQP5B+whnP9Q9OK3IKt8WquDDqX2Xgo2go/t+BTKqFa2Bfhw= =Aen+ -----END PGP SIGNATURE----- --nextPart2645065.dOs2k3PN8f--