Groups > linux.debian.maint.python > #12724 > unrolled thread

Python3 -dbg packages

Started by	Matthias Klose <doko@debian.org>
First post	2020-07-06 20:40 +0200
Last post	2021-09-15 15:30 +0200
Articles	7 — 4 participants

Back to article view | Back to linux.debian.maint.python

  Python3 -dbg packages Matthias Klose <doko@debian.org> - 2020-07-06 20:40 +0200
    Re: Python3 -dbg packages Dmitry Shachnev <mitya57@debian.org> - 2020-07-07 10:50 +0200
    Re: Python3 -dbg packages Matthias Klose <doko@debian.org> - 2021-09-03 11:00 +0200
      Re: Python3 -dbg packages Scott Talbert <swt@techie.net> - 2021-09-13 16:20 +0200
        Re: Python3 -dbg packages Matthias Klose <doko@debian.org> - 2021-09-14 08:40 +0200
          Re: Python3 -dbg packages Matthias Klose <doko@debian.org> - 2021-09-15 12:50 +0200
            Re: Python3 -dbg packages Sandro Tosi <morph@debian.org> - 2021-09-15 15:30 +0200

#12724 — Python3 -dbg packages

From	Matthias Klose <doko@debian.org>
Date	2020-07-06 20:40 +0200
Subject	Python3 -dbg packages
Message-ID	<ApEc2-7AK-31@gated-at.bofh.it>

Python 3.8 upstream now has a common ABI for normal and debug extension builds,
so it is technically possible to load a debug extension in the normal
interpreter, or to load a normal extension in the debug interpreter.  In Debian,
debug extensions are shipped with a different name, and only loaded by the
corresponding interpreter.  We could change / simply the current setup, but I
first wanted to know how many people are still using the debug builds.  The
reason for the separate debug builds allowed debugging of stuff in modules
further down the Python stack, without having to rebuild the whole stack. There
are several solutions how to simplify the packaging, I'm not sure how much the
dbg extensions are still used ... There are several scenarios:

 - Keep the current setup (-dbg packages need to be available to
   run them).

 - Allow the debug interpreter to load normal debug extensions (but
   load a debug extension if it's available by default).  That would
   allow building debug extensions without having debug extensions
   built for all it's dependencies, maybe requiring changes in the
   dependencies of a package.

 - Stop building debug extensions, and telling developers to
   build extensions in debug mode, if they need them.  That would
   probably be inline with everything else shipped in Debian.

 - Stop building debug extensions, and also stop building the Python
   debug interpreter.  You would need to rebuild the interpreter
   itself to have meaningful debug sessions.  I'm not preferring
   this solution.

I'm currently tending to implement the second scenario, but if people think that
having the -dbg packages available is still useful, then also opt for the third
option.

Matthias

[toc] | [next] | [standalone]

#12730

From	Dmitry Shachnev <mitya57@debian.org>
Date	2020-07-07 10:50 +0200
Message-ID	<ApRsB-7iL-3@gated-at.bofh.it>
In reply to	#12724

[Multipart message — attachments visible in raw view] — view raw

Hi Matthias!

On Mon, Jul 06, 2020 at 08:33:39PM +0200, Matthias Klose wrote:
> Python 3.8 upstream now has a common ABI for normal and debug extension
> builds, so it is technically possible to load a debug extension in
> the normal interpreter, or to load a normal extension in the debug
> interpreter.

Packages from the PyQt5 stack (pyqt5, pyqt5chart, pyqt5webengine, qscintilla2)
have recently started to use PEP 384 limited ABI (the built files are named
*.abi3.so). However, when a debug interpreter is used for build, the build
system disables this feature and builds a normal *.cpython-38d-*.so file:

https://riverbankcomputing.com/hg/sip/file/6169324910f8/sipbuild/buildable.py#l60

Because of this, I decided to still build -dbg versions for these packages.

Do you know how much the -dbg interpreter is compatible with *.abi3.so files?
I just checked, and apparently it can import them, but maybe there is some
benefit in keeping the normal debug versions too?

--
Dmitry Shachnev

[toc] | [prev] | [next] | [standalone]

#13504

From	Matthias Klose <doko@debian.org>
Date	2021-09-03 11:00 +0200
Message-ID	<CTddf-2Yt-1@gated-at.bofh.it>
In reply to	#12724

On 7/6/20 8:33 PM, Matthias Klose wrote:
> Python 3.8 upstream now has a common ABI for normal and debug extension builds,
> so it is technically possible to load a debug extension in the normal
> interpreter, or to load a normal extension in the debug interpreter.  In Debian,
> debug extensions are shipped with a different name, and only loaded by the
> corresponding interpreter.  We could change / simply the current setup, but I
> first wanted to know how many people are still using the debug builds.  The
> reason for the separate debug builds allowed debugging of stuff in modules
> further down the Python stack, without having to rebuild the whole stack. There
> are several solutions how to simplify the packaging, I'm not sure how much the
> dbg extensions are still used ... There are several scenarios:
> 
>  - Keep the current setup (-dbg packages need to be available to
>    run them).
> 
>  - Allow the debug interpreter to load normal debug extensions (but
>    load a debug extension if it's available by default).  That would
>    allow building debug extensions without having debug extensions
>    built for all it's dependencies, maybe requiring changes in the
>    dependencies of a package.
> 
>  - Stop building debug extensions, and telling developers to
>    build extensions in debug mode, if they need them.  That would
>    probably be inline with everything else shipped in Debian.
> 
>  - Stop building debug extensions, and also stop building the Python
>    debug interpreter.  You would need to rebuild the interpreter
>    itself to have meaningful debug sessions.  I'm not preferring
>    this solution.
> 
> I'm currently tending to implement the second scenario, but if people think that
> having the -dbg packages available is still useful, then also opt for the third
> option.

Let's address this before we start adding Python 3.10 as a supported Python3
version. Starting with the third option.  I'll file bug reports for the
following packages:

basemap
bottleneck
cbflib
dbus-python
gpyfft
gst-python1.0
h5py
kiwisolver
libgpuarray
libkdtree++
libtorrent-rasterbar
libxml2
lxml
markupsafe
matplotlib
meliae
mpi4py
netifaces
nipy
numexpr
numpy
pairtools
pillow
pillow-sane
psycopg2
pyao
pycairo
pychm
pycuda
pycurl
pyepr
pyfai
pyfuse3
pygccjit
pygobject
pyicu
pymad
pymca
pynfft
pyopencl
pyqt5
pyqt5chart
pyqt5-sip
pyqt5webengine
pysendfile
pystemmer
pytables
python3-defaults
python3-stdlib-extensions
python-aiohttp
python-apsw
python-apt
python-bsddb3
python-cffi
python-djvulibre
python-dmidecode
python-fabio
python-fisx
python-gevent
python-greenlet
python-ldap
python-levenshtein
python-librtmp
python-llfuse
python-ltfatpy
python-multidict
python-mysqldb
python-psutil
python-pygraphviz
python-pylibacl
python-pyxattr
python-regex
python-reportlab
python-setproctitle
python-sfml
pyyaml
pyzmq
qscintilla2
reprozip
rrdtool
scipy
silx
simplejson
sip4
sip5
sip6
storm
thrift
twisted
uvloop
xrayutilities
zope.interface


dd-list:

Alexander Wirt <formorer@debian.org>
   rrdtool (U)

Alexandre Marie <alexandre.marie@synchrotron-soleil.fr>
   python-fisx (U)
   silx (U)
   xrayutilities (U)

Andreas Beckmann <anbe@debian.org>
   pycuda (U)
   pyopencl (U)

Andrew Starr-Bochicchio <asb@debian.org>
   libtorrent-rasterbar (U)

Antoni Villalonga <antoni@friki.cat>
   pairtools (U)

Antonio Valentino <antonio.valentino@tiscali.it>
   numexpr (U)
   pyepr (U)
   pytables (U)
   python-ltfatpy (U)

APT Development Team <deity@lists.debian.org>
   python-apt

Aron Xu <aron@debian.org>
   libxml2 (U)

Brian May <bam@debian.org>
   python-mysqldb (U)

Christoph Berg <myon@debian.org>
   psycopg2 (U)

Colin Watson <cjwatson@debian.org>
   storm (U)

Cristian Greco <cristian@debian.org>
   libtorrent-rasterbar

Dave Beckett <dajobe@debian.org>
   pycairo (U)

David Cournapeau <cournape@gmail.com>
   scipy (U)

Debian Games Team <pkg-games-devel@lists.alioth.debian.org>
   python-sfml

Debian GIS Project <pkg-grass-devel@lists.alioth.debian.org>
   pyepr

Debian GNOME Maintainers <pkg-gnome-maintainers@lists.alioth.debian.org>
   pygobject

Debian Med Packaging Team <debian-med-packaging@lists.alioth.debian.org>
   nipy
   pairtools

Debian NVIDIA Maintainers <pkg-nvidia-devel@lists.alioth.debian.org>
   pycuda

Debian OpenCL Maintainers <pkg-opencl-devel@lists.alioth.debian.org>
   pyopencl

Debian Python Modules Team <python-modules-team@lists.alioth.debian.org>
   bottleneck
   kiwisolver (U)
   markupsafe (U)
   netifaces
   pycairo
   pychm (U)
   pyicu (U)
   pyqt5-sip
   pysendfile (U)
   python-dmidecode (U)
   python-ldap
   python-multidict (U)
   python-mysqldb
   python-regex (U)

Debian Python Team <team+python@tracker.debian.org>
   basemap (U)
   matplotlib (U)
   numpy (U)
   psycopg2
   pycurl
   pyfuse3
   pyqt5
   pyqt5chart
   pyqt5webengine
   pystemmer
   python-aiohttp
   python-cffi
   python-fisx
   python-levenshtein (U)
   python-llfuse (U)
   python-psutil (U)
   python-pygraphviz (U)
   python-setproctitle
   pyyaml
   pyzmq
   qscintilla2
   scipy
   simplejson
   sip4
   sip5
   sip6
   storm
   twisted
   uvloop (U)
   zope.interface

Debian QA Group <packages@qa.debian.org>
   libkdtree++
   python-bsddb3
   python-djvulibre

Debian RRDtool Team <pkg-rrdtool-maint@lists.alioth.debian.org>
   rrdtool

Debian Science Maintainers <debian-science-maintainers@lists.alioth.debian.org>
   cbflib
   gpyfft
   h5py
   libgpuarray
   mpi4py
   numexpr
   pyfai
   pymca
   pynfft
   pytables
   python-fabio
   python-ltfatpy
   reprozip
   silx
   xrayutilities

Debian XML/SGML Group <debian-xml-sgml-pkgs@lists.alioth.debian.org>
   libxml2

Dmitry Shachnev <mitya57@debian.org>
   pyqt5 (U)
   pyqt5-sip (U)
   pyqt5webengine (U)
   sip4 (U)
   sip5 (U)
   sip6 (U)

Eugen Wintersberger <eugen.wintersberger@gmail.com>
   xrayutilities (U)

Fabio Tranchitella <kobold@debian.org>
   psycopg2 (U)

Francesco Paolo Lovergine <frankie@debian.org>
   pyfuse3 (U)

Ghe Rivero <ghe@debian.org>
   pysendfile

Ghislain Antony Vaillant <ghisvail@gmail.com>
   bottleneck (U)
   h5py (U)
   libgpuarray (U)
   pynfft (U)
   reprozip (U)

Gordon Ball <gordon@chronitis.net>
   python-setproctitle (U)

Gudjon I. Gudjonsson <gudjon@gudjon.org>
   qscintilla2 (U)

Iain Lane <laney@debian.org>
   pygobject (U)

Iustin Pop <iustin@debian.org>
   python-pylibacl
   python-pyxattr

James Cowgill <jcowgill@debian.org>
   python-sfml (U)

Jamie Wilkinson <jaq@debian.org>
   pyao
   pymad

Jean-Michel Vourgère <nirgal@debian.org>
   rrdtool (U)

Jelmer Vernooĳ <jelmer@debian.org>
   meliae

Jeremy Bicha <jbicha@debian.org>
   pygobject (U)

Jerome Kieffer <jerome.kieffer@esrf.fr>
   pyfai (U)
   python-fabio (U)
   silx (U)

Joel Rosdahl <joel@debian.org>
   python-apsw

Jonas Meurer <mejo@debian.org>
   python-mysqldb (U)

Julian Andres Klode <jak@debian.org>
   python-apt (U)

Julian Taylor <jtaylor.debian@googlemail.com>
   pyzmq (U)

Laszlo Boszormenyi (GCS) <gcs@debian.org>
   pyicu
   python-gevent
   python-greenlet
   pyzmq (U)
   thrift

Laurent Bigonville <bigon@debian.org>
   pygobject (U)

Loic Minier <lool@dooz.org>
   dbus-python (U)

Maintainers of GStreamer packages <gst-python1.0@packages.debian.org>
   gst-python1.0

Mario Izquierdo (mariodebian) <mariodebian@gmail.com>
   netifaces (U)

Matthew Grant <matt@mattgrant.net.nz>
   python-setproctitle (U)

Matthias Klose <doko@debian.org>
   lxml
   pillow
   pillow-sane
   pygccjit
   python-reportlab
   python3-defaults
   python3-stdlib-extensions
   twisted (U)

Michael Hanke <michael.hanke@gmail.com>
   mpi4py (U)
   nipy (U)

Michael Hudson-Doyle <mwhudson@debian.org>
   pyyaml (U)

Michael Vogt <mvo@debian.org>
   python-apt (U)

Mo Zhou <cdluminate@gmail.com>
   h5py (U)

Morten Kjeldgaard <mok0@ubuntu.com>
   cbflib (U)

Nikolaus Rath <Nikolaus@rath.org>
   pyfuse3 (U)
   python-llfuse

Ondrej Certik <ondrej@certik.cz>
   scipy (U)

Paul Tagliamonte <paultag@debian.org>
   python-aiohttp (U)

Picca Frédéric-Emmanuel <picca@debian.org>
   gpyfft (U)
   pyfai (U)
   pymca (U)
   python-fabio (U)
   python-fisx (U)
   silx (U)
   xrayutilities (U)

Pierre-Elliott Bécue <peb@debian.org>
   zope.interface (U)

Pietro Battiston <me@pietrobattiston.it>
   bottleneck (U)

Piotr Ożarowski <piotr@debian.org>
   markupsafe
   python-aiohttp (U)
   python-multidict
   python3-defaults (U)
   simplejson (U)
   uvloop

Rebecca N. Palmer <rebecca_palmer@zoho.com>
   libgpuarray (U)

Sandro Tosi <morph@debian.org>
   basemap
   kiwisolver
   matplotlib
   numpy
   pychm
   python-dmidecode
   python-levenshtein
   python-psutil
   python-pygraphviz
   python-regex

Scott Kitterman <scott@kitterman.com>
   psycopg2 (U)
   pyyaml (U)

Scott Talbert <swt@techie.net>
   pycurl (U)

Sebastian Dröge <slomo@debian.org>
   dbus-python (U)
   gst-python1.0 (U)

Sebastien Bacher <seb128@debian.org>
   pygobject (U)

Sebastien Delafond <seb@debian.org>
   xrayutilities (U)

Simon McVittie <smcv@debian.org>
   dbus-python (U)

Sjoerd Simons <sjoerd@debian.org>
   dbus-python (U)

Stefan Breunig <stefan-debian@yrden.de>
   python-librtmp

Stefano Rivera <stefanor@debian.org>
   pystemmer (U)
   python-cffi (U)
   python3-defaults (U)

Stephen Kitt <skitt@debian.org>
   pyqt5chart (U)

Teemu Ikonen <tpikonen@gmail.com>
   cbflib (U)

Thomas Goirand <zigo@debian.org>
   netifaces (U)
   python-mysqldb (U)
   simplejson (U)

Tianon Gravi <admwiggin@gmail.com>
   python-aiohttp (U)

Tomasz Rybak <serpent@debian.org>
   pycuda (U)
   pyopencl (U)

Torsten Marek <shlomme@debian.org>
   pycairo (U)
   qscintilla2 (U)
   sip4 (U)

Utopia Maintenance Team <pkg-utopia-maintainers@lists.alioth.debian.org>
   dbus-python

Varun Hiremath <varun@debian.org>
   scipy (U)

Vincent Bernat <bernat@debian.org>
   pyzmq (U)

Wen Heping <wenheping@gmail.com>
   numexpr (U)

Willem van den Akker <wvdakker@wilsoft.nl>
   python-ldap (U)

William Grzybowski <william@grzy.org>
   python-aiohttp (U)

Yaroslav Halchenko <debian@onerussian.com>
   mpi4py (U)
   nipy (U)
   numexpr (U)
   pytables (U)
   reprozip (U)

YunQiang Su <wzssyqa@gmail.com>
   libxml2 (U)

[toc] | [prev] | [next] | [standalone]

#13517

From	Scott Talbert <swt@techie.net>
Date	2021-09-13 16:20 +0200
Message-ID	<CWUYp-7nW-3@gated-at.bofh.it>
In reply to	#13504

On Fri, 3 Sep 2021, Matthias Klose wrote:

>> Python 3.8 upstream now has a common ABI for normal and debug extension builds,
>> so it is technically possible to load a debug extension in the normal
>> interpreter, or to load a normal extension in the debug interpreter.  In Debian,
>> debug extensions are shipped with a different name, and only loaded by the
>> corresponding interpreter.  We could change / simply the current setup, but I
>> first wanted to know how many people are still using the debug builds.  The
>> reason for the separate debug builds allowed debugging of stuff in modules
>> further down the Python stack, without having to rebuild the whole stack. There
>> are several solutions how to simplify the packaging, I'm not sure how much the
>> dbg extensions are still used ... There are several scenarios:
>>
>>  - Keep the current setup (-dbg packages need to be available to
>>    run them).
>>
>>  - Allow the debug interpreter to load normal debug extensions (but
>>    load a debug extension if it's available by default).  That would
>>    allow building debug extensions without having debug extensions
>>    built for all it's dependencies, maybe requiring changes in the
>>    dependencies of a package.
>>
>>  - Stop building debug extensions, and telling developers to
>>    build extensions in debug mode, if they need them.  That would
>>    probably be inline with everything else shipped in Debian.
>>
>>  - Stop building debug extensions, and also stop building the Python
>>    debug interpreter.  You would need to rebuild the interpreter
>>    itself to have meaningful debug sessions.  I'm not preferring
>>    this solution.
>>
>> I'm currently tending to implement the second scenario, but if people think that
>> having the -dbg packages available is still useful, then also opt for the third
>> option.
>
> Let's address this before we start adding Python 3.10 as a supported Python3
> version. Starting with the third option.  I'll file bug reports for the
> following packages:

Just to confirm on this: if we currently have a python module that builds 
a -dbg package, we can now drop this in favor of the automatically 
generated -dbgsym package for debugging?

Scott

[toc] | [prev] | [next] | [standalone]

#13518

From	Matthias Klose <doko@debian.org>
Date	2021-09-14 08:40 +0200
Message-ID	<CXagO-tt-23@gated-at.bofh.it>
In reply to	#13517

On 9/13/21 4:02 PM, Scott Talbert wrote:
> On Fri, 3 Sep 2021, Matthias Klose wrote:
> 
>>> Python 3.8 upstream now has a common ABI for normal and debug extension builds,
>>> so it is technically possible to load a debug extension in the normal
>>> interpreter, or to load a normal extension in the debug interpreter.  In Debian,
>>> debug extensions are shipped with a different name, and only loaded by the
>>> corresponding interpreter.  We could change / simply the current setup, but I
>>> first wanted to know how many people are still using the debug builds.  The
>>> reason for the separate debug builds allowed debugging of stuff in modules
>>> further down the Python stack, without having to rebuild the whole stack. There
>>> are several solutions how to simplify the packaging, I'm not sure how much the
>>> dbg extensions are still used ... There are several scenarios:
>>>
>>>  - Keep the current setup (-dbg packages need to be available to
>>>    run them).
>>>
>>>  - Allow the debug interpreter to load normal debug extensions (but
>>>    load a debug extension if it's available by default).  That would
>>>    allow building debug extensions without having debug extensions
>>>    built for all it's dependencies, maybe requiring changes in the
>>>    dependencies of a package.
>>>
>>>  - Stop building debug extensions, and telling developers to
>>>    build extensions in debug mode, if they need them.  That would
>>>    probably be inline with everything else shipped in Debian.
>>>
>>>  - Stop building debug extensions, and also stop building the Python
>>>    debug interpreter.  You would need to rebuild the interpreter
>>>    itself to have meaningful debug sessions.  I'm not preferring
>>>    this solution.
>>>
>>> I'm currently tending to implement the second scenario, but if people think that
>>> having the -dbg packages available is still useful, then also opt for the third
>>> option.
>>
>> Let's address this before we start adding Python 3.10 as a supported Python3
>> version. Starting with the third option.  I'll file bug reports for the
>> following packages:
> 
> Just to confirm on this: if we currently have a python module that builds a -dbg
> package, we can now drop this in favor of the automatically generated -dbgsym
> package for debugging?

yes, ideally, if there are dependencies on your -dbg in the archive, these
should be removed first.  Sorry, I didn't file these bug reports yet.

Matthias

[toc] | [prev] | [next] | [standalone]

#13519

From	Matthias Klose <doko@debian.org>
Date	2021-09-15 12:50 +0200
Message-ID	<CXAEi-8ur-7@gated-at.bofh.it>
In reply to	#13518

On 9/14/21 8:36 AM, Matthias Klose wrote:
> On 9/13/21 4:02 PM, Scott Talbert wrote:
>> On Fri, 3 Sep 2021, Matthias Klose wrote:
>>
>>>> Python 3.8 upstream now has a common ABI for normal and debug extension builds,
>>>> so it is technically possible to load a debug extension in the normal
>>>> interpreter, or to load a normal extension in the debug interpreter.  In Debian,
>>>> debug extensions are shipped with a different name, and only loaded by the
>>>> corresponding interpreter.  We could change / simply the current setup, but I
>>>> first wanted to know how many people are still using the debug builds.  The
>>>> reason for the separate debug builds allowed debugging of stuff in modules
>>>> further down the Python stack, without having to rebuild the whole stack. There
>>>> are several solutions how to simplify the packaging, I'm not sure how much the
>>>> dbg extensions are still used ... There are several scenarios:
>>>>
>>>>  - Keep the current setup (-dbg packages need to be available to
>>>>    run them).
>>>>
>>>>  - Allow the debug interpreter to load normal debug extensions (but
>>>>    load a debug extension if it's available by default).  That would
>>>>    allow building debug extensions without having debug extensions
>>>>    built for all it's dependencies, maybe requiring changes in the
>>>>    dependencies of a package.
>>>>
>>>>  - Stop building debug extensions, and telling developers to
>>>>    build extensions in debug mode, if they need them.  That would
>>>>    probably be inline with everything else shipped in Debian.
>>>>
>>>>  - Stop building debug extensions, and also stop building the Python
>>>>    debug interpreter.  You would need to rebuild the interpreter
>>>>    itself to have meaningful debug sessions.  I'm not preferring
>>>>    this solution.
>>>>
>>>> I'm currently tending to implement the second scenario, but if people think that
>>>> having the -dbg packages available is still useful, then also opt for the third
>>>> option.
>>>
>>> Let's address this before we start adding Python 3.10 as a supported Python3
>>> version. Starting with the third option.  I'll file bug reports for the
>>> following packages:
>>
>> Just to confirm on this: if we currently have a python module that builds a -dbg
>> package, we can now drop this in favor of the automatically generated -dbgsym
>> package for debugging?
> 
> yes, ideally, if there are dependencies on your -dbg in the archive, these
> should be removed first.  Sorry, I didn't file these bug reports yet.

Now filed as
https://bugs.debian.org/cgi-bin/pkgreport.cgi?tag=pydbg-removal;users=debian-python@lists.debian.org

[toc] | [prev] | [next] | [standalone]

#13524

From	Sandro Tosi <morph@debian.org>
Date	2021-09-15 15:30 +0200
Message-ID	<CXD97-1Ei-1@gated-at.bofh.it>
In reply to	#13519

> Now filed as
> https://bugs.debian.org/cgi-bin/pkgreport.cgi?tag=pydbg-removal;users=debian-python@lists.debian.org

why "Severity: serious"? none of them violates the policy:
https://www.debian.org/Bugs/Developer#severities; please adjust to
normal or important. thanks

-- 
Sandro "morph" Tosi
My website: http://sandrotosi.me/
Me at Debian: http://wiki.debian.org/SandroTosi
Twitter: https://twitter.com/sandrotosi

[toc] | [prev] | [standalone]

csiph-web

Python3 -dbg packages

Contents

#12724 — Python3 -dbg packages

#12730

#13504

#13517

#13518

#13519

#13524