Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.bugs.dist > #1266065 > unrolled thread

Bug#1118055: python3-pysmi: might trigger: AttributeError: module 'importlib' has no attribute 'machinery'

Back to article view | Back to linux.debian.bugs.dist

Contents

  Bug#1118055: python3-pysmi: might trigger: AttributeError: module 'importlib' has no attribute 'machinery' Cyril Brulebois <cyril@debamax.com> - 2025-10-14 14:20 +0200
    Bug#1118055: python3-pysmi: might trigger: AttributeError: module 'importlib' has no attribute 'machinery' Cyril Brulebois <cyril@debamax.com> - 2025-10-14 16:30 +0200
    Bug#1118055: python3-pysmi: might trigger: AttributeError: module 'importlib' has no attribute 'machinery' Colin Watson <cjwatson@debian.org> - 2025-10-14 23:20 +0200

#1266065 — Bug#1118055: python3-pysmi: might trigger: AttributeError: module 'importlib' has no attribute 'machinery'

Package: python3-pysmi
Version: 0.3.4-3
Severity: important

Hi,

I've just spotted something a little weird while working on a customer
package that ships a number of MIBs that are processed via pysmi at
build time:
 - the package builds fine in a trixie devel schroot;
 - the package fails to build in a clean trixie cowbuilder chroot.

At first I suspected some build-depends might be missing (it declares
debhelper and python3-pysmi only), but the error is in importlib, which
is shipped alongside Python:

    Traceback (most recent call last):
      File "/usr/lib/python3/dist-packages/pysmi/searcher/pyfile.py", line 15, in <module>
        SOURCE_SUFFIXES = importlib.machinery.SOURCE_SUFFIXES
                          ^^^^^^^^^^^^^^^^^^^
    AttributeError: module 'importlib' has no attribute 'machinery'
    During handling of the above exception, another exception occurred:

I've tried replicating this with a bare trixie chroot, and I'm getting
similar results with either trixie or bookworm after setting them up
this way:

    suite=bookworm # or trixie
    sudo debootstrap $suite $suite
    sudo chroot $suite
    apt-get install -y python3

Non-interactively:

    root@tokyo:/# python3 -c 'import importlib; print(importlib.machinery)'
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
    AttributeError: module 'importlib' has no attribute 'machinery'

Interactively:

    root@tokyo:/# python3
    Python 3.11.2 (main, Apr 28 2025, 14:11:48) [GCC 12.2.0] on linux
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import importlib; print(importlib.machinery)
    <module 'importlib.machinery' (frozen)>

Using dir() shows slightly different results:

    root@tokyo:/# python3 -c 'import importlib; print(dir(importlib))'
    ['_RELOADING', '__all__', '__builtins__', '__cached__', '__doc__', '__file__', '__import__', '__loader__', '__name__', '__package__', '__path__', '__spec__', '_bootstrap', '_bootstrap_external', '_imp', '_pack_uint32', '_unpack_uint32', 'import_module', 'invalidate_caches', 'reload', 'sys']

    root@tokyo:/# python3
    Python 3.13.5 (main, Jun 25 2025, 18:55:22) [GCC 14.2.0] on linux
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import importlib; print(dir(importlib))
    ['_RELOADING', '__all__', '__builtins__', '__cached__', '__doc__', '__file__', '__import__', '__loader__', '__name__', '__package__', '__path__', '__spec__', '_abc', '_bootstrap', '_bootstrap_external', '_imp', '_pack_uint32', '_unpack_uint32', 'import_module', 'invalidate_caches', 'machinery', 'reload', 'sys', 'util']

Extra entries: _abc, machinery, and util.


As mentioned in introduction, the interactive part is not necessarily
the critical bit, as that might work within devel schroots (I'm not sure
what the changes are, different set of packages, environment variables,
something else…).

Anyway, regardless of the exact details, since pysmi is in the middle of
some autodetection, it would probably be best not to rely on the exact
details of importlib's bootstrap (I only glanced at its internals)…

There are four occurrences of the following, with slight variations:

    try:
        import importlib
    
        try:
            SOURCE_SUFFIXES = importlib.machinery.SOURCE_SUFFIXES
            BYTECODE_SUFFIXES = importlib.machinery.BYTECODE_SUFFIXES
    
        except Exception:
            raise ImportError()

(found in pysmi/searcher/pyfile.py, pysmi/searcher/pypackage.py,
pysmi/writer/pyfile.py, and pysmi/borrower/pyfile.py)

It seems something based on the following instead could work more
reliably:

    from importlib.machinery import SOURCE_SUFFIXES, BYTECODE_SUFFIXES

Ditto for the util bit (based on the dir() output above), only found in
pysmi/searcher/pypackage.py):

    from importlib.util import MAGIC_NUMBER

since the same behaviour can be observed with the “bare” importlib
import:

    root@tokyo:/# python3 -c 'import importlib; print(importlib.util)'
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
        import importlib; print(importlib.util)
                                ^^^^^^^^^^^^^^
    AttributeError: module 'importlib' has no attribute 'util'

    root@tokyo:/# python3
    Python 3.13.5 (main, Jun 25 2025, 18:55:22) [GCC 14.2.0] on linux
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import importlib; print(importlib.util)
    <module 'importlib.util' (frozen)>


Thanks for considering, and sorry for not debugging this all the way.


Cheers,
-- 
Cyril Brulebois -- Debian Consultant @ DEBAMAX -- https://debamax.com/

[toc] | [next] | [standalone]

#1266077

[Multipart message — attachments visible in raw view] — view raw

Hi,

Cyril Brulebois <cyril@debamax.com> (2025-10-14):
> It seems something based on the following instead could work more
> reliably:
> 
>     from importlib.machinery import SOURCE_SUFFIXES, BYTECODE_SUFFIXES
> 
> Ditto for the util bit (based on the dir() output above), only found in
> pysmi/searcher/pypackage.py):
> 
>     from importlib.util import MAGIC_NUMBER

Since I'd like to avoid touching the overall build chroot management
logic (i.e. I'd like to avoid injecting a patched package in some way or
another), I've opted for declaring a submodule, patching/unpatching
manually, and with a PYTHONPATH pointing to the pysmi submodule tweaked
with the attached patch, I can build the many-MIBs-package inside the CI
successfully.

That particular codepath only uses a portion of the patched pysmi code
(pysmi/searcher/pyfile.py), so please don't consider the overall patch
battle-tested.


Cheers,
-- 
Cyril Brulebois -- Debian Consultant @ DEBAMAX -- https://debamax.com/

[toc] | [prev] | [next] | [standalone]

#1266136

On Tue, Oct 14, 2025 at 02:17:37PM +0200, Cyril Brulebois wrote:
>I've just spotted something a little weird while working on a customer
>package that ships a number of MIBs that are processed via pysmi at
>build time:
> - the package builds fine in a trixie devel schroot;
> - the package fails to build in a clean trixie cowbuilder chroot.
>
>At first I suspected some build-depends might be missing (it declares
>debhelper and python3-pysmi only), but the error is in importlib, which
>is shipped alongside Python:
>
>    Traceback (most recent call last):
>      File "/usr/lib/python3/dist-packages/pysmi/searcher/pyfile.py", line 15, in <module>
>        SOURCE_SUFFIXES = importlib.machinery.SOURCE_SUFFIXES
>                          ^^^^^^^^^^^^^^^^^^^
>    AttributeError: module 'importlib' has no attribute 'machinery'

Importing a module doesn't necessarily mean importing its submodules 
(unless importlib/__init__.py did something like `from . import 
machinery`, which it doesn't).  However, this might well be masked in 
many installations by things like /usr/lib/python3/dist-packages/*.pth 
files that import importlib.machinery and thus add it to sys.modules, 
and some of those do exist.  strace would confirm.

>There are four occurrences of the following, with slight variations:
>
>    try:
>        import importlib
>
>        try:
>            SOURCE_SUFFIXES = importlib.machinery.SOURCE_SUFFIXES
>            BYTECODE_SUFFIXES = importlib.machinery.BYTECODE_SUFFIXES
>
>        except Exception:
>            raise ImportError()
>
>(found in pysmi/searcher/pyfile.py, pysmi/searcher/pypackage.py,
>pysmi/writer/pyfile.py, and pysmi/borrower/pyfile.py)
>
>It seems something based on the following instead could work more
>reliably:
>
>    from importlib.machinery import SOURCE_SUFFIXES, BYTECODE_SUFFIXES

Or just replace `import importlib` with `import importlib.machinery` 
(and `import importlib.util` as needed, etc.).  Importing a submodule 
implicitly imports its parent module, just not vice versa.

-- 
Colin Watson (he/him)                              [cjwatson@debian.org]

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.bugs.dist

csiph-web