Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.maint.python > #16410 > unrolled thread

electrum: How to enable autopkgtest?

Back to article view | Back to linux.debian.maint.python

Contents

  electrum: How to enable autopkgtest? Soren Stoutner <soren@debian.org> - 2024-10-29 21:50 +0100
    Re: electrum: How to enable autopkgtest? Andrey Rakhmatullin <wrar@debian.org> - 2024-10-30 06:50 +0100
      Re: electrum: How to enable autopkgtest? Soren Stoutner <soren@debian.org> - 2024-10-30 07:20 +0100
        Re: electrum: How to enable autopkgtest? Andrey Rakhmatullin <wrar@debian.org> - 2024-10-30 07:50 +0100
        Re: electrum: How to enable autopkgtest? Soren Stoutner <soren@debian.org> - 2024-10-30 08:10 +0100
          Re: electrum: How to enable autopkgtest? Andrey Rakhmatullin <wrar@debian.org> - 2024-10-30 08:20 +0100
            Re: electrum: How to enable autopkgtest? Soren Stoutner <soren@debian.org> - 2024-10-30 08:30 +0100
              Re: electrum: How to enable autopkgtest? Andrey Rakhmatullin <wrar@debian.org> - 2024-10-30 08:40 +0100
                Re: electrum: How to enable autopkgtest? Soren Stoutner <soren@debian.org> - 2024-10-31 20:50 +0100
                  Re: electrum: How to enable autopkgtest? weepingclown <weepingclown@disroot.org> - 2024-10-31 21:40 +0100
                    Re: electrum: How to enable autopkgtest? Soren Stoutner <soren@debian.org> - 2024-10-31 21:50 +0100
                      Re: electrum: How to enable autopkgtest? weepingclown <weepingclown@disroot.org> - 2024-10-31 21:50 +0100
                        Re: electrum: How to enable autopkgtest? Soren Stoutner <soren@debian.org> - 2024-10-31 22:10 +0100
          Re: electrum: How to enable autopkgtest? Hefee <hefee@debian.org> - 2024-10-31 19:50 +0100
            Re: electrum: How to enable autopkgtest? Soren Stoutner <soren@debian.org> - 2024-10-31 20:30 +0100
    Re: electrum: How to enable autopkgtest? Antonio Terceiro <terceiro@debian.org> - 2024-10-30 14:10 +0100

#16410 — electrum: How to enable autopkgtest?

[Multipart message — attachments visible in raw view] — view raw

I recently requested the upstream maintainers add the tests directory to their 
signed tarball releases.  I am now in the process of trying to enable those 
tests in the Debian package.

I added a Build-Depend on "python3-pytest <!nocheck>” and added “Testsuite: 
autopkgtest-pkg-pybuild” to debian/control.  This enables the tests during 
build time, but I receive the following error during autopkgtest:

E   ModuleNotFoundError: No module named ‘electrum.gui.qml’

https://salsa.debian.org/cryptocoin-team/electrum/-/jobs/6505015#L677

I believe I could fix this problem by adding the following to debian/rules:

export PYBUILD_NAME=electrum

However, this breaks the current splitting of the files into two binary 
packages, electrum and python3-electrum, using .install files.

My question is, what is the canonical way to handle this?  Is there some 
variation of the “export PYBUILD_NAME” command that preserves the contents of 
the two binary packages?  Or, is there some other command I am missing that 
will allow autopkgtest to import the built modules?

-- 
Soren Stoutner
soren@debian.org

[toc] | [next] | [standalone]

#16411

[Multipart message — attachments visible in raw view] — view raw

On Tue, Oct 29, 2024 at 01:43:50PM -0700, Soren Stoutner wrote:
> I added a Build-Depend on "python3-pytest <!nocheck>” and added “Testsuite: 
> autopkgtest-pkg-pybuild” to debian/control.  This enables the tests during 
> build time, but I receive the following error during autopkgtest:
> 
> E   ModuleNotFoundError: No module named ‘electrum.gui.qml’

The same happens when I install the package manually and try that import.

> https://salsa.debian.org/cryptocoin-team/electrum/-/jobs/6505015#L677
> 
> I believe I could fix this problem by adding the following to debian/rules:
> 
> export PYBUILD_NAME=electrum

Why would this fix this problem?

> My question is, what is the canonical way to handle this?

Fixing the package so that it provides that module or patching the tests
so that they don't require it. 

-- 
WBR, wRAR

[toc] | [prev] | [next] | [standalone]

#16412

[Multipart message — attachments visible in raw view] — view raw

Andrey,

On Tuesday, October 29, 2024 10:47:47 PM MST Andrey Rakhmatullin wrote:
> On Tue, Oct 29, 2024 at 01:43:50PM -0700, Soren Stoutner wrote:
> > I added a Build-Depend on "python3-pytest <!nocheck>” and added “Testsuite:
> > autopkgtest-pkg-pybuild” to debian/control.  This enables the tests during
> > build time, but I receive the following error during autopkgtest:
> > 
> > E   ModuleNotFoundError: No module named ‘electrum.gui.qml’
> 
> The same happens when I install the package manually and try that import.
> 
> > https://salsa.debian.org/cryptocoin-team/electrum/-/jobs/6505015#L677
> > 
> > I believe I could fix this problem by adding the following to debian/rules:
> > 
> > export PYBUILD_NAME=electrum
> 
> Why would this fix this problem?
> 
> > My question is, what is the canonical way to handle this?
> 
> Fixing the package so that it provides that module or patching the tests
> so that they don't require it.

The tests run successfully during build.

"688 passed, 3 skipped, 3 warnings in 98.99s (0:01:38)”

https://salsa.debian.org/cryptocoin-team/electrum/-/jobs/6505007#L1548

However, I should note that there are a significant number of warnings that various 
modules are importable packages but missing from setuptools' `packages` configuration.

It is very possible that I am missing some important plumbing in the packaging to expose 
these properly, as I am fairly new to using Python tests or Python at all (my programming 
background is in other languages).  I would imagine upstream only runs these tests at 
build time, so the upstream developers might not have included the plumbing for them to 
be imported in other environments.  My previous understanding was that "Testsuite: 
autopkgtest-pkg-pybuild” was designed to automatically run build tests in the autopkgtest 
environment, but perhaps in this case the tests are not compatible with autopkgtest.  If so, 
is the best way to handle it to simply not use autopkgtest?  Or is there some massaging of 
the packaging I can do that will make them compatible?

-- 
Soren Stoutner
soren@debian.org

[toc] | [prev] | [next] | [standalone]

#16413

[Multipart message — attachments visible in raw view] — view raw

On Tue, Oct 29, 2024 at 11:11:44PM -0700, Soren Stoutner wrote:
> > > I added a Build-Depend on "python3-pytest <!nocheck>” and added “Testsuite:
> > > autopkgtest-pkg-pybuild” to debian/control.  This enables the tests during
> > > build time, but I receive the following error during autopkgtest:
> > > 
> > > E   ModuleNotFoundError: No module named ‘electrum.gui.qml’
> > 
> > The same happens when I install the package manually and try that import.
> > 
> > > https://salsa.debian.org/cryptocoin-team/electrum/-/jobs/6505015#L677
> > > 
> > > I believe I could fix this problem by adding the following to debian/rules:
> > > 
> > > export PYBUILD_NAME=electrum
> > 
> > Why would this fix this problem?
> > 
> > > My question is, what is the canonical way to handle this?
> > 
> > Fixing the package so that it provides that module or patching the tests
> > so that they don't require it.
> 
> The tests run successfully during build.

Sure, as that module is present in the package source.

The purpose of autopkgtests is to test the installed package. Your package
fails those tests because those tests expect certain modules to be
installed while they are not. Only you can know whether those modules not
being installed is a bug or not, and depending on that there are two ways
to solve the autopkgtests failure. And it looks like you definitely know
the answer, as it was you who added the code that explicitly removes those
modules from the package.

> However, I should note that there are a significant number of warnings that various 
> modules are importable packages but missing from setuptools' `packages` configuration.
> 
> It is very possible that I am missing some important plumbing in the packaging to expose 
> these properly,

No, this is irrelevant, as all modules are installed by the upstream
build system.

> as I am fairly new to using Python tests or Python at all (my programming 
> background is in other languages).  I would imagine upstream only runs these tests at 
> build time, so the upstream developers might not have included the plumbing for them to 
> be imported in other environments.  

No.

-- 
WBR, wRAR

[toc] | [prev] | [next] | [standalone]

#16414

[Multipart message — attachments visible in raw view] — view raw

On Tuesday, October 29, 2024 11:11:44 PM MST Soren Stoutner wrote:
> > > My question is, what is the canonical way to handle this?
> > 
> > Fixing the package so that it provides that module or patching the tests
> > so that they don't require it.
> 
> The tests run successfully during build.
> 
> "688 passed, 3 skipped, 3 warnings in 98.99s (0:01:38)”
> 
> https://salsa.debian.org/cryptocoin-team/electrum/-/jobs/6505007#L1548

Looking at things more deeply, the underlying cause is that the upstream 
source contains GUI environments for both Linux (Qt) and Android (QML).  In 
the build environment, all these files are present (this was the key I was 
missing).  But, in the Debian package, the QML files have been removed (because 
they aren’t needed).

(Andrey, as I’m writing this I see you just sent me an email explaining the 
same thing.)

The test in question is test_qml_types.py.

If I disable it in debian/rules using the following command the test no longer 
runs during build.

export PYBUILD_TEST_ARGS=-k 'not test_qml_types.py’

However, that doesn’t fix the autopkgtest problem.  I still receive the 
following error:

I: pybuild base:311: cd /tmp/autopkgtest-lxc.0qtzuqhg/downtmp/autopkgtest_tmp/
build; python3.12 -m pytest -k 'not test_qml_types.py'
============================= test session starts 
==============================
platform linux -- Python 3.12.7, pytest-8.3.3, pluggy-1.5.0
rootdir: /tmp/autopkgtest-lxc.0qtzuqhg/downtmp/autopkgtest_tmp/build
plugins: typeguard-4.3.0
collected 688 items / 1 error
==================================== ERRORS 
====================================
___________________ ERROR collecting tests/test_qml_types.py 
___________________
ImportError while importing test module '/tmp/autopkgtest-lxc.0qtzuqhg/
downtmp/autopkgtest_tmp/build/tests/test_qml_types.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib/python3.12/importlib/__init__.py:90: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests/test_qml_types.py:5: in <module>
    from electrum.gui.qml.qetypes import QEAmount
E   ModuleNotFoundError: No module named ‘electrum.gui.qml’


https://salsa.debian.org/cryptocoin-team/electrum/-/jobs/6506576#L662

I assume this is because autopkgtests enumerates all of the imports, even for 
tests it isn’t going to run.

Is there a recommended way to disable this test entirely, including the 
import?  I would assume at least one of the following would work:

1.  Use some other command (of which I am not aware) instead of "export 
PYBUILD_TEST_ARGS=-k” that causes autopkgtest to ignore the file entirely.
2.  Patch out the import line in the test using debian/patches.
3.  Exclude the entire test file when repacking the upstream source.

-- 
Soren Stoutner
soren@debian.org

[toc] | [prev] | [next] | [standalone]

#16415

[Multipart message — attachments visible in raw view] — view raw

On Wed, Oct 30, 2024 at 12:06:32AM -0700, Soren Stoutner wrote:
> If I disable it in debian/rules using the following command the test no longer 
> runs during build.
> 
> export PYBUILD_TEST_ARGS=-k 'not test_qml_types.py’

The correct way is --ignore tests/test_qml_types.py



-- 
WBR, wRAR

[toc] | [prev] | [next] | [standalone]

#16416

[Multipart message — attachments visible in raw view] — view raw

On Wednesday, October 30, 2024 12:16:33 AM MST Andrey Rakhmatullin wrote:
> On Wed, Oct 30, 2024 at 12:06:32AM -0700, Soren Stoutner wrote:
> > If I disable it in debian/rules using the following command the test no
> > longer runs during build.
> > 
> > export PYBUILD_TEST_ARGS=-k 'not test_qml_types.py’
> 
> The correct way is --ignore tests/test_qml_types.py

Thank you.  That works perfectly.

Do you think it would be helpful to add that to the wiki at:

https://wiki.debian.org/Python/Pybuild#CUSTOMIZATION

Should it replace the information about the -k command or are they 
complimentary, with each having advantages in different scenarios?

-- 
Soren Stoutner
soren@debian.org

[toc] | [prev] | [next] | [standalone]

#16417

[Multipart message — attachments visible in raw view] — view raw

On Wed, Oct 30, 2024 at 12:28:27AM -0700, Soren Stoutner wrote:
> > > If I disable it in debian/rules using the following command the test no
> > > longer runs during build.
> > > 
> > > export PYBUILD_TEST_ARGS=-k 'not test_qml_types.py’
> > 
> > The correct way is --ignore tests/test_qml_types.py
> 
> Thank you.  That works perfectly.
> 
> Do you think it would be helpful to add that to the wiki at:
> 
> https://wiki.debian.org/Python/Pybuild#CUSTOMIZATION

Yes.

> Should it replace the information about the -k command or are they 
> complimentary, with each having advantages in different scenarios?

They are complimentary, --ignore skips the file at the test collection
step (https://docs.pytest.org/en/stable/example/pythoncollection.html), -k
and -m select/deselect specific tests
(https://docs.pytest.org/en/stable/example/markers.html).


-- 
WBR, wRAR

[toc] | [prev] | [next] | [standalone]

#16426

[Multipart message — attachments visible in raw view] — view raw

On Wednesday, October 30, 2024 12:34:39 AM MST Andrey Rakhmatullin wrote:
> > Do you think it would be helpful to add that to the wiki at:
> > 
> > https://wiki.debian.org/Python/Pybuild#CUSTOMIZATION
> 
> Yes.
> 
> > Should it replace the information about the -k command or are they
> > complimentary, with each having advantages in different scenarios?
> 
> They are complimentary, --ignore skips the file at the test collection
> step (https://docs.pytest.org/en/stable/example/pythoncollection.html), -k
> and -m select/deselect specific tests
> (https://docs.pytest.org/en/stable/example/markers.html).

I added the following text to the wiki:

*The `'-k $exp'` command skips running the test, but the tests are still 
included in the collection. This will cause failures if the tests have 
includes that cannot be satisfied. In that case, the `--ignore` command can be 
used instead, which skips adding the test to the collection.

{{{
export PYBUILD_TEST_ARGS=--ignore path/to/test.py
}}}

-- 
Soren Stoutner
soren@debian.org

[toc] | [prev] | [next] | [standalone]

#16427

Hi,

this is not entirely correct as while '-k' can be used to skip specific tests with 'not <test>' it can also be used to only include tests matching a specific pattern (in both cases it is just matching the pattern given). It is probably better to rephrase it to something like "The `-k not <test>` flag can be used to skip running specific tests, but they are still collected.....". 

Best,
Ananthu

On 31 October 2024 7:43:42 pm UTC, Soren Stoutner <soren@debian.org> wrote:
>*The `'-k $exp'` command skips running the test, but the tests are still 
>included in the collection. This will cause failures if the tests have 
>includes that cannot be satisfied. In that case, the `--ignore` command can be 
>used instead, which skips adding the test to the collection.
>
>{{{
>export PYBUILD_TEST_ARGS=--ignore path/to/test.py
>}}}
>

[toc] | [prev] | [next] | [standalone]

#16428

[Multipart message — attachments visible in raw view] — view raw

On Thursday, October 31, 2024 1:37:22 PM MST weepingclown wrote:
> Hi,
> 
> this is not entirely correct as while '-k' can be used to skip specific tests
> with 'not <test>' it can also be used to only include tests matching a
> specific pattern (in both cases it is just matching the pattern given). It is
> probably better to rephrase it to something like "The `-k not <test>` flag
> can be used to skip running specific tests, but they are still
> collected.....".

That makes sense.  I would recommend you update the Wiki with that wording.

> On 31 October 2024 7:43:42 pm UTC, Soren Stoutner <soren@debian.org> wrote:
> >*The `'-k $exp'` command skips running the test, but the tests are still
> >included in the collection. This will cause failures if the tests have
> >includes that cannot be satisfied. In that case, the `--ignore` command can
> >be used instead, which skips adding the test to the collection.
> >
> >{{{
> >export PYBUILD_TEST_ARGS=--ignore path/to/test.py
> >}}}


-- 
Soren Stoutner
soren@debian.org

[toc] | [prev] | [next] | [standalone]

#16429

Hi,

Sure, I can do that later. The time here now is late and I am quite tired, and given it is me, I would probably forget to do it later. That is the only reason why I haven't done it already and left a note here so that at least someone will remember it xD

Best,
Ananthu

On 31 October 2024 8:39:54 pm UTC, Soren Stoutner <soren@debian.org> wrote:
>On Thursday, October 31, 2024 1:37:22 PM MST weepingclown wrote:
>That makes sense.  I would recommend you update the Wiki with that wording.

[toc] | [prev] | [next] | [standalone]

#16431

[Multipart message — attachments visible in raw view] — view raw

On Thursday, October 31, 2024 1:47:03 PM MST weepingclown wrote:
> Hi,
> 
> Sure, I can do that later. The time here now is late and I am quite tired, 
and
> given it is me, I would probably forget to do it later. That is the only
> reason why I haven't done it already and left a note here so that at least
> someone will remember it xD

Part of the reason why it would probably be best for you or someone who has a 
full grasp of the -k command to make the edit is that what I wrote was a 
continuation of the previous section discussing how to use -k to disable 
tests.  That section doesn’t make any reference to using -k to include tests.  
So, a complete edit will probably need to include modifying the previous 
paragraphs to be comprehensively correct, which is probably best done by 
someone who has more experience with -k than I do.

-- 
Soren Stoutner
soren@debian.org

[toc] | [prev] | [next] | [standalone]

#16423

[Multipart message — attachments visible in raw view] — view raw

Hey,

> Looking at things more deeply, the underlying cause is that the upstream
> source contains GUI environments for both Linux (Qt) and Android (QML).  In
> the build environment, all these files are present (this was the key I was
> missing).  But, in the Debian package, the QML files have been removed
> (because they aren’t needed).

I'm wondering, if the QML GUI is really only usable on Android. Did you 
checked, that it is really not usable on a Desktop? If the QML GUI is 
optimized for phones, that may still be interesting for Mobian ( Debian for 
mobile devices).

I did a fast code check and at least the QML Gui uses Qt 6, the Qt Widget Gui 
used  the old Qt 5 ( that has EOL since May 2023).

If you include the QML GUI you properly want to use  debhelper-sequence-
qmldeps to detect QML depdenencies in the qml files. Me created dh_qmldeps the 
last weeks, as more and more Qt based apps using qml.

Regards,

hefee

[toc] | [prev] | [next] | [standalone]

#16425

[Multipart message — attachments visible in raw view] — view raw

On Thursday, October 31, 2024 11:40:28 AM MST Hefee wrote:
> Hey,
> 
> > Looking at things more deeply, the underlying cause is that the upstream
> > source contains GUI environments for both Linux (Qt) and Android (QML).  
In
> > the build environment, all these files are present (this was the key I was
> > missing).  But, in the Debian package, the QML files have been removed
> > (because they aren’t needed).
> 
> I'm wondering, if the QML GUI is really only usable on Android. Did you
> checked, that it is really not usable on a Desktop? If the QML GUI is
> optimized for phones, that may still be interesting for Mobian ( Debian for
> mobile devices).

When I asked upstream about it a while back, they said they couldn’t imagine 
it being useful in the Debian package, but perhaps it might be useful on 
something like Mobian.  I don’t know if the QML implementation is tied in any 
specific way to the Android version of the app.

If you have an interest in that direction, I would appreciate a co-maintainer 
that would take on the task.  You could communicate with upstream (I find them 
very helpful) and test its functionality on a Mobian device.

-- 
Soren Stoutner
soren@debian.org

[toc] | [prev] | [next] | [standalone]

#16418

[Multipart message — attachments visible in raw view] — view raw

On Tue, Oct 29, 2024 at 01:43:50PM -0700, Soren Stoutner wrote:
> I recently requested the upstream maintainers add the tests directory to their 
> signed tarball releases.  I am now in the process of trying to enable those 
> tests in the Debian package.
> 
> I added a Build-Depend on "python3-pytest <!nocheck>” and added “Testsuite: 
> autopkgtest-pkg-pybuild” to debian/control.  This enables the tests during 
> build time, but I receive the following error during autopkgtest:
> 
> E   ModuleNotFoundError: No module named ‘electrum.gui.qml’
> 
> https://salsa.debian.org/cryptocoin-team/electrum/-/jobs/6505015#L677
> 
> I believe I could fix this problem by adding the following to debian/rules:
> 
> export PYBUILD_NAME=electrum
> 
> However, this breaks the current splitting of the files into two binary 
> packages, electrum and python3-electrum, using .install files.
> 
> My question is, what is the canonical way to handle this?  Is there some 
> variation of the “export PYBUILD_NAME” command that preserves the contents of 
> the two binary packages?  Or, is there some other command I am missing that 
> will allow autopkgtest to import the built modules?

If you check the artfifacts in the pipeline, your python3-electron does
not contain the module that the test is trying to import:

$ dpkg --contents python3-electrum_4.5.8+ds-1+salsaci+20241029+49_all.deb | grep qml.*py
-rw-r--r-- root/root      4142 2024-10-18 00:56 ./usr/lib/python3/dist-packages/electrum/plugins/labels/qml.py
-rw-r--r-- root/root      4559 2024-10-18 00:56 ./usr/lib/python3/dist-packages/electrum/plugins/trustedcoin/qml.py

This is why the import fails, it has nothing to do with PYBUILD_NAME.

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.maint.python

csiph-web