Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.maint.python > #7961 > unrolled thread

Multiarch-renamed python extensions not found during autopkgtest testing

Back to article view | Back to linux.debian.maint.python

Contents

  Multiarch-renamed python extensions not found during autopkgtest  testing Afif Elghraoui <afif@ghraoui.name> - 2015-12-09 08:30 +0100
    Bug#798066: Multiarch-renamed python extensions not found during autopkgtest testing Afif Elghraoui <afif@ghraoui.name> - 2015-12-11 08:00 +0100
      Re: Bug#798066: Multiarch-renamed python extensions not found during  autopkgtest testing Antonio Terceiro <terceiro@debian.org> - 2015-12-11 12:20 +0100
        Re: Bug#798066: Multiarch-renamed python extensions not found during  autopkgtest testing Afif Elghraoui <afif@ghraoui.name> - 2015-12-12 10:30 +0100

#7961 — Multiarch-renamed python extensions not found during autopkgtest testing

Hi, debian-python,

For Debian Med's python packages that have compiled extensions, I
noticed that test suites run via autopkgtest fail because the package
cannot find those extensions, as they've been renamed with the multiarch
triplet.

It seems to only be a problem with autopkgtest because I have in one
case manually installed such a package and successfully ran its test
suite. I therefore reported #798066 [1] against autopkgtest, but quite
some time has elapsed since then and I'd like to get this resolved.

Does anyone know why this might be the case? Autopkgtest for these
packages is currently not very helpful because there are many spurious
failures due to this issue. There are more details in my original bug
report [1].

Many thanks and regards
Afif

1. http://bugs.debian.org/798066

-- 
Afif Elghraoui | عفيف الغراوي
http://afif.ghraoui.name

[toc] | [next] | [standalone]

#7964 — Bug#798066: Multiarch-renamed python extensions not found during autopkgtest testing

Hi,

على الأربعاء  9 كانون الأول 2015 ‫05:25، كتب Antonio Terceiro:
> autopkgtest does not do anything special wrt dependencies, it will
> install exactly what you told it to in debian/tests/control
[...]
> 
> I am therefore closing this bug.

The problem is not that something is not installed. The problem is that
the multiarch configuration for python is not right in autopkgtest (or
schroot). During the package build, dh-python renames the compiled
extension to contain the multiarch triplet. For some reason, only in
autopkgtest, python is not properly configured to find the extension
after it has been renamed, so autopkgtest runs requiring the compiled
extensions fail.

So should we continue to have hacks like this to get autopkgtest to find
multiarch-renamed extensions?

cd /usr/lib/python2.7/dist-packages/pysam
gnutype=`dpkg-architecture -qDEB_TARGET_GNU_TYPE`
for so in *.${gnutype}.so ; do sudo ln -sf $so `basename $so
.${gnutype}.so`.so ; done

http://anonscm.debian.org/cgit/debian-med/python-pysam.git/tree/debian/tests/run-nose-tests

regards
Afif

-- 
Afif Elghraoui | عفيف الغراوي
http://afif.ghraoui.name

[toc] | [prev] | [next] | [standalone]

#7965 — Re: Bug#798066: Multiarch-renamed python extensions not found during autopkgtest testing

[Multipart message — attachments visible in raw view] — view raw

On Thu, Dec 10, 2015 at 10:52:05PM -0800, Afif Elghraoui wrote:
> Hi,
> 
> على الأربعاء  9 كانون الأول 2015 ‫05:25، كتب Antonio Terceiro:
> > autopkgtest does not do anything special wrt dependencies, it will
> > install exactly what you told it to in debian/tests/control
> [...]
> > 
> > I am therefore closing this bug.
> 
> The problem is not that something is not installed. The problem is that
> the multiarch configuration for python is not right in autopkgtest (or
> schroot). During the package build, dh-python renames the compiled
> extension to contain the multiarch triplet. For some reason, only in
> autopkgtest, python is not properly configured to find the extension
> after it has been renamed, so autopkgtest runs requiring the compiled
> extensions fail.

ci.debian.net does not use schroot anymore, since a few weeks ago. and
autopkgtest does not do anything special to packages, it just installs
them on an otherwise regular testbed, be it a schroot chroot, an lxc
container (what ci.debian.net currently uses), and so on.

> So should we continue to have hacks like this to get autopkgtest to find
> multiarch-renamed extensions?
> 
> cd /usr/lib/python2.7/dist-packages/pysam
> gnutype=`dpkg-architecture -qDEB_TARGET_GNU_TYPE`
> for so in *.${gnutype}.so ; do sudo ln -sf $so `basename $so
> .${gnutype}.so`.so ; done
> 
> http://anonscm.debian.org/cgit/debian-med/python-pysam.git/tree/debian/tests/run-nose-tests

Again, there is nothing special in the autopkgtest test beds. I also
don't see other python packages that contain compiled extensions needing
to do this sort of thing.

I tried python-pysam here, and after some trial and error, I can also reproduce
the same issue outside of autopkgtest. The issue is that Python load path is
being confused by the fact that you are on root of the source package:

# pwd
/tmp/python-pysam-0.8.4+ds
# ls
AUTHORS  INSTALL     MANIFEST.in  THANKS     debian  install-CGAT-tools.sh  pysam.py          samtools  setup.cfg  tests
COPYING  KNOWN_BUGS  README.rst   benchmark  doc     pysam                  requirements.txt  save      setup.py   win32
# python
Python 2.7.11 (default, Dec  9 2015, 00:29:25) 
[GCC 5.3.1 20151205] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import pysam
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "pysam/__init__.py", line 1, in <module>
    from pysam.libchtslib import *
ImportError: No module named libchtslib
>>> 
# cd /
# python
Python 2.7.11 (default, Dec  9 2015, 00:29:25) 
[GCC 5.3.1 20151205] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import pysam 
>>> 
# 

So your problem has nothing to do autopkgtest, other than the fact that
autopkgtest always starts the tests from the root of the source package.

-- 
Antonio Terceiro <terceiro@debian.org>

[toc] | [prev] | [next] | [standalone]

#7967 — Re: Bug#798066: Multiarch-renamed python extensions not found during autopkgtest testing

[Multipart message — attachments visible in raw view] — view raw

على الجمعـة 11 كانون الأول 2015 ‫03:09، كتب Antonio Terceiro:

> I also
> don't see other python packages that contain compiled extensions needing
> to do this sort of thing.
>

I have a few other packages that have this problem -- python-cobra is
one. For two other packages (python-pbh5tools and kineticstools), I
disabled tests that require the compiled extensions because they were
failing for this reason.


> I tried python-pysam here, and after some trial and error, I can also reproduce
> the same issue outside of autopkgtest. The issue is that Python load path is
> being confused by the fact that you are on root of the source package:
> 
[...]

Many thanks for that information. I can reproduce that effect as well
and things are making much more sense now. Now, I just need to find a
proper solution to this python module priority problem.

> 
> So your problem has nothing to do autopkgtest, other than the fact that
> autopkgtest always starts the tests from the root of the source package.
> 

Thanks-- I didn't set out to bash autopkgtest with this. I just wanted
to know why the tests weren't running properly and how to resolve that
problem.


Many thanks and regards
Afif

-- 
Afif Elghraoui | عفيف الغراوي
http://afif.ghraoui.name

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.maint.python

csiph-web