Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.bugs.rc > #273478 > unrolled thread

Bug#953832: drmaa: autopkgtest failure: RuntimeError: Could not find drmaa library

Back to article view | Back to linux.debian.bugs.rc

Contents

  Bug#953832: drmaa: autopkgtest failure: RuntimeError: Could not find drmaa library Paul Gevers <elbrus@debian.org> - 2020-03-13 23:20 +0100
    Processed: cannot allocate memory in static TLS block (Was:  Bug#953832: drmaa: autopkgtest failure: RuntimeError: Could not find drmaa  library) "Debian Bug Tracking System" <owner@bugs.debian.org> - 2020-03-14 14:40 +0100
    Bug#953832: cannot allocate memory in static TLS block (Was: Bug#953832: drmaa: autopkgtest failure: RuntimeError: Could not find drmaa library) Andreas Tille <tille@debian.org> - 2020-03-14 14:40 +0100
      Bug#953832: cannot allocate memory in static TLS block Sergio Durigan Junior <sergiodj@debian.org> - 2020-03-14 22:30 +0100
        Bug#953832: cannot allocate memory in static TLS block Andreas Tille <tille@debian.org> - 2020-03-15 10:40 +0100
    Bug#953832: cannot allocate memory in static TLS block Andreas Tille <tille@debian.org> - 2020-03-17 13:40 +0100
    Bug#953832: marked as done (cannot allocate memory in static TLS  block) "Debian Bug Tracking System" <owner@bugs.debian.org> - 2020-05-12 18:10 +0200

#273478 — Bug#953832: drmaa: autopkgtest failure: RuntimeError: Could not find drmaa library

[Multipart message — attachments visible in raw view] — view raw

Source: drmaa
Version: 0.7.9-1
X-Debbugs-CC: debian-ci@lists.debian.org
Severity: serious
User: debian-ci@lists.debian.org
Usertags: fails-always

Dear maintainers,

With a recent upload of drmaa you added an autopkgtest, great. However,
it fails. I copied some of the output at the bottom of this report.

Currently this is blocking the migration to testing [1]. Can you please
investigate the situation and fix it?

More information about this bug and the reason for filing it can be found on
https://wiki.debian.org/ContinuousIntegration/RegressionEmailInformation

Paul

[1] https://qa.debian.org/excuses.php?package=drmaa

https://ci.debian.net/data/autopkgtest/testing/amd64/d/drmaa/4335284/log.gz

autopkgtest [03:10:24]: test autodep8-python3: set -e ; for py in
$(py3versions -r 2>/dev/null) ; do cd "$AUTOPKGTEST_TMP" ; echo "Testing
with $py:" ; $py -c "import drmaa; print(drmaa)" ; done
autopkgtest [03:10:24]: test autodep8-python3: [-----------------------
Testing with python3.8:
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib/python3/dist-packages/drmaa/__init__.py", line 65, in
<module>
    from .session import JobInfo, JobTemplate, Session
  File "/usr/lib/python3/dist-packages/drmaa/session.py", line 39, in
<module>
    from drmaa.helpers import (adapt_rusage, Attribute,
attribute_names_iterator,
  File "/usr/lib/python3/dist-packages/drmaa/helpers.py", line 36, in
<module>
    from drmaa.wrappers import (drmaa_attr_names_t, drmaa_attr_values_t,
  File "/usr/lib/python3/dist-packages/drmaa/wrappers.py", line 52, in
<module>
    raise RuntimeError(('Could not find drmaa library.  Please specify
its ' +
RuntimeError: Could not find drmaa library.  Please specify its full
path using the environment variable DRMAA_LIBRARY_PATH
autopkgtest [03:10:24]: test autodep8-python3: -----------------------]

[toc] | [next] | [standalone]

#273552 — Processed: cannot allocate memory in static TLS block (Was: Bug#953832: drmaa: autopkgtest failure: RuntimeError: Could not find drmaa library)

Processing control commands:

> tags -1 help
Bug #953832 [src:drmaa] drmaa: autopkgtest failure: RuntimeError: Could not find drmaa library
Added tag(s) help.
> retitle -1 cannot allocate memory in static TLS block
Bug #953832 [src:drmaa] drmaa: autopkgtest failure: RuntimeError: Could not find drmaa library
Changed Bug title to 'cannot allocate memory in static TLS block' from 'drmaa: autopkgtest failure: RuntimeError: Could not find drmaa library'.

-- 
953832: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=953832
Debian Bug Tracking System
Contact owner@bugs.debian.org with problems

[toc] | [prev] | [next] | [standalone]

#273553 — Bug#953832: cannot allocate memory in static TLS block (Was: Bug#953832: drmaa: autopkgtest failure: RuntimeError: Could not find drmaa library)

Control: tags -1 help
Control: retitle -1 cannot allocate memory in static TLS block

Hi Paul,

On Fri, Mar 13, 2020 at 11:09:31PM +0100, Paul Gevers wrote:
> <module>
>     raise RuntimeError(('Could not find drmaa library.  Please specify
> its ' +
> RuntimeError: Could not find drmaa library.  Please specify its full
> path using the environment variable DRMAA_LIBRARY_PATH

I've fixed this in Git.  Unfortunately I get a new issue when importing
drmaa

$ python3
Python 3.7.6 (default, Jan 19 2020, 22:34:52) 
[GCC 9.2.1 20200117] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import drmaa
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/andreas/debian-maintain/salsa/med-team/python-drmaa/drmaa/__init__.py", line 65, in <module>
    from .session import JobInfo, JobTemplate, Session
  File "/home/andreas/debian-maintain/salsa/med-team/python-drmaa/drmaa/session.py", line 39, in <module>
    from drmaa.helpers import (adapt_rusage, Attribute, attribute_names_iterator,
  File "/home/andreas/debian-maintain/salsa/med-team/python-drmaa/drmaa/helpers.py", line 36, in <module>
    from drmaa.wrappers import (drmaa_attr_names_t, drmaa_attr_values_t,
  File "/home/andreas/debian-maintain/salsa/med-team/python-drmaa/drmaa/wrappers.py", line 58, in <module>
    _lib = CDLL(libpath, mode=RTLD_GLOBAL)
  File "/usr/lib/python3.7/ctypes/__init__.py", line 364, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: /usr/lib/x86_64-linux-gnu/libjemalloc.so.2: cannot allocate memory in static TLS block


Any help would be welcome

     Andreas.

-- 
http://fam-tille.de

[toc] | [prev] | [next] | [standalone]

#273590 — Bug#953832: cannot allocate memory in static TLS block

[Multipart message — attachments visible in raw view] — view raw

On Saturday, March 14 2020, Andreas Tille wrote:

> On Fri, Mar 13, 2020 at 11:09:31PM +0100, Paul Gevers wrote:
>> <module>
>>     raise RuntimeError(('Could not find drmaa library.  Please specify
>> its ' +
>> RuntimeError: Could not find drmaa library.  Please specify its full
>> path using the environment variable DRMAA_LIBRARY_PATH
>
> I've fixed this in Git.  Unfortunately I get a new issue when importing
> drmaa
>
> $ python3
> Python 3.7.6 (default, Jan 19 2020, 22:34:52) 
> [GCC 9.2.1 20200117] on linux
> Type "help", "copyright", "credits" or "license" for more information.
>>>> import drmaa
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
>   File "/home/andreas/debian-maintain/salsa/med-team/python-drmaa/drmaa/__init__.py", line 65, in <module>
>     from .session import JobInfo, JobTemplate, Session
>   File "/home/andreas/debian-maintain/salsa/med-team/python-drmaa/drmaa/session.py", line 39, in <module>
>     from drmaa.helpers import (adapt_rusage, Attribute, attribute_names_iterator,
>   File "/home/andreas/debian-maintain/salsa/med-team/python-drmaa/drmaa/helpers.py", line 36, in <module>
>     from drmaa.wrappers import (drmaa_attr_names_t, drmaa_attr_values_t,
>   File "/home/andreas/debian-maintain/salsa/med-team/python-drmaa/drmaa/wrappers.py", line 58, in <module>
>     _lib = CDLL(libpath, mode=RTLD_GLOBAL)
>   File "/usr/lib/python3.7/ctypes/__init__.py", line 364, in __init__
>     self._handle = _dlopen(self._name, mode)
> OSError: /usr/lib/x86_64-linux-gnu/libjemalloc.so.2: cannot allocate memory in static TLS block
>
>
> Any help would be welcome

This is an issue with jemalloc's handling of the TLS model when being
dlopened..  See:

  https://github.com/jemalloc/jemalloc/issues/1237

The recommended way to build a libjemalloc that is suitable for being
dlopened is to use '--disable-initial-exec-tls' when building it.  Take
a look at the INSTALL.md file, and look for this option:

  https://github.com/jemalloc/jemalloc/blob/dev/INSTALL.md

There is a way to workaround this bug by doing an LD_PRELOAD of
libjemalloc when invoking python, but this will only mask the problem
and we can't expect users to do/know this.

The way I see it, you can try to convince jemalloc's maintainer to
enable that flag.

BTW, the reason 'find_library' can't find drmaa's library is because the
.so is being installed in a non-standard directory.  I don't know why
the package was made like this, though.

Thanks,

-- 
Sergio
GPG key ID: 237A 54B1 0287 28BF 00EF  31F4 D0EB 7628 65FC 5E36
Please send encrypted e-mail if possible
http://sergiodj.net/

[toc] | [prev] | [next] | [standalone]

#273615 — Bug#953832: cannot allocate memory in static TLS block

Hi Faidon,

could you imagine to build jemalloc with --disable-initial-exec-tls
as Sergio suggests below to fix the issue in drmaa (and possibly other
packages)?

Should I open a separate bug report against jemalloc to request this?

Kind regards

      Andreas.

On Sat, Mar 14, 2020 at 05:18:49PM -0400, Sergio Durigan Junior wrote:
> > $ python3
> > Python 3.7.6 (default, Jan 19 2020, 22:34:52) 
> > [GCC 9.2.1 20200117] on linux
> > Type "help", "copyright", "credits" or "license" for more information.
> >>>> import drmaa
> > Traceback (most recent call last):
> >   File "<stdin>", line 1, in <module>
> >   File "/home/andreas/debian-maintain/salsa/med-team/python-drmaa/drmaa/__init__.py", line 65, in <module>
> >     from .session import JobInfo, JobTemplate, Session
> >   File "/home/andreas/debian-maintain/salsa/med-team/python-drmaa/drmaa/session.py", line 39, in <module>
> >     from drmaa.helpers import (adapt_rusage, Attribute, attribute_names_iterator,
> >   File "/home/andreas/debian-maintain/salsa/med-team/python-drmaa/drmaa/helpers.py", line 36, in <module>
> >     from drmaa.wrappers import (drmaa_attr_names_t, drmaa_attr_values_t,
> >   File "/home/andreas/debian-maintain/salsa/med-team/python-drmaa/drmaa/wrappers.py", line 58, in <module>
> >     _lib = CDLL(libpath, mode=RTLD_GLOBAL)
> >   File "/usr/lib/python3.7/ctypes/__init__.py", line 364, in __init__
> >     self._handle = _dlopen(self._name, mode)
> > OSError: /usr/lib/x86_64-linux-gnu/libjemalloc.so.2: cannot allocate memory in static TLS block
> 
> This is an issue with jemalloc's handling of the TLS model when being
> dlopened..  See:
> 
>   https://github.com/jemalloc/jemalloc/issues/1237
> 
> The recommended way to build a libjemalloc that is suitable for being
> dlopened is to use '--disable-initial-exec-tls' when building it.  Take
> a look at the INSTALL.md file, and look for this option:
> 
>   https://github.com/jemalloc/jemalloc/blob/dev/INSTALL.md
> 
> There is a way to workaround this bug by doing an LD_PRELOAD of
> libjemalloc when invoking python, but this will only mask the problem
> and we can't expect users to do/know this.
> 
> The way I see it, you can try to convince jemalloc's maintainer to
> enable that flag.
> 
> BTW, the reason 'find_library' can't find drmaa's library is because the
> .so is being installed in a non-standard directory.  I don't know why
> the package was made like this, though.
> 
> Thanks,
> 
> -- 
> Sergio
> GPG key ID: 237A 54B1 0287 28BF 00EF  31F4 D0EB 7628 65FC 5E36
> Please send encrypted e-mail if possible
> http://sergiodj.net/



-- 
http://fam-tille.de

[toc] | [prev] | [next] | [standalone]

#273764 — Bug#953832: cannot allocate memory in static TLS block

Hi Faidon,

On Tue, Mar 17, 2020 at 01:29:59PM +0200, Faidon Liambotis wrote:
> Hi Andreas,
> 
> Thanks for reaching out. It sounds like this is already reported as
> #951704 (Cc'ed now).

OK.

> I'll need to give this a closer look, but I hope I
> can have an update within the next couple of weeks. Does this work?

Well, drmaa is certainly not the most important package inside Debian so
I see no reason to push you.  But it would be great to have all those
Python3 issues from the table sooner or later since it generated noise
for the most active Python 2->3 migrators.  Just take the time you need.

See you

      Andreas. 

-- 
http://fam-tille.de

[toc] | [prev] | [next] | [standalone]

#278173 — Bug#953832: marked as done (cannot allocate memory in static TLS block)

[Multipart message — attachments visible in raw view] — view raw

Your message dated Tue, 12 May 2020 16:03:27 +0000
with message-id <E1jYXNP-000Ck7-3l@fasolo.debian.org>
and subject line Bug#953832: fixed in drmaa 0.7.9-2
has caused the Debian Bug report #953832,
regarding cannot allocate memory in static TLS block
to be marked as done.

This means that you claim that the problem has been dealt with.
If this is not the case it is now your responsibility to reopen the
Bug report if necessary, and/or fix the problem forthwith.

(NB: If you are a system administrator and have no idea what this
message is talking about, this may indicate a serious mail system
misconfiguration somewhere. Please contact owner@bugs.debian.org
immediately.)


-- 
953832: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=953832
Debian Bug Tracking System
Contact owner@bugs.debian.org with problems

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.bugs.rc

csiph-web