Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.maint.python > #12138 > unrolled thread

Bug#937262: Help needed (Was: pdb2pqr: Python2 removal in sid/bullseye)

Back to article view | Back to linux.debian.maint.python

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Bug#937262: Help needed (Was: pdb2pqr: Python2 removal in sid/bullseye) Andreas Tille <andreas@an3as.eu> - 2019-12-12 17:20 +0100
    Re: Help with scons needed (Was: Help needed (Was: pdb2pqr: Python2  removal in sid/bullseye)) Scott Talbert <swt@techie.net> - 2019-12-12 21:50 +0100
      Bug#937262: Help with scons needed (Was: Help needed (Was: pdb2pqr: Python2 removal in sid/bullseye)) Andreas Tille <andreas@an3as.eu> - 2019-12-13 10:00 +0100
        Re: Bug#937262: Help with scons needed (Was: Help needed (Was:  pdb2pqr: Python2 removal in sid/bullseye)) Andrey Rahmatullin <wrar@debian.org> - 2019-12-13 11:30 +0100
          Re: Bug#937262: Help with scons needed (Was: Help needed (Was:  pdb2pqr: Python2 removal in sid/bullseye)) Andreas Tille <andreas@an3as.eu> - 2019-12-13 13:10 +0100
        Bug#937262: Help with scons needed (Was: Help needed (Was: pdb2pqr: Python2 removal in sid/bullseye)) Andrey Rahmatullin <wrar@debian.org> - 2019-12-13 13:30 +0100
          Bug#937262: Help with scons needed (Was: Help needed (Was: pdb2pqr: Python2 removal in sid/bullseye)) Andreas Tille <tille@debian.org> - 2019-12-13 13:50 +0100
            Bug#937262: Help with scons needed (Was: Help needed (Was: pdb2pqr: Python2 removal in sid/bullseye)) Andreas Tille <tille@debian.org> - 2019-12-13 16:00 +0100
              Re: Bug#937262: Help with scons needed (Was: Help needed (Was:  pdb2pqr: Python2 removal in sid/bullseye)) Andrey Rahmatullin <wrar@debian.org> - 2019-12-13 19:00 +0100
                Bug#937262: Help needed (Was: pdb2pqr: Python2 removal in sid/bullseye) Andrey Rahmatullin <wrar@debian.org> - 2019-12-13 22:40 +0100

#12138 — Bug#937262: Help needed (Was: pdb2pqr: Python2 removal in sid/bullseye)

Control: tags -1 help

Hi,

it seems pdb2pqr is orphaned upstream.  However, it seems to be worth
keeping inside Debian thus I tried my luck to port it to Python3 in
Git[1].  Unfortunately the build runs into

      scons: Building targets ...
      CopySubAction("pdb2pqr.py", "pdb2pqr.py.in")
      scons: *** [pdb2pqr.py] TypeError : a bytes-like object is required, not 'str'
      Traceback (most recent call last):
        File "/usr/lib/scons/SCons/Action.py", line 1209, in execute
          result = self.execfunction(target=target, source=rsources, env=env)
        File "/usr/lib/scons/SCons/Action.py", line 1371, in __call__
          return self.parent.actfunc(*args, **kw)
        File "./site_scons/site_init.py", line 123, in CopySubAction
          contents = contents.replace(k, v)
      TypeError: a bytes-like object is required, not 'str'
      scons: building terminated because of errors.

I wonder whether it might just be a scons issue.  Please note that I'm
using scons 3.1.1-4 from experimental that is supposed to run with
Python3.

Any hint would be welcome.

Kind regards

       Andreas.


[1] https://salsa.debian.org/med-team/pdb2pqr

-- 
http://fam-tille.de

[toc] | [next] | [standalone]

#12139 — Re: Help with scons needed (Was: Help needed (Was: pdb2pqr: Python2 removal in sid/bullseye))

On Thu, 12 Dec 2019, Andreas Tille wrote:

>> On Thu, 12 Dec 2019, Andreas Tille wrote:
>>
>> I don't see any Python3 changes in that repository.  Did you push your
>> changes?
>
> Argh, its pushed now.
>
>> Anyway, the problem is likely in CopySubAction in site_scons/site_init.py.
>>
>> On line 111, the file 'sourcefile' is opened as binary.  Then, when then
>> next line, 'contents = r.read()' is executed, contents ends up as a bytes
>> object.  Thus on line 123, when 'contents = contents.replace(k, v)' is
>> executed, contents is a bytes object, whereas k and v are strings.  You
>> can't mix strings and bytes objects like that in Python 3.
>>
>> You could perhaps try opening the file as a text file instead (remove the
>> 'b') from the open function call.
>
> That hint was helpful anyway and I get further now.  I think now the
> problem is to convince scons to install in $(CURDIR)/debian/tmp which
> seems to try rather /usr/share/pdb2pqr directly:

Looks like the debian/rules file is specifying /usr/share/pdb2pqr as 
PREFIX:

https://salsa.debian.org/med-team/pdb2pqr/blob/master/debian/rules#L33

Scott

[toc] | [prev] | [next] | [standalone]

#12140 — Bug#937262: Help with scons needed (Was: Help needed (Was: pdb2pqr: Python2 removal in sid/bullseye))

Hi Scott,

On Thu, Dec 12, 2019 at 04:34:09PM -0500, Scott Talbert wrote:
> On Thu, 12 Dec 2019, Andreas Tille wrote:
> > mkdir -p /build/pdb2pqr-2.1.1+dfsg/debian/tmp/usr/share/pdb2pqr
> > scons \
> >        URL="http://localhost/pdb2pqr/" \
> >        PREFIX="/build/pdb2pqr-2.1.1+dfsg/debian/tmp/usr/share/pdb2pqr"
> > scons: Reading SConscript files ...
> > not using opal
> > scons: done reading SConscript files.
> > scons: Building targets ...
> > CopySubAction("pdb2pqr.py", "pdb2pqr.py.in")
> > scons: *** [pdb2pqr.py] Can't write target file pdb2pqr.py
> > scons: building terminated because of errors.
> 
> I think you need to change the other open() call in that function to write
> in text mode also.

Thanks a lot for your patience and your always helpful hints.  After
changing this and some other issues with the C++ code I was (hopefully)
able to solve myself[1] I'm facing the next stumbling stone which is
most probably easy to solve for someone more experienced than me:


...
scons \
        URL="http://localhost/pdb2pqr/" \
        PREFIX="/usr/share/pdb2pqr"
scons: Reading SConscript files ...
not using opal 
scons: done reading SConscript files.
scons: Building targets ...
CopySubAction("pdb2pqr.py", "pdb2pqr.py.in")
Chmod("pdb2pqr.py", 0755)
CopySubAction("apbs_cgi.cgi", "apbs_cgi.py")
Chmod("apbs_cgi.cgi", 0755)
CopySubAction("visualize.cgi", "visualize.py")
Chmod("visualize.cgi", 0755)
CopySubAction("querystatus.cgi", "querystatus.py")
Chmod("querystatus.cgi", 0755)
CopySubAction("src/aconf.py", "src/aconf.py.in")
CopySubAction("html/server.html", "html/server.html.in")
Copy("pdb2pqr.cgi", "pdb2pqr.py")
g++ -o pdb2pka/substruct/Algorithms.os -c -g -fPIC -I/usr/include/python3.7m -I/usr/lib/python3/dist-packages/numpy/core/include pdb2pka/substruct/Algorithms.cpp
In file included from /usr/include/python3.7m/numpy/ndarraytypes.h:1830,
                 from /usr/include/python3.7m/numpy/ndarrayobject.h:12,
                 from /usr/include/python3.7m/numpy/arrayobject.h:4,
                 from pdb2pka/substruct/Algorithms.cpp:43:
/usr/include/python3.7m/numpy/npy_1_7_deprecated_api.h:17:2: warning: #warning "Using deprecated NumPy API, disable it with " "#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]
   17 | #warning "Using deprecated NumPy API, disable it with " \
      |  ^~~~~~~
pdb2pka/substruct/Algorithms.cpp: In function 'PyObject* initAlgorithms()':
pdb2pka/substruct/Algorithms.cpp:333:1: warning: control reaches end of non-void function [-Wreturn-type]
  333 | };
      | ^
g++ -o pdb2pka/substruct/Algorithms.cpython-37m-x86_64-linux-gnu.so -Wl,-z,relro -Wl,-z,now -shared pdb2pka/substruct/Algorithms.os -L/usr/lib -lpython3.7
/usr/bin/ld: cannot find -lpython3.7
collect2: error: ld returned 1 exit status
scons: *** [pdb2pka/substruct/Algorithms.cpython-37m-x86_64-linux-gnu.so] Error 1
scons: building terminated because of errors.

TARGETS: ['pdb2pqr.py', 'apbs_cgi.cgi', 'visualize.cgi', 'querystatus.cgi', 'src/aconf.py', 'html/server.html', 'pdb2pqr.cgi', 'pdb2pka/substruct/Algorithms.cpython-37m-x86_64-linux-gnu.so', 'pdb2pka/_pMC_mult.cpython-37m-x86_64-linux-gnu.so']

========================
Configuration Parameters
========================

Version: 2.1.1
Install directory: /usr/share/pdb2pqr/
pdb2pka and ligand support: True
Path to the website directory: http://localhost/pdb2pqr/
PDB2PQR jobs run via the web interface will be forked on the server.

The preferred way to configure the build is by editing the file build_config.py

Run scons with the python3 that you intend to use with pdb2pqr.
For example: "scons" will setup pdb2pqr to be run with Debian default Python3 interpreter

Run "scons install" to install pdb2pqr in /usr/share/pdb2pqr/

Run "scons basic-test" for a basic functionality test
Run "scons advanced-test" for a single test of ligand and PROPKA support. Requires numpy and PDB2PKA support compiled.
Run "scons complete-test" for a complete test of all functionality EXCEPT PDB2PKA. Requires numpy and PDB2PKA support compiled.
Run "scons pdb2pka-test" for a test of PDB2PKA functionality.
    Requires numpy, PDB2PKA support compiled AND the APBS python3 libraries compiled and installed in the pdb2pka directory.

To setup a web service create a symbolic link to /usr/share/pdb2pqr/ that enables you to view http://localhost/pdb2pqr/ after running "scons install"

Run "scons msvs" to build Visual Studio projects for the Algorithms and pMC_mult modules.
VS project generation is not well supported in scons. Resulting projects should build using NMAKE but cannot be used for debugging.
The resulting projects will need to modified to use VS natively to compile the code or debug.

FAILED!!!!
Failed building pdb2pka/substruct/Algorithms.cpython-37m-x86_64-linux-gnu.so: Error 1
...


So how can I enable proper linking to -lpython3.7 which is obviously not
in the library search path (but package libpython3.7 is definitely
installed in the pbuilder chroot).

Any further hint?

Kind regards and thanks again for all your help

     Andreas.


[1] https://salsa.debian.org/med-team/pdb2pqr/commit/2497134056ff92e64a5953e1b8dea3179d5aa377
 
-- 
http://fam-tille.de

[toc] | [prev] | [next] | [standalone]

#12141 — Re: Bug#937262: Help with scons needed (Was: Help needed (Was: pdb2pqr: Python2 removal in sid/bullseye))

[Multipart message — attachments visible in raw view] — view raw

On Fri, Dec 13, 2019 at 09:49:47AM +0100, Andreas Tille wrote:
> So how can I enable proper linking to -lpython3.7 which is obviously not
> in the library search path (but package libpython3.7 is definitely
> installed in the pbuilder chroot).
To link to a library you need to install its -dev package (this is not just
for libpython*).

-- 
WBR, wRAR

[toc] | [prev] | [next] | [standalone]

#12142 — Re: Bug#937262: Help with scons needed (Was: Help needed (Was: pdb2pqr: Python2 removal in sid/bullseye))

On Fri, Dec 13, 2019 at 03:23:19PM +0500, Andrey Rahmatullin wrote:
> On Fri, Dec 13, 2019 at 09:49:47AM +0100, Andreas Tille wrote:
> > So how can I enable proper linking to -lpython3.7 which is obviously not
> > in the library search path (but package libpython3.7 is definitely
> > installed in the pbuilder chroot).
> To link to a library you need to install its -dev package (this is not just
> for libpython*).

Thus python3-dev is in the Build-Depends, yes.  The library is just not found
by the linker.

Kind regards

      Andreas.

-- 
http://fam-tille.de

[toc] | [prev] | [next] | [standalone]

#12143 — Bug#937262: Help with scons needed (Was: Help needed (Was: pdb2pqr: Python2 removal in sid/bullseye))

[Multipart message — attachments visible in raw view] — view raw

On Fri, Dec 13, 2019 at 09:49:47AM +0100, Andreas Tille wrote:
> g++ -o pdb2pka/substruct/Algorithms.cpython-37m-x86_64-linux-gnu.so -Wl,-z,relro -Wl,-z,now -shared pdb2pka/substruct/Algorithms.os -L/usr/lib -lpython3.7
> /usr/bin/ld: cannot find -lpython3.7
Actually, it's a different problem, sorry.
There is no -lpython3.7, only -lpython3.7m. If the build system used
pkgconfig it would know that. I guess the library name is hardcoded
instead? I see that it got -I/usr/include/python3.7m from somewhere, so
it's not completely broken.

-- 
WBR, wRAR

[toc] | [prev] | [next] | [standalone]

#12144 — Bug#937262: Help with scons needed (Was: Help needed (Was: pdb2pqr: Python2 removal in sid/bullseye))

Hi Andrey,

On Fri, Dec 13, 2019 at 05:18:45PM +0500, Andrey Rahmatullin wrote:
> On Fri, Dec 13, 2019 at 09:49:47AM +0100, Andreas Tille wrote:
> > g++ -o pdb2pka/substruct/Algorithms.cpython-37m-x86_64-linux-gnu.so -Wl,-z,relro -Wl,-z,now -shared pdb2pka/substruct/Algorithms.os -L/usr/lib -lpython3.7
> > /usr/bin/ld: cannot find -lpython3.7
> Actually, it's a different problem, sorry.
> There is no -lpython3.7, only -lpython3.7m. If the build system used
> pkgconfig it would know that. I guess the library name is hardcoded
> instead? I see that it got -I/usr/include/python3.7m from somewhere, so
> it's not completely broken.

Thanks for that additional hint.  I can confirm that I checked whether
pkgconfig can be used before I was asking here.  But we seem to agree
that this is not the case.  I admit I have no real clue how to convince
scons to link to the correct library name. :-(

Kind regards

      Andreas.

-- 
http://fam-tille.de

[toc] | [prev] | [next] | [standalone]

#12145 — Bug#937262: Help with scons needed (Was: Help needed (Was: pdb2pqr: Python2 removal in sid/bullseye))

On Fri, Dec 13, 2019 at 03:16:32PM +0100, Andreas Tille wrote:
> 
> ...
> g++ -o pdb2pka/substruct/Algorithms.cpython-37m-x86_64-linux-gnu.so -Wl,-z,relro -Wl,-z,now -shared pdb2pka/substruct/Algorithms.os -L/usr/lib -lpython3.7m -lpython3.7
> /usr/bin/ld: cannot find -lpython3.7
> collect2: error: ld returned 1 exit status
> ...
> 
> AAArgh, now we just need to get rid of the unwanted stuff.  Any scons
> expert around?

I think at least this is solved now:

   https://salsa.debian.org/med-team/pdb2pqr/commit/1f4ee901456641140ef18ca8e91e4701e1175410

I might come back with other issues

     Andreas.

-- 
http://fam-tille.de

[toc] | [prev] | [next] | [standalone]

#12146 — Re: Bug#937262: Help with scons needed (Was: Help needed (Was: pdb2pqr: Python2 removal in sid/bullseye))

[Multipart message — attachments visible in raw view] — view raw

On Fri, Dec 13, 2019 at 03:49:22PM +0100, Andreas Tille wrote:
> > g++ -o pdb2pka/substruct/Algorithms.cpython-37m-x86_64-linux-gnu.so -Wl,-z,relro -Wl,-z,now -shared pdb2pka/substruct/Algorithms.os -L/usr/lib -lpython3.7m -lpython3.7
> > /usr/bin/ld: cannot find -lpython3.7
> > collect2: error: ld returned 1 exit status
> > ...
> > 
> > AAArgh, now we just need to get rid of the unwanted stuff.  Any scons
> > expert around?
> 
> I think at least this is solved now:
> 
>    https://salsa.debian.org/med-team/pdb2pqr/commit/1f4ee901456641140ef18ca8e91e4701e1175410

Nice catch, "env.Append(LIBS=[python_lib])" where "python_lib = 'python' +
gcv('VERSION')" is definitely the cause.

-- 
WBR, wRAR

[toc] | [prev] | [next] | [standalone]

#12147

[Multipart message — attachments visible in raw view] — view raw

On Fri, Dec 13, 2019 at 10:10:03PM +0100, Andreas Tille wrote:
> i$ pdb2pqr
> Traceback (most recent call last):
>   File "/usr/bin/pdb2pqr", line 52, in <module>
>     from main import mainCommand
>   File "/usr/share/pdb2pqr/main.py", line 77, in <module>
>     import extensions
>   File "/usr/share/pdb2pqr/extensions/__init__.py", line 56, in <module>
>     extDict[extName] = __import__(extName,globals(),locals(),[], -1)
> ValueError: level must be >= 0

"""
level specifies whether to use absolute or relative imports. The default
is -1 which indicates both absolute and relative imports will be
attempted. 0 means only perform absolute imports.  Positive values for
level indicate the number of parent directories to search relative to the
directory of the module calling __import__().
"""
https://docs.python.org/2.7/library/functions.html#__import__

-1 was removed in 3.3 as implicit relative import are not supported in
3.x. As this code seems to use relative imports you need to change -1 to
1.

> So I tried:
> 
> --- a/extensions/__init__.py
> +++ b/extensions/__init__.py
> @@ -53,7 +53,7 @@ _extList = [name for _, name, _ in pkgutil.iter_modules(__path__)]
>  extDict = {}
>  
>  for extName in _extList:
> -    extDict[extName] = __import__(extName,globals(),locals(),[], -1)
> +    extDict[extName] = __import__(extName,globals(),locals(),[], 0)^M

Mindlessly changing the code is almost always a bad idea...

>   File "/usr/share/pdb2pqr/extensions/__init__.py", line 57, in <module>
>     extDict[extName] = __import__(extName,globals(),locals(),[], 0)
> ModuleNotFoundError: No module named 'chi'
This is expected as it now tries to do "import chi". With 1 it should try
"from . import chi". This fails later, as the source also has implicit
relative imports that needs to be fixed, for example "from aconf import"
in src/utilities.py.

-- 
WBR, wRAR

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.maint.python

csiph-web