Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.maint.python > #14836 > unrolled thread

Helping hands needed to upgrade scipy

Back to article view | Back to linux.debian.maint.python

Contents

  Helping hands needed to upgrade scipy Andreas Tille <andreas@an3as.eu> - 2023-01-17 18:20 +0100
    Re: Helping hands needed to upgrade scipy Jerome Kieffer <Jerome.Kieffer@esrf.fr> - 2023-01-17 18:50 +0100
    Re: Helping hands needed to upgrade scipy Drew Parsons <dparsons@debian.org> - 2023-01-18 02:50 +0100
      Re: Helping hands needed to upgrade scipy Andreas Tille <tille@debian.org> - 2023-01-18 08:30 +0100

#14836 — Helping hands needed to upgrade scipy

Hi,

I upgraded the experimental branch of scipy to contain the submodules
upstream includes.  I'm *not* convinced that we need them all -
hopefully we can get rid of boost (but upstream has some patches here)
and scipy-mathjax.

For the moment I've tried to build the experimental branch and ended
up with some issue where cython can't be found (should be cython3 for
sure)[1].

Unfortunately my spare time is occupied now by other more urgent tasks.
I'd be happy if someone else could review / pick up / continue from
here.

Kind regards
    Andreas.


[1] https://salsa.debian.org/python-team/packages/scipy/-/jobs/3809903

-- 
http://fam-tille.de

[toc] | [next] | [standalone]

#14838

Hi Andreas,

I am an upstream author of software packages (fabio & pyFAI) which took
the same build system as scipy (meson-python) and I can confirm you
there are issues with the debian packaging:

* finding cython (apparently it should be fixed upstream in meson)
* choose the proper python: sid has currently py3.10 and 3.11
  co-installed and I found out the wrong one was used when building:
https://github.com/mesonbuild/meson/issues/11248

I hope this helps,

Jerome


On Tue, 17 Jan 2023 18:12:42 +0100
Andreas Tille <andreas@an3as.eu> wrote:

> Hi,
> 
> I upgraded the experimental branch of scipy to contain the submodules
> upstream includes.  I'm *not* convinced that we need them all -
> hopefully we can get rid of boost (but upstream has some patches here)
> and scipy-mathjax.
> 
> For the moment I've tried to build the experimental branch and ended
> up with some issue where cython can't be found (should be cython3 for
> sure)[1].
> 
> Unfortunately my spare time is occupied now by other more urgent tasks.
> I'd be happy if someone else could review / pick up / continue from
> here.
> 
> Kind regards
>     Andreas.
> 
> 
> [1] https://salsa.debian.org/python-team/packages/scipy/-/jobs/3809903
> 
> -- 
> http://fam-tille.de
> 

[toc] | [prev] | [next] | [standalone]

#14845

On 2023-01-17 18:12, Andreas Tille wrote:
> Hi,
> 
> I upgraded the experimental branch of scipy to contain the submodules
> upstream includes.  I'm *not* convinced that we need them all -
> hopefully we can get rid of boost (but upstream has some patches here)
> and scipy-mathjax.
> 
> For the moment I've tried to build the experimental branch and ended
> up with some issue where cython can't be found (should be cython3 for
> sure)[1].
> 
> Unfortunately my spare time is occupied now by other more urgent tasks.
> I'd be happy if someone else could review / pick up / continue from
> here.


The problem was not meson as such, but that upstream is only half using 
meson.  They must have decided they don't trust the way meson handles 
cython, and set up their own custom build tools for it, see 
https://github.com/scipy/scipy/pull/15407

That is, they hard-coded the cython executable name in 
scipy/_build_utils/cythoner.py

They actually did the same already in the old setuptools build 
(tools/cythonize.py), but we didn't notice since they had the breaking 
cython call wrapped in a try block which inspected a Cython class from 
inside python.  I raised an Issue about  whether cythoner.py should do 
the same in https://github.com/scipy/scipy/issues/17808
We could argue the fault is debian's for not providing a cython 
executable (it does have -2 and -3 options for choosing the python 
version to work with).

I figure also we should use python3-mesonpy, since it's what upstream 
uses. Best not to diverge too far from their build method I think.

I've pushed commits updating cythoner.py to point cythoner.py at 
cython3.  The main build now completes. Have to clean up test handling 
next: "FileNotFoundError: [Errno 2] No such file or directory: 
'/projects/python/build/scipy/.coveragerc'

Drew

[toc] | [prev] | [next] | [standalone]

#14846

Hi Drew,

Am Wed, Jan 18, 2023 at 02:45:31AM +0100 schrieb Drew Parsons:
> On 2023-01-17 18:12, Andreas Tille wrote:
> > Hi,
> > 
> > I upgraded the experimental branch of scipy to contain the submodules
> > upstream includes.  I'm *not* convinced that we need them all -
> > hopefully we can get rid of boost (but upstream has some patches here)
> > and scipy-mathjax.
> > 
> > For the moment I've tried to build the experimental branch and ended
> > up with some issue where cython can't be found (should be cython3 for
> > sure)[1].
> > 
> > Unfortunately my spare time is occupied now by other more urgent tasks.
> > I'd be happy if someone else could review / pick up / continue from
> > here.
> 
> 
> The problem was not meson as such, but that upstream is only half using
> meson.  They must have decided they don't trust the way meson handles
> cython, and set up their own custom build tools for it, see
> https://github.com/scipy/scipy/pull/15407
> 
> That is, they hard-coded the cython executable name in
> scipy/_build_utils/cythoner.py
> 
> They actually did the same already in the old setuptools build
> (tools/cythonize.py), but we didn't notice since they had the breaking
> cython call wrapped in a try block which inspected a Cython class from
> inside python.  I raised an Issue about  whether cythoner.py should do the
> same in https://github.com/scipy/scipy/issues/17808
> We could argue the fault is debian's for not providing a cython executable
> (it does have -2 and -3 options for choosing the python version to work
> with).

Thanks for the clarification.

> I figure also we should use python3-mesonpy, since it's what upstream uses.
> Best not to diverge too far from their build method I think.

I'm perfectly fine with this approach.  That's why I initially pushed
meson-python to new.  I've just dput meson-python_0.12.0-2_source to let
it migrate to testing.  My attempt to use plain meson (as per upstream
readme) was just to shorten the time while meson-python was in new
queue.

> I've pushed commits updating cythoner.py to point cythoner.py at cython3.
> The main build now completes. Have to clean up test handling next:
> "FileNotFoundError: [Errno 2] No such file or directory:
> '/projects/python/build/scipy/.coveragerc'

Nice.  Hope my preparation (of meson-python and the multi-source tarball)
helped to move forward.  Please double check whether we really need the
boost code copy. 

Kind regards

     Andreas.

-- 
http://fam-tille.de

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.maint.python

csiph-web