Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.maint.python > #15926 > unrolled thread

newer dask release

Back to article view | Back to linux.debian.maint.python

Contents

  newer dask release Étienne Mollier <emollier@debian.org> - 2024-06-14 23:20 +0200
    Re: newer dask release Diane Trout <diane@ghic.org> - 2024-06-15 04:50 +0200
      Re: newer dask release Étienne Mollier <emollier@debian.org> - 2024-06-15 11:00 +0200
        Re: newer dask release Julian Gilbey <jdg@debian.org> - 2024-06-16 15:00 +0200
          Re: newer dask release Étienne Mollier <emollier@debian.org> - 2024-06-16 23:00 +0200
      Re: newer dask release Étienne Mollier <emollier@debian.org> - 2024-06-16 23:00 +0200
        Re: newer dask release Julian Gilbey <julian@d-and-j.net> - 2024-06-17 09:10 +0200
          Re: newer dask release Étienne Mollier <emollier@debian.org> - 2024-06-17 20:10 +0200
    Re: newer dask release Étienne Mollier <emollier@debian.org> - 2024-06-16 22:50 +0200

#15926 — newer dask release

[Multipart message — attachments visible in raw view] — view raw

Hi Diane, Hi Julian

I'm wrapping up that email as it seems to me there could be some
activity on the dask package from several people at once.

I happen to have a look at the dask.dataframe import issues,
which manifest as at least #1068422, #1069821, and #1069359.
The import problem looks fixed in 2024.5.2, but the new version
also introduced a couple of issues:

  * the following change[1] is needed to fix a test failure[2].

    [1]: https://github.com/dask/dask/pull/11177
    [2]: https://github.com/dask/dask/issues/11176

  * dask.distributed failed its supposedly flaky autopkgtest
    with an error which suggests the two packages might have to
    be uploaded in a lockstep:

    	____________ ERROR collecting protocol/tests/test_highlevelgraph.py ____________
    	/usr/lib/python3/dist-packages/dask/dataframe/__init__.py:22: in _dask_expr_enabled
    	    import dask_expr  # noqa: F401
    	E   ModuleNotFoundError: No module named 'dask_expr'

    Besides, dask.distributed build time tests also regress the
    same way.

  * napari is failing to build from source, but I haven't
    checked yet whether this was caused by introduction of the
    new dask or something else.

Otherwise, according to my tests on reverse dependencies and
reverse build-dependencies on amd64, the new dask did not
introduce regressions.  The majority of packages saw their tests
and builds pass, and the rest had existing failures already.

If that helps, I have the package upgrade staging on my drive,
and may push to salsa after a good night of sleep.  In case you
see reasons I missed for not bumping version too soon, or if you
already went through the upgrade steps already, don't hesitate
to tell me to hold my horses.

I did not focus a lot on the sphinxdoc issue described in the
newly opened #1073183.  I'm not very good with dealing with
sphinxdoc, and would be more tempted to copy the bare rst files
than getting the html files back on tracks.

Have a nice day,  :)
-- 
  .''`.  Étienne Mollier <emollier@debian.org>
 : :' :  pgp: 8f91 b227 c7d6 f2b1 948c  8236 793c f67e 8f0d 11da
 `. `'   sent from /dev/pts/1, please excuse my verbosity
   `-    on air: Suspyre - Siren (One Last Breath)

[toc] | [next] | [standalone]

#15927

[Multipart message — attachments visible in raw view] — view raw

On Fri, 2024-06-14 at 23:16 +0200, Étienne Mollier wrote:
> Hi Diane, Hi Julian
> 
> I'm wrapping up that email as it seems to me there could be some
> activity on the dask package from several people at once.


Yeah I saw that and decided I was overloaded and stopped doing
anything.

Historically it was pretty important to dask and dask.distributed to be
released together, upstream intends them to be matching versions, but I
didn't know how to set the the dependnecy version strings to require
that.



> 
> I happen to have a look at the dask.dataframe import issues,
> which manifest as at least #1068422, #1069821, and #1069359.
> The import problem looks fixed in 2024.5.2, but the new version
> also introduced a couple of issues:
> 
>   * the following change[1] is needed to fix a test failure[2].
> 
>     [1]: https://github.com/dask/dask/pull/11177
>     [2]: https://github.com/dask/dask/issues/11176
> 
>   * dask.distributed failed its supposedly flaky autopkgtest
>     with an error which suggests the two packages might have to
>     be uploaded in a lockstep:

Yes very much yes. Is there a way to nag anyone trying to upload dask
to go check the harder to update distributed before uploading?


> 
> If that helps, I have the package upgrade staging on my drive,
> and may push to salsa after a good night of sleep.  In case you
> see reasons I missed for not bumping version too soon, or if you
> already went through the upgrade steps already, don't hesitate
> to tell me to hold my horses.

That all seems promising to me.

> 
> I did not focus a lot on the sphinxdoc issue described in the
> newly opened #1073183.  I'm not very good with dealing with
> sphinxdoc, and would be more tempted to copy the bare rst files
> than getting the html files back on tracks.
> 

If you want to push what you've done I might have time sunday night/
monday to look at the sphinx problem.

Dask is probably complicated enough to be a good candidate for team
maintenance.

Diane

[toc] | [prev] | [next] | [standalone]

#15929

[Multipart message — attachments visible in raw view] — view raw

Hi Diane,

Diane Trout, on 2024-06-14:
> On Fri, 2024-06-14 at 23:16 +0200, Étienne Mollier wrote:
> > Hi Diane, Hi Julian
> > 
> > I'm wrapping up that email as it seems to me there could be some
> > activity on the dask package from several people at once.
> 
> 
> Yeah I saw that and decided I was overloaded and stopped doing
> anything.
> 
> Historically it was pretty important to dask and dask.distributed to be
> released together, upstream intends them to be matching versions, but I
> didn't know how to set the the dependnecy version strings to require
> that.

It makes sense.  The construction pretty much looks like the
latter is supposed to be a Python submodule of the former.

> > I happen to have a look at the dask.dataframe import issues,
> > which manifest as at least #1068422, #1069821, and #1069359.
> > The import problem looks fixed in 2024.5.2, but the new version
> > also introduced a couple of issues:
> > 
> >   * the following change[1] is needed to fix a test failure[2].
> > 
> >     [1]: https://github.com/dask/dask/pull/11177
> >     [2]: https://github.com/dask/dask/issues/11176
> > 
> >   * dask.distributed failed its supposedly flaky autopkgtest
> >     with an error which suggests the two packages might have to
> >     be uploaded in a lockstep:
> 
> Yes very much yes. Is there a way to nag anyone trying to upload dask
> to go check the harder to update distributed before uploading?

A mention about the interdependency in d/README.source may help,
but getting dask.distributed autopkgtest to not be flaky anymore
would prevent migration of a mismatched distribution to testing,
and raise some alarms in the maneuver; that is, assuming version
mismatch is going to cause test failures in dask.distributed in
every cases.

> > If that helps, I have the package upgrade staging on my drive,
> > and may push to salsa after a good night of sleep.  In case you
> > see reasons I missed for not bumping version too soon, or if you
> > already went through the upgrade steps already, don't hesitate
> > to tell me to hold my horses.
> 
> That all seems promising to me.
> 
> > 
> > I did not focus a lot on the sphinxdoc issue described in the
> > newly opened #1073183.  I'm not very good with dealing with
> > sphinxdoc, and would be more tempted to copy the bare rst files
> > than getting the html files back on tracks.
> > 
> 
> If you want to push what you've done I might have time sunday night/
> monday to look at the sphinx problem.

Okay, I pushed my work in progress on Salsa, so anyone can build
on top of it.  In the meantime I can change target and see to
move dask.distributed forward.

> Dask is probably complicated enough to be a good candidate for team
> maintenance.
> 
> Diane

Thank you for your time putting the additional context!

Have a nice day,  :)
-- 
  .''`.  Étienne Mollier <emollier@debian.org>
 : :' :  pgp: 8f91 b227 c7d6 f2b1 948c  8236 793c f67e 8f0d 11da
 `. `'   sent from /dev/pts/3, please excuse my verbosity
   `-

[toc] | [prev] | [next] | [standalone]

#15930

On Sat, Jun 15, 2024 at 10:50:17AM +0200, Étienne Mollier wrote:
> > Historically it was pretty important to dask and dask.distributed to be
> > released together, upstream intends them to be matching versions, but I
> > didn't know how to set the the dependnecy version strings to require
> > that.
> 
> It makes sense.  The construction pretty much looks like the
> latter is supposed to be a Python submodule of the former.

You could do something like this in the dask.distributed source
package:

Source: dask.distributed
Build-Depends: python3-dask (>> 2024.5.2), python3-dask (<< 2024.5.3~)

and in the binary python3-dask.distributed package similarly:

Depends: python3-dask (>> 2024.5.2), python3-dask (<< 2024.5.3~)

This forces the dask version to match the dask.distributed version.  I
don't know how tight the version numbers need to be, but this works to
keep the packages in step with each other, and will prevent an updated
dask package from migrating if the dask.distributed package has not
been updated in parallel.  Probably more effective than adding
something to README.source (though that will certainly help a bit).

Best wishes,

   Julian

[toc] | [prev] | [next] | [standalone]

#15941

[Multipart message — attachments visible in raw view] — view raw

Hi Julian,

Julian Gilbey, on 2024-06-16:
> On Sat, Jun 15, 2024 at 10:50:17AM +0200, Étienne Mollier wrote:
> > > Historically it was pretty important to dask and dask.distributed to be
> > > released together, upstream intends them to be matching versions, but I
> > > didn't know how to set the the dependnecy version strings to require
> > > that.
> > 
> > It makes sense.  The construction pretty much looks like the
> > latter is supposed to be a Python submodule of the former.
> 
> You could do something like this in the dask.distributed source
> package:
> 
> Source: dask.distributed
> Build-Depends: python3-dask (>> 2024.5.2), python3-dask (<< 2024.5.3~)
> 
> and in the binary python3-dask.distributed package similarly:
> 
> Depends: python3-dask (>> 2024.5.2), python3-dask (<< 2024.5.3~)
> 
> This forces the dask version to match the dask.distributed version.  I
> don't know how tight the version numbers need to be, but this works to
> keep the packages in step with each other, and will prevent an updated
> dask package from migrating if the dask.distributed package has not
> been updated in parallel.  Probably more effective than adding
> something to README.source (though that will certainly help a bit).

Thanks for the hint!  I was not initially happy with the manual
setting of the version, at least for the build dependency, but
thinking a second time about this, it would prompt upload
attempts for newer upstream versions to lookup dask as well as
dask.distributed.  So all in all, this is a fair plan.

Have a nice day,  :)
-- 
  .''`.  Étienne Mollier <emollier@debian.org>
 : :' :  pgp: 8f91 b227 c7d6 f2b1 948c  8236 793c f67e 8f0d 11da
 `. `'   sent from /dev/pts/2, please excuse my verbosity
   `-    on air: Refugee - Grand Canyon: The Source/Theme for t…

[toc] | [prev] | [next] | [standalone]

#15942

[Multipart message — attachments visible in raw view] — view raw

Hi Diane,

Diane Trout, on 2024-06-14:
> On Fri, 2024-06-14 at 23:16 +0200, Étienne Mollier wrote:
> > I did not focus a lot on the sphinxdoc issue described in the
> > newly opened #1073183.  I'm not very good with dealing with
> > sphinxdoc, and would be more tempted to copy the bare rst files
> > than getting the html files back on tracks.
> 
> If you want to push what you've done I might have time sunday night/
> monday to look at the sphinx problem.
> 
> Dask is probably complicated enough to be a good candidate for team
> maintenance.

Thank you for your help with dask documentation, took time to
polish what I could and uploaded dask.  dask.distributed upload
is coming soon, just the time for the last pass of local build
and tests to finish.

Have a nice day,  :)
-- 
  .''`.  Étienne Mollier <emollier@debian.org>
 : :' :  pgp: 8f91 b227 c7d6 f2b1 948c  8236 793c f67e 8f0d 11da
 `. `'   sent from /dev/pts/2, please excuse my verbosity
   `-    on air: Refugee - Grand Canyon: The Source/Theme for t…

[toc] | [prev] | [next] | [standalone]

#15947

On Sun, Jun 16, 2024 at 10:57:22PM +0200, Étienne Mollier wrote:
> Thank you for your help with dask documentation, took time to
> polish what I could and uploaded dask.  dask.distributed upload
> is coming soon, just the time for the last pass of local build
> and tests to finish.
> 
> Have a nice day,  :)

Hi Étienne,

Unfortunately, the new dask.distributed FTBFS:
https://buildd.debian.org/status/package.php?p=dask.distributed

:-(

   Julian

[toc] | [prev] | [next] | [standalone]

#15950

[Multipart message — attachments visible in raw view] — view raw

Hi Julian,

Julian Gilbey, on 2024-06-17:
> On Sun, Jun 16, 2024 at 10:57:22PM +0200, Étienne Mollier wrote:
> > Thank you for your help with dask documentation, took time to
> > polish what I could and uploaded dask.  dask.distributed upload
> > is coming soon, just the time for the last pass of local build
> > and tests to finish.
> > 
> > Have a nice day,  :)
> 
> Hi Étienne,
> 
> Unfortunately, the new dask.distributed FTBFS:
> https://buildd.debian.org/status/package.php?p=dask.distributed

Hmm, I see, it looks like my "isolation" by erasing resov.conf
is just insufficient to catch attempts to make direct use of IP
addresses.  I retried with an unshare chroot, and I already
start seeing the same test items failing.

This prompted me to adjust my configuration to forcefully use my
lingering unshare a bit more often.  Thanks for the follow up!

Have a nice day,  :)
-- 
  .''`.  Étienne Mollier <emollier@debian.org>
 : :' :  pgp: 8f91 b227 c7d6 f2b1 948c  8236 793c f67e 8f0d 11da
 `. `'   sent from /dev/pts/7, please excuse my verbosity
   `-    on air: Rush - Fly By Night

[toc] | [prev] | [next] | [standalone]

#15939

[Multipart message — attachments visible in raw view] — view raw

Étienne Mollier, on 2024-06-14:
>   * napari is failing to build from source, but I haven't
>     checked yet whether this was caused by introduction of the
>     new dask or something else.

After further tests, this turns out to be a pre-existing bug,
and is not a regression introduced by the newer dask, so I
opened #1073504.  I did not identify regressions introduced by
dask.distributed 2024.5.2 neither, at least not on amd64.  This
is very encouraging.

Have a nice day,  :)
-- 
  .''`.  Étienne Mollier <emollier@debian.org>
 : :' :  pgp: 8f91 b227 c7d6 f2b1 948c  8236 793c f67e 8f0d 11da
 `. `'   sent from /dev/pts/2, please excuse my verbosity
   `-

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.maint.python

csiph-web