Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.maint.python > #9789 > unrolled thread

Bug#729956: Forwarded upstream

Back to article view | Back to linux.debian.maint.python

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Bug#729956: Forwarded upstream Andreas Tille <tille@debian.org> - 2017-09-06 22:30 +0200
    Bug#729956: Forwarded upstream Yuri D'Elia <wavexx@thregr.org> - 2017-09-06 22:40 +0200
      Bug#729956: Forwarded upstream Andreas Tille <tille@debian.org> - 2017-09-06 23:00 +0200
        Bug#729956: git-dpm (was Re: Bug#729956: Forwarded upstream) Yuri D'Elia <wavexx@thregr.org> - 2017-09-06 23:20 +0200
          Re: git-dpm (was Re: Bug#729956: Forwarded upstream) Scott Kitterman <debian@kitterman.com> - 2017-09-07 00:50 +0200
            Re: git-dpm (was Re: Bug#729956: Forwarded upstream) Brian May <brian@linuxpenguins.xyz> - 2017-09-07 01:20 +0200
              Re: git-dpm (was Re: Bug#729956: Forwarded upstream) Yuri D'Elia <wavexx@thregr.org> - 2017-09-08 00:50 +0200
            Re: git-dpm (was Re: Bug#729956: Forwarded upstream) Brian May <brian@linuxpenguins.xyz> - 2017-09-07 01:30 +0200
              Re: git-dpm (was Re: Bug#729956: Forwarded upstream) Scott Kitterman <debian@kitterman.com> - 2017-09-07 07:00 +0200
                Re: git-dpm (was Re: Bug#729956: Forwarded upstream) Brian May <brian@linuxpenguins.xyz> - 2017-09-07 07:30 +0200
    Bug#729956: Forwarded upstream Diane Trout <diane@ghic.org> - 2017-09-07 00:30 +0200
      Bug#729956: Forwarded upstream Matthias Klose <doko@debian.org> - 2017-09-07 06:20 +0200
      Bug#729956: Forwarded upstream Andreas Tille <andreas@an3as.eu> - 2017-09-07 06:30 +0200
        Bug#729956: Forwarded upstream Diane Trout <diane@ghic.org> - 2017-09-07 06:40 +0200
          Bug#729956: Forwarded upstream Andreas Tille <andreas@an3as.eu> - 2017-09-07 08:40 +0200
            Bug#729956: Forwarded upstream Diane Trout <diane@ghic.org> - 2017-09-08 08:40 +0200
    Bug#729956: Python 3 Statsmodels & Pandas Diane Trout <diane@ghic.org> - 2017-09-16 22:50 +0200
      Bug#729956: Python 3 Statsmodels & Pandas Yuri D'Elia <wavexx@thregr.org> - 2017-09-16 23:10 +0200
        Bug#729956: Python 3 Statsmodels & Pandas Diane Trout <diane@ghic.org> - 2017-09-16 23:30 +0200
          Bug#729956: Python 3 Statsmodels & Pandas Yuri D'Elia <wavexx@thregr.org> - 2017-09-17 00:00 +0200
            Re: skimage (was: Re: Bug#729956: Python 3 Statsmodels & Pandas) Yaroslav Halchenko <yoh@debian.org> - 2017-10-13 00:10 +0200
            skimage (was: Re: Bug#729956: Python 3 Statsmodels & Pandas) Yuri D'Elia <wavexx@thregr.org> - 2017-10-13 00:40 +0200
          Re: Python 3 Statsmodels & Pandas Stuart Prescott <stuart@debian.org> - 2017-09-18 04:00 +0200
            Re: Python 3 Statsmodels & Pandas Diane Trout <diane@ghic.org> - 2017-09-18 07:20 +0200
              Re: Python 3 Statsmodels & Pandas Stuart Prescott <stuart@debian.org> - 2017-09-18 10:10 +0200
                Re: Python 3 Statsmodels & Pandas Gordon Ball <gordon@chronitis.net> - 2017-09-18 19:30 +0200
                  Re: Python 3 Statsmodels & Pandas Matthias Klose <doko@debian.org> - 2017-09-18 21:30 +0200
                    Re: Python 3 Statsmodels & Pandas Diane Trout <diane@ghic.org> - 2017-09-19 06:50 +0200
                Re: Python 3 Statsmodels & Pandas Andreas Tille <andreas@an3as.eu> - 2017-09-21 16:30 +0200
                  Re: Python 3 Statsmodels & Pandas Ghislain Vaillant <ghisvail@gmail.com> - 2017-09-21 17:00 +0200
                  Re: Python 3 Statsmodels & Pandas Diane Trout <diane@ghic.org> - 2017-09-21 23:50 +0200
                    Re: Python 3 Statsmodels & Pandas Diane Trout <diane@ghic.org> - 2017-09-22 00:10 +0200
                      Re: Python 3 Statsmodels & Pandas Yaroslav Halchenko <debian@onerussian.com> - 2017-09-22 04:30 +0200
                        Re: Python 3 Statsmodels & Pandas Diane Trout <diane@ghic.org> - 2017-09-22 07:20 +0200
                          Re: Python 3 Statsmodels & Pandas Diane Trout <diane@ghic.org> - 2017-09-24 20:30 +0200
                            Re: Python 3 Statsmodels & Pandas Andreas Tille <andreas@fam-tille.de> - 2017-09-25 00:10 +0200
                              Re: Python 3 Statsmodels & Pandas Yaroslav Halchenko <debian@onerussian.com> - 2017-09-25 01:00 +0200
                                Re: Python 3 Statsmodels & Pandas Diane Trout <diane@ghic.org> - 2017-09-25 01:20 +0200
                            Re: Python 3 Statsmodels & Pandas Diane Trout <diane@ghic.org> - 2017-09-25 08:50 +0200
                              Re: Python 3 Statsmodels & Pandas Andreas Tille <andreas@an3as.eu> - 2017-09-25 09:50 +0200
                                Re: Python 3 Statsmodels & Pandas Diane Trout <diane@ghic.org> - 2017-09-25 19:40 +0200
                                  Re: Python 3 Statsmodels & Pandas Andreas Tille <andreas@an3as.eu> - 2017-09-25 19:50 +0200
                                Re: Python 3 Statsmodels & Pandas Diane Trout <diane@ghic.org> - 2017-09-26 06:40 +0200
                                  Re: Python 3 Statsmodels & Pandas Andreas Tille <andreas@fam-tille.de> - 2017-09-26 09:20 +0200
                                    Re: Python 3 Statsmodels & Pandas Diane Trout <diane@ghic.org> - 2017-09-26 20:40 +0200
                                      Re: Python 3 Statsmodels & Pandas Yaroslav Halchenko <debian@onerussian.com> - 2017-09-26 22:50 +0200
                                      Re: Python 3 Statsmodels & Pandas Andreas Tille <andreas@an3as.eu> - 2017-09-26 23:00 +0200
                                        Re: Python 3 Statsmodels & Pandas Diane Trout <diane@ghic.org> - 2017-09-26 23:50 +0200
                                          Re: Python 3 Statsmodels & Pandas Andreas Tille <andreas@fam-tille.de> - 2017-09-27 08:20 +0200
                                        Re: Python 3 Statsmodels & Pandas Diane Trout <diane@ghic.org> - 2017-09-27 07:50 +0200
                                          Re: Python 3 Statsmodels & Pandas Andreas Tille <andreas@fam-tille.de> - 2017-09-27 09:00 +0200
                                            Re: Python 3 Statsmodels & Pandas Diane Trout <diane@ghic.org> - 2017-09-27 09:30 +0200
                                              Re: Python 3 Statsmodels & Pandas Andreas Tille <andreas@an3as.eu> - 2017-09-27 10:10 +0200
                                                Re: Python 3 Statsmodels & Pandas Andreas Tille <andreas@an3as.eu> - 2017-09-27 12:20 +0200
                                                  Re: Python 3 Statsmodels & Pandas "Rebecca N. Palmer" <rebecca_palmer@zoho.com> - 2017-09-29 23:40 +0200
                                                    Re: Python 3 Statsmodels & Pandas Diane Trout <diane@ghic.org> - 2017-09-30 01:00 +0200
                                                      Re: Python 3 Statsmodels & Pandas "Rebecca N. Palmer" <rebecca_palmer@zoho.com> - 2017-09-30 10:20 +0200
                                                      Re: Python 3 Statsmodels & Pandas Ghislain Vaillant <ghisvail@gmail.com> - 2017-09-30 11:20 +0200
                                                      Re: Python 3 Statsmodels & Pandas Dmitry Shachnev <mitya57@debian.org> - 2017-09-30 11:30 +0200
                                                        Re: Python 3 Statsmodels & Pandas Diane Trout <diane@ghic.org> - 2017-09-30 18:00 +0200
                                                          Re: Python 3 Statsmodels & Pandas "Rebecca N. Palmer" <rebecca_palmer@zoho.com> - 2017-09-30 18:40 +0200
                        Re: Python 3 Statsmodels & Pandas Andreas Tille <andreas@an3as.eu> - 2017-09-22 08:20 +0200
                          Re: Python 3 Statsmodels & Pandas Yaroslav Halchenko <debian@onerussian.com> - 2017-09-23 16:10 +0200
                            Re: Python 3 Statsmodels & Pandas Andreas Tille <andreas@an3as.eu> - 2017-09-24 13:30 +0200
                              Re: Python 3 Statsmodels & Pandas Yaroslav Halchenko <debian@onerussian.com> - 2017-09-25 01:00 +0200
                    Re: Python 3 Statsmodels & Pandas Yaroslav Halchenko <yoh@onerussian.com> - 2017-09-22 00:20 +0200
                    Re: Python 3 Statsmodels & Pandas Piotr Ożarowski <piotr@debian.org> - 2017-09-22 11:00 +0200
                      Re: Python 3 Statsmodels & Pandas Diane Trout <diane@ghic.org> - 2017-09-22 21:40 +0200
                        Re: Python 3 Statsmodels & Pandas Ghislain Vaillant <ghisvail@gmail.com> - 2017-09-23 11:50 +0200
                      Re: Python 3 Statsmodels & Pandas Yaroslav Halchenko <debian@onerussian.com> - 2017-09-23 16:00 +0200
                        Re: Python 3 Statsmodels & Pandas Piotr Ożarowski <piotr@debian.org> - 2017-09-25 11:20 +0200
        Bug#729956: Python 3 Statsmodels & Pandas Matthias Klose <doko@debian.org> - 2017-09-17 11:20 +0200

Page 2 of 4 — ← Prev page 1 [2] 3 4  Next page →

#9928 — Re: skimage (was: Re: Bug#729956: Python 3 Statsmodels & Pandas)

On Tue, 10 Oct 2017, Yuri D'Elia wrote:

> On Sat, Sep 16 2017, Yuri D'Elia wrote:
> > Looking at python3-skimage-lib (which also requires a rebuild), it seems
> > that the package failed to pass some tests.

> > Bug #868582 even includes a patch to update to 0.13 [and disables some
> > test failures].

> Has anyone had a chance to look at skimage btw?

my bad 

will look into updating skimage now. thanks for the buzz

-- 
Yaroslav O. Halchenko
Center for Open Neuroscience     http://centerforopenneuroscience.org
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834                       Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik        

[toc] | [prev] | [next] | [standalone]

#9931 — skimage (was: Re: Bug#729956: Python 3 Statsmodels & Pandas)

On Sat, Sep 16 2017, Yuri D'Elia wrote:
> Looking at python3-skimage-lib (which also requires a rebuild), it seems
> that the package failed to pass some tests.
>
> Bug #868582 even includes a patch to update to 0.13 [and disables some
> test failures].

Has anyone had a chance to look at skimage btw?

[toc] | [prev] | [next] | [standalone]

#9814 — Re: Python 3 Statsmodels & Pandas

Hi Diane & other science+python people,

Thanks for spending time on pandas and statsmodels. They are two very valuable 
but very painful packages to work on.

I've not looked at pandas and statsmodels packages carefully beyond "why 
hasn't pandas migrated this time!?" investigations but the circular 
relationship between pandas and statsmodels and its associated problems has 
come up several times before. It seems to be an on-going point of pain that 
sucks in quite a lot of developer time.

On Saturday, 16 September 2017 14:18:10 AEST Diane Trout wrote:
> My solution was to use build-profiles to flag the test dependency with
> !nocheck

this is, of course, a very elegant solution and exactly what build profiles 
are for...

I wonder though, is it really sustainable given this problem keeps coming up?

For the maintainer upload, you can do this bootstrapping with build profiles. 
For other archs, however, I don't believe we can tell the buildd to use build 
profiles so we'd need it to built by hand on porter boxes for every arch and 
then uploaded.

Is it feaible to completely break this circular dependency? If it is only 
needed for tests, would be possible to disable the build-time tests and rely 
on the tests run on ci.d.n instead? If it's used for generating documentation 
too, then would creating a separate source package for the documentation that 
build-depends on statsmodels and pandas be possible such that there is now no 
circular dependency?

(thinking aloud)

cheers
Stuart


-- 
Stuart Prescott    http://www.nanonanonano.net/   stuart@nanonanonano.net
Debian Developer   http://www.debian.org/         stuart@debian.org
GPG fingerprint    90E2 D2C1 AD14 6A1B 7EBB 891D BBC1 7EBB 1396 F2F7

[toc] | [prev] | [next] | [standalone]

#9815 — Re: Python 3 Statsmodels & Pandas

> On Saturday, 16 September 2017 14:18:10 AEST Diane Trout wrote:
> > My solution was to use build-profiles to flag the test dependency
> > with
> > !nocheck
> 
> this is, of course, a very elegant solution and exactly what build
> profiles 
> are for...
> 
> I wonder though, is it really sustainable given this problem keeps
> coming up?

I just did it that way because it was the least disruptive change I
could make that would let me build and test the package.

Also I wasn't sure how much magic was happening on the Debian build
servers.

> 
> Is it feaible to completely break this circular dependency? If it is
> only 
> needed for tests, would be possible to disable the build-time tests
> and rely 

In my experience I'm much more likely to to notice a build time test
failure than one from the CI system. Though I did adjust my pbuilder
config to fail on test failures in the autopkgtests so I already solved
my own concern... 

What do other people think? If there are autopkgtests, should you still
let dh_auto_test run tests?

Diane

[toc] | [prev] | [next] | [standalone]

#9816 — Re: Python 3 Statsmodels & Pandas

Hi Diane,

On Sunday, 17 September 2017 22:14:18 AEST Diane Trout wrote:
> I just did it that way because it was the least disruptive change I
> could make that would let me build and test the package.

Sure, that's entirely sensible.

> In my experience I'm much more likely to to notice a build time test
> failure than one from the CI system. 

Absolutely agreed. I'm thinking that this is a rather exceptional situation 
because of the circular dependency and the fall-out that we regularly see from 
that.

> What do other people think? If there are autopkgtests, should you still
> let dh_auto_test run tests?

(I know I'm not the 'other people' from whom you solicit replies, I was just 
perhaps unclear in what I was musing on before so I want to be more clear now)

Like you, I would *normally* run all tests in both places: buildd tests catch 
severe problems right now; ci.d.n reruns the tests each time new versions of 
dependencies are uploaded too and later breakage is caught.

Perhaps this is not a normal situation, however. Either one of "circular 
dependencies" or "packages that often FTBFS on one or more architectures" 
would be unusual enough to justify doing something different. All I am 
wondering (from my position of ignorance!) if in this case, perhaps the tests 
that cause the circular dependency can be disabled or xfailed, with the 
remaining tests run as normal.

cheers
Stuart


-- 
Stuart Prescott    http://www.nanonanonano.net/   stuart@nanonanonano.net
Debian Developer   http://www.debian.org/         stuart@debian.org
GPG fingerprint    90E2 D2C1 AD14 6A1B 7EBB 891D BBC1 7EBB 1396 F2F7

[toc] | [prev] | [next] | [standalone]

#9817 — Re: Python 3 Statsmodels & Pandas

On 18/09/17 09:48, Stuart Prescott wrote:
> Hi Diane,
> 
> On Sunday, 17 September 2017 22:14:18 AEST Diane Trout wrote:
>> I just did it that way because it was the least disruptive change I
>> could make that would let me build and test the package.
> 
> Sure, that's entirely sensible.
> 
>> In my experience I'm much more likely to to notice a build time test
>> failure than one from the CI system. 
> 
> Absolutely agreed. I'm thinking that this is a rather exceptional situation 
> because of the circular dependency and the fall-out that we regularly see from 
> that.
> 
>> What do other people think? If there are autopkgtests, should you still
>> let dh_auto_test run tests?
> 
> (I know I'm not the 'other people' from whom you solicit replies, I was just 
> perhaps unclear in what I was musing on before so I want to be more clear now)
> 
> Like you, I would *normally* run all tests in both places: buildd tests catch 
> severe problems right now; ci.d.n reruns the tests each time new versions of 
> dependencies are uploaded too and later breakage is caught.
> 
> Perhaps this is not a normal situation, however. Either one of "circular 
> dependencies" or "packages that often FTBFS on one or more architectures" 
> would be unusual enough to justify doing something different. All I am 
> wondering (from my position of ignorance!) if in this case, perhaps the tests 
> that cause the circular dependency can be disabled or xfailed, with the 
> remaining tests run as normal.

I now prefer to use autopkgtest in place of build-time tests; the former
has the advantages:

 * testing the installed package rather than the source tree, which
ensures that the binary package has sufficient dependencies and the
correct (or at least sufficient) contents
 * avoids dependency loops, as noted
 * avoids the need for workarounds required because of the location of
the files being tested at build time (see nbconvert issues with
entry_points not being found at build time)
 * can detect external breakage between uploads via ci.d.n

The downsides are that:

 * boilerplate is required (copying test files out of the source tree to
$AUTOPKGTEST_TMP to avoid testing against the source tree, instead of
the installed package)
 * extended build times if using autopkgtest as a post-build hook
(installing a chroot multiple times, if using sbuild/similar), and other
devs aren't guaranteed to run such tests at build/upload time

I wonder if it is possible (or a worthwhile use of time) to try and
provide an extended autodep8 python mode which runs the equivalent of
dh_auto_test as an autopkgtest, for packages which would opt-in with, eg
"Testsuite: autopkgtest-pkg-python-pytest|nose", rather than the basic
namespace-can-be-imported test.

Gordon

[toc] | [prev] | [next] | [standalone]

#9818 — Re: Python 3 Statsmodels & Pandas

On 18.09.2017 19:24, Gordon Ball wrote:
> On 18/09/17 09:48, Stuart Prescott wrote:
>> Hi Diane,
>>
>> On Sunday, 17 September 2017 22:14:18 AEST Diane Trout wrote:
>>> I just did it that way because it was the least disruptive change I
>>> could make that would let me build and test the package.
>>
>> Sure, that's entirely sensible.
>>
>>> In my experience I'm much more likely to to notice a build time test
>>> failure than one from the CI system. 
>>
>> Absolutely agreed. I'm thinking that this is a rather exceptional situation 
>> because of the circular dependency and the fall-out that we regularly see from 
>> that.
>>
>>> What do other people think? If there are autopkgtests, should you still
>>> let dh_auto_test run tests?
>>
>> (I know I'm not the 'other people' from whom you solicit replies, I was just 
>> perhaps unclear in what I was musing on before so I want to be more clear now)
>>
>> Like you, I would *normally* run all tests in both places: buildd tests catch 
>> severe problems right now; ci.d.n reruns the tests each time new versions of 
>> dependencies are uploaded too and later breakage is caught.
>>
>> Perhaps this is not a normal situation, however. Either one of "circular 
>> dependencies" or "packages that often FTBFS on one or more architectures" 
>> would be unusual enough to justify doing something different. All I am 
>> wondering (from my position of ignorance!) if in this case, perhaps the tests 
>> that cause the circular dependency can be disabled or xfailed, with the 
>> remaining tests run as normal.
> 
> I now prefer to use autopkgtest in place of build-time tests; the former
> has the advantages:
> 
>  * testing the installed package rather than the source tree, which
> ensures that the binary package has sufficient dependencies and the
> correct (or at least sufficient) contents
>  * avoids dependency loops, as noted
>  * avoids the need for workarounds required because of the location of
> the files being tested at build time (see nbconvert issues with
> entry_points not being found at build time)
>  * can detect external breakage between uploads via ci.d.n
> 
> The downsides are that:
> 
>  * boilerplate is required (copying test files out of the source tree to
> $AUTOPKGTEST_TMP to avoid testing against the source tree, instead of
> the installed package)
>  * extended build times if using autopkgtest as a post-build hook
> (installing a chroot multiple times, if using sbuild/similar), and other
> devs aren't guaranteed to run such tests at build/upload time
> 
> I wonder if it is possible (or a worthwhile use of time) to try and
> provide an extended autodep8 python mode which runs the equivalent of
> dh_auto_test as an autopkgtest, for packages which would opt-in with, eg
> "Testsuite: autopkgtest-pkg-python-pytest|nose", rather than the basic
> namespace-can-be-imported test.

the biggest downside with this approach is that you *completely* skip any
testing on other architectures than amd64.  Is that what you really want? Dear
porters, have fun where to search for bugs in packages without testsuites!

autopkg testing is not a replacement for runnning tests at build time from my
point of view, but to run a different set of tests.

It's a waste of build time on the autopkg testers if you need to build the
package again and then run against the tests in the build directory.  Many
testsuites are not made for autpkg testing from the installed place from what I
can see.

Matthias

[toc] | [prev] | [next] | [standalone]

#9821 — Re: Python 3 Statsmodels & Pandas

> the biggest downside with this approach is that you *completely* skip
> any
> testing on other architectures than amd64.  Is that what you really
> want? Dear
> porters, have fun where to search for bugs in packages without
> testsuites!

Ok you convinced me. dh_auto_tests stay.

Is there anything that can streamline rebuilding when there are cycles?
Do the build servers know to take advantage of the profiles to break
dependency chains?

Diane

[toc] | [prev] | [next] | [standalone]

#9824 — Re: Python 3 Statsmodels & Pandas

Hi,

On Mon, Sep 18, 2017 at 05:48:48PM +1000, Stuart Prescott wrote:
> On Sunday, 17 September 2017 22:14:18 AEST Diane Trout wrote:
> > I just did it that way because it was the least disruptive change I
> > could make that would let me build and test the package.
> 
> Sure, that's entirely sensible.
> 
> > In my experience I'm much more likely to to notice a build time test
> > failure than one from the CI system. 
> 
> Absolutely agreed. I'm thinking that this is a rather exceptional situation 
> because of the circular dependency and the fall-out that we regularly see from 
> that.
> 
> > What do other people think? If there are autopkgtests, should you still
> > let dh_auto_test run tests?
> 
> (I know I'm not the 'other people' from whom you solicit replies, I was just 
> perhaps unclear in what I was musing on before so I want to be more clear now)
> 
> Like you, I would *normally* run all tests in both places: buildd tests catch 
> severe problems right now; ci.d.n reruns the tests each time new versions of 
> dependencies are uploaded too and later breakage is caught.
> 
> Perhaps this is not a normal situation, however. Either one of "circular 
> dependencies" or "packages that often FTBFS on one or more architectures" 
> would be unusual enough to justify doing something different. All I am 
> wondering (from my position of ignorance!) if in this case, perhaps the tests 
> that cause the circular dependency can be disabled or xfailed, with the 
> remaining tests run as normal.

If my poor opinion counts:  For the moment we should run those tests in
the build process than can be easily be run.  Everything else should
probably be sorted out later (in autopkgtest or another later upload if
somebody has a clue how we can solve the circular depenendecies).

We somehow need to get some working spatstats to continue with other
packages.

Thanks to all who contributed

    Andreas.

-- 
http://fam-tille.de

[toc] | [prev] | [next] | [standalone]

#9825 — Re: Python 3 Statsmodels & Pandas

On 21/09/17 15:22, Andreas Tille wrote:
> We somehow need to get some working spatstats to continue with other
> packages.

I second Andreas' opinion. In principle, we'd want all tests to run at 
build and integration times and we might achieve that at some point 
during the Buster cycle.

Right now, this situation with pandas and statsmodels is slowing work on 
other scientific packages down significantly.

Cheers,
Ghis

[toc] | [prev] | [next] | [standalone]

#9826 — Re: Python 3 Statsmodels & Pandas

> If my poor opinion counts:  For the moment we should run those tests
> in
> the build process than can be easily be run.  Everything else should
> probably be sorted out later (in autopkgtest or another later upload
> if
> somebody has a clue how we can solve the circular depenendecies).
> 
> We somehow need to get some working spatstats to continue with other
> packages.
> 

Status:

[X] Pandas builds with nocheck, nodoc
[X] Statsmodels builds with Python 3 using above pandas
[X] Pandas tests pass with statsmodels for Python 2 & 3 installed.
[ ] Pandas builds docs with statsmodels installed

My most recent build error was about pandoc not being available.

Unfortunately the tests take a long time enough that I can write this
email before I know if adding pandoc fixed the problem.

dh_auto_tests run Python 2.7, 3.5, 3.6 tests, and then autopkgtests
runs them again.

I posted the larger fixes to pandas I've done to the appropriate bugs

#875807 python3-pandas FTBFS: 3 timezone unit tests fail
#875805 python3-pandas: Please break circular dependency

There's a few more minor patches on my laptop that I haven't attached
to a bug for pandas.

* Updating standards version
* using debhelper 10
* switching sphinx doc build to use python3
* and deleting a few more build files in dh_clean target.

I made larger changes to statsmodels, by using pybuild instead of the
previous multiple targets in debian/rules.

All of those changes are currently on alioth in detrout-python3.

When all these tests pass, shall I add myself to uploaders and release?
or does someone else want to review first?

Diane

[toc] | [prev] | [next] | [standalone]

#9827 — Re: Python 3 Statsmodels & Pandas

On Thu, 2017-09-21 at 17:56 -0400, Yaroslav Halchenko wrote:
> If you could allow to review would be great.
> Thanks for all the work.
> I was btw also trying to build with the patch you shared yesterday

Once I have all the changes for pandas would you like me to put them on
a branch on alioth? Or should I send them via format-patch somewhere?

Also it looked to me like you were changing debian/changelog with each
change? (Some people just use the git log and then apply all the
updates to d/changelog in one go with git buildpackage.

Diane

[toc] | [prev] | [next] | [standalone]

#9829 — Re: Python 3 Statsmodels & Pandas

On Thu, 21 Sep 2017, Diane Trout wrote:

> On Thu, 2017-09-21 at 17:56 -0400, Yaroslav Halchenko wrote:
> > If you could allow to review would be great.
> > Thanks for all the work.
> > I was btw also trying to build with the patch you shared yesterday

> Once I have all the changes for pandas would you like me to put them on
> a branch on alioth? Or should I send them via format-patch somewhere?

I could work with either.

I will upload now -2 (sorry if we duplicated some efforts) with
following changes (diff for changelog):

diff -Nru pandas-0.20.3/debian/changelog pandas-0.20.3/debian/changelog
--- pandas-0.20.3/debian/changelog      2017-07-10 20:00:59.000000000 -0400
+++ pandas-0.20.3/debian/changelog      2017-09-21 16:11:29.000000000 -0400
@@ -1,3 +1,14 @@
+pandas (0.20.3-2) unstable; urgency=medium
+
+  * debian/control
+    - boosted policy to 4.0.0 (I think we should be ok)
+    - drop statsmodels from build-depends to altogether avoid the circular
+      build-depends (Closes: #875805)
+  * Diane Trout:
+    - Add dateutil-2.6.1-fixed-ambiguous-tz-dst-be.patch (Closes: #875807)
+
+ -- Yaroslav Halchenko <debian@onerussian.com>  Thu, 21 Sep 2017 16:11:29 -0400

I could also add you to pkg-exppsy team under which we still have the "active"
vcs for pandas.

> Also it looked to me like you were changing debian/changelog with each
> change? (Some people just use the git log and then apply all the
> updates to d/changelog in one go with git buildpackage.

I prefer indeed reflecting each change in a changelog with a commit and
using debcommit to commit it ;)

-- 
Yaroslav O. Halchenko
Center for Open Neuroscience     http://centerforopenneuroscience.org
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834                       Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik        

[toc] | [prev] | [next] | [standalone]

#9830 — Re: Python 3 Statsmodels & Pandas

I managed to merge the more important doc changes.

I have a patch to switch doc building to using Python 3 components, as
there's a goal of removing the Python 2 components at some point.

The there's the patch for the nodoc/nocheck build profile, as well as
adding a bunch of other dependencies for doc building.

After more testing I'll try posting them. But it probably wont be until
tomorrow.

diff -Nru pandas-0.20.3/debian/changelog pandas-
> 0.20.3/debian/changelog
> --- pandas-0.20.3/debian/changelog      2017-07-10 20:00:59.000000000
> -0400
> +++ pandas-0.20.3/debian/changelog      2017-09-21 16:11:29.000000000
> -0400
> @@ -1,3 +1,14 @@
> +pandas (0.20.3-2) unstable; urgency=medium
> +
> +  * debian/control
> +    - boosted policy to 4.0.0 (I think we should be ok)
> +    - drop statsmodels from build-depends to altogether avoid the
> circular

While I was looking at the policy updating checklist in 4.0.1 they
renamed XS-Testsuite to Testsuite. I think that's the only change I
noticed.

The version of debhelper and debian/compat is also pretty old.


> 
> I could also add you to pkg-exppsy team under which we still have the
> "active"
> vcs for pandas.

Thank you that sounds useful.

Diane


> > 

[toc] | [prev] | [next] | [standalone]

#9841 — Re: Python 3 Statsmodels & Pandas

Status with statsmodels almost done

Trying to deal with jquery.

leaving command

 	-rm ./build/html/_static/jquery.js

causes a build failure now.
leaving it in causes a lintain privacy error.

there's also lintain warnings about a missing hardening flag, and no
doc-base registration.

At some point tests in ./statsmodels/imputation/tests/test_mice.py were
failing. but I'm not sure if its still having trouble.

Kid really wants to go to the park right now, so I wont be able to
touch it for several hours.

I also need to make good patches for sending to alioth, so here's a
snapshot. (with the jquery deletion rule commented out)

I think you should get a package if you build this.

http://chaos.caltech.edu/~diane/debian/statsmodels_0.8.0-3.1.dsc

Diane

[toc] | [prev] | [next] | [standalone]

#9843 — Re: Python 3 Statsmodels & Pandas

On Sun, Sep 24, 2017 at 11:24:10AM -0700, Diane Trout wrote:
> Status with statsmodels almost done
> 
> Trying to deal with jquery.
> 
> leaving command
> 
>  	-rm ./build/html/_static/jquery.js
> 
> causes a build failure now.

Without checking the source:  I'm usually doing something like

override_dh_install:
	dh_install
	find debian -name jquery.js -delete

This should avoid build failures.

> there's also lintain warnings about a missing hardening flag,

There are a number of false positives.  By no means this should be a
blocker for upload in the current situation.  If you are very picky you
might open a wishlist bug report to keep a record of this issue.

> and no
> doc-base registration.

Nice to have but no real blocker for an upload.
 
> Kid really wants to go to the park right now, so I wont be able to
> touch it for several hours.

:-)  Send hugs to your kid. ;-)
 
> I also need to make good patches for sending to alioth, so here's a
> snapshot. (with the jquery deletion rule commented out)
> 
> I think you should get a package if you build this.
> 
> http://chaos.caltech.edu/~diane/debian/statsmodels_0.8.0-3.1.dsc

Thanks for your work on this

     Andreas.

-- 
http://fam-tille.de

[toc] | [prev] | [next] | [standalone]

#9844 — Re: Python 3 Statsmodels & Pandas

On Sun, 24 Sep 2017, Andreas Tille wrote:

> On Sun, Sep 24, 2017 at 11:24:10AM -0700, Diane Trout wrote:
> > Status with statsmodels almost done

> > Trying to deal with jquery.

> > leaving command

> >  	-rm ./build/html/_static/jquery.js

> > causes a build failure now.

> Without checking the source:  I'm usually doing something like

> override_dh_install:
> 	dh_install
> 	find debian -name jquery.js -delete

> This should avoid build failures.

FTR  not sure how that could have caused a build failure since leading '-'
in makefile means "ignore the failure"

-- 
Yaroslav O. Halchenko
Center for Open Neuroscience     http://centerforopenneuroscience.org
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834                       Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik        

[toc] | [prev] | [next] | [standalone]

#9846 — Re: Python 3 Statsmodels & Pandas

If the jquery.js file is deleted something later errors out.  My guess is it's listed in one of the .install files but I haven't looked yet

On September 24, 2017 3:51:30 PM PDT, Yaroslav Halchenko <debian@onerussian.com> wrote:
>
>On Sun, 24 Sep 2017, Andreas Tille wrote:
>
>> On Sun, Sep 24, 2017 at 11:24:10AM -0700, Diane Trout wrote:
>> > Status with statsmodels almost done
>
>> > Trying to deal with jquery.
>
>> > leaving command
>
>> >  	-rm ./build/html/_static/jquery.js
>
>> > causes a build failure now.
>
>> Without checking the source:  I'm usually doing something like
>
>> override_dh_install:
>> 	dh_install
>> 	find debian -name jquery.js -delete
>
>> This should avoid build failures.
>
>FTR  not sure how that could have caused a build failure since leading
>'-'
>in makefile means "ignore the failure"

[toc] | [prev] | [next] | [standalone]

#9847 — Re: Python 3 Statsmodels & Pandas

On Sun, 2017-09-24 at 11:24 -0700, Diane Trout wrote:
> Status with statsmodels almost done
> 
> Trying to deal with jquery.
> 
> leaving command
> 
>  	-rm ./build/html/_static/jquery.js
> 
> causes a build failure now.
> leaving it in causes a lintain privacy error.

I found the problem, I started building the notebooks and those
templates refer to jquery and requirejs in addition to mathjax.

(So I need to replace those with local copies)

In my first try I misunderstood what the Build-Depends-Indep field was
for and so I rewrote history to fix one of my commits. so ..

everything I've done is now detrout-python3-try2, and hopefully the
other branch can get deleted. (sorry about this)

The remaining issues are:

* Some of the doc pages call get_rdataset, and there's no network
access in the builder so those calls fail. (ugliest error)
* There's a lintian warning about no bindnow.
* there's one intersphinx reference that's not in debian.
* I couldn't get override_dh_auto_build-indep to actually build the
docs. I shoved the doc building into override_dh_installdocs

Diane

[toc] | [prev] | [next] | [standalone]

#9849 — Re: Python 3 Statsmodels & Pandas

Hi Diane,

On Sun, Sep 24, 2017 at 11:45:43PM -0700, Diane Trout wrote:
> The remaining issues are:
> 
> * Some of the doc pages call get_rdataset, and there's no network
> access in the builder so those calls fail. (ugliest error)

Can you pre-fetch the data and provide it in debian/datasets?

> * There's a lintian warning about no bindnow.

I would ignore this for the moment.

> * there's one intersphinx reference that's not in debian.

If it is not very important I would probably exclude the piece of
documentation which is affected.  What exact reference is this?  Can we
help by packaging something else that might be needed later not only
inside the docs but also in code?

Kind regards

      Andreas. 

-- 
http://fam-tille.de

[toc] | [prev] | [next] | [standalone]

Page 2 of 4 — ← Prev page 1 [2] 3 4  Next page →

Back to top | Article view | linux.debian.maint.python

csiph-web