Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.maint.python > #9789 > unrolled thread

Bug#729956: Forwarded upstream

Back to article view | Back to linux.debian.maint.python

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Bug#729956: Forwarded upstream Andreas Tille <tille@debian.org> - 2017-09-06 22:30 +0200
    Bug#729956: Forwarded upstream Yuri D'Elia <wavexx@thregr.org> - 2017-09-06 22:40 +0200
      Bug#729956: Forwarded upstream Andreas Tille <tille@debian.org> - 2017-09-06 23:00 +0200
        Bug#729956: git-dpm (was Re: Bug#729956: Forwarded upstream) Yuri D'Elia <wavexx@thregr.org> - 2017-09-06 23:20 +0200
          Re: git-dpm (was Re: Bug#729956: Forwarded upstream) Scott Kitterman <debian@kitterman.com> - 2017-09-07 00:50 +0200
            Re: git-dpm (was Re: Bug#729956: Forwarded upstream) Brian May <brian@linuxpenguins.xyz> - 2017-09-07 01:20 +0200
              Re: git-dpm (was Re: Bug#729956: Forwarded upstream) Yuri D'Elia <wavexx@thregr.org> - 2017-09-08 00:50 +0200
            Re: git-dpm (was Re: Bug#729956: Forwarded upstream) Brian May <brian@linuxpenguins.xyz> - 2017-09-07 01:30 +0200
              Re: git-dpm (was Re: Bug#729956: Forwarded upstream) Scott Kitterman <debian@kitterman.com> - 2017-09-07 07:00 +0200
                Re: git-dpm (was Re: Bug#729956: Forwarded upstream) Brian May <brian@linuxpenguins.xyz> - 2017-09-07 07:30 +0200
    Bug#729956: Forwarded upstream Diane Trout <diane@ghic.org> - 2017-09-07 00:30 +0200
      Bug#729956: Forwarded upstream Matthias Klose <doko@debian.org> - 2017-09-07 06:20 +0200
      Bug#729956: Forwarded upstream Andreas Tille <andreas@an3as.eu> - 2017-09-07 06:30 +0200
        Bug#729956: Forwarded upstream Diane Trout <diane@ghic.org> - 2017-09-07 06:40 +0200
          Bug#729956: Forwarded upstream Andreas Tille <andreas@an3as.eu> - 2017-09-07 08:40 +0200
            Bug#729956: Forwarded upstream Diane Trout <diane@ghic.org> - 2017-09-08 08:40 +0200
    Bug#729956: Python 3 Statsmodels & Pandas Diane Trout <diane@ghic.org> - 2017-09-16 22:50 +0200
      Bug#729956: Python 3 Statsmodels & Pandas Yuri D'Elia <wavexx@thregr.org> - 2017-09-16 23:10 +0200
        Bug#729956: Python 3 Statsmodels & Pandas Diane Trout <diane@ghic.org> - 2017-09-16 23:30 +0200
          Bug#729956: Python 3 Statsmodels & Pandas Yuri D'Elia <wavexx@thregr.org> - 2017-09-17 00:00 +0200
            Re: skimage (was: Re: Bug#729956: Python 3 Statsmodels & Pandas) Yaroslav Halchenko <yoh@debian.org> - 2017-10-13 00:10 +0200
            skimage (was: Re: Bug#729956: Python 3 Statsmodels & Pandas) Yuri D'Elia <wavexx@thregr.org> - 2017-10-13 00:40 +0200
          Re: Python 3 Statsmodels & Pandas Stuart Prescott <stuart@debian.org> - 2017-09-18 04:00 +0200
            Re: Python 3 Statsmodels & Pandas Diane Trout <diane@ghic.org> - 2017-09-18 07:20 +0200
              Re: Python 3 Statsmodels & Pandas Stuart Prescott <stuart@debian.org> - 2017-09-18 10:10 +0200
                Re: Python 3 Statsmodels & Pandas Gordon Ball <gordon@chronitis.net> - 2017-09-18 19:30 +0200
                  Re: Python 3 Statsmodels & Pandas Matthias Klose <doko@debian.org> - 2017-09-18 21:30 +0200
                    Re: Python 3 Statsmodels & Pandas Diane Trout <diane@ghic.org> - 2017-09-19 06:50 +0200
                Re: Python 3 Statsmodels & Pandas Andreas Tille <andreas@an3as.eu> - 2017-09-21 16:30 +0200
                  Re: Python 3 Statsmodels & Pandas Ghislain Vaillant <ghisvail@gmail.com> - 2017-09-21 17:00 +0200
                  Re: Python 3 Statsmodels & Pandas Diane Trout <diane@ghic.org> - 2017-09-21 23:50 +0200
                    Re: Python 3 Statsmodels & Pandas Diane Trout <diane@ghic.org> - 2017-09-22 00:10 +0200
                      Re: Python 3 Statsmodels & Pandas Yaroslav Halchenko <debian@onerussian.com> - 2017-09-22 04:30 +0200
                        Re: Python 3 Statsmodels & Pandas Diane Trout <diane@ghic.org> - 2017-09-22 07:20 +0200
                          Re: Python 3 Statsmodels & Pandas Diane Trout <diane@ghic.org> - 2017-09-24 20:30 +0200
                            Re: Python 3 Statsmodels & Pandas Andreas Tille <andreas@fam-tille.de> - 2017-09-25 00:10 +0200
                              Re: Python 3 Statsmodels & Pandas Yaroslav Halchenko <debian@onerussian.com> - 2017-09-25 01:00 +0200
                                Re: Python 3 Statsmodels & Pandas Diane Trout <diane@ghic.org> - 2017-09-25 01:20 +0200
                            Re: Python 3 Statsmodels & Pandas Diane Trout <diane@ghic.org> - 2017-09-25 08:50 +0200
                              Re: Python 3 Statsmodels & Pandas Andreas Tille <andreas@an3as.eu> - 2017-09-25 09:50 +0200
                                Re: Python 3 Statsmodels & Pandas Diane Trout <diane@ghic.org> - 2017-09-25 19:40 +0200
                                  Re: Python 3 Statsmodels & Pandas Andreas Tille <andreas@an3as.eu> - 2017-09-25 19:50 +0200
                                Re: Python 3 Statsmodels & Pandas Diane Trout <diane@ghic.org> - 2017-09-26 06:40 +0200
                                  Re: Python 3 Statsmodels & Pandas Andreas Tille <andreas@fam-tille.de> - 2017-09-26 09:20 +0200
                                    Re: Python 3 Statsmodels & Pandas Diane Trout <diane@ghic.org> - 2017-09-26 20:40 +0200
                                      Re: Python 3 Statsmodels & Pandas Yaroslav Halchenko <debian@onerussian.com> - 2017-09-26 22:50 +0200
                                      Re: Python 3 Statsmodels & Pandas Andreas Tille <andreas@an3as.eu> - 2017-09-26 23:00 +0200
                                        Re: Python 3 Statsmodels & Pandas Diane Trout <diane@ghic.org> - 2017-09-26 23:50 +0200
                                          Re: Python 3 Statsmodels & Pandas Andreas Tille <andreas@fam-tille.de> - 2017-09-27 08:20 +0200
                                        Re: Python 3 Statsmodels & Pandas Diane Trout <diane@ghic.org> - 2017-09-27 07:50 +0200
                                          Re: Python 3 Statsmodels & Pandas Andreas Tille <andreas@fam-tille.de> - 2017-09-27 09:00 +0200
                                            Re: Python 3 Statsmodels & Pandas Diane Trout <diane@ghic.org> - 2017-09-27 09:30 +0200
                                              Re: Python 3 Statsmodels & Pandas Andreas Tille <andreas@an3as.eu> - 2017-09-27 10:10 +0200
                                                Re: Python 3 Statsmodels & Pandas Andreas Tille <andreas@an3as.eu> - 2017-09-27 12:20 +0200
                                                  Re: Python 3 Statsmodels & Pandas "Rebecca N. Palmer" <rebecca_palmer@zoho.com> - 2017-09-29 23:40 +0200
                                                    Re: Python 3 Statsmodels & Pandas Diane Trout <diane@ghic.org> - 2017-09-30 01:00 +0200
                                                      Re: Python 3 Statsmodels & Pandas "Rebecca N. Palmer" <rebecca_palmer@zoho.com> - 2017-09-30 10:20 +0200
                                                      Re: Python 3 Statsmodels & Pandas Ghislain Vaillant <ghisvail@gmail.com> - 2017-09-30 11:20 +0200
                                                      Re: Python 3 Statsmodels & Pandas Dmitry Shachnev <mitya57@debian.org> - 2017-09-30 11:30 +0200
                                                        Re: Python 3 Statsmodels & Pandas Diane Trout <diane@ghic.org> - 2017-09-30 18:00 +0200
                                                          Re: Python 3 Statsmodels & Pandas "Rebecca N. Palmer" <rebecca_palmer@zoho.com> - 2017-09-30 18:40 +0200
                        Re: Python 3 Statsmodels & Pandas Andreas Tille <andreas@an3as.eu> - 2017-09-22 08:20 +0200
                          Re: Python 3 Statsmodels & Pandas Yaroslav Halchenko <debian@onerussian.com> - 2017-09-23 16:10 +0200
                            Re: Python 3 Statsmodels & Pandas Andreas Tille <andreas@an3as.eu> - 2017-09-24 13:30 +0200
                              Re: Python 3 Statsmodels & Pandas Yaroslav Halchenko <debian@onerussian.com> - 2017-09-25 01:00 +0200
                    Re: Python 3 Statsmodels & Pandas Yaroslav Halchenko <yoh@onerussian.com> - 2017-09-22 00:20 +0200
                    Re: Python 3 Statsmodels & Pandas Piotr Ożarowski <piotr@debian.org> - 2017-09-22 11:00 +0200
                      Re: Python 3 Statsmodels & Pandas Diane Trout <diane@ghic.org> - 2017-09-22 21:40 +0200
                        Re: Python 3 Statsmodels & Pandas Ghislain Vaillant <ghisvail@gmail.com> - 2017-09-23 11:50 +0200
                      Re: Python 3 Statsmodels & Pandas Yaroslav Halchenko <debian@onerussian.com> - 2017-09-23 16:00 +0200
                        Re: Python 3 Statsmodels & Pandas Piotr Ożarowski <piotr@debian.org> - 2017-09-25 11:20 +0200
        Bug#729956: Python 3 Statsmodels & Pandas Matthias Klose <doko@debian.org> - 2017-09-17 11:20 +0200

Page 3 of 4 — ← Prev page 1 2 [3] 4  Next page →

#9852 — Re: Python 3 Statsmodels & Pandas

On Mon, 2017-09-25 at 09:44 +0200, Andreas Tille wrote:
> 
> > * Some of the doc pages call get_rdataset, and there's no network
> > access in the builder so those calls fail. (ugliest error)
> 
> Can you pre-fetch the data and provide it in debian/datasets?

Looks like it'll take a bit of patching. There is a caching mechanism,
but it's off by default.

> > * there's one intersphinx reference that's not in debian.
> 
> If it is not very important I would probably exclude the piece of
> documentation which is affected.  What exact reference is this?  Can
> we
> help by packaging something else that might be needed later not only
> inside the docs but also in code?

intersphinx is used for cross references, so there's two places in the
local docs which show plain text, and hyperlinks on the official site.

If you look at:

http://www.statsmodels.org/stable/dev/git_notes.html#merging-vs-rebasin
g

in the sentence "One great place to start learning about rebase is
rebasing without tears." the phrase "rebasing without tears" is a link
to the pydagogue site, and in my local package it's just plain text.

I'm planning on ignoring it for now.

Diane

[toc] | [prev] | [next] | [standalone]

#9853 — Re: Python 3 Statsmodels & Pandas

Hi Diane,

On Mon, Sep 25, 2017 at 10:38:00AM -0700, Diane Trout wrote:
> 
> I'm planning on ignoring it for now.

Sounds sensible.  Thanks a lot for your work on this

      Andreas.

-- 
http://fam-tille.de

[toc] | [prev] | [next] | [standalone]

#9854 — Re: Python 3 Statsmodels & Pandas

On Mon, 2017-09-25 at 09:44 +0200, Andreas Tille wrote:
> Hi Diane,
> 
> On Sun, Sep 24, 2017 at 11:45:43PM -0700, Diane Trout wrote:
> > The remaining issues are:
> > 
> > * Some of the doc pages call get_rdataset, and there's no network
> > access in the builder so those calls fail. (ugliest error)
> 
> Can you pre-fetch the data and provide it in debian/datasets?

I made the changes and cached the downloaded zip files....

and then realized isn't this redistributing datasets?

Don't we need to verify the license before uploading?

Below is what I've found so far, (before getting tired of licensing
issues)

Any thoughts about how to handle this?

Here's a list of the file names from the include-binaries file I
created via caching.

datasets.csv.zip                    
csv,HistData,Guerry.csv.zip         
doc,HistData,rst,Guerry.rst.zip     
csv,COUNT,medpar.csv.zip            
doc,COUNT,rst,medpar.rst.zip        
csv,car,Duncan.csv.zip              
doc,car,rst,Duncan.rst.zip          
csv,robustbase,starsCYG.csv.zip     
doc,robustbase,rst,starsCYG.rst.zip 
doc,car,rst,Moore.rst.zip           
csv,vcd,Arthritis.csv.zip           
doc,vcd,rst,Arthritis.rst.zip       
csv,MASS,epil.csv.zip               
doc,MASS,rst,epil.rst.zip           
csv,geepack,dietox.csv.zip          
doc,geepack,rst,dietox.rst.zip      

The files are being downloaded from this github repository.

https://github.com/vincentarelbundock/Rdatasets a useful index of the
datasets is 
http://vincentarelbundock.github.com/Rdatasets/datasets.html

Guerry.csv is probably safe as its from "Essay on the Moral Statistics
of France" published 1833.

medpar 2016's license is here:
https://www.healthdata.gov/dataset/medpar-limited-data-set-lds-hospital
-national
and is listed as "Open Data Commons Open Database License"
https://opendatacommons.org/licenses/odbl/1.0/

Duncan is the Duncan's Occupational Prestige Data from 1950.
Couldn't find a license

starsCYG is Data for the Hertzsprung-Russell Diagram of the star
cluster CYG OB1
http://ugrad.stat.ubc.ca/R/library/rrcov/html/stars.html
Couldn't find a license

Moore is from Moore, J. C., Jr. and Krupat, E. (1971) 
Relationship between source status, authoritarianism and conformity in
a social setting.
Couldn't find a license


Diane

[toc] | [prev] | [next] | [standalone]

#9855 — Re: Python 3 Statsmodels & Pandas

On Mon, Sep 25, 2017 at 09:36:43PM -0700, Diane Trout wrote:
> > > * Some of the doc pages call get_rdataset, and there's no network
> > > access in the builder so those calls fail. (ugliest error)
> > 
> > Can you pre-fetch the data and provide it in debian/datasets?
> 
> I made the changes and cached the downloaded zip files....
> 
> and then realized isn't this redistributing datasets?
> 
> Don't we need to verify the license before uploading?

In principle yes.
 
> Below is what I've found so far, (before getting tired of licensing
> issues)
> 
> Any thoughts about how to handle this?

I wonder how this was dealt with before?  If that much data sets were
needed to build the docs, how did the doc generation process worked
before?

My thought is that we should shift anything to a later point of
development which would cause a delay in fixing all RC bugs -
specifically #873512 remains open.  I have pinged upstream about this.
If we migrate to ship also python3, we need extra cycles through new and
may be new bugs to fix.

BTW, I'd love if you would merge your work to master branch.  I'm a bit
confused by the amount of branches and lost track which one to look at.
 
> Here's a list of the file names from the include-binaries file I
> created via caching.
> 
> datasets.csv.zip                    
> csv,HistData,Guerry.csv.zip         
> doc,HistData,rst,Guerry.rst.zip     
> csv,COUNT,medpar.csv.zip            
> doc,COUNT,rst,medpar.rst.zip        
> csv,car,Duncan.csv.zip              
> doc,car,rst,Duncan.rst.zip          
> csv,robustbase,starsCYG.csv.zip     
> doc,robustbase,rst,starsCYG.rst.zip 
> doc,car,rst,Moore.rst.zip           
> csv,vcd,Arthritis.csv.zip           
> doc,vcd,rst,Arthritis.rst.zip       
> csv,MASS,epil.csv.zip               
> doc,MASS,rst,epil.rst.zip           
> csv,geepack,dietox.csv.zip          
> doc,geepack,rst,dietox.rst.zip      
> 
> The files are being downloaded from this github repository.
> 
> https://github.com/vincentarelbundock/Rdatasets a useful index of the
> datasets is 
> http://vincentarelbundock.github.com/Rdatasets/datasets.html

This reminds me to the debian/README.source files ftpmaster once
suggested for R packages[1].  May be that's an apropriate way to
document the licenses?  Feel free to find examples for instance
in the package r-cran-ape.
 
> Guerry.csv is probably safe as its from "Essay on the Moral Statistics
> of France" published 1833.
> 
> medpar 2016's license is here:
> https://www.healthdata.gov/dataset/medpar-limited-data-set-lds-hospital
> -national
> and is listed as "Open Data Commons Open Database License"
> https://opendatacommons.org/licenses/odbl/1.0/
> 
> Duncan is the Duncan's Occupational Prestige Data from 1950.
> Couldn't find a license
> 
> starsCYG is Data for the Hertzsprung-Russell Diagram of the star
> cluster CYG OB1
> http://ugrad.stat.ubc.ca/R/library/rrcov/html/stars.html
> Couldn't find a license
> 
> Moore is from Moore, J. C., Jr. and Krupat, E. (1971) 
> Relationship between source status, authoritarianism and conformity in
> a social setting.
> Couldn't find a license

So may be we should patch the docs to ignore those data sets without a
license.

Thanks a lot for your work on this package

    Andreas.


[1] http://lists.debian.org/debian-devel/2013/09/msg00332.html 

-- 
http://fam-tille.de

[toc] | [prev] | [next] | [standalone]

#9856 — Re: Python 3 Statsmodels & Pandas

> > Below is what I've found so far, (before getting tired of licensing
> > issues)
> > 
> > Any thoughts about how to handle this?
> 
> I wonder how this was dealt with before?  If that much data sets were
> needed to build the docs, how did the doc generation process worked
> before?

Perhaps the earlier packages weren't built using a tool that disables
network access? the get_rdataset function will attempt to download the
file first.

Although it hurts reproducibility, that is certainly a simpler
solution.

> 
> BTW, I'd love if you would merge your work to master branch.  I'm a
> bit
> confused by the amount of branches and lost track which one to look
> at.

I verified which was which and deleted the obsolete branch.

use detrout-python3-try2 (the only detrout branch that should be left)

> 
> > The files are being downloaded from this github repository.
> > 
> > https://github.com/vincentarelbundock/Rdatasets a useful index of
> > the
> > datasets is 
> > http://vincentarelbundock.github.com/Rdatasets/datasets.html
> 
> This reminds me to the debian/README.source files ftpmaster once
> suggested for R packages[1].  May be that's an apropriate way to
> document the licenses?  Feel free to find examples for instance
> in the package r-cran-ape.

I really wish that listing the citation for these early scientific
datasets counted as complying with the license.

Diane

[toc] | [prev] | [next] | [standalone]

#9857 — Re: Python 3 Statsmodels & Pandas

Thanks for digging into this and sorry I have missed that.  I typically
add export http*_proxy  to prevent any network interactions but I guess
didn't get that far with statsmodels.

FWIW, for dipy package I now ask upstream to provide me e.g.
    dipy_0.12.0.orig-doc-examples.tar.gz
where there are examples, which require lengthy computation to produce doc pieces

To resolve this issue quickly, I might have also simply patched/excluded
those examples/docs which require external datasets (especially with
incompatible terms), from building.

-- 
Yaroslav O. Halchenko
Center for Open Neuroscience     http://centerforopenneuroscience.org
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834                       Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik        

[toc] | [prev] | [next] | [standalone]

#9858 — Re: Python 3 Statsmodels & Pandas

Hi Diane,

On Tue, Sep 26, 2017 at 11:36:17AM -0700, Diane Trout wrote:
> > BTW, I'd love if you would merge your work to master branch.  I'm a
> > bit
> > confused by the amount of branches and lost track which one to look
> > at.
> 
> I verified which was which and deleted the obsolete branch.
> 
> use detrout-python3-try2 (the only detrout branch that should be left)

Thanks.  Just let me know if you want me to try rebuilding your work.

> > This reminds me to the debian/README.source files ftpmaster once
> > suggested for R packages[1].  May be that's an apropriate way to
> > document the licenses?  Feel free to find examples for instance
> > in the package r-cran-ape.
> 
> I really wish that listing the citation for these early scientific
> datasets counted as complying with the license.

Since it is accepted for the R packages and the data are refering
to R data I do not see any reason why this should not be accepted.

As far as I understood Yaroslav he agrees with me that if all
fails patching the docs to not use the data would be a temporary
solution otherwise.

While I have not tried to build the current status I wonder what you
think about #873512.  I'm perfectly fine with your solution to exclude
some tests - I just wanted to give a hint that there is a potential
upstream patch.

Kind regards

     Andreas.

-- 
http://fam-tille.de

[toc] | [prev] | [next] | [standalone]

#9859 — Re: Python 3 Statsmodels & Pandas

> While I have not tried to build the current status I wonder what you
> think about #873512.  I'm perfectly fine with your solution to
> exclude
> some tests - I just wanted to give a hint that there is a potential
> upstream patch.

I think I looked at the upstream commits that fixed it,

https://github.com/statsmodels/statsmodels/pull/3402/files

and discovered that it required too much back porting to easily apply
to 0.8.0

Since the error is just you can't pickle (save object state) a few data
types, I was assuming a user could just avoid the missing functionality
until there's a new upstream release.

I was more concerned about remembering to remove disabling the tests
after its fixed, in case something else goes wrong later.

Diane

[toc] | [prev] | [next] | [standalone]

#9862 — Re: Python 3 Statsmodels & Pandas

Dear Diane,

On Tue, Sep 26, 2017 at 02:49:12PM -0700, Diane Trout wrote:
> 
> > While I have not tried to build the current status I wonder what you
> > think about #873512.  I'm perfectly fine with your solution to
> > exclude
> > some tests - I just wanted to give a hint that there is a potential
> > upstream patch.
> 
> I think I looked at the upstream commits that fixed it,
> 
> https://github.com/statsmodels/statsmodels/pull/3402/files
> 
> and discovered that it required too much back porting to easily apply
> to 0.8.0
> 
> Since the error is just you can't pickle (save object state) a few data
> types, I was assuming a user could just avoid the missing functionality
> until there's a new upstream release.

OK, as long as we get this bug of statsmodels fixed soon that's
perfectly fine.
 
> I was more concerned about remembering to remove disabling the tests
> after its fixed, in case something else goes wrong later.

I guess some patch will not apply on the new version and once we think
about this patch we will remember (hopefully ;-) ).

Thanks again

       Andreas. 

-- 
http://fam-tille.de

[toc] | [prev] | [next] | [standalone]

#9861 — Re: Python 3 Statsmodels & Pandas

> Since it is accepted for the R packages and the data are refering
> to R data I do not see any reason why this should not be accepted.

I traced back from Rdatasets to the original R packages.

Every one of the packages are licensed as some combination of GPL-2 and
GPL-3

However it's likely that some of the datasets are not copyrightable.

Here's my badly formatted notes of the name of the dataset
and the link to the R package documentation, and what GPL license it's
available under.

How should this be formatted to be shipped with Debian?

https://ghic.org/~diane/debian/statsmodels.datasets.README.txt

Does it go in README.source? or in upstream/metadata? or something
else?

I could also ask one of the campus librarians to help review the
datasets to determine if they should or shouldn't be copyrightable.

My current WIP patch for the documentation is here:
The Debian copyright file still needs to be updated, and the citations
need to be listed.

https://ghic.org/~diane/debian/statsmodels-0001-add-patch-use-cached-da
tasets-and-cache-all-of-the-g.patch

Diane

[toc] | [prev] | [next] | [standalone]

#9863 — Re: Python 3 Statsmodels & Pandas

Dear Diane,

On Tue, Sep 26, 2017 at 10:47:21PM -0700, Diane Trout wrote:
> 
> > Since it is accepted for the R packages and the data are refering
> > to R data I do not see any reason why this should not be accepted.
> 
> I traced back from Rdatasets to the original R packages.
> 
> Every one of the packages are licensed as some combination of GPL-2 and
> GPL-3

As far as I understood CRAN only free software is permitted there.
So if there is something on CRAN we should be able to distribute
it license wise.  (I've fought one exception (locfit), thought).

> However it's likely that some of the datasets are not copyrightable.

+1
 
> Here's my badly formatted notes of the name of the dataset
> and the link to the R package documentation, and what GPL license it's
> available under.
> 
> How should this be formatted to be shipped with Debian?
> 
> https://ghic.org/~diane/debian/statsmodels.datasets.README.txt

I think regarding formatting and context its perfectly fine.
 
> Does it go in README.source? or in upstream/metadata? or something
> else?

I think there is no "right" place to do it - but ftpmaster is used
to read README.source in these cases.  So why not using it?

> I could also ask one of the campus librarians to help review the
> datasets to determine if they should or shouldn't be copyrightable.

I think ftpmaster is the instance who needs to decide this in our case
(and due to the change of the binary packages it will be processed by
them).  I'd like to repeat myself:  We really need to get statsmodels
into testing soon since otherwise a lot of packages will removed from
there and I'm specifically picky about one rdependency (metaphlan2)
which requires always manual work by the release team due some complex
dependency relations.  So getting the statsmodels uploaded *soon* has
way higher priority for me currently than beeing more picky than
ftpmaster about copyrights of some datasets in documentation that could
be probably read online as well since it creates work for several
people.  If we could upload statsmodels today (or latest tomorrow)
I would immediately ping ftpmaster to check it quickly to enable us
removing those pieces if they don't like it.

> My current WIP patch for the documentation is here:
> The Debian copyright file still needs to be updated, and the citations
> need to be listed.
> 
> https://ghic.org/~diane/debian/statsmodels-0001-add-patch-use-cached-da
> tasets-and-cache-all-of-the-g.patch

I `git am`-ed this to your branch (any reason to not push it?)  BTW,
please take over the changelog entry under your ID.  I did not
contribute anything relevant.

The bad news is that when I tried to build the branch + your patch
(as I said, please push to enable more easy testing for others) I'm
running into:


...
Finished /build/statsmodels-0.8.0/docs/source/examples/notebooks/generated/statespace_arma_0.html
Finished /build/statsmodels-0.8.0/docs/source/examples/notebooks/generated/pca_fertility_factors.html
Finished /build/statsmodels-0.8.0/docs/source/examples/notebooks/generated/ols.html
Finished /build/statsmodels-0.8.0/docs/source/examples/notebooks/generated/regression_plots.html
concurrent.futures.process._RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/usr/lib/python3.5/concurrent/futures/process.py", line 175, in _process_worker
    r = call_item.fn(*call_item.args, **call_item.kwargs)
  File "/usr/lib/python3.5/concurrent/futures/process.py", line 153, in _process_chunk
    return [fn(*args) for args in chunk]
  File "/usr/lib/python3.5/concurrent/futures/process.py", line 153, in <listcomp>
    return [fn(*args) for args in chunk]
  File "../tools/nbgenerate.py", line 97, in do_one
    kernel_name=kernel_name)
  File "../tools/nbgenerate.py", line 55, in execute_nb
    ep.preprocess(nb, {'metadta': {'path': 'notebooks/'}})
  File "/usr/lib/python3/dist-packages/nbconvert/preprocessors/execute.py", line 141, in preprocess
    cwd=path)
  File "/usr/lib/python3/dist-packages/jupyter_client/manager.py", line 452, in start_new_kernel
    kc.wait_for_ready(timeout=startup_timeout)
  File "/usr/lib/python3/dist-packages/jupyter_client/blocking/client.py", line 120, in wait_for_ready
    raise RuntimeError('Kernel died before replying to kernel_info')
RuntimeError: Kernel died before replying to kernel_info
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "../tools/nbgenerate.py", line 165, in <module>
    main()
  File "../tools/nbgenerate.py", line 162, in main
    kernel_name=args.kernel_name)
  File "../tools/nbgenerate.py", line 123, in do
    for dst in pool.map(func, nbs):
  File "/usr/lib/python3.5/concurrent/futures/_base.py", line 556, in result_iterator
    yield future.result()
  File "/usr/lib/python3.5/concurrent/futures/_base.py", line 398, in result
    return self.__get_result()
  File "/usr/lib/python3.5/concurrent/futures/_base.py", line 357, in __get_result
    raise self._exception
RuntimeError: Kernel died before replying to kernel_info
Makefile:62: recipe for target 'html' failed
make[2]: *** [html] Error 1
make[2]: Leaving directory '/build/statsmodels-0.8.0/docs'
debian/rules:35: recipe for target 'override_dh_installdocs' failed
make[1]: *** [override_dh_installdocs] Error 2
make[1]: Leaving directory '/build/statsmodels-0.8.0'
debian/rules:16: recipe for target 'binary' failed
make: *** [binary] Error 2
dpkg-buildpackage: error: fakeroot debian/rules binary gave error exit status 2


Any idea what might be the problem here?

Kind regards

       Andreas.

-- 
http://fam-tille.de

[toc] | [prev] | [next] | [standalone]

#9864 — Re: Python 3 Statsmodels & Pandas

On Wed, 2017-09-27 at 08:34 +0200, Andreas Tille wrote:
> 
> > 
> > https://ghic.org/~diane/debian/statsmodels.datasets.README.txt
> 
> I think regarding formatting and context its perfectly fine.
>  
> > Does it go in README.source? or in upstream/metadata? or something
> > else?
> 
> I think there is no "right" place to do it - but ftpmaster is used
> to read README.source in these cases.  So why not using it?
> 

I added the readme to README.source with a bit of explanatory text.

all changes are pushed.


> The bad news is that when I tried to build the branch + your patch
> (as I said, please push to enable more easy testing for others) I'm
> running into:
> 

Unfortunately that happened to me once.... (out of 4 or 5 builds)

and searching for the error message led me to

https://github.com/jupyter/jupyter_client/issues/154

There's also few other issues that also involve the Kernel dying error
message.

So it seems like a sporadic upstream bug?

I need to sleep now.

Try again? hopefully it'll work. Probably the stack trace and version
numbers of the jupyter components need to be filed with upstream.

Diane

[toc] | [prev] | [next] | [standalone]

#9865 — Re: Python 3 Statsmodels & Pandas

Dear Diane,

On Wed, Sep 27, 2017 at 12:20:13AM -0700, Diane Trout wrote:
> > 
> > I think there is no "right" place to do it - but ftpmaster is used
> > to read README.source in these cases.  So why not using it?
> > 
> 
> I added the readme to README.source with a bit of explanatory text.
> 
> all changes are pushed.

Seems the updated README.source escaped your push. ;-)
 
> > The bad news is that when I tried to build the branch + your patch
> > (as I said, please push to enable more easy testing for others) I'm
> > running into:
> > 
> 
> Unfortunately that happened to me once.... (out of 4 or 5 builds)
> 
> and searching for the error message led me to
> 
> https://github.com/jupyter/jupyter_client/issues/154
> 
> There's also few other issues that also involve the Kernel dying error
> message.
> 
> So it seems like a sporadic upstream bug?

Hmmmm, if it is suspected to be a timing issue I was checking for
--parallel switch but it is not set.  Any better idea how to reliably
build the docs?
 
> I need to sleep now.

Sleep well - I hope we get this settled tomorrow.

> Try again? hopefully it'll work. Probably the stack trace and version
> numbers of the jupyter components need to be filed with upstream.

I can try again later.

I'd also merge your detrout-python3-try2 branch into debian to prepare
the next upload.

Thanks for all your work on this

       Andreas.

-- 
http://fam-tille.de

[toc] | [prev] | [next] | [standalone]

#9866 — Re: Python 3 Statsmodels & Pandas

Hi again,

On Wed, Sep 27, 2017 at 10:05:15AM +0200, Andreas Tille wrote:
> > I added the readme to README.source with a bit of explanatory text.
> > 
> > all changes are pushed.
> 
> Seems the updated README.source escaped your push. ;-)

I think I've restored it from the link you posted below and added some
information to README.source.
  
> > There's also few other issues that also involve the Kernel dying error
> > message.
> > 
> > So it seems like a sporadic upstream bug?

I was able to build the package several times in a sequence now.
  
> > I need to sleep now.
> 
> Sleep well - I hope we get this settled tomorrow.

I hope my upload might relax your next day a bit. ;-)
 
> > Try again? hopefully it'll work. Probably the stack trace and version
> > numbers of the jupyter components need to be filed with upstream.
> 
> I'd also merge your detrout-python3-try2 branch into debian to prepare
> the next upload.

I've found some time, did some polishing and uploaded after merging your
branch into debian branch (please git pull and may be you delete your
branch to keep the repository more simple).

As I said I'll ping ftpmaster via IRC once the package will show up
in the new queue.

Thanks for all your work

      Andreas.

-- 
http://fam-tille.de

[toc] | [prev] | [next] | [standalone]

#9877 — Re: Python 3 Statsmodels & Pandas

statsmodels passed NEW...and FTBFS with

dh: unable to load addon sphinxdoc: Can't locate 
Debian/Debhelper/Sequence/sphinxdoc.pm in @INC

That file is in the sphinx-common package, which suggests that at least 
some of the sphinx dependencies need to be Build-Depends and not just 
Build-Depends-Indep.

Meanwhile, pandas FTBFS with several failing tests (all involving 
datetime) on arm*/mips*el and a broken attempt to skip a test (trying to 
raise nose.SkipTest without importing nose) on mips/s390x.

[toc] | [prev] | [next] | [standalone]

#9878 — Re: Python 3 Statsmodels & Pandas

On Fri, 2017-09-29 at 22:05 +0100, Rebecca N. Palmer wrote:
> statsmodels passed NEW...and FTBFS with
> 
> dh: unable to load addon sphinxdoc: Can't locate 
> Debian/Debhelper/Sequence/sphinxdoc.pm in @INC
> That file is in the sphinx-common package, which suggests that at
> least 
> some of the sphinx dependencies need to be Build-Depends and not
> just 
> Build-Depends-Indep.

Oops.

I thought that it wouldn't try using sphinx when not building doc
packages. But apparently being listed in "dh --with" counts as being
used.

I wonder if it's better to filter sphinxdoc out of the dh line, install
sphinx-common, or just always install python3-sphinx?

I have some ideas of what to do, I should be able to fix this soon.

> 
> Meanwhile, pandas FTBFS with several failing tests (all involving 
> datetime) on arm*/mips*el and a broken attempt to skip a test (trying
> to 
> raise nose.SkipTest without importing nose) on mips/s390x.
> 

I don't know what was wrong with pandas & datetime. I was putting off 
/ hoping someone else would log into a porterbox to try to figure out
what's actually wrong.

As for nose, python{,3}-nose is listed a build-depends <!nocheck>? Why
is it failing on importing nose? it should either be there or it
shouldn't be running tests?

Diane

[toc] | [prev] | [next] | [standalone]

#9879 — Re: Python 3 Statsmodels & Pandas

On 29/09/17 23:55, Diane Trout wrote:
> I wonder if it's better to filter sphinxdoc out of the dh line, install
> sphinx-common, or just always install python3-sphinx?

For local testing of this, use pbuilder --debbuildopts -B

Given earlier messages that we want this working ASAP, always installing 
python3-sphinx is the lowest-risk option.

> As for nose, python{,3}-nose is listed a build-depends <!nocheck>? Why
> is it failing on importing nose? it should either be there or it
> shouldn't be running tests?

It's not failing on importing nose - it's failing because the patch 
tries to skip a test *without* importing nose.

[toc] | [prev] | [next] | [standalone]

#9880 — Re: Python 3 Statsmodels & Pandas

On 29/09/17 23:55, Diane Trout wrote:
> On Fri, 2017-09-29 at 22:05 +0100, Rebecca N. Palmer wrote:
>> statsmodels passed NEW...and FTBFS with
>>
>> dh: unable to load addon sphinxdoc: Can't locate
>> Debian/Debhelper/Sequence/sphinxdoc.pm in @INC
>> That file is in the sphinx-common package, which suggests that at
>> least
>> some of the sphinx dependencies need to be Build-Depends and not
>> just
>> Build-Depends-Indep.
> 
> Oops.
> 
> I thought that it wouldn't try using sphinx when not building doc
> packages. But apparently being listed in "dh --with" counts as being
> used.
> 
> I wonder if it's better to filter sphinxdoc out of the dh line, install
> sphinx-common, or just always install python3-sphinx?
> 
> I have some ideas of what to do, I should be able to fix this soon.

Either always install python3-sphinx (no harm in doing so), or use 
sphinx-common + python3-sphinx <!nodoc> + nodoc guards in d/rules.

There is no need for more complicated logic.

Ghis

[toc] | [prev] | [next] | [standalone]

#9881 — Re: Python 3 Statsmodels & Pandas

[Multipart message — attachments visible in raw view] — view raw

Hi Diane,

On Fri, Sep 29, 2017 at 03:55:13PM -0700, Diane Trout wrote:
> Oops.
>
> I thought that it wouldn't try using sphinx when not building doc
> packages. But apparently being listed in "dh --with" counts as being
> used.

Yes, because dh tries to open Sequece/sphinxdoc.pm which is provided by
sphinx-common.

> I wonder if it's better to filter sphinxdoc out of the dh line, install
> sphinx-common, or just always install python3-sphinx?

Adding sphinx-common to B-D and keeping python3-sphinx in B-D-Indep is
probably the easiest solution. Also you can try not relying on --with
at all and manually call dh_sphinxdoc when the build is arch-indep.

--
Dmitry Shachnev

[toc] | [prev] | [next] | [standalone]

#9886 — Re: Python 3 Statsmodels & Pandas

[Multipart message — attachments visible in raw view] — view raw

On Sat, 2017-09-30 at 12:26 +0300, Dmitry Shachnev wrote:
> 
> 
> > I wonder if it's better to filter sphinxdoc out of the dh line,
> > install
> > sphinx-common, or just always install python3-sphinx?
> 
> Adding sphinx-common to B-D and keeping python3-sphinx in B-D-Indep
> is
> probably the easiest solution. Also you can try not relying on --with
> at all and manually call dh_sphinxdoc when the build is arch-indep.
> 

It took me two tries, but statsmodels 0.8.0-6 builds everywhere pandas
builds now.

I am curious if Rebecca is using pandas on a non-intel architecture
though (was wondering how she noticed pandas hadn't built)

Diane

[toc] | [prev] | [next] | [standalone]

Page 3 of 4 — ← Prev page 1 2 [3] 4  Next page →

Back to top | Article view | linux.debian.maint.python

csiph-web