Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.maint.python > #9037 > unrolled thread

Binary naming for Django Related Packages

Back to article view | Back to linux.debian.maint.python

Contents

  Binary naming for Django Related Packages Scott Kitterman <debian@kitterman.com> - 2016-11-28 17:20 +0100
    Re: Binary naming for Django Related Packages Barry Warsaw <barry@debian.org> - 2016-11-28 17:40 +0100
      Re: Binary naming for Django Related Packages Piotr Ożarowski <piotr@debian.org> - 2016-11-28 18:00 +0100
        Re: Binary naming for Django Related Packages Raphael Hertzog <hertzog@debian.org> - 2016-11-29 13:50 +0100
      Re: Binary naming for Django Related Packages Thomas Goirand <zigo@debian.org> - 2016-12-03 17:10 +0100
        Re: Binary naming for Django Related Packages Jeremy Stanley <fungi@yuggoth.org> - 2016-12-03 17:50 +0100
    Re: Binary naming for Django Related Packages Julien Puydt <julien.puydt@laposte.net> - 2016-11-28 17:50 +0100
    Re: Binary naming for Django Related Packages Piotr Ożarowski <piotr@debian.org> - 2016-11-28 18:00 +0100
      Re: Binary naming for Django Related Packages Scott Kitterman <debian@kitterman.com> - 2016-11-28 18:10 +0100
        Re: Binary naming for Django Related Packages Raphael Hertzog <hertzog@debian.org> - 2016-11-29 14:00 +0100
          Re: Binary naming for Django Related Packages Scott Kitterman <debian@kitterman.com> - 2016-11-29 14:20 +0100
            Re: Binary naming for Django Related Packages Piotr Ożarowski <piotr@debian.org> - 2016-11-29 14:50 +0100
              Re: Binary naming for Django Related Packages Scott Kitterman <debian@kitterman.com> - 2016-11-30 06:20 +0100
                Re: Binary naming for Django Related Packages Raphael Hertzog <hertzog@debian.org> - 2016-12-07 11:50 +0100
                  Re: Binary naming for Django Related Packages Scott Kitterman <debian@kitterman.com> - 2017-01-18 07:50 +0100
                    Re: Binary naming for Django Related Packages IOhannes m zmölnig (Debian/GNU)  <umlaeute@debian.org> - 2017-01-18 10:50 +0100
    Re: Binary naming for Django Related Packages Thomas Goirand <zigo@debian.org> - 2016-12-03 17:10 +0100

#9037 — Binary naming for Django Related Packages

[Multipart message — attachments visible in raw view] — view raw

I've recently done some Django related packaging for the first time and 
noticed that we have organically (as far as I can tell) grown a slightly 
different naming convention for such packages.  Instead of python*-foo, we use 
python*-django-foo.

I think this is a reasonable approach and followed it in the new packages I've 
recently done.

I decided to check and see how common the approach is.  Here's what I found in 
Sid:

Start with django: 7
Start w/django, not transitional: 2
Start with django3: 0

Start with python-django (excluding -doc): 136
Start with python3-django: 84

I think it would make sense to add this to the Python policy so how we're 
doing it is documented.  I am attaching a proposed diff.  I made it a should 
because there are two non-DPMT packages that don't follow this rule and I 
think it's late in the cycle to be adding to must policy requirements.

Please let me know what you think.  I'm open to suggestions on wording.  I'd 
like to get this done in the next week and do a python-defaults upload with 
this and a few minor (non-policy) changes that are pending.

Scott K

@@ -534,6 +534,13 @@
      This requirement also applies to extension modules; binaries for all
      the supported Python versions should be included in a single package.
 
+     As a special exception to the `python3-' and `python-' binary naming
+     policy, Python modules intended for use with Django (`python3-django'/
+     `python-django') should add django to their binary package names to
+     make it clear they are intended for use with Django and not general
+     purpose Python modules, i.e.  `python3-django-' and `python-django-'
+     respectively.
+

[toc] | [next] | [standalone]

#9038

[Multipart message — attachments visible in raw view] — view raw

On Nov 28, 2016, at 11:11 AM, Scott Kitterman wrote:

>@@ -534,6 +534,13 @@
>      This requirement also applies to extension modules; binaries for all
>      the supported Python versions should be included in a single package.
> 
>+     As a special exception to the `python3-' and `python-' binary naming
>+     policy, Python modules intended for use with Django (`python3-django'/
>+     `python-django') should add django to their binary package names to
>+     make it clear they are intended for use with Django and not general
>+     purpose Python modules, i.e.  `python3-django-' and `python-django-'
>+     respectively.

+1 but I have a question since I'm not a hardcore Django developer.

In many cases we have namespace packages, e.g. zope.*, flufl.*, etc.  Usually
these will be called python-<main>.<sub>, e.g. python-flufl.i18n.

Is there any risk of having confusing names because of a conflict between a
3rd party Django module and a Django subpackage?  e.g. python3-django-foo
vs. python3-django.foo.

I'm sure it's a non-issue in practice.

Cheers,
-Barry

[toc] | [prev] | [next] | [standalone]

#9040

[Barry Warsaw, 2016-11-28]
> Is there any risk of having confusing names because of a conflict between a
> 3rd party Django module and a Django subpackage?  e.g. python3-django-foo
> vs. python3-django.foo.
> 
> I'm sure it's a non-issue in practice.

this is a huge issue IMHO beacause Django submodules use global
namespace and thus any unique django submodule name takes not so unique
Python module name (i.e. they're installed under
/usr/lib/python3/dist-packages/ now, not under
/usr/lib/python3/dist-packages/django)
-- 
Piotr Ożarowski                         Debian GNU/Linux Developer
www.ozarowski.pl          www.griffith.cc           www.debian.org
GPG Fingerprint: 1D2F A898 58DA AF62 1786 2DF7 AEF6 F1A2 A745 7645

[toc] | [prev] | [next] | [standalone]

#9047

On Mon, 28 Nov 2016, Piotr Ożarowski wrote:
> [Barry Warsaw, 2016-11-28]
> > Is there any risk of having confusing names because of a conflict between a
> > 3rd party Django module and a Django subpackage?  e.g. python3-django-foo
> > vs. python3-django.foo.
> > 
> > I'm sure it's a non-issue in practice.
> 
> this is a huge issue IMHO beacause Django submodules use global
> namespace and thus any unique django submodule name takes not so unique
> Python module name (i.e. they're installed under
> /usr/lib/python3/dist-packages/ now, not under
> /usr/lib/python3/dist-packages/django)

This is true but it's still a non-issue in practice because that kind of
conflict is usually detected and thus avoided at the pypi level.

And checking for file conflict is part of the job of the packager and we
have QA tools doing that kind of work too.

Cheers,
-- 
Raphaël Hertzog ◈ Debian Developer

Support Debian LTS: http://www.freexian.com/services/debian-lts.html
Learn to master Debian: http://debian-handbook.info/get/

[toc] | [prev] | [next] | [standalone]

#9058

On 11/28/2016 05:30 PM, Barry Warsaw wrote:
> On Nov 28, 2016, at 11:11 AM, Scott Kitterman wrote:
> 
>> @@ -534,6 +534,13 @@
>>      This requirement also applies to extension modules; binaries for all
>>      the supported Python versions should be included in a single package.
>>
>> +     As a special exception to the `python3-' and `python-' binary naming
>> +     policy, Python modules intended for use with Django (`python3-django'/
>> +     `python-django') should add django to their binary package names to
>> +     make it clear they are intended for use with Django and not general
>> +     purpose Python modules, i.e.  `python3-django-' and `python-django-'
>> +     respectively.
> 
> +1 but I have a question since I'm not a hardcore Django developer.
> 
> In many cases we have namespace packages, e.g. zope.*, flufl.*, etc.  Usually
> these will be called python-<main>.<sub>, e.g. python-flufl.i18n.
> 
> Is there any risk of having confusing names because of a conflict between a
> 3rd party Django module and a Django subpackage?  e.g. python3-django-foo
> vs. python3-django.foo.
> 
> I'm sure it's a non-issue in practice.
> 
> Cheers,
> -Barry

Because of problems when doing imports in Python3 (in a venv, the system
module wont be loaded if it's there and there's already something in the
venv), we should attempt to discourage upstream to use namespaced
modules. This indeed could prevent from running unit tests. That's what
has been discovered in the OpenStack world, and now all the oslo libs
aren't using namespace (though we've kept the dot for the egg-names).

Cheers,

Thomas Goirand (zigo)

[toc] | [prev] | [next] | [standalone]

#9060

On 2016-12-03 17:01:45 +0100 (+0100), Thomas Goirand wrote:
[...]
> Because of problems when doing imports in Python3 (in a venv, the system
> module wont be loaded if it's there and there's already something in the
> venv), we should attempt to discourage upstream to use namespaced
> modules. This indeed could prevent from running unit tests. That's what
> has been discovered in the OpenStack world, and now all the oslo libs
> aren't using namespace (though we've kept the dot for the egg-names).

To clarify, the main issue encountered there was a conflict over
namespace-level init when some modules were editable installs.
Historical details of the decision are outlined at:

<URL: https://specs.openstack.org/openstack/oslo-specs/specs/kilo/drop-namespace-packages.html#problem-description >

-- 
Jeremy Stanley

[toc] | [prev] | [next] | [standalone]

#9039

+1

On 28/11/2016 17:11, Scott Kitterman wrote:
> <a sensible policy change>

Snark on #debian-python

[toc] | [prev] | [next] | [standalone]

#9041

[Scott Kitterman, 2016-11-28]
> I've recently done some Django related packaging for the first time and 
> noticed that we have organically (as far as I can tell) grown a slightly 
> different naming convention for such packages.  Instead of python*-foo, we use 
> python*-django-foo.
> 
> I think this is a reasonable approach and followed it in the new packages I've 
> recently done.
> 
> I decided to check and see how common the approach is.  Here's what I found in 
> Sid:
> 
> Start with django: 7
> Start w/django, not transitional: 2
> Start with django3: 0
> 
> Start with python-django (excluding -doc): 136
> Start with python3-django: 84
> 
> I think it would make sense to add this to the Python policy so how we're 
> doing it is documented.  I am attaching a proposed diff.  I made it a should 
> because there are two non-DPMT packages that don't follow this rule and I 
> think it's late in the cycle to be adding to must policy requirements.
> 
> Please let me know what you think.  I'm open to suggestions on wording.  I'd 
> like to get this done in the next week and do a python-defaults upload with 
> this and a few minor (non-policy) changes that are pending.

-1 from me


If Django packages have no use outside Django¹, they should be moved out²
of public dist-packages IMO. If they are useful, "-django" part is misleading.

[¹] dash suggest they're not in django namespace, otherwise binary
    package name would be python3-django.foo
    (or python3-django.ext.foo, like in flask?)
[²] sys.path.append('/usr/lib/python3/django-packages/') in
    django/__init__.py if django import always prepends other imports
    (python3-django- namespace would be tolerable then, I guess)
-- 
Piotr Ożarowski                         Debian GNU/Linux Developer
www.ozarowski.pl          www.griffith.cc           www.debian.org
GPG Fingerprint: 1D2F A898 58DA AF62 1786 2DF7 AEF6 F1A2 A745 7645

[toc] | [prev] | [next] | [standalone]

#9042

On Monday, November 28, 2016 05:50:24 PM Piotr Ożarowski wrote:
> [Scott Kitterman, 2016-11-28]
> 
> > I've recently done some Django related packaging for the first time and
> > noticed that we have organically (as far as I can tell) grown a slightly
> > different naming convention for such packages.  Instead of python*-foo, we
> > use python*-django-foo.
> > 
> > I think this is a reasonable approach and followed it in the new packages
> > I've recently done.
> > 
> > I decided to check and see how common the approach is.  Here's what I
> > found in Sid:
> > 
> > Start with django: 7
> > Start w/django, not transitional: 2
> > Start with django3: 0
> > 
> > Start with python-django (excluding -doc): 136
> > Start with python3-django: 84
> > 
> > I think it would make sense to add this to the Python policy so how we're
> > doing it is documented.  I am attaching a proposed diff.  I made it a
> > should because there are two non-DPMT packages that don't follow this
> > rule and I think it's late in the cycle to be adding to must policy
> > requirements.
> > 
> > Please let me know what you think.  I'm open to suggestions on wording. 
> > I'd like to get this done in the next week and do a python-defaults
> > upload with this and a few minor (non-policy) changes that are pending.
> 
> -1 from me
> 
> 
> If Django packages have no use outside Django¹, they should be moved out²
> of public dist-packages IMO. If they are useful, "-django" part is
> misleading.
> 
> [¹] dash suggest they're not in django namespace, otherwise binary
>     package name would be python3-django.foo
>     (or python3-django.ext.foo, like in flask?)
> [²] sys.path.append('/usr/lib/python3/django-packages/') in
>     django/__init__.py if django import always prepends other imports
>     (python3-django- namespace would be tolerable then, I guess)

Some, but not all of them, are shipped as django_foo, so for those, I think 
python*-django-foo is actually correct.  

>From a django perspective, it makes a difference.  As an example, django-
python-hstore (which I just packaged with python*-django-hstore binaries) is 
django_hstore.  When added to Django INSTALLED_APPS, it's added as 
django_hstore.  The django. namespace is for things shipped with Django, not 
third-party packages.

Asking upstreams to rename to use django_ where they don't will complicate 
things for upgrades, so I don't think that's a great idea.

I'm not one of the python-django uploaders, so we'd need their feedback on 
[2].  I think something like that is a reasonable compromise if they are 
willing to support it.

Thanks,

Scott K

[toc] | [prev] | [next] | [standalone]

#9048

On Mon, 28 Nov 2016, Scott Kitterman wrote:
> > > Please let me know what you think.  I'm open to suggestions on wording. 
> > > I'd like to get this done in the next week and do a python-defaults
> > > upload with this and a few minor (non-policy) changes that are pending.

+1 from me. I'm actually the one who started this convention when I
packaged the first Django extension. When I look for available Django
extension, I like to be able to rely on the prefix.

> > [²] sys.path.append('/usr/lib/python3/django-packages/') in
> >     django/__init__.py if django import always prepends other imports
> >     (python3-django- namespace would be tolerable then, I guess)
> 
> I'm not one of the python-django uploaders, so we'd need their feedback on 
> [2].  I think something like that is a reasonable compromise if they are 
> willing to support it.

I certainly don't want to introduce this Debian-specific difference, no.
Django applications/extensions are meant to be managed via "pip" and they
must be available in the global namespace. I would not be surprised that
some of the extensions actually rely on being available globally...

I don't see any benefit to this change. The global namespace pollution
already exists at the upstream level, while we have to handle potential
conflicts, it's not up to us to preventively curate the namespace when
upstream has not followed the best practice (i.e. the "django_" prefix
in the module name).

Cheers,
-- 
Raphaël Hertzog ◈ Debian Developer

Support Debian LTS: http://www.freexian.com/services/debian-lts.html
Learn to master Debian: http://debian-handbook.info/get/

[toc] | [prev] | [next] | [standalone]

#9049

On Tuesday, November 29, 2016 01:52:07 PM Raphael Hertzog wrote:
> On Mon, 28 Nov 2016, Scott Kitterman wrote:
> > > > Please let me know what you think.  I'm open to suggestions on
> > > > wording.
> > > > I'd like to get this done in the next week and do a python-defaults
> > > > upload with this and a few minor (non-policy) changes that are
> > > > pending.
> 
> +1 from me. I'm actually the one who started this convention when I
> packaged the first Django extension. When I look for available Django
> extension, I like to be able to rely on the prefix.
> 
> > > [²] sys.path.append('/usr/lib/python3/django-packages/') in
> > > 
> > >     django/__init__.py if django import always prepends other imports
> > >     (python3-django- namespace would be tolerable then, I guess)
> > 
> > I'm not one of the python-django uploaders, so we'd need their feedback on
> > [2].  I think something like that is a reasonable compromise if they are
> > willing to support it.
> 
> I certainly don't want to introduce this Debian-specific difference, no.
> Django applications/extensions are meant to be managed via "pip" and they
> must be available in the global namespace. I would not be surprised that
> some of the extensions actually rely on being available globally...
> 
> I don't see any benefit to this change. The global namespace pollution
> already exists at the upstream level, while we have to handle potential
> conflicts, it's not up to us to preventively curate the namespace when
> upstream has not followed the best practice (i.e. the "django_" prefix
> in the module name).

Thanks for the feedback.  I think that eliminates Piotr's options 2.

Personally, I think policy is at its best documenting current practice rather 
than to drive it, that's why I made the initial proposal.  There's an 
exception to the usual rule that is virtually universally applied, so I 
believe we ought to document it.

Piotr: Is there some language that acknowledges the situation as unusual, even 
if it doesn't fully bless it that you'd be comfortable with in policy so we 
can at least document current practice?

Scott K

[toc] | [prev] | [next] | [standalone]

#9050

[Scott Kitterman, 2016-11-29]
> Piotr: Is there some language that acknowledges the situation as unusual, even 
> if it doesn't fully bless it that you'd be comfortable with in policy so we 
> can at least document current practice?

if module name is foo, name of the binary package should be python3-foo,
not python3-bar-foo. There is no change needed on the upstream side or
in django/__init__.py - it's just about naming binary packages the way
our policy recommends. Adjusting policy to document good changes we came
up in practice is good. Documenting bad ones is not.

That said, I don't have a veto vote so I will just have to deal with it.
-- 
Piotr Ożarowski                         Debian GNU/Linux Developer
www.ozarowski.pl          www.griffith.cc           www.debian.org
GPG Fingerprint: 1D2F A898 58DA AF62 1786 2DF7 AEF6 F1A2 A745 7645

[toc] | [prev] | [next] | [standalone]

#9051

On Tuesday, November 29, 2016 02:40:06 PM Piotr Ożarowski wrote:
> [Scott Kitterman, 2016-11-29]
> 
> > Piotr: Is there some language that acknowledges the situation as unusual,
> > even if it doesn't fully bless it that you'd be comfortable with in
> > policy so we can at least document current practice?
> 
> if module name is foo, name of the binary package should be python3-foo,
> not python3-bar-foo. There is no change needed on the upstream side or
> in django/__init__.py - it's just about naming binary packages the way
> our policy recommends. Adjusting policy to document good changes we came
> up in practice is good. Documenting bad ones is not.
> 
> That said, I don't have a veto vote so I will just have to deal with it.

I can see your point.  I can see Raphael's too.

I do think that upstream third parties using django_foo represents some kind 
of best practice that should be documented and encouraged (but not directly by 
Debian).  

Raphael, do you think that the upstream Django project might be willing to 
make some kind of best practices for naming third party django packages?  If 
they did that, then that would give us a basis for Debian maintainers talking 
to their upstreams about moving to django_.

Over time, that would result in python-django-foo being the correct name 
without any kind of Python policy exception (and we'd limit this to being 
temporary).

How's that?

Scott K

[toc] | [prev] | [next] | [standalone]

#9076

Hi,

On Wed, 30 Nov 2016, Scott Kitterman wrote:
> Raphael, do you think that the upstream Django project might be willing to 
> make some kind of best practices for naming third party django packages?  If 
> they did that, then that would give us a basis for Debian maintainers talking 
> to their upstreams about moving to django_.

They already partly do that, see:
https://docs.djangoproject.com/en/1.10/intro/reusable-apps/#packaging-your-app

They recommend a "django-" prefix in the PyPi package name. But they say
nothing about the Python module name and the sample just bundles a "polls"
module in a "django-polls" package.

Thus I posted this to gather their feedback on the need to recommend the
prefix on the name of the module too:
https://groups.google.com/forum/#!topic/django-developers/f8yNRkn6Fpo

Cheers,
-- 
Raphaël Hertzog ◈ Debian Developer

Support Debian LTS: http://www.freexian.com/services/debian-lts.html
Learn to master Debian: http://debian-handbook.info/get/

[toc] | [prev] | [next] | [standalone]

#9169

[Multipart message — attachments visible in raw view] — view raw

On Wednesday, December 07, 2016 11:43:29 AM Raphael Hertzog wrote:
> Hi,
> 
> On Wed, 30 Nov 2016, Scott Kitterman wrote:
> > Raphael, do you think that the upstream Django project might be willing to
> > make some kind of best practices for naming third party django packages? 
> > If they did that, then that would give us a basis for Debian maintainers
> > talking to their upstreams about moving to django_.
> 
> They already partly do that, see:
> https://docs.djangoproject.com/en/1.10/intro/reusable-apps/#packaging-your-a
> pp
> 
> They recommend a "django-" prefix in the PyPi package name. But they say
> nothing about the Python module name and the sample just bundles a "polls"
> module in a "django-polls" package.
> 
> Thus I posted this to gather their feedback on the need to recommend the
> prefix on the name of the module too:
> https://groups.google.com/forum/#!topic/django-developers/f8yNRkn6Fpo

Thanks.  Since not everyone liked my first attempt at this, I softened it a 
bit (please see the attached).  We're going to have at least one more python-
defaults upload and I'd like to get this resolved.

Scott K

[toc] | [prev] | [next] | [standalone]

#9172

On 2017-01-18 07:46, Scott Kitterman wrote:
> +··········named·django_packagename·upstream.··These·are·then·packaged·as
> +··········<package>python3-django-package</package>·and

please use "package" vs "packagename" consistently.
e.g. an upstream named "django_packagename" should be packaged as
"python3-django-packagename".

It's kind of obvious, but I think the policy should be precise.

(and probably use "<packagename>" or "$packagename" or something else to
mark it as variable)

gfmadr
IOhannes

[toc] | [prev] | [next] | [standalone]

#9057

On 11/28/2016 05:11 PM, Scott Kitterman wrote:
> I've recently done some Django related packaging for the first time and 
> noticed that we have organically (as far as I can tell) grown a slightly 
> different naming convention for such packages.  Instead of python*-foo, we use 
> python*-django-foo.
> 
> I think this is a reasonable approach and followed it in the new packages I've 
> recently done.
> 
> I decided to check and see how common the approach is.  Here's what I found in 
> Sid:
> 
> Start with django: 7
> Start w/django, not transitional: 2
> Start with django3: 0
> 
> Start with python-django (excluding -doc): 136
> Start with python3-django: 84
> 
> I think it would make sense to add this to the Python policy so how we're 
> doing it is documented.  I am attaching a proposed diff.  I made it a should 
> because there are two non-DPMT packages that don't follow this rule and I 
> think it's late in the cycle to be adding to must policy requirements.
> 
> Please let me know what you think.  I'm open to suggestions on wording.  I'd 
> like to get this done in the next week and do a python-defaults upload with 
> this and a few minor (non-policy) changes that are pending.
> 
> Scott K
> 
> @@ -534,6 +534,13 @@
>       This requirement also applies to extension modules; binaries for all
>       the supported Python versions should be included in a single package.
>  
> +     As a special exception to the `python3-' and `python-' binary naming
> +     policy, Python modules intended for use with Django (`python3-django'/
> +     `python-django') should add django to their binary package names to
> +     make it clear they are intended for use with Django and not general
> +     purpose Python modules, i.e.  `python3-django-' and `python-django-'
> +     respectively.

IMO, what should drive the binary package name is what upstream sets as
egg name, so that we don't have to do pydists-overrides. The current
global Python policy is to use the Python module names (ie: what one
would write when doing an import), which IMO is wrong, because we got to
do so many substitutions to have the Depends correct. If our policy was
to use egg-name for package names, we wouldn't have any substitution to
calculate.

Anyway, I don't see why Django modules should be an exception to any
rule we choose. If upstream is missing the "django-" prefix, then we
should suggest it.

Thomas Goirand (zigo)

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.maint.python

csiph-web