Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.maint.python > #8154 > unrolled thread

mkdocs locale error building djangorestframework

Back to article view | Back to linux.debian.maint.python

Contents

  mkdocs locale error building djangorestframework Brian May <brian@linuxpenguins.xyz> - 2016-01-26 01:30 +0100
    Re: mkdocs locale error building djangorestframework Ben Hutchings <ben@decadent.org.uk> - 2016-01-26 01:50 +0100
      Re: mkdocs locale error building djangorestframework Brian May <bam@debian.org> - 2016-01-26 01:50 +0100
        Re: mkdocs locale error building djangorestframework Brian May <bam@debian.org> - 2016-01-26 02:20 +0100
          Re: mkdocs locale error building djangorestframework Ben Hutchings <ben@decadent.org.uk> - 2016-01-26 02:30 +0100
            Re: mkdocs locale error building djangorestframework Brian May <bam@debian.org> - 2016-01-26 02:30 +0100
            Re: mkdocs locale error building djangorestframework Michael Tautschnig <mt@debian.org> - 2016-01-26 03:30 +0100
        Re: mkdocs locale error building djangorestframework Ben Hutchings <ben@decadent.org.uk> - 2016-01-26 02:20 +0100
          Re: mkdocs locale error building djangorestframework Barry Warsaw <barry@debian.org> - 2016-01-26 05:20 +0100
            Re: mkdocs locale error building djangorestframework Julien Cristau <jcristau@debian.org> - 2016-01-28 14:10 +0100
          Re: mkdocs locale error building djangorestframework Alexandre Viau <aviau@debian.org> - 2016-01-26 16:10 +0100

#8154 — mkdocs locale error building djangorestframework

Hello,

In building djangorestframework, it uses a package called mkdocs.

mkdocs in turn uses a package called python3-click to do its command
line processing.

python3-click has checks to ensure it is called with a non-ASCII
locale. Or if it is an ASCII locale, it raises a fatal error. Or at
least it tries to raise a fatal error.

This is documented in the click documentation:
http://click.pocoo.org/5/python3/

An added complication is at the moment the code to display the fatal
error is broken under Python 3 - see #812713. You can't compare byte
objects and str objects.

The problem is that when building djangorestframework, I need to have a
utf8 locale available. C.UTF-8 is fine. However, I don't think there is
any guarantee (??) that a utf8 locale will be available when building
packages. This I believe is the root cause of #812672.

So should I somehow be trying to get the UTF-8 locale (is this possible
from debian/rules?), or should I submit another bug report against
python3-click saying that it really should work in ASCII environments?

Regards
-- 
Brian May <brian@linuxpenguins.xyz>
https://linuxpenguins.xyz/brian/

[toc] | [next] | [standalone]

#8155

[Multipart message — attachments visible in raw view] — view raw

On Tue, 2016-01-26 at 11:29 +1100, Brian May wrote:
[...]
> The problem is that when building djangorestframework, I need to have a
> utf8 locale available. C.UTF-8 is fine. However, I don't think there is
> any guarantee (??) that a utf8 locale will be available when building
> packages. This I believe is the root cause of #812672.
>
> So should I somehow be trying to get the UTF-8 locale (is this possible
> from debian/rules?), or should I submit another bug report against
> python3-click saying that it really should work in ASCII environments?

C.UTF-8 is built into glibc so I believe you can assume it's present in
all Debian installations (from wheezy onward).

(The linux source package depends on this already for reproducible
docs.)

Ben.

-- 
Ben Hutchings
Q.  Which is the greater problem in the world today, ignorance or apathy?
A.  I don't know and I couldn't care less.

[toc] | [prev] | [next] | [standalone]

#8156

Ben Hutchings <ben@decadent.org.uk> writes:

> C.UTF-8 is built into glibc so I believe you can assume it's present in
> all Debian installations (from wheezy onward).

Ok, this is getting more and more complicated. The only reason I can see
for the FTBFS in #812672 is because the C.UTF-8 locale doesn't exist. It
shouldn't even be reaching that line of code otherwise.

Maybe something to do with building with pbuilder as opposed to sbuild?
-- 
Brian May <bam@debian.org>

[toc] | [prev] | [next] | [standalone]

#8157

Ben Hutchings <ben@decadent.org.uk> writes:

> That's not the problem at all.  Read the error message again.  Read the
> source line it points to.  Now look at where rv comes from:

Yes, that is how it would appear.

However, all this code does is try to work out the appropriate fatal
error message that should get displayed.

i.e. we should never have executed this code in the first place.

(I did file a bug on the python3 issue however)
-- 
Brian May <bam@debian.org>

[toc] | [prev] | [next] | [standalone]

#8159

[Multipart message — attachments visible in raw view] — view raw

On Tue, 2016-01-26 at 12:16 +1100, Brian May wrote:
> Ben Hutchings <ben@decadent.org.uk> writes:
> 
> > That's not the problem at all.  Read the error message again.  Read the
> > source line it points to.  Now look at where rv comes from:
> 
> Yes, that is how it would appear.
> 
> However, all this code does is try to work out the appropriate fatal
> error message that should get displayed.
> 
> i.e. we should never have executed this code in the first place.
> 
> (I did file a bug on the python3 issue however)

Oh, well that's probably because you only set LANG and it's being
overridden by LC_ALL.  Use a bigger hammer: set LC_ALL yourself.

Ben.

-- 
Ben Hutchings
Q.  Which is the greater problem in the world today, ignorance or apathy?
A.  I don't know and I couldn't care less.

[toc] | [prev] | [next] | [standalone]

#8160

Ben Hutchings <ben@decadent.org.uk> writes:

> Oh, well that's probably because you only set LANG and it's being
> overridden by LC_ALL.  Use a bigger hammer: set LC_ALL yourself.

Yes, somebody mentioned this on the BTS also. I very much suspect this
will be the solution.

Thanks
-- 
Brian May <bam@debian.org>

[toc] | [prev] | [next] | [standalone]

#8161

[Multipart message — attachments visible in raw view] — view raw

On Tue, Jan 26, 2016 at  1:23:19 +0000, Ben Hutchings wrote:
> On Tue, 2016-01-26 at 12:16 +1100, Brian May wrote:
> > Ben Hutchings <ben@decadent.org.uk> writes:
> > 
> > > That's not the problem at all.  Read the error message again.  Read the
> > > source line it points to.  Now look at where rv comes from:
> > 
> > Yes, that is how it would appear.
> > 
> > However, all this code does is try to work out the appropriate fatal
> > error message that should get displayed.
> > 
> > i.e. we should never have executed this code in the first place.
> > 
> > (I did file a bug on the python3 issue however)
> 
> Oh, well that's probably because you only set LANG and it's being
> overridden by LC_ALL.  Use a bigger hammer: set LC_ALL yourself.
> 

For the wider record: yes, pbuilder sets LC_ALL to C, which has been/is being
discussed in #376404 and #725788. It's just a problem surfacing with python3 and
will affect a number of packages I'm afraid.

Best,
Michael

[toc] | [prev] | [next] | [standalone]

#8158

[Multipart message — attachments visible in raw view] — view raw

On Tue, 2016-01-26 at 11:49 +1100, Brian May wrote:
> Ben Hutchings <ben@decadent.org.uk> writes:
> 
> > C.UTF-8 is built into glibc so I believe you can assume it's present in
> > all Debian installations (from wheezy onward).
> 
> Ok, this is getting more and more complicated. The only reason I can see
> for the FTBFS in #812672 is because the C.UTF-8 locale doesn't exist. It
> shouldn't even be reaching that line of code otherwise.
> 
> Maybe something to do with building with pbuilder as opposed to sbuild?

That's not the problem at all.  Read the error message again.  Read the
source line it points to.  Now look at where rv comes from:

>>> import subprocess
>>> rv = subprocess.Popen(['locale', '-a'], stdout=subprocess.PIPE,
...                       stderr=subprocess.PIPE).communicate()[0]
>>> type(rv)
<class 'bytes'>

This is clearly a bug in python3-click.

Ben.

-- 
Ben Hutchings
Q.  Which is the greater problem in the world today, ignorance or apathy?
A.  I don't know and I couldn't care less.

[toc] | [prev] | [next] | [standalone]

#8162

[Multipart message — attachments visible in raw view] — view raw

On Jan 26, 2016, at 01:12 AM, Ben Hutchings wrote:

>That's not the problem at all.  Read the error message again.  Read the
>source line it points to.  Now look at where rv comes from:
>
>>>> import subprocess
>>>> rv = subprocess.Popen(['locale', '-a'], stdout=subprocess.PIPE,  
>...                       stderr=subprocess.PIPE).communicate()[0]
>>>> type(rv)  
><class 'bytes'>

For Python 3, try adding `universal_newlines=True` to any subprocess call.
You'll get back a str.

Cheers,
-Barry

[toc] | [prev] | [next] | [standalone]

#8175

On Mon, Jan 25, 2016 at 23:13:49 -0500, Barry Warsaw wrote:

> On Jan 26, 2016, at 01:12 AM, Ben Hutchings wrote:
> 
> >That's not the problem at all.  Read the error message again.  Read the
> >source line it points to.  Now look at where rv comes from:
> >
> >>>> import subprocess
> >>>> rv = subprocess.Popen(['locale', '-a'], stdout=subprocess.PIPE,  
> >...                       stderr=subprocess.PIPE).communicate()[0]
> >>>> type(rv)  
> ><class 'bytes'>
> 
> For Python 3, try adding `universal_newlines=True` to any subprocess call.
> You'll get back a str.
> 
Or a UnicodeDecodeError.

Cheers,
Julien

[toc] | [prev] | [next] | [standalone]

#8164

On Mon, Jan 25, 2016 at 8:12 PM, Ben Hutchings <ben@decadent.org.uk> wrote:
> On Tue, 2016-01-26 at 11:49 +1100, Brian May wrote:
>>>> import subprocess
>>>> rv = subprocess.Popen(['locale', '-a'], stdout=subprocess.PIPE,
> ...                       stderr=subprocess.PIPE).communicate()[0]
>>>> type(rv)
> <class 'bytes'>
>
> This is clearly a bug in python3-click.

Yes it was.

I have just uploaded a fix.

--
Alexandre Viau
aviau@debian.org

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.maint.python

csiph-web