Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.maint.python > #10380 > unrolled thread

Strange encoding problem in setup.py of biopython 1.71

Started byAndreas Tille <andreas@an3as.eu>
First post2018-04-05 16:20 +0200
Last post2018-04-05 16:50 +0200
Articles 10 — 6 participants

Back to article view | Back to linux.debian.maint.python

Contents

  Strange encoding problem in setup.py of biopython 1.71 Andreas Tille <andreas@an3as.eu> - 2018-04-05 16:20 +0200
    Re: Strange encoding problem in setup.py of biopython 1.71 Piotr Ożarowski <piotr@debian.org> - 2018-04-05 16:40 +0200
    Re: Strange encoding problem in setup.py of biopython 1.71 Andreas Tille <andreas@an3as.eu> - 2018-04-05 16:40 +0200
      Re: Strange encoding problem in setup.py of biopython 1.71 Klaus Zimmermann <klaus_zimmermann@gmx.de> - 2018-04-05 16:50 +0200
        Re: Strange encoding problem in setup.py of biopython 1.71 Peter Cock <p.j.a.cock@googlemail.com> - 2018-04-05 17:30 +0200
          Re: Strange encoding problem in setup.py of biopython 1.71 Andreas Tille <tille@debian.org> - 2018-04-06 09:20 +0200
            Re: Strange encoding problem in setup.py of biopython 1.71 Peter Cock <p.j.a.cock@googlemail.com> - 2018-04-06 11:40 +0200
              Re: Strange encoding problem in setup.py of biopython 1.71 Piotr Ożarowski <piotr@debian.org> - 2018-04-07 00:00 +0200
              Re: Strange encoding problem in setup.py of biopython 1.71 Peter Cock <p.j.a.cock@googlemail.com> - 2018-04-13 13:20 +0200
      Re: Strange encoding problem in setup.py of biopython 1.71 Olivier Sallou <olivier.sallou@irisa.fr> - 2018-04-05 16:50 +0200

#10380 — Strange encoding problem in setup.py of biopython 1.71

FromAndreas Tille <andreas@an3as.eu>
Date2018-04-05 16:20 +0200
SubjectStrange encoding problem in setup.py of biopython 1.71
Message-ID<vBdU5-5Ck-1@gated-at.bofh.it>
Hi,

I imported BioPython version 1.71 into the packaging Git[1] and tried to
build it.  Unfortunately I'm stumbling upon some encoding problem in
(maybe?) setup.py:


...
dh_auto_clean
        pybuild --clean -i python{version} -p 2.7
I: pybuild base:217: python2.7 setup.py clean 
running clean
removing '/home/andreas/debian-maintain/salsa/med-team/build-area/python-biopython-1.71+dfsg/.pybuild/cpython2_2.7/build' (and everything under it)
'build/bdist.linux-x86_64' does not exist -- can't clean it
'build/scripts-2.7' does not exist -- can't clean it
        pybuild --clean -i python{version} -p 3.6
I: pybuild base:217: python3.6 setup.py clean 
Traceback (most recent call last):
  File "setup.py", line 435, in <module>
    readme_rst = handle.read()
  File "/usr/lib/python3.6/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 7116: ordinal not in range(128)
E: pybuild pybuild:336: clean: plugin distutils failed with: exit code=1: python3.6 setup.py clean 
dh_auto_clean: pybuild --clean -i python{version} -p 3.6 returned exit code 13
...


I admit I have no idea how to fix this.  Any idea?

Kind regards

        Andreas.


[1] https://salsa.debian.org/med-team/python-biopython

-- 
http://fam-tille.de

[toc] | [next] | [standalone]

#10382

FromPiotr Ożarowski <piotr@debian.org>
Date2018-04-05 16:40 +0200
Message-ID<vBeds-5Ip-13@gated-at.bofh.it>
In reply to#10380
[Andreas Tille, 2018-04-05]
> Traceback (most recent call last):
>   File "setup.py", line 435, in <module>
>     readme_rst = handle.read()
>   File "/usr/lib/python3.6/encodings/ascii.py", line 26, in decode
>     return codecs.ascii_decode(input, self.errors)[0]
> UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 7116: ordinal not in range(128)
> E: pybuild pybuild:336: clean: plugin distutils failed with: exit code=1: python3.6 setup.py clean 
> dh_auto_clean: pybuild --clean -i python{version} -p 3.6 returned exit code 13
> ...

in setup.py, if you decide to keep supporting Python 2, change lines 434-435 into:

  with open("README.rst", 'rb') as handle:
      readme_rst = handle.read().decode('UTF-8')

or if you finally let it go, change only line 434:

  with open("README.rst", encoding='UTF-8') as handle:
-- 
GPG: 1D2F A898 58DA AF62 1786 2DF7 AEF6 F1A2 A745 7645

[toc] | [prev] | [next] | [standalone]

#10383

FromAndreas Tille <andreas@an3as.eu>
Date2018-04-05 16:40 +0200
Message-ID<vBeds-5Ip-17@gated-at.bofh.it>
In reply to#10380
Hi Olivier,

On Thu, Apr 05, 2018 at 04:19:37PM +0200, Olivier Sallou wrote:
> 
> On 04/05/2018 04:10 PM, Andreas Tille wrote:
> > (maybe?) setup.py:
> >
> >
> > ...
> > dh_auto_clean
> >         pybuild --clean -i python{version} -p 2.7
> > I: pybuild base:217: python2.7 setup.py clean 
> > running clean
> > removing '/home/andreas/debian-maintain/salsa/med-team/build-area/python-biopython-1.71+dfsg/.pybuild/cpython2_2.7/build' (and everything under it)
> > 'build/bdist.linux-x86_64' does not exist -- can't clean it
> > 'build/scripts-2.7' does not exist -- can't clean it
> >         pybuild --clean -i python{version} -p 3.6
> > I: pybuild base:217: python3.6 setup.py clean 
> > Traceback (most recent call last):
> >   File "setup.py", line 435, in <module>
> >     readme_rst = handle.read()
> >   File "/usr/lib/python3.6/encodings/ascii.py", line 26, in decode
> >     return codecs.ascii_decode(input, self.errors)[0]
> > UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 7116: ordinal not in range(128)
> > E: pybuild pybuild:336: clean: plugin distutils failed with: exit code=1: python3.6 setup.py clean 
> > dh_auto_clean: pybuild --clean -i python{version} -p 3.6 returned exit code 13
> this is usually related to non ascii characters in file (readme.rst?) ,
> if you find some, you can patch file to remove them.

Is there any efficient method to seek for this kind of non-ascii
characters?

Kind regards

      Andreas.

-- 
http://fam-tille.de

[toc] | [prev] | [next] | [standalone]

#10384

FromKlaus Zimmermann <klaus_zimmermann@gmx.de>
Date2018-04-05 16:50 +0200
Message-ID<vBen8-5LS-7@gated-at.bofh.it>
In reply to#10383
Hi Andreas,

here, the culprit is
U+2019 : RIGHT SINGLE QUOTATION MARK {single comma quotation mark}
that is used as an apostrophe in two places in "Apple's".

I found this by looking for 0xe2 (from your error message) in an hexdump
(od -A d -xc README.rst |less ) at the named position (7116).

Cheers
Klaus


On 05/04/18 16:34, Andreas Tille wrote:
> Hi Olivier,
> 
> On Thu, Apr 05, 2018 at 04:19:37PM +0200, Olivier Sallou wrote:
>>
>> On 04/05/2018 04:10 PM, Andreas Tille wrote:
>>> (maybe?) setup.py:
>>>
>>>
>>> ...
>>> dh_auto_clean
>>>         pybuild --clean -i python{version} -p 2.7
>>> I: pybuild base:217: python2.7 setup.py clean 
>>> running clean
>>> removing '/home/andreas/debian-maintain/salsa/med-team/build-area/python-biopython-1.71+dfsg/.pybuild/cpython2_2.7/build' (and everything under it)
>>> 'build/bdist.linux-x86_64' does not exist -- can't clean it
>>> 'build/scripts-2.7' does not exist -- can't clean it
>>>         pybuild --clean -i python{version} -p 3.6
>>> I: pybuild base:217: python3.6 setup.py clean 
>>> Traceback (most recent call last):
>>>   File "setup.py", line 435, in <module>
>>>     readme_rst = handle.read()
>>>   File "/usr/lib/python3.6/encodings/ascii.py", line 26, in decode
>>>     return codecs.ascii_decode(input, self.errors)[0]
>>> UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 7116: ordinal not in range(128)
>>> E: pybuild pybuild:336: clean: plugin distutils failed with: exit code=1: python3.6 setup.py clean 
>>> dh_auto_clean: pybuild --clean -i python{version} -p 3.6 returned exit code 13
>> this is usually related to non ascii characters in file (readme.rst?) ,
>> if you find some, you can patch file to remove them.
> 
> Is there any efficient method to seek for this kind of non-ascii
> characters?
> 
> Kind regards
> 
>       Andreas.
>

[toc] | [prev] | [next] | [standalone]

#10392

FromPeter Cock <p.j.a.cock@googlemail.com>
Date2018-04-05 17:30 +0200
Message-ID<vBeZQ-6hA-11@gated-at.bofh.it>
In reply to#10384
Confirmed - not sure if my earlier email made it?

https://github.com/biopython/biopython/commit/6421d365a511c0de71748737ba6e1cd8518658ac

Can you pull in that commit as a patch for the Biopython 1.71 package?

Peter

On Thu, Apr 5, 2018 at 3:39 PM, Klaus Zimmermann
<klaus_zimmermann@gmx.de> wrote:
> Hi Andreas,
>
> here, the culprit is
> U+2019 : RIGHT SINGLE QUOTATION MARK {single comma quotation mark}
> that is used as an apostrophe in two places in "Apple's".
>
> I found this by looking for 0xe2 (from your error message) in an hexdump
> (od -A d -xc README.rst |less ) at the named position (7116).
>
> Cheers
> Klaus
>
>
> On 05/04/18 16:34, Andreas Tille wrote:
>> Hi Olivier,
>>
>> On Thu, Apr 05, 2018 at 04:19:37PM +0200, Olivier Sallou wrote:
>>>
>>> On 04/05/2018 04:10 PM, Andreas Tille wrote:
>>>> (maybe?) setup.py:
>>>>
>>>>
>>>> ...
>>>> dh_auto_clean
>>>>         pybuild --clean -i python{version} -p 2.7
>>>> I: pybuild base:217: python2.7 setup.py clean
>>>> running clean
>>>> removing '/home/andreas/debian-maintain/salsa/med-team/build-area/python-biopython-1.71+dfsg/.pybuild/cpython2_2.7/build' (and everything under it)
>>>> 'build/bdist.linux-x86_64' does not exist -- can't clean it
>>>> 'build/scripts-2.7' does not exist -- can't clean it
>>>>         pybuild --clean -i python{version} -p 3.6
>>>> I: pybuild base:217: python3.6 setup.py clean
>>>> Traceback (most recent call last):
>>>>   File "setup.py", line 435, in <module>
>>>>     readme_rst = handle.read()
>>>>   File "/usr/lib/python3.6/encodings/ascii.py", line 26, in decode
>>>>     return codecs.ascii_decode(input, self.errors)[0]
>>>> UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 7116: ordinal not in range(128)
>>>> E: pybuild pybuild:336: clean: plugin distutils failed with: exit code=1: python3.6 setup.py clean
>>>> dh_auto_clean: pybuild --clean -i python{version} -p 3.6 returned exit code 13
>>> this is usually related to non ascii characters in file (readme.rst?) ,
>>> if you find some, you can patch file to remove them.
>>
>> Is there any efficient method to seek for this kind of non-ascii
>> characters?
>>
>> Kind regards
>>
>>       Andreas.
>>
>
>

[toc] | [prev] | [next] | [standalone]

#10396

FromAndreas Tille <tille@debian.org>
Date2018-04-06 09:20 +0200
Message-ID<vBtPb-7Uw-5@gated-at.bofh.it>
In reply to#10392
Hi Peter,

On Thu, Apr 05, 2018 at 04:19:50PM +0100, Peter Cock wrote:
> 
> https://github.com/biopython/biopython/commit/6421d365a511c0de71748737ba6e1cd8518658ac
> 
> Can you pull in that commit as a patch for the Biopython 1.71 package?

Thanks.  I'm using this as patch.  I can confirm that this works now
(I'm using the other patch suggested here[1] in addition to deal with
potential other encoding issues.
 
I'll upload soon (I had some other issues with the packaging which are
not related).

Kind regards

    Andreas.


[1] https://lists.debian.org/debian-python/2018/04/msg00023.html

-- 
http://fam-tille.de

[toc] | [prev] | [next] | [standalone]

#10397

FromPeter Cock <p.j.a.cock@googlemail.com>
Date2018-04-06 11:40 +0200
Message-ID<vBw0G-O7-7@gated-at.bofh.it>
In reply to#10396
Thanks Andreas,

Piotr, I think your fix would be worth including in Biopython.

Do you want to submit this as a pull request, or give me
permission to do so by email?

We'd need you to agree to both the Biopython License Agreement
and the 3-clause BSD licence, and say if you'd like to be
acknowledged by name or not.

Regards,

Peter

On Fri, Apr 6, 2018 at 8:16 AM, Andreas Tille <tille@debian.org> wrote:
> Hi Peter,
>
> On Thu, Apr 05, 2018 at 04:19:50PM +0100, Peter Cock wrote:
>>
>> https://github.com/biopython/biopython/commit/6421d365a511c0de71748737ba6e1cd8518658ac
>>
>> Can you pull in that commit as a patch for the Biopython 1.71 package?
>
> Thanks.  I'm using this as patch.  I can confirm that this works now
> (I'm using the other patch suggested here[1] in addition to deal with
> potential other encoding issues.
>
> I'll upload soon (I had some other issues with the packaging which are
> not related).
>
> Kind regards
>
>     Andreas.
>
>
> [1] https://lists.debian.org/debian-python/2018/04/msg00023.html
>
> --
> http://fam-tille.de

[toc] | [prev] | [next] | [standalone]

#10398

FromPiotr Ożarowski <piotr@debian.org>
Date2018-04-07 00:00 +0200
Message-ID<vBHyO-8kN-7@gated-at.bofh.it>
In reply to#10397

[Multipart message — attachments visible in raw view] — view raw

[Peter Cock, 2018-04-06]
> Piotr, I think your fix would be worth including in Biopython.
> 
> Do you want to submit this as a pull request, or give me
> permission to do so by email?

go ahead

> We'd need you to agree to both the Biopython License Agreement
> and the 3-clause BSD licence, and say if you'd like to be
> acknowledged by name or not.

OK (and no need to mention my name)

[toc] | [prev] | [next] | [standalone]

#10417

FromPeter Cock <p.j.a.cock@googlemail.com>
Date2018-04-13 13:20 +0200
Message-ID<vE4Uh-2Wo-1@gated-at.bofh.it>
In reply to#10397
I ended up doing something similar to catch any future
problematic characters:

https://github.com/biopython/biopython/pull/1607

Peter


On Fri, Apr 6, 2018 at 10:36 AM, Peter Cock <p.j.a.cock@googlemail.com> wrote:
> Thanks Andreas,
>
> Piotr, I think your fix would be worth including in Biopython.
>
> Do you want to submit this as a pull request, or give me
> permission to do so by email?
>
> We'd need you to agree to both the Biopython License Agreement
> and the 3-clause BSD licence, and say if you'd like to be
> acknowledged by name or not.
>
> Regards,
>
> Peter
>
> On Fri, Apr 6, 2018 at 8:16 AM, Andreas Tille <tille@debian.org> wrote:
>> Hi Peter,
>>
>> On Thu, Apr 05, 2018 at 04:19:50PM +0100, Peter Cock wrote:
>>>
>>> https://github.com/biopython/biopython/commit/6421d365a511c0de71748737ba6e1cd8518658ac
>>>
>>> Can you pull in that commit as a patch for the Biopython 1.71 package?
>>
>> Thanks.  I'm using this as patch.  I can confirm that this works now
>> (I'm using the other patch suggested here[1] in addition to deal with
>> potential other encoding issues.
>>
>> I'll upload soon (I had some other issues with the packaging which are
>> not related).
>>
>> Kind regards
>>
>>     Andreas.
>>
>>
>> [1] https://lists.debian.org/debian-python/2018/04/msg00023.html
>>
>> --
>> http://fam-tille.de

[toc] | [prev] | [next] | [standalone]

#10386

FromOlivier Sallou <olivier.sallou@irisa.fr>
Date2018-04-05 16:50 +0200
Message-ID<vBen8-5LS-11@gated-at.bofh.it>
In reply to#10383

On 04/05/2018 04:34 PM, Andreas Tille wrote:
> Hi Olivier,
>
> On Thu, Apr 05, 2018 at 04:19:37PM +0200, Olivier Sallou wrote:
>> On 04/05/2018 04:10 PM, Andreas Tille wrote:
>>> (maybe?) setup.py:
>>>
>>>
>>> ...
>>> dh_auto_clean
>>>         pybuild --clean -i python{version} -p 2.7
>>> I: pybuild base:217: python2.7 setup.py clean 
>>> running clean
>>> removing '/home/andreas/debian-maintain/salsa/med-team/build-area/python-biopython-1.71+dfsg/.pybuild/cpython2_2.7/build' (and everything under it)
>>> 'build/bdist.linux-x86_64' does not exist -- can't clean it
>>> 'build/scripts-2.7' does not exist -- can't clean it
>>>         pybuild --clean -i python{version} -p 3.6
>>> I: pybuild base:217: python3.6 setup.py clean 
>>> Traceback (most recent call last):
>>>   File "setup.py", line 435, in <module>
>>>     readme_rst = handle.read()
>>>   File "/usr/lib/python3.6/encodings/ascii.py", line 26, in decode
>>>     return codecs.ascii_decode(input, self.errors)[0]
>>> UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 7116: ordinal not in range(128)
>>> E: pybuild pybuild:336: clean: plugin distutils failed with: exit code=1: python3.6 setup.py clean 
>>> dh_auto_clean: pybuild --clean -i python{version} -p 3.6 returned exit code 13
>> this is usually related to non ascii characters in file (readme.rst?) ,
>> if you find some, you can patch file to remove them.
> Is there any efficient method to seek for this kind of non-ascii
> characters?
I found none  :-(  but looking at file content
>
> Kind regards
>
>       Andreas.
>

-- 
Olivier Sallou
Univ Rennes, Inria, CNRS, IRISA
Irisa, Campus de Beaulieu
F-35042 RENNES - FRANCE
Tel: 02.99.84.71.95

gpg key id: 4096R/326D8438  (keyring.debian.org)
Key fingerprint = 5FB4 6F83 D3B9 5204 6335  D26D 78DC 68DB 326D 8438

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.maint.python

csiph-web