Re: unicodedata with chr() not the same between python 3.4 and 3.5

Path	csiph.com!fu-berlin.de!uni-berlin.de!not-for-mail
From	Chris Angelico <rosuav@gmail.com>
Newsgroups	comp.lang.python
Subject	Re: unicodedata with chr() not the same between python 3.4 and 3.5
Date	Wed, 23 Dec 2015 02:42:11 +1100
Lines	51
Message-ID	<mailman.61.1450798939.2237.python-list@python.org> (permalink)
References	<CALyJZZXFXcCfQq8CnVEH-OkqvACa6HJMvMnmWbZSrntTMWW7bA@mail.gmail.com>
Mime-Version	1.0
Content-Type	text/plain; charset=UTF-8
X-Trace	news.uni-berlin.de F3OSVtTqDK+fL5JTZuIPIQtnpU2SmVB+r+Wv7eEaZFFg==
Return-Path	<rosuav@gmail.com>
X-Original-To	python-list@python.org
Delivered-To	python-list@mail.python.org
X-Spam-Status	OK 0.019
X-Spam-Evidence	'H': 0.96; 'S': 0.00; 'cc:addr:python-list': 0.09; 'integers': 0.09; 'subject:same': 0.09; 'url:unicode': 0.09; 'python': 0.10; 'subject:not': 0.11; 'subject:python': 0.14; 'wed,': 0.15; '23,': 0.16; 'from:addr:rosuav': 0.16; 'from:name:chris angelico': 0.16; 'received:io': 0.16; 'received:psf.io': 0.16; 'subject:between': 0.16; 'wrote:': 0.16; '>>>': 0.20; '2015': 0.20; 'cc:2**0': 0.20; 'cc:addr:python.org': 0.20; 'am,': 0.23; 'code,': 0.23; 'dec': 0.23; 'import': 0.24; 'header:In-Reply-To:1': 0.24; 'linux': 0.26; 'points': 0.27; 'message-id:@mail.gmail.com': 0.27; "skip:' 10": 0.28; 'skip:u 20': 0.28; 'skip:( 20': 0.28; "i'm": 0.30; 'code': 0.30; 'changed': 0.33; 'source': 0.33; 'received:google.com': 0.35; 'files,': 0.35; 'mapping': 0.35; 'newer': 0.35; 'unicode': 0.35; 'something': 0.35; 'but': 0.36; 'url:org': 0.36; 'received:209.85': 0.36; 'subject:: ': 0.37; 'two': 0.37; 'received:209.85.213': 0.37; 'received:209': 0.38; 'why': 0.39; 'data': 0.39; 'sure': 0.39; 'subject:the': 0.39; 'rather': 0.39; 'subject:with': 0.40; 'some': 0.40; 'your': 0.60; 'more': 0.63; 'url:0': 0.63; 'between': 0.65; 'here': 0.66; '3.4': 0.84; 'category.': 0.84; 'chrisa': 0.84; 'difference.': 0.84; 'to:none': 0.91; '2014,': 0.91
DKIM-Signature	v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:cc :content-type; bh=utNLVQ6LIbjueZtoigM2ZuM6+QDwkyUmQJaZhs0kQFw=; b=rKKIMGr6Brgrx7YZSvx6UPLbPXlRUcJm8OHR8y4ZvOquSmkNdqLwyNUpwHSODdSc3H /ckS/+4fW+hj2raEQo/vEm1UFqHc06dgxuKk1fbidA7jMncR4x1UDkYzfHwUBUAa3rXb 7goiJh68UqV3Q77t3YyfvPbILnBQm9uL1GVCVAmBCviUmGbpMjrawY2SOGxrwZsSFrpr U+7w1RE9JEVLdxU3hhkOlZlORB84nA8od8f+3oHGGVLN4zgrv6ehl+GYDRaTYgEKKyxg S7MMUqoKZBUnd/bGmt5cndBymGTposm0faXjy+6kA1+mOtk5cbb5Nti4T06ULCZl82Hq b0gA==
X-Received	by 10.50.66.179 with SMTP id g19mr7522828igt.94.1450798931063; Tue, 22 Dec 2015 07:42:11 -0800 (PST)
In-Reply-To	<CALyJZZXFXcCfQq8CnVEH-OkqvACa6HJMvMnmWbZSrntTMWW7bA@mail.gmail.com>
X-BeenThere	python-list@python.org
X-Mailman-Version	2.1.20+
Precedence	list
List-Id	General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe	<https://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive	<http://mail.python.org/pipermail/python-list/>
List-Post	<mailto:python-list@python.org>
List-Help	<mailto:python-list-request@python.org?subject=help>
List-Subscribe	<https://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Xref	csiph.com comp.lang.python:100733

Show key headers only | View raw

On Wed, Dec 23, 2015 at 2:27 AM, Vincent Davis <vincent@vincentdavis.net> wrote:
> I was expecting the code below to be the same between python3.4 and 3.5. I
> need a mapping between the integers and unicode that is consistant between
> 3.4 and 3.5
>
>>>>
> import unicodedata
>>>>
> u = ''.join(chr(i) for i in range(65536) if (unicodedata.category(chr(i))
> in ('Lu', 'Ll')))[945:965]

Not sure why you're slicing it like this, but it makes little
difference. The significant thing here is that the newer Pythons are
shipping newer Unicode data files, and some code points have changed
category.

rosuav@sikorsky:~$ python3.4
Python 3.4.2 (default, Oct  8 2014, 10:45:20)
[GCC 4.9.1] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import unicodedata
>>> unicodedata.unidata_version
'6.3.0'
>>>
rosuav@sikorsky:~$ python3.5
Python 3.5.0b1+ (default:7255af1a1c50+, May 26 2015, 00:39:06)
[GCC 4.9.2] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import unicodedata
>>> unicodedata.unidata_version
'7.0.0'
>>>
rosuav@sikorsky:~$ python3.6
Python 3.6.0a0 (default:6e114c4023f5, Dec 20 2015, 19:15:28)
[GCC 4.9.2] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import unicodedata
>>> unicodedata.unidata_version
'8.0.0'
>>>

Have a read here of what changed in those two major versions:

http://unicode.org/versions/Unicode7.0.0/
http://unicode.org/versions/Unicode8.0.0/

I'm not sure what the best way is to create the mapping you want, but
I would advise freezing it to a specific set of codepoints in your
source code, rather than depending on something external.

ChrisA

Back to comp.lang.python | Previous | Next | Find similar | Unroll thread

Thread

Re: unicodedata with chr() not the same between python 3.4 and 3.5 Chris Angelico <rosuav@gmail.com> - 2015-12-23 02:42 +1100

csiph-web