Path: csiph.com!usenet.pasdenom.info!news.etla.org!news.stack.nl!newsfeed.xs4all.nl!newsfeed3.news.xs4all.nl!xs4all!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail Return-Path: X-Original-To: python-list@python.org Delivered-To: python-list@mail.python.org X-Spam-Status: OK 0.002 X-Spam-Evidence: '*H*': 1.00; '*S*': 0.00; 'true,': 0.05; 'subject:Python': 0.06; 'subject: -- ': 0.07; 'string': 0.09; 'methods,': 0.09; 'received:80.91': 0.09; 'received:80.91.229': 0.09; 'received:gmane.org': 0.09; 'received:list': 0.09; 'type,': 0.09; '(now': 0.16; 'article.': 0.16; 'finney': 0.16; 'received:80.91.229.3': 0.16; 'received:plane.gmane.org': 0.16; 'subject:Unicode': 0.16; 'subject:handling': 0.16; 'code.': 0.18; "python's": 0.19; 'header:User-Agent:1': 0.23; 'example.': 0.24; 'laws': 0.24; 'unicode': 0.24; '(or': 0.24; 'least': 0.26; 'gets': 0.27; 'header:X-Complaints-To:1': 0.27; 'point': 0.28; "doesn't": 0.30; 'characters': 0.30; 'needed.': 0.30; 'code': 0.31; 'writes:': 0.31; 'text': 0.33; 'reader': 0.33; 'knowledge': 0.35; "can't": 0.35; 'case,': 0.35; 'but': 0.35; 'two': 0.37; 'being': 0.38; 'ben': 0.38; 'whatever': 0.38; 'to:addr:python-list': 0.38; 'to:addr:python.org': 0.39; 'received:org': 0.40; 'affect': 0.61; 'more': 0.64; 'skip:\xe2 10': 0.65; 'specialized': 0.65; '8bit%:46': 0.78; 'batchelder': 0.84; 'is)': 0.84; 'universe': 0.84; 'visually': 0.84 X-Injected-Via-Gmane: http://gmane.org/ To: python-list@python.org From: Ben Finney Subject: Re: Python Unicode handling wins again -- mostly Date: Tue, 03 Dec 2013 09:56:57 +1100 References: <529934dc$0$29993$c3e8da3$5496439d@news.astraweb.com> <529CEFB1.2030007@stoneleaf.us> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Gmane-NNTP-Posting-Host: rasputin.madmonks.org X-Public-Key-ID: 0xAC128405 X-Public-Key-Fingerprint: 517C F14B B2F3 98B0 CB35 4855 B8B2 4C06 AC12 8405 X-Public-Key-URL: http://www.benfinney.id.au/contact/bfinney-gpg.asc X-Post-From: Ben Finney User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.4 (gnu/linux) Cancel-Lock: sha1:J66uzuOORfB6dy4/aMLrag07ghg= X-BeenThere: python-list@python.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: General discussion list for the Python programming language List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Newsgroups: comp.lang.python Message-ID: Lines: 25 NNTP-Posting-Host: 2001:888:2000:d::a6 X-Trace: 1386025030 news.xs4all.nl 15911 [2001:888:2000:d::a6]:42544 X-Complaints-To: abuse@xs4all.nl Xref: csiph.com comp.lang.python:60898 Ned Batchelder writes: > This is where my knowledge about Unicode gets fuzzy. Isn't it the > case that some grapheme clusters (or whatever the right word is) can't > be normalized down to a single code point? Characters can accept many > accents, for example. That's true, but doesn't affect the point being made: that one can have both “sequence of Unicode code points” in Python's ‘unicode’ (now ‘str’) type, and also deal with “sequence of text the reader will see”. > In that case, you can't always normalize and use the existing string > methods, but would need more specialized code. Specialised code may not be needed. It will at least be true that “any two code-point sequences which normalise to the same value will be visually the same for the reader”, which is an important assertion for addressing the complaints from Mortoray's article. -- \ “Pray, v. To ask that the laws of the universe be annulled in | `\ behalf of a single petitioner confessedly unworthy.” —Ambrose | _o__) Bierce, _The Devil's Dictionary_, 1906 | Ben Finney