Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > comp.lang.python > #81218
| Path | csiph.com!usenet.pasdenom.info!weretis.net!feeder1.news.weretis.net!feeder.erje.net!eu.feeder.erje.net!newsfeed.xs4all.nl!newsfeed2a.news.xs4all.nl!xs4all!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail |
|---|---|
| Return-Path | <python.list@tim.thechases.com> |
| X-Original-To | python-list@python.org |
| Delivered-To | python-list@mail.python.org |
| X-Spam-Status | OK 0.040 |
| X-Spam-Evidence | '*H*': 0.92; '*S*': 0.00; 'insert': 0.05; 'character,': 0.09; 'subject:2.7': 0.09; "''.join(c": 0.16; '-tkc': 0.16; 'distinct': 0.16; 'from:addr:python.list': 0.16; 'from:addr:tim.thechases.com': 0.16; 'from:name:tim chase': 0.16; 'received:10.216': 0.16; 'subject:unicode': 0.16; 'subject:python': 0.16; 'weird': 0.16; 'wrote:': 0.18; 'example': 0.22; 'this?': 0.23; 'string,': 0.24; 'unicode': 0.24; 'looks': 0.24; 'this:': 0.26; 'header:In-Reply-To:1': 0.27; 'character': 0.29; 'characters': 0.30; 'code': 0.31; "skip:' 10": 0.31; "d'aprano": 0.31; 'steven': 0.31; 'text': 0.33; 'subject:time': 0.33; 'sense': 0.34; 'subject: (': 0.35; 'display': 0.35; 'something': 0.35; 'form.': 0.35; 'test': 0.35; 'but': 0.35; 'combination': 0.36; 'subject:one': 0.36; 'should': 0.36; 'application': 0.37; 'two': 0.37; 'received:10': 0.37; 'handle': 0.38; 'to:addr:python-list': 0.38; 'rather': 0.38; 'to:addr:python.org': 0.39; 'even': 0.60; 'remove': 0.60; 'middle': 0.60; 'happen': 0.63; 'skip:n 10': 0.64; 'subject:more': 0.64; 'received:46': 0.66; 'combining': 0.68; 'as:': 0.81; 'perspective,': 0.84; 'routines': 0.84; 'canonical': 0.91 |
| X-Sender-Id | wwwh|x-authuser|tim@thechases.com |
| X-Sender-Id | wwwh|x-authuser|tim@thechases.com |
| X-MC-Relay | Neutral |
| X-MailChannels-SenderId | wwwh|x-authuser|tim@thechases.com |
| X-MailChannels-Auth-Id | wwwh |
| X-MC-Loop-Signature | 1416585453458:1381203137 |
| X-MC-Ingress-Time | 1416585453458 |
| Date | Fri, 21 Nov 2014 10:00:10 -0600 |
| From | Tim Chase <python.list@tim.thechases.com> |
| To | python-list@python.org |
| Subject | Re: python 2.7 and unicode (one more time) |
| In-Reply-To | <546f58db$0$12990$c3e8da3$5496439d@news.astraweb.com> |
| References | <m4kcu8$ta8$1@ger.gmane.org> <m4kn66$ma3$1@ger.gmane.org> <1416491999.209592.193345213.3A6FF9A4@webmail.messagingengine.com> <m4l20h$toc$1@ger.gmane.org> <CAPTjJmqjUGmTA-0JXrkeKYLeShmT=mOE=c38o8YD7XheCK_5Kw@mail.gmail.com> <m4l53q$ff4$1@ger.gmane.org> <mailman.16029.1416502283.18130.python-list@python.org> <87ppch8zvx.fsf@elektro.pacujo.net> <546e880d$0$12995$c3e8da3$5496439d@news.astraweb.com> <mailman.16072.1416532215.18130.python-list@python.org> <546f58db$0$12990$c3e8da3$5496439d@news.astraweb.com> |
| X-Mailer | Claws Mail 3.8.1 (GTK+ 2.24.10; x86_64-pc-linux-gnu) |
| Mime-Version | 1.0 |
| Content-Type | text/plain; charset=UTF-8 |
| Content-Transfer-Encoding | quoted-printable |
| X-AuthUser | tim@thechases.com |
| X-BeenThere | python-list@python.org |
| X-Mailman-Version | 2.1.15 |
| Precedence | list |
| List-Id | General discussion list for the Python programming language <python-list.python.org> |
| List-Unsubscribe | <https://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe> |
| List-Archive | <http://mail.python.org/pipermail/python-list/> |
| List-Post | <mailto:python-list@python.org> |
| List-Help | <mailto:python-list-request@python.org?subject=help> |
| List-Subscribe | <https://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe> |
| Newsgroups | comp.lang.python |
| Message-ID | <mailman.16107.1416585466.18130.python-list@python.org> (permalink) |
| Lines | 60 |
| NNTP-Posting-Host | 2001:888:2000:d::a6 |
| X-Trace | 1416585466 news.xs4all.nl 2910 [2001:888:2000:d::a6]:55035 |
| X-Complaints-To | abuse@xs4all.nl |
| Xref | csiph.com comp.lang.python:81218 |
Show key headers only | View raw
On 2014-11-22 02:23, Steven D'Aprano wrote:
> LATIN SMALL LETTER E
> COMBINING CIRCUMFLEX ACCENT
>
> then my application should treat that as a single "character" and
> display it as:
>
> LATIN SMALL LETTER E WITH CIRCUMFLEX
>
> which looks like this: ê
>
> rather than two distinct "characters" eˆ
>
> Now, that specific example is a no-brainer, because the Unicode
> normalization routines will handle the conversion. But not every
> combination of accented characters has a canonical combined form.
> What about something like this?
>
> 'w\N{COMBINING CIRCUMFLEX ACCENT}\N{COMBINING OGONEK}\N{COMBINING
> CARON}'
>
> If I insert a character into my string, I want to be able to insert
> before the w or after the caron, but not in the middle of those
> three code points.
Things get even weirder if you have
'\N{LATIN SMALL LETTER E WITH CIRCUMFLEX}\N{COMBINING
OGONEK}\N{COMBINING CARON}'
and when you try to do comparisons like
s1 = '\N{LATIN SMALL LETTER E WITH CIRCUMFLEX}\N{COMBINING OGONEK}'
s2 = 'e\N{COMBINING CIRCUMFLEX ACCENT}\N{COMBINING OGONEK}'
s3 = 'e\N{COMBINING OGONEK}\N{COMBINING CIRCUMFLEX ACCENT}'
print(s1 == s2)
print(s1 == s3)
print(s2 == s3)
Then you also have the case where you want to edit text and the user
wants to remove the COMBINING OGONEK from the character, so you *do*
want to do something akin to
s4 = ''.join(c for c in s3 if c != '\N{COMBINING OGONEK}')
And yet, weird things happen if you try to remove the circumflex:
for test in (s1, s2, s3):
print(test == ''.join(
c for c in test if c != '\N{COMBINING CIRCUMFLEX ACCENT}'
)
They all make sense if you understand what's going on under the hood,
but from a visual/conceptual perspective, something feels amiss.
-tkc
Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread
Re: python 2.7 and unicode (one more time) Michael Torrie <torriem@gmail.com> - 2014-11-20 09:51 -0700
Re: python 2.7 and unicode (one more time) Marko Rauhamaa <marko@pacujo.net> - 2014-11-20 20:56 +0200
Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-21 10:49 +1100
Re: python 2.7 and unicode (one more time) Marko Rauhamaa <marko@pacujo.net> - 2014-11-21 08:36 +0200
Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-21 18:02 +1100
Re: python 2.7 and unicode (one more time) Marko Rauhamaa <marko@pacujo.net> - 2014-11-21 09:14 +0200
Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-21 18:29 +1100
Re: python 2.7 and unicode (one more time) Marko Rauhamaa <marko@pacujo.net> - 2014-11-21 10:16 +0200
Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-21 19:24 +1100
Re: python 2.7 and unicode (one more time) Rustom Mody <rustompmody@gmail.com> - 2014-11-21 08:03 -0800
Re: python 2.7 and unicode (one more time) Marko Rauhamaa <marko@pacujo.net> - 2014-11-21 18:36 +0200
Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-22 03:52 +1100
Re: python 2.7 and unicode (one more time) Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-11-22 15:38 +1100
Re: python 2.7 and unicode (one more time) Marko Rauhamaa <marko@pacujo.net> - 2014-11-22 08:40 +0200
Re: python 2.7 and unicode (one more time) Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-11-23 00:50 +1100
Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-23 01:17 +1100
Re: python 2.7 and unicode (one more time) Marko Rauhamaa <marko@pacujo.net> - 2014-11-22 16:18 +0200
Re: python 2.7 and unicode (one more time) Roy Smith <roy@panix.com> - 2014-11-22 09:44 -0500
Re: python 2.7 and unicode (one more time) Marko Rauhamaa <marko@pacujo.net> - 2014-11-22 16:56 +0200
Re: python 2.7 and unicode (one more time) wxjmfauth@gmail.com - 2014-11-22 07:44 -0800
Re: python 2.7 and unicode (one more time) Rustom Mody <rustompmody@gmail.com> - 2014-11-22 07:57 -0800
Re: python 2.7 and unicode (one more time) Marko Rauhamaa <marko@pacujo.net> - 2014-11-22 19:49 +0200
Re: python 2.7 and unicode (one more time) Mark Lawrence <breamoreboy@yahoo.co.uk> - 2014-11-22 18:17 +0000
Re: python 2.7 and unicode (one more time) wxjmfauth@gmail.com - 2014-11-22 11:32 -0800
Re: python 2.7 and unicode (one more time) wxjmfauth@gmail.com - 2014-11-22 11:17 -0800
Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-23 07:17 +1100
Re: python 2.7 and unicode (one more time) Mark Lawrence <breamoreboy@yahoo.co.uk> - 2014-11-22 22:04 +0000
Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-23 09:31 +1100
Re: python 2.7 and unicode (one more time) Mark Lawrence <breamoreboy@yahoo.co.uk> - 2014-11-22 23:38 +0000
Re: python 2.7 and unicode (one more time) wxjmfauth@gmail.com - 2014-11-23 04:52 -0800
Re: python 2.7 and unicode (one more time) random832@fastmail.us - 2014-11-22 20:52 -0500
Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-23 13:11 +1100
Re: python 2.7 and unicode (one more time) random832@fastmail.us - 2014-11-22 22:10 -0500
Re: python 2.7 and unicode (one more time) Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2014-11-23 11:33 -0500
Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-24 03:39 +1100
Re: python 2.7 and unicode (one more time) random832@fastmail.us - 2014-11-23 13:13 -0500
Re: python 2.7 and unicode (one more time) Dave Angel <d@davea.name> - 2014-11-23 15:31 -0500
Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-24 09:22 +1100
Re: python 2.7 and unicode (one more time) Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2014-11-23 20:24 -0500
Re: python 2.7 and unicode (one more time) random832@fastmail.us - 2014-11-24 00:05 -0500
Re: python 2.7 and unicode (one more time) Gregory Ewing <greg.ewing@canterbury.ac.nz> - 2014-11-24 11:51 +1300
Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-24 09:59 +1100
Re: python 2.7 and unicode (one more time) Marko Rauhamaa <marko@pacujo.net> - 2014-11-24 08:57 +0200
Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-24 18:18 +1100
Re: python 2.7 and unicode (one more time) Marko Rauhamaa <marko@pacujo.net> - 2014-11-24 09:40 +0200
Re: python 2.7 and unicode (one more time) Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-11-25 09:56 +1100
Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-25 10:08 +1100
Re: python 2.7 and unicode (one more time) Marko Rauhamaa <marko@pacujo.net> - 2014-11-25 08:23 +0200
Re: python 2.7 and unicode (one more time) Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-11-25 22:56 +1100
Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-26 00:07 +1100
Re: python 2.7 and unicode (one more time) wxjmfauth@gmail.com - 2014-11-25 06:26 -0800
Re: python 2.7 and unicode (one more time) random832@fastmail.us - 2014-11-22 20:35 -0500
Re: python 2.7 and unicode (one more time) Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-11-23 17:17 +1100
Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-23 17:37 +1100
Re: python 2.7 and unicode (one more time) wxjmfauth@gmail.com - 2014-11-23 07:40 -0800
Re: python 2.7 and unicode (one more time) Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-11-21 11:32 +1100
Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-21 12:10 +1100
Re: python 2.7 and unicode (one more time) Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-11-22 02:23 +1100
Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-22 02:40 +1100
Re: python 2.7 and unicode (one more time) Tim Chase <python.list@tim.thechases.com> - 2014-11-21 10:00 -0600
Re: python 2.7 and unicode (one more time) random832@fastmail.us - 2014-11-20 20:31 -0500
Re: python 2.7 and unicode (one more time) Chris Angelico <rosuav@gmail.com> - 2014-11-21 12:39 +1100
csiph-web