Path: csiph.com!fu-berlin.de!uni-berlin.de!not-for-mail
From: Chris Angelico <rosuav@gmail.com>
Newsgroups: comp.lang.python
Subject: Re: Pyhon 2.x or 3.x, which is faster?
Date: Fri, 11 Mar 2016 07:07:09 +1100
Lines: 39
Message-ID: <mailman.153.1457640433.15725.python-list@python.org>
References: <mailman.238.1457265255.20602.python-list@python.org> <mailman.49.1457383632.10335.python-list@python.org> <nbkvq2$j5s$1@dont-email.me> <mailman.7.1457395587.15725.python-list@python.org> <nbl81v$hii$1@dont-email.me> <mailman.16.1457399576.15725.python-list@python.org> <nblapf$ofi$1@dont-email.me> <mailman.28.1457405184.15725.python-list@python.org> <nbmbnp$hsi$1@dont-email.me> <mailman.45.1457453433.15725.python-list@python.org> <nbn871$8np$1@dont-email.me> <56df6761$0$1588$c3e8da3$5496439d@news.astraweb.com> <mailman.72.1457512871.15725.python-list@python.org> <nbp36e$560$1@dont-email.me> <mailman.96.1457558042.15725.python-list@python.org> <nbqaja$5g5$1@dont-email.me> <mailman.98.1457566570.15725.python-list@python.org> <nbqgmi$mha$1@dont-email.me> <mailman.108.1457595070.15725.python-list@python.org> <nbrms1$26p$1@dont-email.me> <mailman.123.1457612163.15725.python-list@python.org> <nbrq7b$eeu$1@dont-email.me> <mailman.127.1457615320.15725.python-list@python.org> <nbrvpj$5lm$1@dont-email.me>
Mime-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
In-Reply-To: <nbrvpj$5lm$1@dont-email.me>
Precedence: list
Xref: csiph.com comp.lang.python:104549

On Fri, Mar 11, 2016 at 1:22 AM, BartC <bc@freeuk.com> wrote:
>>
>> 1) Unicode support, intrinsic to the language, is crucial, even if
>> BartC refuses to recognize this. Anything released beyond the confines
>> of his personal workspace will need full Unicode support, otherwise it
>> is a problem to the rest of the world, and should be destroyed with
>> fire. Thanks.
>
>
> I don't agree. If I distribute some text in the form of a series of ASCII
> byte values (eg. classic TXT format, with either kind of line separator),
> then that same data can be directly interpreted as UTF-8.

What you call "classic TXT format" is still an encoding, which means
you're acknowledging the difference between characters and bytes -
that's the first step. But you have to be certain that you are
interpreting it as UTF-8, in which case ASCII ceases to be
significant, and what you've done is declare that your file consists
of a stream of UTF-8-encoded Unicode characters, divided into lines
with either U+000D U+000A or just U+000A. That's a nice clear encoding
definition.

And the difference between characters and bytes is only the first step
(albeit the biggest and most important step). You _need_ to make sure
that you're thinking about text as text, and that means being aware of
RTL vs LTR, combining characters, case conversions, collations, etc,
etc, etc, all in terms of Unicode rather than as eight-bit or
seven-bit characters. (For example, a na=C3=AFve MUD client might assume
that one byte is one character is 8 pixels of width. I know this,
because some years ago I wrote one exactly like that (well, the figure
"8" came from measuring the current font, but other than at font
changes, it was fixed). An intelligent Unicode-aware MUD client has to
not only cope with variable width, but also characters that don't have
any width at all, and those that use the same space as their base
character, and those that are placed to the left of the preceding
character.) You can't ignore this, although you might be able to leave
full support for later - but it's a bug until you do.

ChrisA