Path: csiph.com!news.mixmin.net!news.unit0.net!fu-berlin.de!uni-berlin.de!not-for-mail
From: Random832 <random832@fastmail.com>
Newsgroups: comp.lang.python
Subject: Re: Coding systems are political (was Exended ASCII and code pages)
Date: Sat, 28 May 2016 14:05:40 -0400
Lines: 22
Message-ID: <mailman.12.1464458743.1839.python-list@python.org>
References: <mailman.38.1464046609.20402.python-list@python.org> <5743a8d2$0$1587$c3e8da3$5496439d@news.astraweb.com> <ni1o5c$3f6$1@ger.gmane.org> <7D6476C4-9969-4648-AB43-4CE0C7115B9C@icloud.com> <mailman.64.1464115222.20402.python-list@python.org> <dqkviiFr38vU2@mid.individual.net> <C42AA409-D8DF-445A-AB12-A7511C714643@icloud.com> <mailman.80.1464167464.20402.python-list@python.org> <57457c4f$0$11122$c3e8da3@news.astraweb.com> <57461326.5040606@lucidity.plus.com> <nejckblc1vcoarq0aefseqntvgd8f1k8jh@4ax.com> <5746AFA3.9020303@lucidity.plus.com> <mailman.1.1464250289.2277.python-list@python.org> <8737p589bo.fsf@elektro.pacujo.net> <57475CB5.7010102@lucidity.plus.com> <mailman.12.1464294584.2277.python-list@python.org> <87pos8y14e.fsf@elektro.pacujo.net> <5747c071$0$1593$c3e8da3$5496439d@news.astraweb.com> <87fut4xc8w.fsf@elektro.pacujo.net> <5747ed83$0$1620$c3e8da3$5496439d@news.astraweb.com> <8760u0x9qu.fsf@elektro.pacujo.net> <574819d8$0$1598$c3e8da3$5496439d@news.astraweb.com> <1464357085.645851.620574537.560EAC9E@webmail.messagingengine.com> <mailman.22.1464357086.2277.python-list@python.org> <cba2a15d-7f4e-4eb8-9ffa-1605c608bb5e@googlegroups.com> <1464365340.674161.620708793.4CE81D69@webmail.messagingengine.com> <mailman.25.1464365343.2277.python-list@python.org> <37c8a09e-f18b-44ab-8650-0146924bbb9e@googlegroups.com> <1464458740.544484.621455137.2D6EA255@webmail.messagingengine.com>
Mime-Version: 1.0
Content-Type: text/plain
Content-Transfer-Encoding: 7bit
X-Trace: news.uni-berlin.de ZZRO4NJeF6nZlYf/GsXJ4Ah8wkUxhMeSKbZh3y/mvXYA==
Return-Path: <random832@fastmail.com>
X-Original-To: python-list@python.org
Delivered-To: python-list@mail.python.org
X-Spam-Status: OK 0.003
X-Spam-Evidence: '*H*': 0.99; '*S*': 0.00; 'badly': 0.07; 'subject:code': 0.07; 'utf-8': 0.07; '128': 0.09; 'additionally': 0.09; 'encode': 0.09; 'received:internal': 0.09; '*any*': 0.16; 'encodings': 0.16; 'message-id:@webmail.messagingengine.com': 0.16; 'received:10.202': 0.16; 'received:10.202.2': 0.16; 'received:10.202.2.212': 0.16; 'received:66.111': 0.16; 'received:66.111.4': 0.16; 'received:66.111.4.27': 0.16; 'received:io': 0.16; 'received:messagingengine.com': 0.16; 'received:out3-smtp.messagingengine.com': 0.16; 'received:psf.io': 0.16; 'wrote:': 0.16; 'bytes': 0.18; 'otherwise,': 0.20; 'ascii': 0.22; 'sat,': 0.23; 'header:In-Reply-To:1': 0.24; "doesn't": 0.26; 'points': 0.27; 'coding': 0.27; 'least': 0.27; 'opposed': 0.27; 'for,': 0.29; 'character': 0.29; '(including': 0.30; 'code': 0.30; 'compared': 0.30; 'waste': 0.30; "i'll": 0.33; 'unicode': 0.35; 'but': 0.36; 'there': 0.36; '(and': 0.36; 'to:addr:python-list': 0.36; 'subject:: ': 0.37; 'received:10': 0.37; 'two': 0.37; 'being': 0.37; 'say': 0.37; 'received:66': 0.38; 'means': 0.39; "didn't": 0.39; 'to:addr:python.org': 0.40; 'header:Message-Id:1': 0.61; 'more': 0.63; 'information': 0.63; 'therefore': 0.67; 'prime': 0.72; 'chinese': 0.79; 'ucs-2': 0.84; 'density': 0.91; 'asian': 0.93; 'imagine': 0.96
DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=fastmail.com; h= content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-sasl-enc :x-sasl-enc; s=mesmtp; bh=FEorKAlPodzLMbPJ0rQpa6WPlF8=; b=aX3Vjo qGS4W/Rmvq/wOD3sCLjBz9ai/zmOM3YTNiTK8+1MgRY5oLT3Zhw1JPX7fb5KMQdN tJrEuiHdCqJDixrVtIAcvvO91+noacr9KCnilLqq91vU4B/DbnNfYecrWvlW/MRT dr5bH50fsEUTHln8ti0uWeW5Yc5PWsT0jrsUE=
DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d= messagingengine.com; h=content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-sasl-enc:x-sasl-enc; s=smtpout; bh=FEorKAlPodzLMbP J0rQpa6WPlF8=; b=cQ+YVIBFfyt4LIs2PcDav7PmlpZ0j9rakyVfY5+jKoHZfNZ 643wX670dmR8t9z31J+Ju8huUhFr6QugqPBsB2cRgM9VWRhujSOrqVwun766T37j iE00CANiCRm6Xb+qiQKz3uYJHaER9/FeLmsRR5wcTSIMR4b9WI3sjROBhfEY=
X-Sasl-Enc: 84zhnxm3dDsfMfRn4RRih06VhQ8Lvt/4DDiaeTYwjt4/ 1464458740
X-Mailer: MessagingEngine.com Webmail Interface - ajax-b882aaad
In-Reply-To: <37c8a09e-f18b-44ab-8650-0146924bbb9e@googlegroups.com>
X-BeenThere: python-list@python.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe: <https://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive: <http://mail.python.org/pipermail/python-list/>
List-Post: <mailto:python-list@python.org>
List-Help: <mailto:python-list-request@python.org?subject=help>
List-Subscribe: <https://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
X-Mailman-Original-Message-ID: <1464458740.544484.621455137.2D6EA255@webmail.messagingengine.com>
X-Mailman-Original-References: <mailman.38.1464046609.20402.python-list@python.org> <5743a8d2$0$1587$c3e8da3$5496439d@news.astraweb.com> <ni1o5c$3f6$1@ger.gmane.org> <7D6476C4-9969-4648-AB43-4CE0C7115B9C@icloud.com> <mailman.64.1464115222.20402.python-list@python.org> <dqkviiFr38vU2@mid.individual.net> <C42AA409-D8DF-445A-AB12-A7511C714643@icloud.com> <mailman.80.1464167464.20402.python-list@python.org> <57457c4f$0$11122$c3e8da3@news.astraweb.com> <57461326.5040606@lucidity.plus.com> <nejckblc1vcoarq0aefseqntvgd8f1k8jh@4ax.com> <5746AFA3.9020303@lucidity.plus.com> <mailman.1.1464250289.2277.python-list@python.org> <8737p589bo.fsf@elektro.pacujo.net> <57475CB5.7010102@lucidity.plus.com> <mailman.12.1464294584.2277.python-list@python.org> <87pos8y14e.fsf@elektro.pacujo.net> <5747c071$0$1593$c3e8da3$5496439d@news.astraweb.com> <87fut4xc8w.fsf@elektro.pacujo.net> <5747ed83$0$1620$c3e8da3$5496439d@news.astraweb.com> <8760u0x9qu.fsf@elektro.pacujo.net> <574819d8$0$1598$c3e8da3$5496439d@news.astraweb.com> <1464357085.645851.620574537.560EAC9E@webmail.messagingengine.com> <mailman.22.1464357086.2277.python-list@python.org> <cba2a15d-7f4e-4eb8-9ffa-1605c608bb5e@googlegroups.com> <1464365340.674161.620708793.4CE81D69@webmail.messagingengine.com> <mailman.25.1464365343.2277.python-list@python.org> <37c8a09e-f18b-44ab-8650-0146924bbb9e@googlegroups.com>
Xref: csiph.com comp.lang.python:109218

On Sat, May 28, 2016, at 00:46, Rustom Mody wrote:
> Which also means that if the Chinese were to have more say in the
> design of Unicode/ UTF-8 they would likely not waste swathes of prime
> real-estate for almost never used control characters just in the name
> of ASCII compliance

There are only 128 code points in the single-byte range of UTF-8. Only
32 of which are used for, almost-never-used or otherwise, control
characters. What do you imagine they would have put there instead?

At least Unicode doesn't do as badly as the first-draft ISO-UCS, which
didn't allow a C0/C1 control value in *any* position in UCS-2 or UCS-4,
therefore UCS-2 would encode only 192*192=36,864 codepoints as two bytes
(and 64 control characters as one byte), as opposed to UTF-16's 63,488
(including all control characters) two-byte characters.

For completeness, I'll note that conventional East Asian character
coding systems do have a higher information density compared to UTF-8,
but at a cost of not being self-synchronizing. And their single-byte
characters are in fact ASCII and C0/C1 controls, with only Japanese
Shift-
JIS encodings additionally having Katakana as single-byte characters.