Path: csiph.com!usenet.pasdenom.info!aioe.org!news.stack.nl!newsfeed.xs4all.nl!newsfeed2a.news.xs4all.nl!xs4all!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail Return-Path: X-Original-To: python-list@python.org Delivered-To: python-list@mail.python.org X-Spam-Status: OK 0.004 X-Spam-Evidence: '*H*': 0.99; '*S*': 0.00; 'encoding': 0.05; '(of': 0.07; 'incompatible': 0.07; 'lesser': 0.07; 'ascii': 0.09; 'bits': 0.09; 'encode': 0.09; 'handful': 0.09; 'received:internal': 0.09; 'subject:2.7': 0.09; 'used.': 0.09; '8-bit': 0.16; 'alphabet': 0.16; 'encodings': 0.16; 'encodings,': 0.16; 'letters.': 0.16; 'message-id:@webmail.messagingengine.com': 0.16; 'received:10.202': 0.16; 'received:10.202.2': 0.16; 'received:66.111': 0.16; 'received:66.111.4': 0.16; 'received:messagingengine.com': 0.16; 'sheer': 0.16; 'subject:unicode': 0.16; 'sure.': 0.16; 'unicode,': 0.16; 'subject:python': 0.16; 'language': 0.16; 'wrote:': 0.18; 'basically': 0.19; 'thu,': 0.19; 'unicode': 0.24; 'possibly': 0.26; 'supported': 0.26; 'header:In-Reply-To:1': 0.27; 'chris': 0.29; 'characters': 0.30; "i'm": 0.30; 'languages': 0.32; 'subject:time': 0.33; 'could': 0.34; 'received:66': 0.35; 'subject: (': 0.35; 'but': 0.35; 'there': 0.35; 'subject:one': 0.36; 'possible': 0.36; 'half': 0.37; 'two': 0.37; 'received:10': 0.37; 'easily': 0.37; 'nov': 0.38; 'to:addr:python-list': 0.38; 'to:addr:python.org': 0.39; 'skip:u 10': 0.60; 'letters': 0.60; 'most': 0.60; 'from:no real name:2**0': 0.61; 'header:Message- Id:1': 0.63; 'high': 0.63; 'skip:n 10': 0.64; 'subject:more': 0.64; 'different': 0.65; 'due': 0.66; '20,': 0.68; 'combining': 0.68; 'eight': 0.74; '2014,': 0.84; 'bmp,': 0.84; 'packing': 0.84; 'russian,': 0.84; 'whereas': 0.91; 'numerous': 0.93; 'remember,': 0.93 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=fastmail.us; h= message-id:x-sasl-enc:from:to:mime-version :content-transfer-encoding:content-type:subject:date:in-reply-to :references; s=mesmtp; bh=+mG+dL5/TIgoK+O6ASPULGZ6qok=; b=or01An PTMamWegYoN5pj1KfBwogS9DUDsTmKIOSuj/Sre1fvwhWDx2VrgRQqm5HIp4Jvvs zssJzuoouwaArwYpMLdK1yI/alaRskn0xpL1In8Vs8Nu+8yK9OIVWiB647h3SEbS GxLO6C537F62vfiKEHdjvsM7gAmvvvBp7ouJM= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d= messagingengine.com; h=message-id:x-sasl-enc:from:to :mime-version:content-transfer-encoding:content-type:subject :date:in-reply-to:references; s=smtpout; bh=+mG+dL5/TIgoK+O6ASPU LGZ6qok=; b=k1m0hfsdqJBcv77uLPvC70zSv97u9bssmePqwzBuzjKne3nX9EJz lLbwX7AVPMZM363Qe8thPPvUMgB59IXxZE+3EiQxfcC70U0Q/wwXji2tHIAv96L/ IY1XnI3OvbcjLlttZnHRFAgTbWAGQk6h8UMhtmB49qOepf50ntQmTJc= X-Sasl-Enc: NYKgb/R/Mux5Q79nN7+pDgA4MRv0DzA0JZlqDaIes8nm 1416533490 From: random832@fastmail.us To: python-list@python.org MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain X-Mailer: MessagingEngine.com Webmail Interface - ajax-a419c8a0 Subject: Re: python 2.7 and unicode (one more time) Date: Thu, 20 Nov 2014 20:31:30 -0500 In-Reply-To: References: <1416491999.209592.193345213.3A6FF9A4@webmail.messagingengine.com> <87ppch8zvx.fsf@elektro.pacujo.net> <546e880d$0$12995$c3e8da3$5496439d@news.astraweb.com> X-BeenThere: python-list@python.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: General discussion list for the Python programming language List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Newsgroups: comp.lang.python Message-ID: Lines: 23 NNTP-Posting-Host: 2001:888:2000:d::a6 X-Trace: 1416533493 news.xs4all.nl 2966 [2001:888:2000:d::a6]:59530 X-Complaints-To: abuse@xs4all.nl Xref: csiph.com comp.lang.python:81157 On Thu, Nov 20, 2014, at 20:10, Chris Angelico wrote: > 2) Languages which use a different alphabet (eg Cyrillic - Russian, > Bulgarian). You could possibly cram them into an eight-bit encoding > without tipping ASCII out, but I'm not sure. In Unicode, these > languages are all easily supported by the BMP, as they don't use a > huge number of characters each. There are numerous eight-bit encodings that support latin and one other alphabet. Remember, ASCII is a seven-bit encoding, and an eight-bit encoding is basically two seven-bit encodings. The most difficult (of those still possible at all) language to encode in eight bits is actually Vietnamese, which uses the Latin alphabet, due to the sheer number of accented letters used. Windows' encoding of it (along with some other lesser used encodings, all for Vietnamese) is the only 8-bit encoding to use combining accents, in a way unfortunately incompatible with unicode normalization if naively translated, whereas VISCII sacrifices a handful of C0 control characters in addition to fully packing the high half with letters. -- Random832