Path: csiph.com!x330-a1.tempe.blueboxinc.net!usenet.pasdenom.info!aioe.org!feeder.news-service.com!tudelft.nl!txtfeed1.tudelft.nl!newsfeed20.multikabel.net!multikabel.net!newsfeed10.multikabel.net!xlned.com!feeder5.xlned.com!newsfeed.xs4all.nl!newsfeed6.news.xs4all.nl!xs4all!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
Return-Path: <ian.g.kelly@gmail.com>
X-Original-To: python-list@python.org
Delivered-To: python-list@mail.python.org
X-Spam-Status: OK 0.005
X-Spam-Evidence: '*H*': 0.99; '*S*': 0.00; 'abstraction': 0.07; 'python': 0.07; 'only,': 0.09; 'streams.': 0.09; 'utf-8': 0.09; 'pm,': 0.11; 'am,': 0.14; 'wrote:': 0.14; 'mode;': 0.16; 'programmer,': 0.16; 'subject:unicode': 0.16; 'bytes': 0.19; '(which': 0.21; 'header:In-Reply-To:1': 0.22; 'right.': 0.22; 'thu,': 0.22; 'byte': 0.25; 'cases': 0.25; 'received:209.85.161.46': 0.26; 'received:mail- fx0-f46.google.com': 0.26; 'message-id:@mail.gmail.com': 0.28; 'external': 0.29; 'received:209.85.161': 0.29; 'string': 0.29; 'messages.': 0.29; 'unicode': 0.29; 'all.': 0.30; 'characters,': 0.31; 'strings,': 0.31; 'strings.': 0.31; 'however,': 0.31; 'to:addr:python-list': 0.32; 'matters': 0.33; 'worry': 0.33; 'uses': 0.34; 'characters': 0.35; 'think': 0.36; 'represent': 0.37; 'received:209.85': 0.37; 'exactly': 0.37; 'sequence': 0.38; 'received:google.com': 0.38; 'no.': 0.38; 'so,': 0.38; 'to:addr:python.org': 0.39; 'received:209': 0.39; 'header:Received:5': 0.40; '2011': 0.62; 'stream': 0.73; 'way)': 0.91
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:in-reply-to:references:from:date :message-id:subject:to:content-type; bh=EupsINSnfTRb1gWMZgU7M9jiIutuob/t6240oiNoBcM=; b=yGOUFdsIKBMOnOazlxNRDR6mWJkx2jxKE9busEB7EeH8Ii6B8fs1vRa4cQOMuZiraU SiCNuMClfc4SRM/sPfECXX/Q9yxfhAXHDSM6oITpK1S2qmXSZi0rLjK7PvYSxkMphLB+ 11yCKwX6/beIhzMxnbjxMG0qduyU9QK9ywq+g=
DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type; b=lGPjZl2SwQt32QLKz8Ke3ObVIOeWnEDa4y+cp5/4+GcsyeZYvu9EkBKvfGwqDvsTPR m3MFuYPyDDEokwEZ0fldJsayf0C25Ds8mYB12jzOr2IPVaX4Pa35HgT3Qsnm7CRZewsI fQU4RKy02j2zELZidwEYsPBdRtNBanjlLSt18=
MIME-Version: 1.0
In-Reply-To: <3ae7c960dc8cf622fcf95aa48ed9df40.squirrel@webmail.lexicon.net>
References: <OkDyp.2983$M61.450@newsfe07.iad> <mailman.1433.1305151801.9059.python-list@python.org> <vpEyp.981$dL5.736@newsfe08.iad> <mailman.1435.1305157329.9059.python-list@python.org> <KDGyp.180$0t1.7@newsfe04.iad> <mailman.1439.1305167541.9059.python-list@python.org> <874o50k1eb.fsf@benfinney.id.au> <U8Lyp.1000$dL5.14@newsfe08.iad> <3ae7c960dc8cf622fcf95aa48ed9df40.squirrel@webmail.lexicon.net>
From: Ian Kelly <ian.g.kelly@gmail.com>
Date: Thu, 12 May 2011 10:17:33 -0600
Subject: Re: unicode by default
To: python-list@python.org
Content-Type: text/plain; charset=ISO-8859-1
X-BeenThere: python-list@python.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe: <http://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive: <http://mail.python.org/pipermail/python-list>
List-Post: <mailto:python-list@python.org>
List-Help: <mailto:python-list-request@python.org?subject=help>
List-Subscribe: <http://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups: comp.lang.python
Message-ID: <mailman.1476.1305217085.9059.python-list@python.org>
Lines: 19
NNTP-Posting-Host: 82.94.164.166
X-Trace: 1305217085 news.xs4all.nl 41113 [::ffff:82.94.164.166]:37428
X-Complaints-To: abuse@xs4all.nl
Xref: x330-a1.tempe.blueboxinc.net comp.lang.python:5244

On Thu, May 12, 2011 at 1:58 AM, John Machin <sjmachin@lexicon.net> wrote:
> On Thu, May 12, 2011 4:31 pm, harrismh777 wrote:
>
>>
>> So, the UTF-16 UTF-32 is INTERNAL only, for Python
>
> NO. See one of my previous messages. UTF-16 and UTF-32, like UTF-8 are
> encodings for the EXTERNAL representation of Unicode characters in byte
> streams.

Right.  *Under the hood* Python uses UCS-2 (which is not exactly the
same thing as UTF-16, by the way) to represent Unicode strings.
However, this is entirely transparent.  To the Python programmer, a
unicode string is just an abstraction of a sequence of code-points.
You don't need to think about UCS-2 at all.  The only times you need
to worry about encodings are when you're encoding unicode characters
to byte strings, or decoding bytes to unicode characters, or opening a
stream in text mode; and in those cases the only encoding that matters
is the external one.