Path: csiph.com!x330-a1.tempe.blueboxinc.net!newsfeed.hal-mli.net!feeder3.hal-mli.net!feeder.news-service.com!feeder.news-service.com!newsfeed.xs4all.nl!newsfeed5.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail
Return-Path: <rosuav@gmail.com>
X-Original-To: python-list@python.org
Delivered-To: python-list@mail.python.org
X-Spam-Status: OK 0.011
X-Spam-Evidence: '*H*': 0.98; '*S*': 0.00; 'encoded': 0.05; 'abstraction': 0.07; 'bytes.': 0.07; 'python': 0.08; 'bytes,': 0.09; 'subject:parsing': 0.09; 'received:209.85.214.174': 0.13; 'received:mail-iw0-f174.google.com': 0.13; 'stored': 0.13; 'wrote:': 0.15; '127': 0.16; "2's": 0.16; 'abstraction,': 0.16; 'from:addr:rosuav': 0.16; 'from:name:chris angelico': 0.16; 'somehow,': 0.16; 'transmit': 0.16; 'pm,': 0.16; 'bytes': 0.19; 'string,': 0.19; 'otherwise,': 0.19; 'header:In-Reply-To:1': 0.22; 'tue,': 0.23; 'code': 0.24; 'byte': 0.25; 'received:209.85.214': 0.28; 'character': 0.28; 'network,': 0.28; 'message- id:@mail.gmail.com': 0.28; 'unicode': 0.29; 'characters,': 0.30; 'chris': 0.32; 'to:addr:python-list': 0.34; 'there': 0.34; 'characters': 0.34; 'requests': 0.35; 'sequence': 0.37; 'some': 0.37; 'but': 0.37; 'could': 0.37; 'received:google.com': 0.38; 'received:209.85': 0.38; 'subject:: ': 0.38; 'problem.': 0.38; 'to:addr:python.org': 0.39; 'received:209': 0.40; 'called': 0.40; 'did': 0.40; 'where': 0.40; 'music': 0.62; 'piece': 0.62; 'our': 0.63; 'computers': 0.65; 'world': 0.65; 'memory,': 0.67; 'delivered': 0.69; 'subjectcharset:utf-8': 0.72; 'care': 0.73; 'midi': 0.84; 'rusi': 0.84; 'ascii.': 0.91; 'subject:little': 0.91
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=NzyYBWR24I4dom3MFTQrn0o403oUoodvOL3BAb+fkOY=; b=Fz2uUZG1sYO6n5hJojoyRrGPB+YTRLmFEZbRCHzxqPene/7Y/8phmTN6HU6z0tHlT5 wty0kTJWVc0faPc0Sujkcf74gxZQ7isbIAm0PKVJshi2njVqHCHy3doUMgPY6fj4Aw3N 5kwVHcAHPYSwKWfN5NIm2Yz3VvZl8TJbSzSJw=
MIME-Version: 1.0
In-Reply-To: <b8a46135-3d6f-4608-93cc-8278ad040483@u6g2000prc.googlegroups.com>
References: <36037253-086b-4467-a1db-9492d3772e78@r5g2000prf.googlegroups.com> <j01ph6$knt$1@speranza.aioe.org> <4e24c823$0$29981$c3e8da3$5496439d@news.astraweb.com> <j02oth$f4c$1@speranza.aioe.org> <feedc526-0c7b-4296-a011-1943950e6b61@j9g2000prj.googlegroups.com> <4e24f5c7$0$29981$c3e8da3$5496439d@news.astraweb.com> <b8a46135-3d6f-4608-93cc-8278ad040483@u6g2000prc.googlegroups.com>
Date: Tue, 19 Jul 2011 15:36:33 +1000
Subject: =?UTF-8?Q?Re=3A_a_little_parsing_challenge_=E2=98=BA?=
From: Chris Angelico <rosuav@gmail.com>
To: python-list@python.org
Content-Type: text/plain; charset=ISO-8859-1
X-BeenThere: python-list@python.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe: <http://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive: <http://mail.python.org/pipermail/python-list>
List-Post: <mailto:python-list@python.org>
List-Help: <mailto:python-list-request@python.org?subject=help>
List-Subscribe: <http://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups: comp.lang.python
Message-ID: <mailman.1238.1311053796.1164.python-list@python.org>
Lines: 28
NNTP-Posting-Host: 2001:888:2000:d::a6
X-Trace: 1311053796 news.xs4all.nl 23873 [2001:888:2000:d::a6]:34540
X-Complaints-To: abuse@xs4all.nl
Xref: x330-a1.tempe.blueboxinc.net comp.lang.python:9854

On Tue, Jul 19, 2011 at 2:59 PM, rusi <rustompmody@gmail.com> wrote:
> Some evidences of leakiness:
> code point vs character vs byte
> encoding and decoding
> UTF-x and UCS-y
>
> Very important and necessary distinctions? Maybe... But I did not need
> them when my world was built of the 127 bricks of ASCII.

Codepoint vs byte is NOT an abstraction. Unicode consists of
characters, where each character is represented by a number called its
codepoint. Since computers work with bytes, we need a way of encoding
those characters into bytes. It's no different from encoding a piece
of music in bytes, and having it come out as 0x90 0x64 0x40. Are those
bytes an abstraction of the note? No. They're an encoding of a MIDI
message that requests that the note be struck. The note itself is an
abstraction, if you like; but the bytes to create that note could be
delivered in a variety of other ways.

A Python Unicode string, whether it's Python 2's 'unicode' or Python
3's 'str', is a sequence of characters. Since those characters are
stored in memory, they must be encoded somehow, but that's not our
problem. We need only care about encoding when we save those
characters to disk, transmit them across the network, or in some other
way need to store them as bytes. Otherwise, there is no abstraction,
and no leak.

Chris Angelico