Re: Chardet, file, ... and the Flexible String Representation

Path	csiph.com!v102.xanadu-bbs.net!xanadu-bbs.net!feeder.erje.net!eu.feeder.erje.net!news2.arglkargh.de!news.mixmin.net!rt.uk.eu.org!newsfeed.xs4all.nl!newsfeed4.news.xs4all.nl!xs4all!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
Return-Path	<rosuav@gmail.com>
X-Original-To	python-list@python.org
Delivered-To	python-list@mail.python.org
X-Spam-Status	OK 0.004
X-Spam-Evidence	'H': 0.99; 'S': 0.00; 'string.': 0.05; 'subject:file': 0.07; 'suddenly': 0.07; 'string': 0.09; 'integers': 0.09; 'python': 0.11; 'blocks': 0.16; 'does,': 0.16; 'from:addr:rosuav': 0.16; 'from:name:chris angelico': 0.16; 'subject:String': 0.16; 'sat,': 0.16; 'wrote:': 0.18; '(the': 0.22; '>>>': 0.22; 'memory': 0.22; 'unicode': 0.24; 'van': 0.27; 'header:In-Reply-To:1': 0.27; 'am,': 0.29; 'array': 0.29; 'character': 0.29; 'points': 0.29; 'characters': 0.30; 'message- id:@mail.gmail.com': 0.30; 'code': 0.31; 'sep': 0.31; 'sets.': 0.31; 'subject:the': 0.34; 'could': 0.34; 'but': 0.35; 'received:google.com': 0.35; 'to:addr:python-list': 0.38; 'anything': 0.39; 'bad': 0.39; 'to:addr:python.org': 0.39; 'according': 0.40; 'simply': 0.61; 'fact,': 0.69; 'mystery': 0.93; '2013': 0.98
DKIM-Signature	v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type:content-transfer-encoding; bh=R8Rte6biKMnY1rYq6gZ3oIm+7GULJfu6c2O2M43OJtM=; b=eTyHF9Dk2/h+t3LJUb2292Vza2ZdJl523rZfDUAo6d7tnB4vAkrMkqSSC86Dwaczub YGqkAVYwTF9NwhtpGfMpkpH0owQRzUZL2zs6Pa0dUxvXwQuD+7mTk3qasdNKAbLlqebr o3zPWRzgPRDd6t9pGrkuuUrjisRQQfxdiVXtwAsA5/DoN9syaYrXP3Wx+C/sL9wxe9XR 6tdQaabqdaws4iUbDwACAinE+aXKSMUPmhUT350zkNkSIOrEYUnnnS4pfxZc6Fe7xdU2 hvksRrqGuoSqr5UITdz6/hfHku3htmtRZ2fDX8sPdPZ/K9ygTP40ajK1bDT4wKFUACe2 rbgA==
MIME-Version	1.0
X-Received	by 10.52.249.102 with SMTP id yt6mr2525740vdc.21.1378483492306; Fri, 06 Sep 2013 09:04:52 -0700 (PDT)
In-Reply-To	<m2a9jqq7g9.fsf@cochabamba.vanoostrum.org>
References	<4ce85ea8-4a4c-46cf-a546-ad999576a5f7@googlegroups.com> <m2a9jqq7g9.fsf@cochabamba.vanoostrum.org>
Date	Sat, 7 Sep 2013 02:04:52 +1000
Subject	Re: Chardet, file, ... and the Flexible String Representation
From	Chris Angelico <rosuav@gmail.com>
To	python-list@python.org
Content-Type	text/plain; charset=ISO-8859-1
Content-Transfer-Encoding	quoted-printable
X-BeenThere	python-list@python.org
X-Mailman-Version	2.1.15
Precedence	list
List-Id	General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe	<https://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive	<http://mail.python.org/pipermail/python-list/>
List-Post	<mailto:python-list@python.org>
List-Help	<mailto:python-list-request@python.org?subject=help>
List-Subscribe	<https://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups	comp.lang.python
Message-ID	<mailman.127.1378483495.5461.python-list@python.org> (permalink)
Lines	21
NNTP-Posting-Host	2001:888:2000:d::a6
X-Trace	1378483495 news.xs4all.nl 16011 [2001:888:2000:d::a6]:49470
X-Complaints-To	abuse@xs4all.nl
Xref	csiph.com comp.lang.python:53792

Show key headers only | View raw

On Sat, Sep 7, 2013 at 1:46 AM, Piet van Oostrum <piet@vanoostrum.org> wrote:
> The FSR simply stores a Unicode string as an array[*] of ints (the Unicode code points of the characters of the string. That's it. Then it uses a memory-efficient way to store this array of ints. But that has nothing to do with character sets. The same principle could be used for any array of ints.

Python does, in fact, store integers in different-sized blocks of
memory according to size - though not for anything smaller than
32-bit.

>>> sys.getsizeof(100)
14
>>> sys.getsizeof(1000000000000000000000000000000000)
28

So why this is suddenly a bad thing for characters is a mystery none
but he can comprehend.

ChrisA

Thread

Chardet, file, ... and the Flexible String Representation wxjmfauth@gmail.com - 2013-09-06 02:11 -0700
  Re: Chardet, file, ... and the Flexible String Representation Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2013-09-06 10:57 +0000
  Re: Chardet, file, ... and the Flexible String Representation Antoon Pardon <antoon.pardon@rece.vub.ac.be> - 2013-09-06 13:10 +0200
  Re: Chardet, file, ... and the Flexible String Representation Ned Batchelder <ned@nedbatchelder.com> - 2013-09-06 07:02 -0400
  Re: Chardet, file, ... and the Flexible String Representation Piet van Oostrum <piet@vanoostrum.org> - 2013-09-06 11:46 -0400
    Re: Chardet, file, ... and the Flexible String Representation Chris Angelico <rosuav@gmail.com> - 2013-09-07 02:04 +1000
    Re: Chardet, file, ... and the Flexible String Representation random832@fastmail.us - 2013-09-06 12:59 -0400
    Re: Chardet, file, ... and the Flexible String Representation Chris Angelico <rosuav@gmail.com> - 2013-09-07 03:04 +1000
    Re: Chardet, file, ... and the Flexible String Representation wxjmfauth@gmail.com - 2013-09-09 07:28 -0700
      Re: Chardet, file, ... and the Flexible String Representation Ned Batchelder <ned@nedbatchelder.com> - 2013-09-09 12:38 -0400
      Re: Chardet, file, ... and the Flexible String Representation Michael Torrie <torriem@gmail.com> - 2013-09-09 11:05 -0600
        Re: Chardet, file, ... and the Flexible String Representation Steven D'Aprano <steve@pearwood.info> - 2013-09-10 04:58 +0000
      Re: Chardet, file, ... and the Flexible String Representation Terry Reedy <tjreedy@udel.edu> - 2013-09-09 16:47 -0400
      Re: Chardet, file, ... and the Flexible String Representation random832@fastmail.us - 2013-09-10 11:36 -0400
    Re: Chardet, file, ... and the Flexible String Representation random832@fastmail.us - 2013-09-09 14:34 -0400
    Re: Chardet, file, ... and the Flexible String Representation Ian Kelly <ian.g.kelly@gmail.com> - 2013-09-09 13:03 -0600
    Re: Chardet, file, ... and the Flexible String Representation random832@fastmail.us - 2013-09-09 15:27 -0400
    Re: Chardet, file, ... and the Flexible String Representation Serhiy Storchaka <storchaka@gmail.com> - 2013-09-12 00:11 +0300

csiph-web