Path: csiph.com!usenet.pasdenom.info!news.albasani.net!newsfeed.freenet.ag!news2.euro.net!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
Date: Wed, 28 Mar 2012 14:20:50 -0500
From: Evan Driscoll <driscoll@cs.wisc.edu>
User-Agent: Mozilla/5.0 (X11; U; Linux i686 (x86_64); en-US; rv:1.9.2.17) Gecko/20110414 Thunderbird/3.1.10
MIME-Version: 1.0
To: Ross Ridge <rridge@csclub.uwaterloo.ca>
Subject: Re: Re: "convert" string to bytes without changing data (encoding)
References: <9tg21lFmo3U1@mid.dfncis.de> <mailman.1065.1332925364.3037.python-list@python.org> <jkvb5a$88d$1@rumours.uwaterloo.ca> <4f73504c$0$29981$c3e8da3$5496439d@news.astraweb.com> <jkvktq$u16$1@rumours.uwaterloo.ca>
In-Reply-To: <jkvktq$u16$1@rumours.uwaterloo.ca>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Cc: python-list@python.org
Precedence: list
Newsgroups: comp.lang.python
Message-ID: <mailman.1092.1332962455.3037.python-list@python.org>
Lines: 40
NNTP-Posting-Host: 2001:888:2000:d::a6
Xref: csiph.com comp.lang.python:22304

On 01/-10/-28163 01:59 PM, Ross Ridge wrote:
> Steven D'Aprano<steve+comp.lang.python@pearwood.info>  wrote:
>> The right way to convert bytes to strings, and vice versa, is via
>> encoding and decoding operations.
>
> If you want to dictate to the original poster the correct way to do
> things then you don't need to do anything more that.  You don't need to
> pretend like Chris Angelico that there's isn't a direct mapping from
> the his Python 3 implementation's internal respresentation of strings
> to bytes in order to label what he's asking for as being "silly".

That mapping may as well be:

   def get_bytes(some_string):
       import random
       length = random.randint(len(some_string), 5*len(some_string))
       bytes = [0] * length
       for i in xrange(length):
           bytes[i] = random.randint(0, 255)
       return bytes

Of course this is hyperbole, but it's essentially about as much 
guarantee as to what the result is.

As many others have said, the encoding isn't defined, and I would guess 
varies between implementations. (E.g. if Jython and IronPython use their 
host platforms' native strings, both have 16-bit chars and thus probably 
use UTF-16 encoding. I am not sure what CPython uses, but I bet it's 
*not* that.)

It's even guaranteed that the byte representation won't change! If 
something is lazily evaluated or you have a COW string or something, the 
bytes backing it will differ.


So yes, you can say that pretending there's not a mapping of strings to 
internal representation is silly, because there is. However, there's 
nothing you can say about that mapping.

Evan