Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #91299

Re: Fwd: Lossless bulletproof conversion to unicode (backslashing)

Path csiph.com!usenet.pasdenom.info!news.redatomik.org!newsfeed.xs4all.nl!newsfeed4a.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail
Return-Path <techtonik@gmail.com>
X-Original-To python-list@python.org
Delivered-To python-list@mail.python.org
X-Spam-Status OK 0.000
X-Spam-Evidence '*H*': 1.00; '*S*': 0.00; 'anyway.': 0.04; 'binary': 0.05; 'url:bitbucket': 0.05; '(first': 0.09; 'creighton': 0.09; 'from:addr:techtonik': 0.09; 'from:name:anatoly techtonik': 0.09; 'stdout': 0.09; 'stringio': 0.09; 'unicode,': 0.09; 'cc:addr :python-list': 0.10; 'python': 0.11; 'question.': 0.13; 'wed,': 0.15; 'all?': 0.16; 'anatoly': 0.16; 'appears.': 0.16; 'decode': 0.16; 'something.': 0.16; 'subject:unicode': 0.16; 'wrote:': 0.16; 'laura': 0.18; 'switched': 0.18; 'cc:2**0': 0.21; 'cc:addr:python.org': 0.21; 'explicit': 0.22; 'cc:no real name:2**0': 0.23; '2015': 0.23; 'header:In-Reply-To:1': 0.24; 'least': 0.27; 'message-id:@mail.gmail.com': 0.28; "doesn't": 0.28; 'windows,': 0.29; 'mode': 0.31; 'code': 0.31; 'advertised': 0.33; 'received:google.com': 0.34; 'fail': 0.35; 'unicode': 0.35; 'something': 0.35; 'really': 0.35; 'being': 0.36; 'text': 0.36; 'url:org': 0.36; '(and': 0.36; 'subject:: ': 0.37; 'missing': 0.37; 'stuff': 0.38; 'doing': 0.38; 'pm,': 0.39; 'data': 0.40; 'subject: (': 0.40; 'why': 0.40; 'your': 0.60; 'future': 0.61; 'information': 0.62; 'details': 0.63; 'here': 0.66; 'answer.': 0.72; 'obvious': 0.72
DKIM-Signature v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=CHIbfkuzb6K8sT8MzP8WZ+437p21IMAdbN+NGWWTkK4=; b=DDNMKB1W+yT2XW7nIop4yjUsarD777eGvHwK64GIeqw9gpJPZxHTU7yNOUjjTggB3Z ohTCnhuuqu2IV/81cwVxLQfZr/t5P1vaUJsgNKtt4Yx2cl0sXCeC6dAgUA/oiNfHd1iq TY6xN9zLYi0eTxQ1LqZ/T2okhuCYQjw0QkbPvCXpstw/xW6CiEAq/vHR8wPM501XRRb9 ticjGOjpNeQWhaneO3Oq5hC6pUpcmJAR7rvleZ+VkrJT4oeOQiV8qw+oN18jMwQhXrFN mhNHJwMBUmgZRnQv8RBM2N3sHFtKhKBNebC7HtNm5SQeYJKjq82oOmm6/lg3WK7xnUYG DVqw==
X-Received by 10.140.106.101 with SMTP id d92mr38753841qgf.70.1432727580546; Wed, 27 May 2015 04:53:00 -0700 (PDT)
MIME-Version 1.0
In-Reply-To <CAPkN8xJGnut_V0BmEah1ravN3cXESqx7c6U=mOAS0coD3fcPjA@mail.gmail.com>
References <CAPkN8xKTXJu2nhvocG8KuyO1XkJVfK_WsmY6dM=hWsVyg+BVyA@mail.gmail.com> <techtonik@gmail.com> <CAPkN8xK452YWZvVRipTjq05eKthxEE28CB5W5-MN+6DZnXHOGg@mail.gmail.com> <201505271135.t4RBZY6F013836@fido.openend.se> <CAPkN8xJGnut_V0BmEah1ravN3cXESqx7c6U=mOAS0coD3fcPjA@mail.gmail.com>
From anatoly techtonik <techtonik@gmail.com>
Date Wed, 27 May 2015 14:52:40 +0300
Subject Re: Fwd: Lossless bulletproof conversion to unicode (backslashing)
To Laura Creighton <lac@openend.se>
Cc python-list@python.org
Content-Type text/plain; charset=UTF-8
X-BeenThere python-list@python.org
X-Mailman-Version 2.1.20+
Precedence list
List-Id General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe <https://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive <http://mail.python.org/pipermail/python-list/>
List-Post <mailto:python-list@python.org>
List-Help <mailto:python-list-request@python.org?subject=help>
List-Subscribe <https://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups comp.lang.python
Message-ID <mailman.83.1432727585.5151.python-list@python.org> (permalink)
Lines 27
NNTP-Posting-Host 2001:888:2000:d::a6
X-Trace 1432727585 news.xs4all.nl 2914 [2001:888:2000:d::a6]:37935
X-Complaints-To abuse@xs4all.nl
Xref csiph.com comp.lang.python:91299

Show key headers only | View raw

On Wed, May 27, 2015 at 2:47 PM, anatoly techtonik <techtonik@gmail.com> wrote:
> On Wed, May 27, 2015 at 2:35 PM, Laura Creighton <lac@openend.se> wrote:
>> I am missing something.  Why do you need unicode at all?  Why can you
>> not just keep your binary data as binary data?
>
> Good question. From the SCons code I see that we need unicode, because
> we switched to io.StringIO which is advertised as the future (and Python 3
> way of doing things, because Python 3 doesn't have non-unicode StringIO).
>
> A really deep and exhaustive answer.
> advertisement (first link on StringIO vs io.StringIO):
> http://stackoverflow.com/questions/3410309/what-is-the-difference-between-stringio-and-io-stringio-in-python2-7
> peaceful details
> https://bitbucket.org/scons/scons/commits/05d5af305a5d
> gory consequences
> https://bitbucket.org/scons/scons/pull-request/235/fix-tree-all-print-when-build-tree
>
>> I feel like I must be missing something obvious here ...
>
> Not that obvious as it appears.

And the short answer is that we need unicode because we are printing this
information to the stdout, and stdout is opened in text mode at least on
Windows, and without explicit conversion, Python will try to decode stuff
as being `ascii` and fail anyway.
-- 
anatoly t.

Back to comp.lang.python | Previous | Next | Find similar | Unroll thread

Thread

Re: Fwd: Lossless bulletproof conversion to unicode (backslashing) anatoly techtonik <techtonik@gmail.com> - 2015-05-27 14:52 +0300

csiph-web