Re: Unicode failure

Path	csiph.com!fu-berlin.de!uni-berlin.de!not-for-mail
From	Oscar Benjamin <oscar.j.benjamin@gmail.com>
Newsgroups	comp.lang.python
Subject	Re: Unicode failure
Date	Sat, 5 Dec 2015 10:46:20 +0000
Lines	41
Message-ID	<mailman.222.1449312390.14615.python-list@python.org> (permalink)
References	<20151204130738.76313c43@imp> <n3t7jo$ae3$1@ger.gmane.org> <20151205010827.2a82c6bc@imp>
Mime-Version	1.0
Content-Type	text/plain; charset=UTF-8
Content-Transfer-Encoding	quoted-printable
X-Trace	news.uni-berlin.de SxMe5j+Ny55KY9Jpb3D9jwZtCM8zsfV8cm72Ho/Gr7QQ==
Return-Path	<oscar.j.benjamin@gmail.com>
X-Original-To	python-list@python.org
Delivered-To	python-list@mail.python.org
X-Spam-Status	OK 0.000
X-Spam-Evidence	'H': 1.00; 'S': 0.00; '--': 0.07; '-0500': 0.07; 'bytes.': 0.07; 'postgresql': 0.07; 'skip:/ 10': 0.07; 'utf-8': 0.07; 'cc:addr:python-list': 0.09; 'assumed': 0.09; 'coding:': 0.09; 'derived': 0.09; 'encode': 0.09; 'stdout': 0.09; 'str,': 0.09; 'python': 0.10; 'encoding': 0.15; 'cc:name:python list': 0.16; 'display,': 0.16; 'email addr:udel.edu>': 0.16; 'email name:<tjreedy': 0.16; 'fail,': 0.16; 'netbsd': 0.16; 'received:io': 0.16; 'received:psf.io': 0.16; 'redundant,': 0.16; 'reedy': 0.16; 'subject:Unicode': 0.16; 'wrote:': 0.16; 'string': 0.17; '>': 0.18; 'windows': 0.20; '2015': 0.20; 'developer': 0.20; 'cc:addr:python.org': 0.20; 'cc:2*1': 0.22; '3.x': 0.22; 'ascii': 0.22; 'complete,': 0.22; 'trying': 0.22; 'dec': 0.23; 'unix': 0.24; 'header:In-Reply-To:1': 0.24; 'connected': 0.27; 'error': 0.27; 'fri,': 0.27; 'mostly': 0.27; 'message- id:@mail.gmail.com': 0.27; 'device': 0.28; 'away.': 0.29; 'itself,': 0.29; 'thinks': 0.29; 'windows,': 0.29; 'character': 0.29; 'print': 0.30; 'checked': 0.31; 'guess': 0.31; 'rules': 0.31; 'returned': 0.32; 'idle': 0.33; 'right?': 0.33; 'true.': 0.33; 'windows.': 0.33; 'running': 0.34; 'received:google.com': 0.35; 'text': 0.35; 'unicode': 0.35; 'but': 0.36; 'should': 0.36; 'received:209.85': 0.36; 'pm,': 0.36; 'subject:: ': 0.37; 'display': 0.37; 'thought': 0.37; 'received:209': 0.38; 'data': 0.39; 'sure': 0.39; 'some': 0.40; 'entire': 0.61; 'default': 0.61; 'within': 0.64; 'soon': 0.65; '3.4': 0.84; 'oscar': 0.84; 'belt': 0.91
DKIM-Signature	v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=8ztxeprU7TDnuS7F19rUpgsbTVpwZM0ghyUU2MnGBB0=; b=LcR/7DMuVjtVtu+NQx1nrFM4VfbCZTa1aoQMQFgU6i7GVUeq7K6p3xZ9yn1yoHR08a kQEDEf28Wv3BTnjbPWsUEjqvPJnhG3/8uLZU9FPyZZXK0/2+TxGKk3BpzoWhwyvm7e92 2bCZLywYJ2fDeoAOT2nvTI6wEiTu7Rp5KGmWyufPLbA+L76cppEAlqrYdawF8pXnLUYp 6e8CEfroGtBFlHCSrUsh7RLXfSM6dxkec89omJSD1v7ztR6+3RbkD19RXLiNzOG6tznV 48HPQB+dcjTiATZzlV7qRIYIM2VJe2KXpiB75KZ0uvvXFCRjeoDVBJAHFSdYlFBF7a7s GnCw==
X-Received	by 10.25.81.17 with SMTP id f17mr8325419lfb.90.1449312381188; Sat, 05 Dec 2015 02:46:21 -0800 (PST)
In-Reply-To	<20151205010827.2a82c6bc@imp>
X-Content-Filtered-By	Mailman/MimeDel 2.1.20+
X-BeenThere	python-list@python.org
X-Mailman-Version	2.1.20+
Precedence	list
List-Id	General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe	<https://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive	<http://mail.python.org/pipermail/python-list/>
List-Post	<mailto:python-list@python.org>
List-Help	<mailto:python-list-request@python.org?subject=help>
List-Subscribe	<https://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Xref	csiph.com comp.lang.python:100033

Show key headers only | View raw

On 5 Dec 2015 06:10, "D'Arcy J.M. Cain" <darcy@vybenetworks.com> wrote:
>
> On Fri, 4 Dec 2015 18:28:22 -0500
> Terry Reedy <tjreedy@udel.edu> wrote:
> > On 12/4/2015 1:07 PM, D'Arcy J.M. Cain wrote:
> > > I thought that going to Python 3.4 would solve my Unicode issues
> >
> > Within Python itself, that should be mostly true.  As soon as you
> > send text to a display, the rules of the display device take over.
>
> OK but my display (xterm) can display those characters.  I see it when
> I dump unicode text from my database.
>
> > > #! /usr/bin/python3
> > > # -*- coding: UTF-8 -*-
> >
> > Redundant, as this is the default for 3.x
>
> I assumed so but belt and suspenders, right?
>
> > Tk widgets, and hence IDLE windows, will print any character from
> > \u0000 to \uffff without raising, even if the result is blank or �.
> > Higher codepoints fail, but allowing the entire BMP is better than
> > any Windows codepage.
>
> Not sure I follow all this but to be clear, I am not using Tk, Idle or
> Windows.  I guess I should have mentioned that I am on Unix but I
> thought that the hash-bang would have given that away.  To be complete,
> I am running xterms on Xubuntu connected to NetBSD 7.0.  The data is
> coming from a PostgreSQL 9.3.5 database.  I am using a beta of PyGreSQL
> 5.0 (I am the lead developer for it) and I checked and the type
> returned is str, not bytes.  The database encoding is UTF8.

Yeah but the error you showed was from print trying to encode the string as
ASCII. For some reason Python thinks that stdout is ASCII I think. So I
repeat: what is SYS.stdout.encoding? If you're using xterm I think it will
be derived from LANG. So what's LANG?

--
Oscar

Back to comp.lang.python | Previous | Next | Find similar | Unroll thread

Thread

Re: Unicode failure Oscar Benjamin <oscar.j.benjamin@gmail.com> - 2015-12-05 10:46 +0000

csiph-web