Path: csiph.com!usenet.pasdenom.info!weretis.net!feeder1.news.weretis.net!feeder.erje.net!eu.feeder.erje.net!newsfeed.xs4all.nl!newsfeed5.news.xs4all.nl!xs4all!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
Newsgroups: comp.lang.python
Date: Fri, 9 Nov 2012 13:17:39 -0800 (PST)
In-Reply-To: <mailman.3508.1352483285.27098.python-list@python.org>
Complaints-To: groups-abuse@google.com
Injection-Info: glegroupsg2000goo.googlegroups.com; posting-host=108.78.40.138; posting-account=S-UcDQoAAACh3mXdFBHQR00lNytDt6nm
References: <3d4644f8-ab88-41c5-9a52-2a5678dd64c0@googlegroups.com> <mailman.3508.1352483285.27098.python-list@python.org>
User-Agent: G2/1.0
MIME-Version: 1.0
Subject: Re: Printing characters outside of the ASCII range
From: danielk <danielkleinad@gmail.com>
To: comp.lang.python@googlegroups.com
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
Cc: python-list@python.org, d@davea.name, danielk <danielkleinad@gmail.com>
Precedence: list
Message-ID: <mailman.3515.1352495868.27098.python-list@python.org>
Lines: 104
NNTP-Posting-Host: 2001:888:2000:d::a6
Xref: csiph.com comp.lang.python:33052

On Friday, November 9, 2012 12:48:05 PM UTC-5, Dave Angel wrote:
> On 11/09/2012 12:17 PM, danielk wrote:
>=20
> > I'm converting an application to Python 3. The app works fine on Python=
 2.
>=20
> >
>=20
> > Simply put, this simple one-liner:
>=20
> >
>=20
> > print(chr(254))
>=20
> >
>=20
> > errors out with:
>=20
> >
>=20
> > Traceback (most recent call last):
>=20
> >   File "D:\home\python\tst.py", line 1, in <module>
>=20
> >     print(chr(254))
>=20
> >   File "C:\Python33\lib\encodings\cp437.py", line 19, in encode
>=20
> >     return codecs.charmap_encode(input,self.errors,encoding_map)[0]
>=20
> > UnicodeEncodeError: 'charmap' codec can't encode character '\xfe' in po=
sition 0: character maps to <undefined>
>=20
> >
>=20
> > I'm using this character as a delimiter in my application.
>=20
> >
>=20
> > What do I have to do to convert this string so that it does not error o=
ut?
>=20
>=20
>=20
> What character do you want?  What characters do your console handle
>=20
> directly?  What does a "delimiter" mean for your particular console?
>=20
>=20
>=20
> Or are you just printing it for the fun of it, and the real purpose is
>=20
> for further processing, which will not go to the console?
>=20
>=20
>=20
> What kind of things will it be separating?  (strings, bytes ?)  Clearly
>=20
> you originally picked it as something unlikely to occur in those elements=
.
>=20
>=20
>=20
> When those things are combined with a separator between, how are the
>=20
> results going to be used?  Saved to a file?  Printed to console?  What?
>=20
>=20
>=20
> --=20
>=20
>=20
>=20
> DaveA

The database I'm using stores information as a 3-dimensional array. The del=
imiters between elements are chr(252), chr(253) and chr(254). So a record c=
an look like this (example only uses one of the delimiters for simplicity):

name + chr(254) + address + chr(254) + city + chr(254) + st + chr(254) + zi=
p

The other delimiters can be embedded within each field. For example, if the=
re were multiple addresses for 'name' then the 'address' field would look l=
ike this:

addr1 + chr(253) + addr2 + chr(253) + addr3 + etc ...

I use Python to connect to the database using subprocess.Popen to run a ser=
ver process. Python requests 'actions' like 'read' and 'write' to the serve=
r process, whereby the server process performs the actions. Some actions re=
quire that the server send back information in the form of records that con=
tain those delimiters.

I have __str__ and __repr__ methods in the classes but Python is choking on=
 those characters. Surely, I could convert those characters on the server b=
efore sending them to Python and that is what I'm probably going to do, so =
guess I've answered my own question. On Python 2, it just printed the 'exte=
nded' ASCII representation.

I guess the question I have is: How do you tell Python to use a specific en=
coding for 'print' statements when I know there will be characters outside =
of the ASCII range of 0-127?