Path: csiph.com!v102.xanadu-bbs.net!xanadu-bbs.net!feeder.erje.net!eu.feeder.erje.net!newsfeed.xs4all.nl!newsfeed4a.news.xs4all.nl!xs4all!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
Return-Path: <pengyu.ut@gmail.com>
X-Original-To: python-list@python.org
Delivered-To: python-list@mail.python.org
X-Spam-Status: OK 0.002
X-Spam-Evidence: '*H*': 1.00; '*S*': 0.00; 'subject:not': 0.03; "subject:' ": 0.07; 'bash': 0.09; 'encode': 0.09; 'subject:position': 0.09; 'python': 0.11; '"u"': 0.16; 'codec': 0.16; 'hex': 0.16; 'ordinal': 0.16; 'subject: \n ': 0.16; 'do.': 0.18; 'trying': 0.19; 'thanks.': 0.20; 'to:name:python- list@python.org': 0.22; 'print': 0.22; 'error': 0.23; 'character': 0.29; 'message-id:@mail.gmail.com': 0.30; 'file': 0.32; '(most': 0.33; 'skip:# 10': 0.33; "can't": 0.35; 'received:google.com': 0.35; 'there': 0.35; 'hi,': 0.36; 'to:addr:python-list': 0.38; 'recent': 0.39; 'does': 0.39; 'skip:. 10': 0.39; 'subject:can': 0.39; 'to:addr:python.org': 0.39; 'skip:u 10': 0.60; 'such': 0.63
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type :content-transfer-encoding; bh=zctxkm0eTmg1nsKefZPEsDKJXsrksYkoIJS5gqi3UDE=; b=w/TzwCgk+5fev6YBIkPS5DECH+SVaN+ezvXgMvKXPvSGj17MYePEZ2Xt7ZmEyKTaFy SxYvPK7+WOTEoJshbzfm2ZBb0V+b1GY2WywAD/rbeV2AzgqG7zv3AX7ng7/7TdODm7GD 702RRh5SzAOoS2jz1uupspkIloR/CjgSHmLm6pNYY0QEbmAWmvmvP9FvSlerKa+r9Sqb YO9fZdd9HHfgZycKISMKKZ82vIGJfuAxXvECeDa8bY5e//jTmIn7R5u8BwHH1PI3P5oJ ja9iYxD17989ewQV/UcN98y7nrlQ9dJs3d3jdPxRFLH8rpsQMFyGUq5nqBAsnWIBMD/6 TDcg==
MIME-Version: 1.0
X-Received: by 10.50.176.129 with SMTP id ci1mr2106396igc.40.1421205975638; Tue, 13 Jan 2015 19:26:15 -0800 (PST)
Date: Tue, 13 Jan 2015 21:26:15 -0600
Subject: UnicodeEncodeError: 'ascii' codec can't encode character u'\ua000' in position 0: ordinal not in range(128)
From: Peng Yu <pengyu.ut@gmail.com>
To: "python-list@python.org" <python-list@python.org>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
X-BeenThere: python-list@python.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe: <https://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive: <http://mail.python.org/pipermail/python-list/>
List-Post: <mailto:python-list@python.org>
List-Help: <mailto:python-list-request@python.org?subject=help>
List-Subscribe: <https://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups: comp.lang.python
Message-ID: <mailman.17699.1421205984.18130.python-list@python.org>
Lines: 39
NNTP-Posting-Host: 2001:888:2000:d::a6
X-Trace: 1421205984 news.xs4all.nl 2847 [2001:888:2000:d::a6]:46063
X-Complaints-To: abuse@xs4all.nl
Xref: csiph.com comp.lang.python:83731

Hi,

I am trying to understand what does encode() do. What are the hex
representations of "u" in main.py? Why there is UnicodeEncodeError
when main.py is piped to xxd? Why there is no such error when it is
not piped? Thanks.

~$ cat main.py
#!/usr/bin/env python

u =3D unichr(40960) + u'abcd' + unichr(1972)
print u
~$ cat main_encode.py
#!/usr/bin/env python

u =3D unichr(40960) + u'abcd' + unichr(1972)
print u.encode('utf-8')
$ ./main.py
=EA=80=80abcd=DE=B4
~$ cat main.sh
#!/usr/bin/env bash

set -v
./main.py | xxd
./main_encode.py | xxd

~$ ./main.sh
./main.py | xxd
Traceback (most recent call last):
  File "./main.py", line 4, in <module>
    print u
UnicodeEncodeError: 'ascii' codec can't encode character u'\ua000' in
position 0: ordinal not in range(128)
./main_encode.py | xxd
0000000: ea80 8061 6263 64de b40a                 ...abcd...

--=20
Regards,
Peng