Re: "More About Unicode in Python 2 and 3"

Path	csiph.com!usenet.pasdenom.info!aioe.org!news.stack.nl!newsfeed.xs4all.nl!newsfeed1.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail
Return-Path	<python.list@tim.thechases.com>
X-Original-To	python-list@python.org
Delivered-To	python-list@mail.python.org
X-Spam-Status	OK 0.003
X-Spam-Evidence	'H': 0.99; 'S': 0.00; 'encoding': 0.05; 'subject:Python': 0.06; '(all': 0.07; 'skip:\\ 20': 0.07; 'bytes,': 0.09; '-tkc': 0.16; '16))': 0.16; 'call)': 0.16; 'delimiter': 0.16; 'dump': 0.16; 'from:addr:python.list': 0.16; 'from:addr:tim.thechases.com': 0.16; 'from:name:tim chase': 0.16; 'hex': 0.16; 'subject:More': 0.16; 'subject:Unicode': 0.16; 'wrote:': 0.18; "skip:' 30": 0.19; '>>>': 0.22; 'bytes': 0.24; 'source': 0.25; 'header:In-Reply-To:1': 0.27; 'chris': 0.29; 'easier': 0.31; "skip:' 10": 0.31; 'subject:About': 0.31; 'skip:b 30': 0.33; 'but': 0.35; 'there': 0.35; 'charset:us-ascii': 0.36; 'turn': 0.37; 'two': 0.37; 'depends': 0.38; 'to:addr:python-list': 0.38; 'to:addr:python.org': 0.39; 'space': 0.40; 'how': 0.40; 'subject:"': 0.60; 'most': 0.60; "you're": 0.61; 'such': 0.63; 'more': 0.64; 'received:50.22': 0.84
Date	Mon, 6 Jan 2014 05:49:28 -0600
From	Tim Chase <python.list@tim.thechases.com>
To	python-list@python.org
Subject	Re: "More About Unicode in Python 2 and 3"
In-Reply-To	<CAPTjJmqNZyuvVvBu1zD7Ht7YySZ9weDLiOdL=6TNKHhfsqqH7g@mail.gmail.com>
References	<lablra$1mc$2@ger.gmane.org> <labmaj$8u2$1@ger.gmane.org> <lad05k$gf6$1@ger.gmane.org> <CAPTjJmqBeoTLxXiKVcsvk395qgKt+Qv+jF_sOpzi7CgZmBjQcw@mail.gmail.com> <52CA13BD.4050708@stoneleaf.us> <mailman.5001.1388976943.18130.python-list@python.org> <roy-7ED5DF.23241105012014@news.panix.com> <CAPTjJmqNZyuvVvBu1zD7Ht7YySZ9weDLiOdL=6TNKHhfsqqH7g@mail.gmail.com>
X-Mailer	Claws Mail 3.8.1 (GTK+ 2.24.10; x86_64-pc-linux-gnu)
Mime-Version	1.0
Content-Type	text/plain; charset=US-ASCII
Content-Transfer-Encoding	7bit
X-AntiAbuse	This header was added to track abuse, please include it with any abuse report
X-AntiAbuse	Primary Hostname - boston.accountservergroup.com
X-AntiAbuse	Original Domain - python.org
X-AntiAbuse	Originator/Caller UID/GID - [47 12] / [47 12]
X-AntiAbuse	Sender Address Domain - tim.thechases.com
X-Get-Message-Sender-Via	boston.accountservergroup.com: authenticated_id: tim@thechases.com
X-BeenThere	python-list@python.org
X-Mailman-Version	2.1.15
Precedence	list
List-Id	General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe	<https://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive	<http://mail.python.org/pipermail/python-list/>
List-Post	<mailto:python-list@python.org>
List-Help	<mailto:python-list-request@python.org?subject=help>
List-Subscribe	<https://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups	comp.lang.python
Message-ID	<mailman.5012.1389008911.18130.python-list@python.org> (permalink)
Lines	40
NNTP-Posting-Host	2001:888:2000:d::a6
X-Trace	1389008911 news.xs4all.nl 2903 [2001:888:2000:d::a6]:57916
X-Complaints-To	abuse@xs4all.nl
Xref	csiph.com comp.lang.python:63276

Show key headers only | View raw

On 2014-01-06 15:51, Chris Angelico wrote:
> >>> data = b"\x43\x6c\x67\x75\x62\x61" # is there an easier way to
> >>> turn a hex dump into a bytes literal?

Depends on how you source them:

# space separated:
>>> s1 = "43 6c 67 75 62 61"
>>> ''.join(chr(int(pair, 16)) for pair in s1.split())
'Clguba'

# all smooshed together:
>>> s2 = s1.replace(' ','')
>>> s2
'436c67756261'
>>> ''.join(chr(int(s2[i*2:(i+1)*2], 16)) for i in range(len(s2)/2))
'Clguba'

# as \xHH escaped:
>>> s3 = ''.join('\\x'+s2[i*2:(i+1)*2] for i in range(len(s2)/2))
>>> print(s3)
\x43\x6c\x67\x75\x62\x61
>>> print(b3)
b'\\x43\\x6c\\x67\\x75\\x62\\x61'
>>> b3.decode('unicode_escape')
'Clguba'

It might get more complex if you're not just dealing with bytes, or
if you have some other encoding scheme, but "s1" (space-separated, or
some other delimiter such as colon-separated that can be passed
to the .split() call) and "s2" (all smooshed together) are the two I
encounter most frequently.

-tkc

Thread

Re: "More About Unicode in Python 2 and 3" Chris Angelico <rosuav@gmail.com> - 2014-01-06 13:55 +1100
  Re: "More About Unicode in Python 2 and 3" Roy Smith <roy@panix.com> - 2014-01-05 23:24 -0500
    Re: "More About Unicode in Python 2 and 3" Tim Chase <python.list@tim.thechases.com> - 2014-01-05 22:41 -0600
      Re: "More About Unicode in Python 2 and 3" Roy Smith <roy@panix.com> - 2014-01-05 23:49 -0500
        Re: "More About Unicode in Python 2 and 3" Chris Angelico <rosuav@gmail.com> - 2014-01-06 15:59 +1100
    Re: "More About Unicode in Python 2 and 3" Chris Angelico <rosuav@gmail.com> - 2014-01-06 15:51 +1100
    Re: "More About Unicode in Python 2 and 3" Tim Chase <python.list@tim.thechases.com> - 2014-01-06 05:49 -0600
    Re: "More About Unicode in Python 2 and 3" Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-01-07 03:24 +1100
      Re: "More About Unicode in Python 2 and 3" Chris Angelico <rosuav@gmail.com> - 2014-01-07 03:30 +1100
    Re: "More About Unicode in Python 2 and 3" Serhiy Storchaka <storchaka@gmail.com> - 2014-01-06 22:20 +0200
    Re: "More About Unicode in Python 2 and 3" Serhiy Storchaka <storchaka@gmail.com> - 2014-01-06 22:21 +0200
    Re: "More About Unicode in Python 2 and 3" Tim Chase <python.list@tim.thechases.com> - 2014-01-06 14:42 -0600
    Re: "More About Unicode in Python 2 and 3" Mark Lawrence <breamoreboy@yahoo.co.uk> - 2014-01-06 20:47 +0000
    Re: "More About Unicode in Python 2 and 3" Chris Angelico <rosuav@gmail.com> - 2014-01-07 10:06 +1100

csiph-web