Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #15084

Re: Unicode literals and byte string interpretation.

Show all headers | View raw

On Fri, Oct 28, 2011 at 2:05 PM, Fletcher Johnson <flt.johnson@gmail.com> wrote:
> If I create a new Unicode object u'\x82\xb1\x82\xea\x82\xcd' how does
> this creation process interpret the bytes in the byte string? Does it
> assume the string represents a utf-16 encoding, at utf-8 encoding,
> etc...?
>
> For reference the string is これは in the 'shift-jis' encoding.

Encodings define how characters are represented in bytes. I think
probably what you're looking for is a byte string with those hex
values in it, which you can then turn into a Unicode string:

>>> a=b'\x82\xb1\x82\xea\x82\xcd'
>>> unicode(a,"shift-jis")    # use 'str' instead of 'unicode' in Python 3
u'\u3053\u308c\u306f'

The u'....' notation is for Unicode strings, which are not encoded in
any way. The last line of the above is a valid way of entering that
string in your source code, identifying Unicode characters by their
codepoints.

ChrisA

Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Unicode literals and byte string interpretation. Fletcher Johnson <flt.johnson@gmail.com> - 2011-10-27 20:05 -0700
  Re: Unicode literals and byte string interpretation. David Riley <fraveydank@gmail.com> - 2011-10-27 23:37 -0400
  Re: Unicode literals and byte string interpretation. Chris Angelico <rosuav@gmail.com> - 2011-10-28 14:38 +1100
  Re: Unicode literals and byte string interpretation. Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2011-10-28 07:06 +0000
    Re: Unicode literals and byte string interpretation. Fletcher Johnson <flt.johnson@gmail.com> - 2011-10-31 21:01 -0700

csiph-web