Path: csiph.com!usenet.pasdenom.info!aioe.org!news.stack.nl!newsfeed.xs4all.nl!newsfeed5.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail Return-Path: X-Original-To: python-list@python.org Delivered-To: python-list@mail.python.org X-Spam-Status: OK 0.000 X-Spam-Evidence: '*H*': 1.00; '*S*': 0.00; 'ascii': 0.07; 'bytes.': 0.07; 'type,': 0.07; 'width': 0.07; 'subject:How': 0.09; 'python': 0.09; '32-bit': 0.09; 'already.': 0.09; 'cookies': 0.09; 'encode': 0.09; 'immutable': 0.09; 'pep': 0.09; 'subject:()': 0.09; 'subject:string': 0.09; 'subject:using': 0.09; 'terry': 0.09; 'thread,': 0.09; 'stored': 0.10; 'aug': 0.13; 'language': 0.14; 'folks': 0.15; 'consciously': 0.16; 'earlier.': 0.16; 'from:addr:rosuav': 0.16; 'from:name:chris angelico': 0.16; 'range.': 0.16; 'reedy': 0.16; 'sense,': 0.16; 'socket.': 0.16; 'subject:unicode': 0.16; 'subject:variable': 0.16; 'mon,': 0.16; 'string': 0.17; 'wrote:': 0.17; 'bytes': 0.17; 'copied': 0.17; 'string,': 0.17; 'unicode': 0.17; 'memory': 0.18; 'equivalent': 0.20; 'bit': 0.21; 'earlier': 0.21; 'received:209.85.214.174': 0.21; 'nearly': 0.23; 'somewhere': 0.24; 'header': 0.24; 'paul': 0.24; 'header:In-Reply-To:1': 0.25; 'am,': 0.27; 'implemented': 0.27; 'realize': 0.27; 'expansion': 0.27; 'first.': 0.27; 'message-id:@mail.gmail.com': 0.27; 'correct': 0.28; 'fixed': 0.28; 'fine': 0.28; 'obscure': 0.29; 'really,': 0.29; 'smart': 0.29; 'strings,': 0.29; "i'm": 0.29; 'maybe': 0.29; 'usually': 0.30; 'function': 0.30; 'stuff': 0.30; 'more,': 0.32; 'goes': 0.33; 'done,': 0.33; 'impression': 0.33; 'ones,': 0.33; 'to:addr :python-list': 0.33; 'languages': 0.33; 'that,': 0.34; "can't": 0.34; 'received:google.com': 0.34; 'clear': 0.35; 'whatever': 0.35; 'posting': 0.35; 'received:209.85': 0.35; 'something': 0.35; 'but': 0.36; 'characters': 0.36; "wasn't": 0.36; 'possible': 0.37; 'why': 0.37; 'received:209': 0.37; 'subject:: ': 0.38; 'to:addr:python.org': 0.39; 'received:209.85.214': 0.39; 'header:Received:5': 0.40; 'end': 0.40; 'further': 0.61; 'provide': 0.62; 'different': 0.63; 'more': 0.63; 'making': 0.64; '20,': 0.65; 'content,': 0.65; 'internet': 0.71; 'guaranteed': 0.76; 'pike': 0.84; 'subject:value': 0.84; 'viable': 0.84 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=v6VejVx9Vzp94oDfYBKGxYpWJ7KjsaYn676stbDi+CU=; b=TWURwQjBdDAh4kVyG68JYAyYXssRK+8ALNiYsMsUvynN/QoMj0W0ZZ8DScbLGbKgRx SNpqKjSDkOPZ3phdDXrpOAlF7IQDc8GHOpv5HTcDnAgbTz2iwiEbCe5edToQABGN9+px oLysUz4cRJE4fNJt47irEODS3ssrAeXl8PwDVgZhRbhKvKlu6B7ytPivDD6kCY2Cy+rP ib9B2fEmg7/ezdbki+NAjgp7uML/sUuyPJDtEAKPUx494SveieDmSOq2kXzCioRRoGqi 5j5Zjv4a2IpHWBkWC5yIUuhDYRNbZv8KhgcYup0pp35lODrzp1Vf1flxsh3VUNHdvSkp rIgg== MIME-Version: 1.0 In-Reply-To: References: <308df2af-abe7-4043-b199-0a39f440e0ab@googlegroups.com> <502f8a2a$0$29978$c3e8da3$5496439d@news.astraweb.com> <7xehn4vyya.fsf@ruckus.brouhaha.com> <5030832d$0$29978$c3e8da3$5496439d@news.astraweb.com> <7x8vdbmho6.fsf@ruckus.brouhaha.com> Date: Mon, 20 Aug 2012 08:42:53 +1000 Subject: Re: How do I display unicode value stored in a string variable using ord() From: Chris Angelico To: python-list@python.org Content-Type: text/plain; charset=ISO-8859-1 X-BeenThere: python-list@python.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: General discussion list for the Python programming language List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Newsgroups: comp.lang.python Message-ID: Lines: 39 NNTP-Posting-Host: 2001:888:2000:d::a6 X-Trace: 1345416176 news.xs4all.nl 6895 [2001:888:2000:d::a6]:45080 X-Complaints-To: abuse@xs4all.nl Xref: csiph.com comp.lang.python:27439 On Mon, Aug 20, 2012 at 3:34 AM, Terry Reedy wrote: > On 8/19/2012 4:04 AM, Paul Rubin wrote: >> I realize the folks who designed and implemented PEP 393 are very smart >> cookies and considered stuff carefully, while I'm just an internet user >> posting an immediate impression of something I hadn't seen before (I >> still use Python 2.6), but I still have to ask: if the 393 approach >> makes sense, why don't other languages do it? > > Python has often copied or borrowed, with adjustments. This time it is the > first. We will see how it goes, but it has been tested for nearly a year > already. Maybe it wasn't consciously borrowed, but whatever innovation is done, there's usually an obscure beardless language that did it earlier. :) Pike has a single string type, which can use the full Unicode range. If all codepoints are <256, the string width is 8 (measured in bits); if <65536, width is 16; otherwise 32. Using the inbuilt count_memory function (similar to the Python function used somewhere earlier in this thread, but which I can't at present put my finger to), I find that for strings of 16 bytes or more, there's a fixed 20-byte header plus the string content, stored in the correct number of bytes. (Pike strings, like Python ones, are immutable and do not need expansion room.) However, Python goes a bit further by making it VERY clear that this is a mere optimization, and that Unicode strings and bytes strings are completely different beasts. In Pike, it's possible to forget to encode something before (say) writing it to a socket. Everything works fine while you have only ASCII characters in the string, and then breaks when you have a >255 codepoint - or perhaps worse, when you have a 127