Path: csiph.com!usenet.pasdenom.info!weretis.net!feeder4.news.weretis.net!rt.uk.eu.org!newsfeed.xs4all.nl!newsfeed1.news.xs4all.nl!xs4all!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
To: python-list@python.org
From: Mark Lawrence <breamoreboy@yahoo.co.uk>
Subject: Re: Everything you did not want to know about Unicode in Python 3
Date: Tue, 13 May 2014 03:33:02 +0100
References: <mailman.9915.1399907977.18130.python-list@python.org> <8P7cv.78617$Sp6.8377@fx15.am4> <537172eb$0$29980$c3e8da3$5496439d@news.astraweb.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
User-Agent: Mozilla/5.0 (Windows NT 6.1; rv:24.0) Gecko/20100101 Thunderbird/24.5.0
In-Reply-To: <537172eb$0$29980$c3e8da3$5496439d@news.astraweb.com>
Precedence: list
Newsgroups: comp.lang.python
Message-ID: <mailman.9935.1399948395.18130.python-list@python.org>
Lines: 65
NNTP-Posting-Host: 2001:888:2000:d::a6
Xref: csiph.com comp.lang.python:71422

On 13/05/2014 02:18, Steven D'Aprano wrote:
> On Mon, 12 May 2014 17:47:48 +0000, alister wrote:
>
>> On Mon, 12 May 2014 16:19:17 +0100, Mark Lawrence wrote:
>>
>>> This was *NOT* written by our resident unicode expert
>>> http://lucumr.pocoo.org/2014/5/12/everything-about-unicode/
>>>
>>> Posted as I thought it would make a rather pleasant change from
>>> interminable threads about names vs values vs variables vs objects.
>>
>> Surely those example programs are not the pythonoic way to do things or
>> am i missing something?
>
> Feel free to show us your version of "cat" for Python then. Feel free to
> target any version you like. Don't forget to test it against files with
> names and content that:
>
> - aren't valid UTF-8;
>
> - are valid UTF-8, but not valid in the local encoding.
>
>
>
>> if those code samples are anything to go by this guy makes JMF look
>> sensible.
>
> Armin Ronacher is an extremely experienced and knowledgeable Python
> developer, and a Python core developer. He might be wrong, but he's not
> *obviously* wrong.
>
> Unicode is hard, not because Unicode is hard, but because of legacy
> problems. I can create a file on a machine that uses ISO-8859-7 for the
> file name, put JShift-JIS encoded text inside it, transfer it to a
> machine that uses Windows-1251 as the file system encoding, then SSH into
> that machine from a system using Big5, and try to make sense of it. If
> everybody used UTF-8 any time data touched a disk or network, we'd be
> laughing. It would all be so simple.
>
> Reading Armin's post, I think that all that is needed to simplify his
> Python 3 version is:
>
> - have a bytes version of sys.argv (bargv? argvb?) and read
>    the file names from that;
>
> - have a simple way to write bytes to stdout and stderr.
>
> Most programs won't need either of those, but file system utilities will.
>

I think http://bugs.python.org/issue8776 and 
http://bugs.python.org/issue8775 are relevant but both were placed in 
the small round filing cabinet.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask 
what you can do for our language.

Mark Lawrence

---
This email is free from viruses and malware because avast! Antivirus protection is active.
http://www.avast.com