Path: csiph.com!v102.xanadu-bbs.net!xanadu-bbs.net!goblin2!goblin.stu.neva.ru!newsfeed1.swip.net!uio.no!news.tele.dk!news.tele.dk!small.news.tele.dk!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail
Date: Sun, 17 Aug 2014 07:32:07 +0200
From: Dominique Ramaekers <dominique@ramaekers-stassart.be>
User-Agent: Mozilla/5.0 (X11; Linux i686; rv:31.0) Gecko/20100101 Thunderbird/31.0
MIME-Version: 1.0
To: python-list@python.org
Subject: Re: Unicode in cgi-script with apache2
References: <mailman.13038.1408130249.18130.python-list@python.org> <satHv.195207$ze2.61877@fx28.am4> <mailman.13054.1408229123.18130.python-list@python.org> <lsp5ab$sjv$1@dont-email.me>
In-Reply-To: <lsp5ab$sjv$1@dont-email.me>
Content-Type: text/plain; charset=windows-1252; format=flowed
Content-Transfer-Encoding: 7bit
Precedence: list
Newsgroups: comp.lang.python
Message-ID: <mailman.13058.1408253857.18130.python-list@python.org>
Lines: 83
NNTP-Posting-Host: 2001:888:2000:d::a6
Xref: csiph.com comp.lang.python:76414

* My system is a linux-box.

* I've tried using encoding="utf-8". It didn't fix things.

* That print uses sys.stdout would explain, using sys.stdout isn't better.

* My locale and the system-wide locale is UTF-8. Using SetEnv 
PYTHONIOENCODING utf-8 didn't fix things

* The file is encoded UTF-8...

I can not speak for anybody else but in my search I don't believe to 
have read about someone who had the problem on a Windows-system. They 
all used linux (different kinds of flavors) or OS-X... This is the first 
time I've encountered a situation where Windows is better in encoding 
issues :P +1 for Microsoft...

I think that Apache (*nix versions) doesn't tell Python, she's accepting 
UTF-8. Or Python doesn't listen right... Maybe I should place a bug 
report in both projects?


Op 17-08-14 om 04:50 schreef Denis McMahon:
> On Sun, 17 Aug 2014 00:36:14 +0200, Dominique Ramaekers wrote:
>
>> What seems to be the problem:
>> My Script was ok. I know this because in the terminal I got my expected
>> output. Python3 uses UTF-8 coding as a standard. The problem is, when
>> python 'prints' to the apache interface, it translates the string to
>> ascii. (Why, I never found an answer).
> Is the apache server running on a linux or a windows platform?
>
> The problem may not be python, it may be the underlying OS. I wonder if
> apache is spawning a process for python though, and if so whether it is
> in some way constraining the character set available to stdout of the
> spawned process.
>
>  From your other message, the error appears to be a python error on
> reading the input file. For some reason python seems to be trying to
> interpret the file it is reading as ascii.
>
> I wonder if specifying the binary data parameter and / or utf-8 encoding
> when opening the file might help.
>
> eg:
>
> f = open( "/var/www/cgi-data/index.html", "rb" )
> f = open( "/var/www/cgi-data/index.html", "rb", encoding="utf-8" )
> f = open( "/var/www/cgi-data/index.html", "r", encoding="utf-8" )
>
> I've managed to drive down a bit further in the problem:
>
> print() goes to sys.stdout
>
> This is part of what the docs say about sys.stdout:
>
> """
> The character encoding is platform-dependent. Under Windows, if the
> stream is interactive (that is, if its isatty() method returns True), the
> console codepage is used, otherwise the ANSI code page. Under other
> platforms, the locale encoding is used (see locale.getpreferredencoding
> ()).
>
> Under all platforms though, you can override this value by setting the
> PYTHONIOENCODING environment variable before starting Python.
> """
>
> At this point, details of the OS become very significant. If your server
> is running on a windows platform you may need to figure out how to make
> apache set the PYTHONIOENCODING environment variable to "utf-8" (or
> whatever else is appropriate) before calling the python script.
>
> I believe that the following line in your httpd.conf may have the
> required effect.
>
> SetEnv PYTHONIOENCODING utf-8
>
> Of course, if the file is not encoded as utf-8, but rather something
> else, then use that as the encoding in the above suggestions. If the
> server is not running windows, then I'm not sure where the problem might
> be.
>