Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #95641

Re: file.write() of non-ASCII characters differs in Interpreted Python than in script run

Show all headers | View raw

On Tue, Aug 25, 2015 at 2:19 PM, RAH <rene.heymans@gmail.com> wrote:
> Dear All,
>
> I experienced an incomprehensible behavior (I've spent already many hours on this subject): the `file.write('string')` provides an error in run mode and not when interpreted at the console. The string must contain non-ASCII characters. If all ASCII, there is no error.
>
> The following example shows what I can see. I must overlook something because I cannot think Python makes a difference between interpreted and run modes and yet ... Can someone please check that subject.
>
> Thank you in advance.
> René
>
> Code extract from WSGI application (reply.py)
> =============================================
>
>     request_body = environ['wsgi.input'].read(request_body_size)    # bytes
>     rb = request_body.decode()                                      # string
>     d = parse_qs(rb)                                                # dict
>
>     f = open('logbytes', 'ab')
>     g = open('logstr', 'a')
>     h = open('logdict', 'a')
>
>     f.write(request_body)
>     g.write(str(type(request_body)) + '\t' + str(type(rb)) + '\t' + str(type(d)) + '\n')
>     h.write(str(d) + '\n')      <--- line 28 of the application
>
>     h.close()
>     g.close()
>     f.close()
>
>
> Tail of Apache2 error.log
> =========================
>
> [Tue Aug 25 20:24:04.657933 2015] [wsgi:error] [pid 3677:tid 3029764928] [remote 192.168.1.5:27575]   File "reply.py", line 28, in application
> [Tue Aug 25 20:24:04.658001 2015] [wsgi:error] [pid 3677:tid 3029764928] [remote 192.168.1.5:27575]     h.write(str(d) + '\\n')
> [Tue Aug 25 20:24:04.658201 2015] [wsgi:error] [pid 3677:tid 3029764928] [remote 192.168.1.5:27575] UnicodeEncodeError: 'ascii' codec can't encode character '\\xc7' in position 15: ordinal not in range(128)
>

What version of Python is Apache2 using? From the looks of the error,
it is probably using some version of Python2, in which case you'll
need to manually encode the string and pick an encoding for the file
(via an encoding argument to the open function). I'd recommend using
UTF-8.

You can log out the value of sys.version to find out the version number.


> Trying similar code within the Python interpreter
> =================================================
>
> rse@Alibaba:~/test$ python
> Python 3.4.0 (default, Jun 19 2015, 14:18:46)
> [GCC 4.8.2] on linux
> Type "help", "copyright", "credits" or "license" for more information.
>>>> di = {'userName': ['Ça va !']}    <--- A dictionary
>>>> str(di)
> "{'userName': ['Ça va !']}"           <--- and its string representation
>>>> type(str(di))
> <class 'str'>                   <--- Is a string indeed
>>>> fi = open('essai', 'a')
>>>> fi.write(str(di) + '\n')
> 26                              <--- It works well
>>>> fi.close()
>>>>

In this run, you are using Python 3.4, which defaults to UTF-8.

Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

file.write() of non-ASCII characters differs in Interpreted Python than in script run RAH <rene.heymans@gmail.com> - 2015-08-25 14:19 -0700
  Re: file.write() of non-ASCII characters differs in Interpreted Python than in script run Chris Kaynor <ckaynor@zindagigames.com> - 2015-08-25 14:28 -0700
    Re: file.write() of non-ASCII characters differs in Interpreted Python than in script run RAH <rene.heymans@gmail.com> - 2015-08-26 02:12 -0700
      Re: file.write() of non-ASCII characters differs in Interpreted Python than in script run Chris Angelico <rosuav@gmail.com> - 2015-08-26 19:24 +1000
        Re: file.write() of non-ASCII characters differs in Interpreted Python than in script run RAH <rene.heymans@gmail.com> - 2015-08-26 07:24 -0700
  Re: file.write() of non-ASCII characters differs in Interpreted Python than in script run Chris Angelico <rosuav@gmail.com> - 2015-08-26 09:16 +1000
    Re: file.write() of non-ASCII characters differs in Interpreted Python than in script run RAH <rene.heymans@gmail.com> - 2015-08-26 07:18 -0700
  Re: file.write() of non-ASCII characters differs in Interpreted Python than in script run dieter <dieter@handshake.de> - 2015-08-26 07:51 +0200
    Re: file.write() of non-ASCII characters differs in Interpreted Python than in script run RAH <rene.heymans@gmail.com> - 2015-08-26 07:20 -0700
  Re: file.write() of non-ASCII characters differs in Interpreted Python than in script run Pete Dowdell <contact@stridebird.com> - 2015-08-26 14:09 +0700
    Re: file.write() of non-ASCII characters differs in Interpreted Python than in script run RAH <rene.heymans@gmail.com> - 2015-08-26 07:22 -0700
  Re: file.write() of non-ASCII characters differs in Interpreted Python than in script run RAH <rene.heymans@gmail.com> - 2015-08-26 08:02 -0700
    Re: file.write() of non-ASCII characters differs in Interpreted Python than in script run Chris Angelico <rosuav@gmail.com> - 2015-08-27 01:57 +1000
      Re: file.write() of non-ASCII characters differs in Interpreted Python than in script run RAH <rene.heymans@gmail.com> - 2015-08-26 12:23 -0700
        Re: file.write() of non-ASCII characters differs in Interpreted Python than in script run Chris Angelico <rosuav@gmail.com> - 2015-08-27 09:15 +1000
          Re: file.write() of non-ASCII characters differs in Interpreted Python than in script run Marko Rauhamaa <marko@pacujo.net> - 2015-08-27 08:59 +0300
  Re: file.write() of non-ASCII characters differs in Interpreted Python than in script run RAH <rene.heymans@gmail.com> - 2015-08-27 07:01 -0700

csiph-web