Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #18518 > unrolled thread

UnicodeEncodeError when piping stdout, but not when printing directly to the console

Back to article view | Back to comp.lang.python

Contents

  UnicodeEncodeError when piping stdout, but not when printing directly to the console Adam Funk <a24061@ducksburg.com> - 2012-01-04 21:15 +0000
    Re: UnicodeEncodeError when piping stdout, but not when printing directly to the console Peter Otten <__peter__@web.de> - 2012-01-04 22:49 +0100
      Re: UnicodeEncodeError when piping stdout, but not when printing directly to the console Adam Funk <a24061@ducksburg.com> - 2012-01-06 09:12 +0000
        Re: UnicodeEncodeError when piping stdout, but not when printing directly to the console Peter Otten <__peter__@web.de> - 2012-01-06 10:34 +0100
          Re: UnicodeEncodeError when piping stdout, but not when printing directly to the console Adam Funk <a24061@ducksburg.com> - 2012-01-06 14:22 +0000

#18518 — UnicodeEncodeError when piping stdout, but not when printing directly to the console

(I'm using Python 2.7.2+ on Ubuntu.)

When I'm running my program in an xterm, the print command with an
argument containing unicode works fine (it correctly detects my UTF-8
environment).  But when I run it with a pipe or redirect to a file (|
or >), unicode strings fail with the following (for example):

UnicodeEncodeError: 'ascii' codec can't encode character u'\u0107' in position 21: ordinal not in range(128)

How can I force python (preferably within my python program, rather
than having to set something externally) to treat stdout as UTF-8?


Thanks,
Adam


-- 
Nam Sibbyllam quidem Cumis ego ipse oculis meis vidi in ampulla 
pendere, et cum illi pueri dicerent: beable beable beable; respondebat 
illa: doidy doidy doidy.                                   [plorkwort]

[toc] | [next] | [standalone]

#18519

Adam Funk wrote:

> (I'm using Python 2.7.2+ on Ubuntu.)
> 
> When I'm running my program in an xterm, the print command with an
> argument containing unicode works fine (it correctly detects my UTF-8
> environment).  But when I run it with a pipe or redirect to a file (|
> or >), unicode strings fail with the following (for example):
> 
> UnicodeEncodeError: 'ascii' codec can't encode character u'\u0107' in
> position 21: ordinal not in range(128)
> 
> How can I force python (preferably within my python program, rather
> than having to set something externally) to treat stdout as UTF-8?


$ cat force_utf8.py
# -*- coding: utf-8 -*-
import sys

if sys.stdout.encoding is None:
    import codecs
    writer = codecs.getwriter("utf-8")
    sys.stdout = writer(sys.stdout)

print u"Ähnlich üblich nötig"

$ python force_utf8.py
Ähnlich üblich nötig

$ python force_utf8.py | cat
Ähnlich üblich nötig

[toc] | [prev] | [next] | [standalone]

#18592

On 2012-01-04, Peter Otten wrote:

> Adam Funk wrote:

>> How can I force python (preferably within my python program, rather
>> than having to set something externally) to treat stdout as UTF-8?
>
>
> $ cat force_utf8.py
> # -*- coding: utf-8 -*-
> import sys
>
> if sys.stdout.encoding is None:
>     import codecs
>     writer = codecs.getwriter("utf-8")
>     sys.stdout = writer(sys.stdout)
>
> print u"Ähnlich üblich nötig"

That's great, thanks!

I guess issues like this will magically go away when I eventually move
to Python 3?


-- 
Physics is like sex.  Sure, it may give some practical results,
but that's not why we do it.                  [Richard Feynman]

[toc] | [prev] | [next] | [standalone]

#18593

Adam Funk wrote:

> On 2012-01-04, Peter Otten wrote:
> 
>> Adam Funk wrote:
> 
>>> How can I force python (preferably within my python program, rather
>>> than having to set something externally) to treat stdout as UTF-8?
>>
>>
>> $ cat force_utf8.py
>> # -*- coding: utf-8 -*-
>> import sys
>>
>> if sys.stdout.encoding is None:
>>     import codecs
>>     writer = codecs.getwriter("utf-8")
>>     sys.stdout = writer(sys.stdout)
>>
>> print u"Ähnlich üblich nötig"
> 
> That's great, thanks!
> 
> I guess issues like this will magically go away when I eventually move
> to Python 3?

Not "magically", but UTF-8 has become the default encoding...

[toc] | [prev] | [next] | [standalone]

#18604

On 2012-01-06, Peter Otten wrote:

> Adam Funk wrote:
>
>> On 2012-01-04, Peter Otten wrote:
>> 
>>> Adam Funk wrote:
>> 
>>>> How can I force python (preferably within my python program, rather
>>>> than having to set something externally) to treat stdout as UTF-8?
>>>
>>>
>>> $ cat force_utf8.py
>>> # -*- coding: utf-8 -*-
>>> import sys
>>>
>>> if sys.stdout.encoding is None:
>>>     import codecs
>>>     writer = codecs.getwriter("utf-8")
>>>     sys.stdout = writer(sys.stdout)
>>>
>>> print u"Ähnlich üblich nötig"
>> 
>> That's great, thanks!
>> 
>> I guess issues like this will magically go away when I eventually move
>> to Python 3?
>
> Not "magically", but UTF-8 has become the default encoding...

Close enough!



-- 
When Elaine turned 11, her mother sent her to train under
Donald Knuth in his mountain hideaway.         [XKCD 342]

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web