Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #63781

Re: Problem writing some strings (UnicodeEncodeError)

Show all headers | View raw

Paulo da Silva wrote:

> I am using a python3 script to produce a bash script from lots of
> filenames got using os.walk.
> 
> I have a template string for each bash command in which I replace a
> special string with the filename and then write the command to the bash
> script file.
> 
> Something like this:
> 
> shf=open(bashfilename,'w')
> filenames=getfilenames() # uses os.walk
> for fn in filenames:
> ...
> cmd=templ.replace("<fn>",fn)
> shf.write(cmd)
> 
> For certain filenames I got a UnicodeEncodeError exception at
> shf.write(cmd)!
> I use utf-8 and have # -*- coding: utf-8 -*- in the source .py.
> 
> How can I fix this?
> 
> Thanks for any help/comments.

You make it harder to debug your problem by not giving the complete 
traceback. If the error message contains 'surrogates not allowed' like in 
the demo below

>>> with open("tmp.txt", "w") as f:
...     f.write("\udcef")
... 
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
UnicodeEncodeError: 'utf-8' codec can't encode character '\udcef' in 
position 0: surrogates not allowed

you have filenames that are not valid UTF-8 on your harddisk. 

A possible fix would be to use bytes instead of str. For that you need to 
open `bashfilename` in binary mode ("wb") and pass bytes to the os.walk() 
call. 

Or you just go and fix the offending names.

Back to comp.lang.python | Previous | Next — Previous in thread | Next in thread | Find similar | Unroll thread

Thread

Problem writing some strings (UnicodeEncodeError) Paulo da Silva <p_s_d_a_s_i_l_v_a@netcabo.pt> - 2014-01-12 15:36 +0000
  Re: Problem writing some strings (UnicodeEncodeError) Peter Otten <__peter__@web.de> - 2014-01-12 17:23 +0100
    Re: Problem writing some strings (UnicodeEncodeError) Paulo da Silva <p_s_d_a_s_i_l_v_a@netcabo.pt> - 2014-01-12 17:51 +0000
      Re: Problem writing some strings (UnicodeEncodeError) Peter Otten <__peter__@web.de> - 2014-01-12 19:50 +0100
        Re: Problem writing some strings (UnicodeEncodeError) Paulo da Silva <p_s_d_a_s_i_l_v_a@netcabo.pt> - 2014-01-12 19:41 +0000
          Re: Problem writing some strings (UnicodeEncodeError) Peter Otten <__peter__@web.de> - 2014-01-12 21:29 +0100
            Re: Problem writing some strings (UnicodeEncodeError) Paulo da Silva <p_s_d_a_s_i_l_v_a@netcabo.pt> - 2014-01-12 23:53 +0000
              Re: Problem writing some strings (UnicodeEncodeError) Peter Otten <__peter__@web.de> - 2014-01-13 09:48 +0100
              Re: Problem writing some strings (UnicodeEncodeError) Peter Otten <__peter__@web.de> - 2014-01-13 09:58 +0100
                Re: Problem writing some strings (UnicodeEncodeError) Paulo da Silva <p_s_d_a_s_i_l_v_a@netcabo.pt> - 2014-01-13 16:14 +0000
                Re: Problem writing some strings (UnicodeEncodeError) Peter Otten <__peter__@web.de> - 2014-01-13 18:29 +0100
                Re: Problem writing some strings (UnicodeEncodeError) Paulo da Silva <p_s_d_a_s_i_l_v_a@netcabo.pt> - 2014-01-13 18:44 +0000
  Re: Problem writing some strings (UnicodeEncodeError) Emile van Sebille <emile@fenx.com> - 2014-01-12 08:55 -0800

csiph-web