Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #100459 > unrolled thread

subprocess.call with non-ASCII arguments?

Back to article view | Back to comp.lang.python

Contents

  subprocess.call with non-ASCII arguments? Ulli Horlacher <framstag@rus.uni-stuttgart.de> - 2015-12-15 14:25 +0000
    Re: subprocess.call with non-ASCII arguments? Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2015-12-15 10:03 -0500
      Re: subprocess.call with non-ASCII arguments? Ulli Horlacher <framstag@rus.uni-stuttgart.de> - 2015-12-15 17:57 +0000
        Re: subprocess.call with non-ASCII arguments? Dennis Lee Bieber <wlfraed@ix.netcom.com> - 2015-12-15 21:20 -0500
      Re: subprocess.call with non-ASCII arguments? Ulli Horlacher <framstag@rus.uni-stuttgart.de> - 2015-12-16 12:44 +0000
        Re: subprocess.call with non-ASCII arguments? wxjmfauth@gmail.com - 2015-12-16 06:55 -0800
    Re: subprocess.call with non-ASCII arguments? Ulli Horlacher <framstag@rus.uni-stuttgart.de> - 2015-12-15 15:05 +0000
    Re: subprocess.call with non-ASCII arguments? Laura Creighton <lac@openend.se> - 2015-12-15 16:08 +0100

#100459 — subprocess.call with non-ASCII arguments?

(My first posting seems to got lost)

I want to create a zip file within a Python 2.7 program on windows.

My code:

  cmd = ['7za.exe','a','-tzip',archive] + files
  status = subprocess.call(cmd)

leads to:

  File "fexit.py", line 971, in sendfile_retry
    status = subprocess.call(cmd)
  File "C:\Python27\lib\subprocess.py", line 522, in call
    return Popen(*popenargs, **kwargs).wait()
  File "C:\Python27\lib\subprocess.py", line 710, in __init__
    errread, errwrite)
  File "C:\Python27\lib\subprocess.py", line 958, in _execute_child
    startupinfo)
UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 87:
 ordinal not in range(128)


This is because the array "files" contains filenames with non-ASCII
characters.

So, the problem is in subprocess.py, which I cannot modify.


Instead of calling a 7z subprocess with non-ASCII arguments I tried to
call it with a listfile: it starts with a "@" and contains the names of
the files to be packed into the arcive. It is a special 7z feature.

New code:

  fileslist = archive + '.list'
  flo = open(fileslist,'w')
  for file in files: print(file,file=flo)
  flo.close()
  cmd = ['7za.exe','a','-tzip',archive,'@'+fileslist]
  status = subprocess.call(cmd)


But with that I get a new error:

  File "fexit.py", line 959, in sendfile_retry
    for file in files: print(file,file=flo)
UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 8:
  ordinal not in range(128)


I get the same error message, when i use:
      flo = open(fileslist,'wb')
      

How can I tell open() or print() that I want to write non-ASCII ?



-- 
Ullrich Horlacher              Server und Virtualisierung
Rechenzentrum IZUS/TIK         E-Mail: horlacher@tik.uni-stuttgart.de
Universitaet Stuttgart         Tel:    ++49-711-68565868
Allmandring 30a                Fax:    ++49-711-682357
70550 Stuttgart (Germany)      WWW:    http://www.tik.uni-stuttgart.de/

[toc] | [next] | [standalone]

#100460

On Tue, 15 Dec 2015 14:25:50 +0000 (UTC), Ulli Horlacher
<framstag@rus.uni-stuttgart.de> declaimed the following:

>(My first posting seems to got lost)
>
>I want to create a zip file within a Python 2.7 program on windows.
>
>My code:
>
>  cmd = ['7za.exe','a','-tzip',archive] + files
>  status = subprocess.call(cmd)
>
	My first thought would be...

	WHY spawn an OS dependent subprocess...

	Python has a zipfile library that is portable between OS. Along with
libraries for gzip, bzip2, and tarfiles...

-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
    wlfraed@ix.netcom.com    HTTP://wlfraed.home.netcom.com/

[toc] | [prev] | [next] | [standalone]

#100479

Dennis Lee Bieber <wlfraed@ix.netcom.com> wrote:

>         Python has a zipfile library that is portable between OS. Along with
> libraries for gzip, bzip2, and tarfiles...

Ohh.. this is new to me!

https://docs.python.org/2/library/tarfile.html
https://docs.python.org/2/library/zipfile.html

What is missing in the documentation:
Is the tar/zip file generation done in memory or will it be written
directly to disk?

My program creates zip or tar files up to TB size. This cannot be done in
memory.


-- 
Ullrich Horlacher              Server und Virtualisierung
Rechenzentrum IZUS/TIK         E-Mail: horlacher@tik.uni-stuttgart.de
Universitaet Stuttgart         Tel:    ++49-711-68565868
Allmandring 30a                Fax:    ++49-711-682357
70550 Stuttgart (Germany)      WWW:    http://www.tik.uni-stuttgart.de/

[toc] | [prev] | [next] | [standalone]

#100488

On Tue, 15 Dec 2015 17:57:18 +0000 (UTC), Ulli Horlacher
<framstag@rus.uni-stuttgart.de> declaimed the following:

>Dennis Lee Bieber <wlfraed@ix.netcom.com> wrote:
>
>>         Python has a zipfile library that is portable between OS. Along with
>> libraries for gzip, bzip2, and tarfiles...
>
>Ohh.. this is new to me!
>
>https://docs.python.org/2/library/tarfile.html
>https://docs.python.org/2/library/zipfile.html
>
>What is missing in the documentation:
>Is the tar/zip file generation done in memory or will it be written
>directly to disk?
>

	From the help file...

"""
class zipfile.ZipFile(file[, mode[, compression[, allowZip64]]]) 
Open a ZIP file, where file can be either a path to a file (a string) or a
file-like object.
"""

	I'd interpret that to mean that it tends to work with an on-disk file.
If you've given it a file name for the archive it likely is not building a
memory image and writing it all only when closed.


"""
ZipFile.write(filename[, arcname[, compress_type]]) 
Write the file named filename to the archive, giving it the archive name
arcname (by default, this will be the same as filename, but without a drive
letter and with leading path separators removed).
"""

	I'll admit it is not clear if the individual file is first compressed
in memory and then appended to disk file (I'm not that up on the internals
of ZIP files; a true TAR file is just a concatenation of uncompressed
files; .tar.gz indicates that the concatenated file is subsequently
compressed).
-- 
	Wulfraed                 Dennis Lee Bieber         AF6VN
    wlfraed@ix.netcom.com    HTTP://wlfraed.home.netcom.com/

[toc] | [prev] | [next] | [standalone]

#100504

Dennis Lee Bieber <wlfraed@ix.netcom.com> wrote:

> >I want to create a zip file within a Python 2.7 program on windows.
> >
> >My code:
> >
> >  cmd = ['7za.exe','a','-tzip',archive] + files
> >  status = subprocess.call(cmd)
> >
>         My first thought would be...
> 
>         WHY spawn an OS dependent subprocess...
> 
>         Python has a zipfile library that is portable between OS. Along with
> libraries for gzip, bzip2, and tarfiles...

Great hint!

With the python modules zipfile and tarfile I have no more problems with
non-ASCII filenames! It needed a little bit more programming with
os.walk(), because zipfile cannot add directories recursivly.

S:\>python fexit.py * framstag
Container name: test
creating C:\Users\admin\AppData\Local\Temp\fex\test.zip
zip dist\fexit.exe
zip mf.cmd
zip fex.ico
zip fexit.spec
zip build\fexit\fexit.exe.manifest
zip build\fexit\out00-Analysis.toc
zip build\fexit\out00-EXE.toc
zip build\fexit\out00-PKG.pkg
zip build\fexit\out00-PKG.toc
zip build\fexit\out00-PYZ.pyz
zip build\fexit\out00-PYZ.toc
zip build\fexit\out00-Tree.toc
zip build\fexit\out01-Tree.toc
zip build\fexit\warnfexit.txt
zip fexit.py
zip tar.py
zip zip.py

Recipient: framstag@rus.uni-stuttgart.de
test.zip: 13 MB of 13 MB (100%) 28540 kB/s

-- 
Ullrich Horlacher              Server und Virtualisierung
Rechenzentrum IZUS/TIK         E-Mail: horlacher@tik.uni-stuttgart.de
Universitaet Stuttgart         Tel:    ++49-711-68565868
Allmandring 30a                Fax:    ++49-711-682357
70550 Stuttgart (Germany)      WWW:    http://www.tik.uni-stuttgart.de/

[toc] | [prev] | [next] | [standalone]

#100512

Le mercredi 16 décembre 2015 13:44:36 UTC+1, Ulli Horlacher a écrit :
...


Your application will never work. Or more precisely,
it may only work by chance.
I guess, your understanding of non ascii file names
really means *non ascii, but cp1252 characters*, like
ö, Ä, ß, ü, ... (to stick with German.)

Python 3? No, it's not the solution.

[toc] | [prev] | [next] | [standalone]

#100461

Ulli Horlacher <framstag@rus.uni-stuttgart.de> wrote:

> Instead of calling a 7z subprocess with non-ASCII arguments I tried to
> call it with a listfile: it starts with a "@" and contains the names of
> the files to be packed into the arcive. It is a special 7z feature.
> 
> New code:
> 
>   fileslist = archive + '.list'
>   flo = open(fileslist,'w')
>   for file in files: print(file,file=flo)
>   flo.close()
>   cmd = ['7za.exe','a','-tzip',archive,'@'+fileslist]
>   status = subprocess.call(cmd)
> 
> 
> But with that I get a new error:
> 
>   File "fexit.py", line 959, in sendfile_retry
>     for file in files: print(file,file=flo)
> UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 8:
>   ordinal not in range(128)

I found a partial solution:

    for file in files: print(file.encode('utf8'),file=flo)

But this works only for files I get from Tk askopenfilename(), not for
files from sys.argv[]
Then I see:

S:\>python fexit.py -a x.zip C:\Users\admin\_XöX.exe .
files selected:

"C:\Users\admin\_X÷X.exe"
2015-12-07 16:17:15
114 kB

Traceback (most recent call last):
  File "fexit.py", line 2166, in <module>
    wexit(main())
  File "fexit.py", line 260, in main
    status = sendfile_retry(files,recipient,comment)
  File "fexit.py", line 959, in sendfile_retry
    for file in files: print(file.encode('utf8'),file=flo)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xf6 in position 17:
 ordinal not in range(128)


-- 
Ullrich Horlacher              Server und Virtualisierung
Rechenzentrum IZUS/TIK         E-Mail: horlacher@tik.uni-stuttgart.de
Universitaet Stuttgart         Tel:    ++49-711-68565868
Allmandring 30a                Fax:    ++49-711-682357
70550 Stuttgart (Germany)      WWW:    http://www.tik.uni-stuttgart.de/

[toc] | [prev] | [next] | [standalone]

#100462

In a message of Tue, 15 Dec 2015 14:25:50 +0000, Ulli Horlacher writes:
>(My first posting seems to got lost)
>
>I want to create a zip file within a Python 2.7 program on windows.
>
>My code:
>
>  cmd = ['7za.exe','a','-tzip',archive] + files
>  status = subprocess.call(cmd)
>
>leads to:
>
>  File "fexit.py", line 971, in sendfile_retry
>    status = subprocess.call(cmd)
>  File "C:\Python27\lib\subprocess.py", line 522, in call
>    return Popen(*popenargs, **kwargs).wait()
>  File "C:\Python27\lib\subprocess.py", line 710, in __init__
>    errread, errwrite)
>  File "C:\Python27\lib\subprocess.py", line 958, in _execute_child
>    startupinfo)
>UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 87:
> ordinal not in range(128)
>
>
>This is because the array "files" contains filenames with non-ASCII
>characters.
>
>So, the problem is in subprocess.py, which I cannot modify.
>
>
>Instead of calling a 7z subprocess with non-ASCII arguments I tried to
>call it with a listfile: it starts with a "@" and contains the names of
>the files to be packed into the arcive. It is a special 7z feature.
>
>New code:
>
>  fileslist = archive + '.list'
>  flo = open(fileslist,'w')
>  for file in files: print(file,file=flo)
>  flo.close()
>  cmd = ['7za.exe','a','-tzip',archive,'@'+fileslist]
>  status = subprocess.call(cmd)
>
>
>But with that I get a new error:
>
>  File "fexit.py", line 959, in sendfile_retry
>    for file in files: print(file,file=flo)
>UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 8:
>  ordinal not in range(128)
>
>
>I get the same error message, when i use:
>      flo = open(fileslist,'wb')
>      
>
>How can I tell open() or print() that I want to write non-ASCII ?

see if setting the environment variable PYTHONIOENCODING
https://docs.python.org/2/using/cmdline.html

works for you.  No promises, not a windows user.

Laura

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web